Wednesday, April 25, 2018

Spring Boot 2 generic JPA converter to encrypt and decrypt an entity attribute

A recent requirement in my work mandated that certain columns in a table be encrypted when stored in the database. One can use the javax.crypto package from the standard JDK to achieve this by writing service layer code to encrypt and decrypt data for each entity attribute (table column) that required it.

The entire code-base is built using Spring Boot 2 and JPA (Hibernate), so I was looking for a way to achieve the encryption and decryption without having to write code for each entity attribute.  Well, there is such a feature provided by JPA called Converters which provide a way for you to write code once and then apply it to any required table attribute by way of a simple @Converter annotation.

Purpose of this blog post

JPA allows writing 'conversion' code so that an entity attribute (table column) can be converted from one type to another and back. Though this is not a latest feature of JPA nor a bleeding edge topic, I am sharing how I wrote a generic converter for encryption and decryption of a string type table column.

Java and Cryptography

The humble JDK provides a very good cryptography package javax.crypto which provides many facilities to do all things related to cryptography, one of which is to encrypt and decrypt data (or strings). I will not go into the details but just note here that the javax.crypto.Cipher class provides the functionality of a cryptographic cipher or algorithm for encryption and decryption.

How does cryptography work?

I am not a cryptography expert, but understand enough of it to apply it to the problem at hand. Basically I want to encrypt a String and write it to a table column in the database and then decrypt it after reading it so that the other layers in the application can use it.

The crypto facilities basically use an ecryption/decryption algorithm to encrypt/decrypt the string and secure the data by applying a key value, which only you know, to encrpt/decrypt the data. That's it in a nutshell.

For more details on how Java cryptography works please see this excellent tutorial

JPA and Converter

JPA allows you to apply a @Converter annotation to any Entity attribute i.e. to any table column definition in your table defintion. This is possible via the AttributeConveter interface. A class implementing this interface needs to implement the methods from this interface


  • Y convertToDatabaseColumn(X attribute) - Converts the value stored in the entity attribute into the data representation to be stored in the database.
  • X convertToEntityAttribute(Y dbData) -  Converts the data stored in the database column into the value to be stored in the entity attribute. Note that it is the responsibility of the converter writer to specify the correct dbData type for the corresponding column for use by the JDBC driver: i.e., persistence providers are not expected to do such type conversion.

Writing the generic attribute converter

I needed a converter which would essentially take a String type attribute and then encrypt it. So, the method Y convertToDatabaseColumn(X attribute) method needed to take a string and return an encrypted version of it for storage.

The converter also has to read the stored encrypted string  and return the decrypted version. So, the method  X convertToEntityAttribute(Y dbData) needed to take the encrypted String, decrypt it and and return it to the other layers of the JPA mechanism.

One thought that came naturally to mind, was that I didn't want to write code once again when the need arose to encrypt and decrypt some other type of attribute, say a Date or something else. So the base attribute converter had to have just enough code to perform the encryption and decryption of a String. (Note that we want to store the data as an encrypted String in the table column). The other task of converting the entity attribute type to a String and vice versa would have to be done outside this base class to allow flexibility to allow using any attribute entity type.

With these points in mind, here is how I wrote this attribute converter.

Step 1 - Write the encryption and decryption code

As noted before, use Java cryptography to achieve encryption and decryption. For this a Cipher instance is needed with all its properties set appropriately. Then it can be used for encryption and decryption.

import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import java.security.Key;
import java.security.NoSuchAlgorithmException;
import java.security.spec.AlgorithmParameterSpec;

import javax.crypto.Cipher;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

/**
 * Utility class to generate an instance of {@link javax.crypto.Cipher}
 * 
 * @author Sunit Katkar, sunitkatkar@gmail.com
 * @since ver 1.0 (Apr 2018)
 * @version 1.0
 */
public class CipherMaker {

    private static final String CIPHER_INSTANCE_NAME = "AES/CBC/PKCS5Padding";
    private static final String SECRET_KEY_ALGORITHM = "AES";

    /**
     * @param encryptionMode
     *            - decides whether to ecrypt or decrypt data. Values accepted:
     *            {@link Cipher#ENCRYPT_MODE} for encryption and
     *            {@link Cipher#DECRYPT_MODE} for decryption.
     * @param key
     *            - the key to use for encrypting or decrypting data. This can be a
     *            simple String like "MySecretKey" or a more complex, hard to guess
     *            longer string
     * @return
     * @throws InvalidKeyException
     * @throws NoSuchPaddingException
     * @throws NoSuchAlgorithmException
     * @throws InvalidAlgorithmParameterException
     * @throws InvalidKeyException
     * @throws InvalidAlgorithmParameterException
     */
    public Cipher configureAndGetInstance(int encryptionMode, String key)
            throws InvalidKeyException, NoSuchPaddingException, 
            NoSuchAlgorithmException, InvalidAlgorithmParameterException, 
            InvalidKeyException, InvalidAlgorithmParameterException {

        Cipher cipher = Cipher.getInstance(CIPHER_INSTANCE_NAME);
        Key secretKey = new SecretKeySpec(key.getBytes(), SECRET_KEY_ALGORITHM);

        byte[] ivBytes = new byte[cipher.getBlockSize()];
        AlgorithmParameterSpec algorithmParameters = new IvParameterSpec(ivBytes);

        cipher.init(encryptionMode, secretKey, algorithmParameters);
        return cipher;
    }
}

Step 2 - Write the base attribute converter 

As noted before, the attribute converter needs to encrypt and decrypt. So, the CipherMaker class above will be useful here. The source code comments are self-explanatory.

import static org.apache.commons.lang3.StringUtils.isNotEmpty;

import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.Base64;

import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.persistence.AttributeConverter;

/**
 * Abstract base class for implementing the JPA Attribute Converter which will
 * encrypt and decrypt an Entity attribute (table column)
 * 
 * @author Sunit Katkar, sunitkatkar@gmail.com
 * @since ver 1.0 (Apr 2018)
 * @version 1.0
 * @param <X>
 */
public abstract class AbstractEncryptDecryptConverter<X>
  implements AttributeConverter<X, String> {

 /**
  * This is the key required for encryption/decryption. This is defined here
  * for example purpose. In production, this should come from a secure
  * location not accessible easily. In Spring Boot, one possible location is
  * the application.properties file. Though its not the most secure way, it
  * will keep this key out of the actual java code.
  */
 private static final String SECRET_ENCRYPTION_KEY = "MySuperSecretKey";

 /** CipherMaker is needed to configure and create instance of Cipher */
 private CipherMaker cipherMaker;

 /**
  * Constructor
  * 
  * @param cipherMaker
  */
 public AbstractEncryptDecryptConverter(CipherMaker cipherMaker) {
  this.cipherMaker = cipherMaker;
 }

 /**
  * Default constructor
  */
 public AbstractEncryptDecryptConverter() {
  this(new CipherMaker());
 }

 /*
  * (non-Javadoc)
  * 
  * @see
  * javax.persistence.AttributeConverter#convertToDatabaseColumn(java.lang.
  * Object)
  */
 @Override
 public String convertToDatabaseColumn(X attribute) {
  if (isNotEmpty(SECRET_ENCRYPTION_KEY) && isNotNullOrEmpty(attribute)) {
   try {
    Cipher cipher = cipherMaker.configureAndGetInstance(
      Cipher.ENCRYPT_MODE, 
      SECRET_ENCRYPTION_KEY);
    return encryptData(cipher, attribute);
   } catch (NoSuchAlgorithmException 
     | InvalidKeyException
     | InvalidAlgorithmParameterException 
     | BadPaddingException
     | NoSuchPaddingException 
     | IllegalBlockSizeException e) {
    throw new RuntimeException(e);
   }
  }
  return convertEntityAttributeToString(attribute);
 }

 /*
  * (non-Javadoc)
  * 
  * @see
  * javax.persistence.AttributeConverter#convertToEntityAttribute(java.lang.
  * Object)
  */
 @Override
 public X convertToEntityAttribute(String dbData) {
  if (isNotEmpty(SECRET_ENCRYPTION_KEY) && isNotEmpty(dbData)) {
   try {
    Cipher cipher = cipherMaker.configureAndGetInstance(
      Cipher.DECRYPT_MODE, 
      SECRET_ENCRYPTION_KEY);
    return decryptData(cipher, dbData);
   } catch (NoSuchAlgorithmException 
     | InvalidAlgorithmParameterException
     | InvalidKeyException 
     | BadPaddingException
     | NoSuchPaddingException 
     | IllegalBlockSizeException e) {
    throw new RuntimeException(e);
   }
  }
  return convertStringToEntityAttribute(dbData);
 }

 /**
  * The concrete class which implements this abstract class will have to
  * provide the implementation. For simple String encryption, the
  * implementation is simple as apache commons lang StringUtils can be used.
  * But this method was abstracted out as there might be other types of null
  * check technique required when a non String entity is to be encrypted
  * 
  * @param attribute
  * @return
  */
 abstract boolean isNotNullOrEmpty(X attribute);

 /**
  * The concrete class which implements this abstract class will have to
  * provide the implementation. For decryption of a String, its simple as a
  * String has to be returned, but for other non String types some more code
  * might have to be implemented. For example, a Date type of Date string.
  * 
  * @param dbData
  * @return
  */
 abstract X convertStringToEntityAttribute(String dbData);

 /**
  * The concrete class which implements this abstract class will have to
  * provide the implementation. For encryption of a String, its simple as a
  * String has to be returned, but for other non String types some more code
  * might have to be implemented. For example, a Date type of Date string.
  * 
  * @param attribute
  * @return
  */
 abstract String convertEntityAttributeToString(X attribute);

 /**
  * Helper method to encrypt data
  * 
  * @param cipher
  * @param attribute
  * @return
  * @throws IllegalBlockSizeException
  * @throws BadPaddingException
  */
 private String encryptData(Cipher cipher, X attribute)
   throws IllegalBlockSizeException, BadPaddingException {
  byte[] bytesToEncrypt = convertEntityAttributeToString(attribute)
    .getBytes();
  byte[] encryptedBytes = cipher.doFinal(bytesToEncrypt);
  return Base64.getEncoder().encodeToString(encryptedBytes);
 }

 /**
  * Helper method to decrypt data
  * 
  * @param cipher
  * @param dbData
  * @return
  * @throws IllegalBlockSizeException
  * @throws BadPaddingException
  */
 private X decryptData(Cipher cipher, String dbData)
   throws IllegalBlockSizeException, BadPaddingException {
  byte[] bytesToDecrypt = Base64.getDecoder().decode(dbData);
  byte[] decryptedBytes = cipher.doFinal(bytesToDecrypt);
  return convertStringToEntityAttribute(new String(decryptedBytes));
 }
}

Step 3 - Implementing the concrete String encryption/descryption JPA attribute converter

Now that the base class and crypto code is ready, the actual implementation for writing a string ecnryption/decryption JPA converter is quite easy.

/**
 * Concrete implementation of the {@link AbstractEncryptDecryptConverter}
 * abstract class to encrypt/decrypt an entity attribute of type
 * {@link java.lang.String} <br/>
 * Note: This is the class where the {@literal @}Converter annotation is applied
 * 
 * @author Sunit Katkar, sunitkatkar@gmail.com
 * @since ver 1.0 (Apr 2018)
 * @version 1.0 *
 */
@Converter(autoApply = false)
public class StringEncryptDecryptConverter
  extends AbstractEncryptDecryptConverter<String> {

 /**
  * Default constructor initializes with an instance of the
  * {@link CipherMaker} crypto class to get a {@link javax.crypto.Cipher}
  * instance
  */
 public StringEncryptDecryptConverter() {
  this(new CipherMaker());
 }

 /**
  * Constructor
  * 
  * @param cipherMaker
  */
 public StringEncryptDecryptConverter(CipherMaker cipherMaker) {
  super(cipherMaker);
 }

 @Override
 boolean isNotNullOrEmpty(String attribute) {
  return isNotEmpty(attribute);
 }

 @Override
 String convertStringToEntityAttribute(String dbData) {
  // the input is a string and output is a string
  return dbData;
 }

 @Override
 String convertEntityAttributeToString(String attribute) {
  // Here too the input is a string and output is a string
  return attribute;
 }
}

Step 4 - Using the String encryption/descryption JPA attribute converter

Using this JPA converter in a JPA Entity is just a matter of annotating the attribute with the @Convert annotation as shown below.

import java.io.Serializable;

import javax.persistence.Column;
import javax.persistence.Convert;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;

/**
 * JPA Entity which will be saved in the database table named 'employee'. The
 * attribute (or table column) <tt>sensitiveData</tt> needs to be encrypted for
 * storage and decrypted when read.
 * 
 * @author Sunit Katkar, sunitkatkar@gmail.com
 * @since ver 1.0 (Apr 2018)
 * @version 1.0
 * @param <X>
 */
@Entity
@Table(name = "employee")
public class Employee implements Serializable {

 private static final long serialVersionUID = 1L;

 @Id
 @GeneratedValue(strategy = GenerationType.IDENTITY)
 @Column(name = "employee_id")
 private Long id;

 @Column(name = "first_name")
 private String firstName;

 @Column(name = "last_name")
 private String lastName;

 /**
  * This String attribute needs to be encrypted
  */
 @Column(name = "sensitive_data")
 @Convert(converter = StringEncryptDecryptConverter.class)
 private String sensitiveData;

 /**
  * @return the id
  */
 public Long getId() {
  return id;
 }

 /**
  * @param id
  *            the id to set
  */
 public void setId(Long id) {
  this.id = id;
 }

 /**
  * @return the firstName
  */
 public String getFirstName() {
  return firstName;
 }

 /**
  * @param firstName
  *            the firstName to set
  */
 public void setFirstName(String firstName) {
  this.firstName = firstName;
 }

 /**
  * @return the lastName
  */
 public String getLastName() {
  return lastName;
 }

 /**
  * @param lastName
  *            the lastName to set
  */
 public void setLastName(String lastName) {
  this.lastName = lastName;
 }

 /**
  * @return the sensitiveData
  */
 public String getSensitiveData() {
  return sensitiveData;
 }

 /**
  * @param sensitiveData
  *            the sensitveData to set
  */
 public void setSensitiveData(String sensitiveData) {
  this.sensitiveData = sensitiveData;
 }
}

Bonus - Converter to encrypt / decrypt LocalDate

Since the basic infrastructure for encrypting and decrypting JPA entity attributes is created, it can be used for converting other types of entities, for example, an attribute of type java.time.LocalDate. Here is the code to do that.

import static java.time.format.DateTimeFormatter.ISO_DATE;
import static org.apache.commons.lang3.StringUtils.isEmpty;

import java.time.LocalDate;

import javax.persistence.Converter;

/**
 * Concrete implementation of the {@link AbstractEncryptDecryptConverter}
 * abstract class to encrypt/decrypt an entity attribute of type
 * {@link java.time.LocalDate} <br/>
 * Note: This is the class where the {@literal @}Converter annotation is applied
 * 
 * @author Sunit Katkar, sunitkatkar@gmail.com
 * @since ver 1.0 (Apr 2018)
 * @version 1.0 *
 */
@Converter(autoApply = false)
public class LocalDateEncryptDecryptConverter
  extends AbstractEncryptDecryptConverter<LocalDate> {

 public LocalDateEncryptDecryptConverter() {
  this(new CipherMaker());
 }

 public LocalDateEncryptDecryptConverter(CipherMaker cipherMaker) {
  super(cipherMaker);
 }

 @Override
 boolean isNotNullOrEmpty(LocalDate attribute) {
  return attribute != null;
 }

 @Override
 LocalDate convertStringToEntityAttribute(String dbData) {
  return isEmpty(dbData) ? null : LocalDate.parse(dbData, ISO_DATE);
 }

 @Override
 String convertEntityAttributeToString(LocalDate attribute) {
  return ((attribute == null) ? null : attribute.format(ISO_DATE));
 }
}


Resources

The source code for this blog post is available as a Spring Boot 2 project on Github. Click here.

Happy coding :)

12 comments:

Partha said...

Hi Sunit Katkar. Thanks for the information. I want to know the limitation on AES key, and reson behind it.

Sunit said...

@Partha
Thanks for your comment. I just selected AES as it is available in the JDK. There are many other encryption/decryption like 3DES, TwoFish, RSA. Take a look at this link.

Each algorithm has its own merits and demerits. As I said in the blog post that I am not a cryptography expert, I cannot really recommend what algorithm should be used. The purpose of the blog post was showing how we can store values in database columns securely.

I feel that AES as shown in the example is good enough. In production, you will be securing your database access anyway, so accessing your database will be very difficult if not impossible for hackers. Then this second layer of protection via AES will be enough to further safeguard your data. So even if your database falls into the wrong hands, this AES based protection will be good enough to deter further hacking attempts at your data. Offcourse, you need to keep your AES key very protected.

Sai Raghava said...

Hi Sunit, cheers for writing such a beautiful article. Can we have a mechanism to write a generic attribute converter to encrypt different attributes with different encryption keys.

kumar said...
This comment has been removed by the author.
kumar said...

Very good article, May i know how to use this encrypted data in reports?

Nikhil Bhalwankar said...
This comment has been removed by the author.
Nikhil Bhalwankar said...

Very good article. Thanks Sunit.

Sunit said...

@Nikhil Bhalwankar
Thank you for your kind words.

Sunit said...

@Nikhil Bhalwankar
You had asked 'Is there any way to retrieve encrypted field directly using SQL Query?'
Once you encrypt the field, you are just changing the plain text data into an encrypted string and storing it in a column. Yes, you can definitely access the encrypted fields via a regular SQL select statement. But what will you do with that encrypted value? You will need to decrypt it and for that you need to know the algorithm used and the key for the encryption/decryption.

Enes Batur said...

Hi Sunit,

In JPA Entity, if we use AbstractEncryptDecryptConverter as a Convertor instead of StringEncryptDecryptConvertor or LocalDateEncryptDecryptConvertor for the sake of more generic structure, is it capable to find suitable implementation of the abstract class with respect to column data type? Thanks in advance.

rohitchavan said...

In my personal project I encryption the client name using the information above...in front end I wanted to add clin3t name search filter for that in java Spring layer I write native sql query but it's not working because of encryption think... So how I can provide the filter functionality using your encryption technique...

Sunit said...

@rohitchavan,
From your comment what I understand is that you have a table with some records. One of the attributes/columns is 'name' which you have annotated with the @Convertor annotation. Now you wish to run a query and get all the 'name' values which match whatever your UI sends you. So I take it that you have a JPQL query like
select i from MyTable i where i.name LIKE CONCAT('%',:searchPhrase ,'%')");

The encryption/decryption has nothing to do with your querying. Please re-read the blog post and understand that when JPA service/repository layer fetches a value, then the convertor does it job of decrypting the value and giving it back to the JPA service/repository layer in its original unencrypted form. The reverse happens when you ask JPA service/repository layer to save a value. The value gets encrypted and stored in the database.

Please note that your application code has no knowledge of the encryption/decryption happening before saving and fetching the data.

In your case, try to run your JPQL query on an unencrypted field and check if it works. I am sure it will. I suspect that your JPQL query with the LIKE clause is most likely where you need to focus on and ensure it is working as expected.

Hope this helps.