RSA Encryption in Java: Key Pairs, Encrypt/Decrypt & Digital Signatures

RSA is the most widely used asymmetric (public-key) algorithm. A public key encrypts or verifies; the private key decrypts or signs. Java's JCA supports RSA through KeyPairGenerator and Cipher. This article covers key generation, OAEP encryption, digital signatures, key serialisation, and the hybrid encryption pattern used in practice.

Java

RSA Overview: When to Use Asymmetric Cryptography

Operation	Key used	Use case
Encrypt	Recipient's public key	Secure key exchange, small payloads
Decrypt	Recipient's private key	Recover encrypted data
Sign	Signer's private key	Prove authenticity and integrity
Verify	Signer's public key	Confirm signature is genuine

RSA is slow for large data. In practice, use hybrid encryption: encrypt the actual data with AES-GCM, then encrypt the AES key with RSA. TLS does exactly this.

Generating an RSA Key Pair

Java

import java.security.*;
import java.util.Base64;

// Generate a 2048-bit RSA key pair (use 4096 for highest security)
KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
kpg.initialize(2048, new SecureRandom());
KeyPair keyPair = kpg.generateKeyPair();

PublicKey  publicKey  = keyPair.getPublic();
PrivateKey privateKey = keyPair.getPrivate();

// Serialise to Base64-encoded DER format
String pubBase64  = Base64.getEncoder().encodeToString(publicKey.getEncoded());
String privBase64 = Base64.getEncoder().encodeToString(privateKey.getEncoded());
System.out.println("Public  : " + pubBase64.substring(0, 40) + "...");
System.out.println("Private : " + privBase64.substring(0, 40) + "...");

Encrypt & Decrypt with OAEP Padding

Always use OAEP (Optimal Asymmetric Encryption Padding): the older PKCS1v1.5 padding is vulnerable to Bleichenbacher's attack.

Java

import javax.crypto.*;
import javax.crypto.spec.OAEPParameterSpec;
import javax.crypto.spec.PSource;
import java.security.*;
import java.security.spec.*;
import java.util.Base64;

public class RsaEncryption {

    private static final String TRANSFORMATION =
            "RSA/ECB/OAEPWithSHA-256AndMGF1Padding";

    public static byte[] encrypt(byte[] plaintext, PublicKey publicKey) throws Exception {
        Cipher cipher = Cipher.getInstance(TRANSFORMATION);
        cipher.init(Cipher.ENCRYPT_MODE, publicKey);
        return cipher.doFinal(plaintext);
    }

    public static byte[] decrypt(byte[] ciphertext, PrivateKey privateKey) throws Exception {
        Cipher cipher = Cipher.getInstance(TRANSFORMATION);
        cipher.init(Cipher.DECRYPT_MODE, privateKey);
        return cipher.doFinal(ciphertext);
    }

    public static void main(String[] args) throws Exception {
        KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
        kpg.initialize(2048);
        KeyPair kp = kpg.generateKeyPair();

        byte[] message   = "Top secret!".getBytes(StandardCharsets.UTF_8);
        byte[] encrypted = encrypt(message, kp.getPublic());
        byte[] decrypted = decrypt(encrypted, kp.getPrivate());

        System.out.println(new String(decrypted));  // Top secret!
    }
}

RSA-2048 with OAEP-SHA256 can encrypt at most 190 bytes of plaintext (2048 bits − OAEP overhead). For larger data, encrypt the data with AES-GCM and encrypt only the AES key with RSA.

Digital Signatures: Sign & Verify

A digital signature proves that a message was created by the holder of the private key and has not been modified. The Signature class handles signing and verification.

Java

import java.security.*;
import java.util.Base64;

public class RsaSignature {

    private static final String ALGORITHM = "SHA256withRSA";

    public static byte[] sign(byte[] data, PrivateKey privateKey) throws Exception {
        Signature sig = Signature.getInstance(ALGORITHM);
        sig.initSign(privateKey);
        sig.update(data);
        return sig.sign();
    }

    public static boolean verify(byte[] data, byte[] signature, PublicKey publicKey)
            throws Exception {
        Signature sig = Signature.getInstance(ALGORITHM);
        sig.initVerify(publicKey);
        sig.update(data);
        return sig.verify(signature);
    }

    public static void main(String[] args) throws Exception {
        KeyPairGenerator kpg = KeyPairGenerator.getInstance("RSA");
        kpg.initialize(2048);
        KeyPair kp = kpg.generateKeyPair();

        byte[] document  = "Sign this contract.".getBytes(StandardCharsets.UTF_8);
        byte[] signature = sign(document, kp.getPrivate());

        System.out.println("Signature: " + Base64.getEncoder().encodeToString(signature));
        System.out.println("Valid    : " + verify(document, signature, kp.getPublic())); // true

        // Tamper with document
        document[0] = 'X';
        System.out.println("Tampered : " + verify(document, signature, kp.getPublic())); // false
    }
}

Key Serialisation & Deserialisation

Java

import java.security.*;
import java.security.spec.*;
import java.util.Base64;

// Serialise keys to PEM-style Base64
String publicPem  = "-----BEGIN PUBLIC KEY-----\n"
        + Base64.getMimeEncoder(64, new byte[]{'\n'}).encodeToString(publicKey.getEncoded())
        + "\n-----END PUBLIC KEY-----";

String privatePem = "-----BEGIN PRIVATE KEY-----\n"
        + Base64.getMimeEncoder(64, new byte[]{'\n'}).encodeToString(privateKey.getEncoded())
        + "\n-----END PRIVATE KEY-----";

// Deserialise
KeyFactory kf = KeyFactory.getInstance("RSA");

// Public key (X.509 SubjectPublicKeyInfo format)
byte[] pubBytes = Base64.getMimeDecoder().decode(
        publicPem.replace("-----BEGIN PUBLIC KEY-----", "")
                 .replace("-----END PUBLIC KEY-----", "").trim());
PublicKey restoredPub = kf.generatePublic(new X509EncodedKeySpec(pubBytes));

// Private key (PKCS#8 format)
byte[] privBytes = Base64.getMimeDecoder().decode(
        privatePem.replace("-----BEGIN PRIVATE KEY-----", "")
                  .replace("-----END PRIVATE KEY-----", "").trim());
PrivateKey restoredPriv = kf.generatePrivate(new PKCS8EncodedKeySpec(privBytes));

Hybrid Encryption Pattern: RSA + AES

The standard pattern for encrypting arbitrary-length data with RSA:

Step	Operation	Key
1	Generate a random 256-bit AES session key	-
2	Encrypt the plaintext with AES-GCM using the session key	AES session key
3	Encrypt the AES session key with RSA-OAEP	Recipient's RSA public key
4	Send: encrypted AES key + IV + AES ciphertext	-
5	Recipient: decrypt AES key with RSA private key	Recipient's RSA private key
6	Recipient: decrypt data with AES session key	Decrypted AES key

Java

// Sender side
KeyGenerator aesKg = KeyGenerator.getInstance("AES");
aesKg.init(256);
SecretKey sessionKey = aesKg.generateKey();

// Encrypt data with AES-GCM
byte[] encryptedData = aesGcmEncrypt(data, sessionKey);

// Encrypt session key with RSA-OAEP
Cipher rsaCipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
rsaCipher.init(Cipher.ENCRYPT_MODE, recipientPublicKey);
byte[] encryptedKey = rsaCipher.doFinal(sessionKey.getEncoded());

// Transmit: encryptedKey + encryptedData

// Recipient side
rsaCipher.init(Cipher.DECRYPT_MODE, recipientPrivateKey);
byte[] keyBytes = rsaCipher.doFinal(encryptedKey);
SecretKey recoveredKey = new SecretKeySpec(keyBytes, "AES");
byte[] plaintext = aesGcmDecrypt(encryptedData, recoveredKey);