I’m very excited to announce the results of what I have been working on for the past 1,5 years. *drumrolls*

I added support for OpenPGP v6 (rfc9580) in both Bouncy Castle and PGPainless! In this blog post, I want to go over the work in more details.

I want to give a huge shout-out to the fine folks at the Sovereign Tech Agency, who commissioned the work outlined in this article as part of the Sovereign Tech Fund program. Your support made this work possible <3.

OpenPGP v6

PGPainless and Bouncy Castle already support OpenPGP v4 (rfc4880). You might be wondering, what happened to OpenPGP v5? Why the sudden jump to v6? Was the task of versioning left to a Microsoft intern?

The answer is drama. I won’t elaborate too much and instead recommend you to read up on the topic on your own. What matters for me is: GnuPG dropped out of the standardization process and decided to “fork” the OpenPGP protocol under the name LibrePGP. To minimize the negative impact of interoperability issues on the user-base, while still remaining able to improve the OpenPGP protocol, the IETF working group decided to surrender the v5 version number to LibrePGP, defining the new OpenPGP version as v6 instead.

Note: LibrePGP ≠ OpenPGP.

Personally, I dislike the idea of LibrePGP as in my opinion it does more damage than it brings value for users. This post by Andrew Gallagher further illustrates the argument between GnuPG and the IETF.
Ultimately for Bouncy Castle and PGPainless I focused on implementing the IETFs OpenPGP v6 as defined in RFC9580.

For a condensed overview of the new features of OpenPGP v6, I recommend you reading this post by Daniel Huigens from Proton.

Bouncy Castle

For many years now, Bouncy Castle has proven itself to be the library for cryptographic components in the Java ecosystem. It provides implementations of most cryptographic algorithms and hash functions, but also contains components implementing higher level protocols, such as TLS and more recently MLS. It also has an OpenPGP API, which is used internally by PGPainless.

Bouncy Castles OpenPGP API is divided into two main package trees. The bcpg package contains packet definitions and serialization/deserialization code without any advanced OpenPGP logic. The code in this package takes care of placing OpenPGP packet contents as octets in the right order onto the wire, and vice versa parsing incoming byte streams back into plain old java objects (POJOs).

Decoupled from this, there is also the openpgp packet, which handles more complex logic, such as providing decryption of encrypted data packets, generation of encrypted and/or signed messages etc.

Put plainly, while the bcpg package represents what is being processed/produced, the openpgp package handles how this is done.

Bouncy Castle’s OpenPGP API, while being flexible and powerful, unfortunately has always been rather challenging to use – until now… but more on that later.

Since PGPainless internally depends on Bouncy Castle, it was necessary to implement OpenPGP v6 support for Bouncy Castle first, before integrating these changes into PGPainless later on.

Modifying the bcpg package meant studying the new packet versions introduced in rfc9580 really well and extending the existing packet constructors with switch-case statements, altering the parser/serializer methods based on encountered packet version numbers. This work was also quite easy to test, as the rfc provides test vectors for most new packet types. In the process, I was able to increase the code coverage of the packet parsing code quite a bit 🙂

The real meat was in the openpgp package, where the changes were more complex and substantial. Most notably, OpenPGP v6 introduces new packets for an improved method of message encryption. While the old method encrypted the message with a symmetric session key, which was encrypted using either a symmetric key derived from a passprase and/or with the recipients public key(s), the new method introduces an intermediate step. The message is now encrypted using a message key that is derived from the session key.

To illustrate the differences, please enjoy these diagrams showing the process of decrypting a message, which is encrypted symmetrically to a message passphrase. The old method is using one SKESKv4 packet per message passphrase and the message contents are wrapped in a SEIPDv1 packet.

As you can clearly see, with the SKESKv4 packet, the passphrase is passed into an S2K (string-to-key) key derivation function, which results in a symmetric key, which is then used to decrypt an encrypted session key.

Side note: In very legacy systems, there is no encrypted session key, so the result from the S2K function is directly used as session key, but you can usually ignore this detail.

Using the session key, the SEIPDv1 packets encrypted data is simply decrypted using the recovered session key. For integrity protection, a checksum is checked, but that’s not depicted in the diagram.

Now compare this with the new method using one SKESKv6 packet per message passphrase and the message contents stored in a SEIPDv2 packet:

The result of passing the passphrase into the S2K function is piped into an HKDF key derivation function which hashes metadata about the SKESK packet itself, preventing downgrade and crossgrade attacks across different packet versions. The resulting key-encryption-key (kek) is used in an AEAD encryption scheme that takes metadata of the SEIPDv2 packet as additional data to decrypt the encrypted session key.

When decrypting the SEIPDv2 packet to obtain the message contents, the session key is not used directly, but instead a message-key is derived from it by passing in some salt value, which is unique per message, along with some more packet metadata. The resulting message key and IV are used in an AEAD encryption scheme to finally recover the message contents.

Implementing the new method was a pain, as subtlest mistakes cause the decryption process to fail. More than one time, I starred at the debugger for 3 days straight, only to discover that the bug was a misplaced length, or a swapped variable.

Another, more subtle change between the OpenPGP RFCs was the shift away form 64bit key-ids towards 256bit key fingerprints. Since many preexisting APIs within Bouncy Castle and PGPainless historically use Long key-ids in places where now fingerprints need to be passed in, it was necessary to introduce a new data type to unify both types. For this purpose, the KeyIdentifier class was created, which can be based on either a Long or a fingerprint byte array. This allows a smooth migration towards fingerprints, with a clear upgrade path. Eventually, passing in key-ids can be deprecated.

The new encryption methods are exposed to the user via Bouncy Castle’s existing PGPEncryptedDataGenerator class, which now has a new method for enabling message encryption via SEIPDv2 packets.

Symmetric message encryption is obviously not the only new feature introduced by RFC9580. A large chunk of the work was dedicated to adding support for OpenPGP v6 keys, certificates and signatures. It is now possible, to protect both v4 and v6 secret keys using memory hard Argon2 S2K.

This further eliminates a class of attacks known as KOpenPGP, earlier discovered by Bruseghini et al. from Proton. An attacker with access to the locked secret key (for example an encrypted mail hoster) could manipulate the public key parameters in a way that would result in partial or complete secret key leakage when used. OpenPGP v6 keys are no longer vulnerable to this attack, as the public key parameters are fed as additional data into the AEAD encryption scheme.

You can now choose to generate keys based on native ed25519 and curve25519, as well as ed448 and curve448. The legacy encoding formats for ed25519 and curve25519 can also still be used, but not with v6 keys.

Bouncy Castle contains two implementations of the low-level cryptography (JCE and BC). A complex part of adding support for the new curves was to implement conversion of keys between both implementations. This PR in particular does a pretty good job of demonstrating this I think. I’m not a cryptographer, so dealing with random low level crypto APIs to fix complex encoding issues while trying to avoid breaking the existing implementation is fun! I especially enjoyed debugging ed448 key conversion randomly failing until I noticed that the bug is caused by faulty encoding of keys with a leading 0.

Another almost one-line fix to a problem I spent 3 days debugging was submitted in this patch. I accidentally processed the salt in a salted signature by passing it via the update method, which in case of text signatures applied line ending conversions to the salt. This caused rare cases where signature verification failed. Finding the cause was tricky as you can imagine.

A bunch of smaller improvements also made it into Bouncy Castle. For historic reasons, OpenPGP packets can encode their length in multiple different formats. There are two main methods, the old legacy encoding and the new method (introduced in rfc2440, so not exactly new anymore). When encoding a packet, Bouncy Castle used to decide the encoding type in a hard-coded way per each packet type. This meant that for example a public key packet would always be encoded using the old format. As a consequence, it was impossible to round-trip packets byte per byte. Now though, when parsing a packet, Bouncy Castle remembers the encoding format and uses the same format when emitting the packet (unless of course instructed otherwise).

High Level OpenPGP API for Bouncy Castle

Starting with release 1.81, there is now a high level OpenPGP API built into Bouncy Castle itself!

Located in org.bouncycastle.openpgp.api, this API abstracts away the need to manually wrap different streams in order to generate OpenPGP messages, similar to how PGPainless already does it. The API also automatically chooses the best available algorithms depending on preferences and capabilities of the recipients.

The new OpenPGPKeyGenerator automagically sets up fresh keys with sane algorithm preferences and allows generation of both v4 and v6 keys.

The mid-level API already provided the user with ways to gather information about OpenPGP keys via the PGPPublicKeyRing / PGPSecretKeyRing classes. However, these classes did not actually evaluate signatures on the key(s) itself, so you still might end up accidentally using a revoked/expired/rogue subkey for example. The new API introduces a solution to these issues in the form of the OpenPGPCertificate / OpenPGPKey classes. These provide an API to evaluate the key at a certain point in time, and gather information about subkeys and/or user-ids validity. Signatures on the key(s) are evaluated once and the results are cached in order to avoid unnecessary signature calculations. Furthermore, these classes provide ready-made methods for exporting ASCII armored encodings of secret/public key material.

All interfaces from the new API expect the new OpenPGPCertificate / OpenPGPKey classes to be passed in. This further motivates users to migrate to the new types.

Similar to PGPainless, the new API also provides a way for users to easily decrypt messages and perform signature verification. The same goes for message and signature generation. There is also a new OpenPGPKeyEditor class that can be used to modify existing secret keys, e.g. add new user-ids / subkeys, revoke key components or extend expiration periods. If your application has certain requirements when it comes to key strengths/algorithms, you can define these in the OpenPGPPolicy class. This way, signatures made by weak keys can be automatically rejected and weak certificates are not used when generating messages.

Similarly to PGPainless, the new API provides a single entry point for the user, which is the OpenPGPApi class. This class exposes methods to perform different operations. You can have multiple instances of the OpenPGPApi class, each configured individually with its own policy instance.

Here is a very brief proof of concept demonstrating how to use the new API:

        OpenPGPApi api = new BcOpenPGPApi(); // or new JcaOpenPGPApi(new BouncyCastleProvider());

        // generate a fresh key
        OpenPGPKey aliceKey = api.generateKey()
                .ed25519x25519Key("Alice <alice@example.com>")
                .build("sw0rdf1sh".toCharArray());
        // extract the public key (certificate)
        OpenPGPCertificate aliceCert = aliceKey.toCertificate();
        // ASCII armor the cert for easy sharing
        String aliceCertArmored = aliceCert.toAsciiArmoredString();
        assertTrue(aliceCertArmored.startsWith("-----BEGIN PGP PUBLIC KEY BLOCK-----"));

        // encrypt a message
        OpenPGPMessageGenerator encryptor = api.signAndOrEncryptMessage()
                .addSigningKey(aliceKey)
                .addKeyPassphrase("sw0rdf1sh".toCharArray())
                .addEncryptionCertificate(aliceCert);
        ByteArrayOutputStream encOut = new ByteArrayOutputStream();
        OpenPGPMessageOutputStream messageOut = encryptor.open(encOut);
        messageOut.write("Hello, World!\n".getBytes(StandardCharsets.UTF_8));
        messageOut.close();
        String encAsc = encOut.toString();
        assertTrue(encAsc.startsWith("-----BEGIN PGP MESSAGE-----"));

        // decrypt the message
        ByteArrayInputStream messageIn = new ByteArrayInputStream(encOut.toByteArray());
        OpenPGPMessageInputStream decryptIn = api.decryptAndOrVerifyMessage()
                .addDecryptionKey(aliceKey, "sw0rdf1sh".toCharArray())
                .addVerificationCertificate(aliceCert)
                .process(messageIn);
        byte[] decrypted = decryptIn.readAllBytes();
        decryptIn.close(); // <- important!
        assertArrayEquals("Hello, World!\n".getBytes(StandardCharsets.UTF_8), decrypted);
        OpenPGPMessageInputStream.Result result = decryptIn.getResult();
        assertTrue(result.getSignatures().get(0).isValid());

        // create a detached signature
        byte[] data = "Foo Bar Baz".getBytes(StandardCharsets.UTF_8);
        OpenPGPSignature.OpenPGPDocumentSignature signature = api.createDetachedSignature()
                .addSigningKey(aliceKey)
                .addKeyPassphrase("sw0rdf1sh".toCharArray())
                .sign(new ByteArrayInputStream(data))
                .get(0);
        String armoredSig = signature.toAsciiArmoredString();
        assertTrue(armoredSig.startsWith("-----BEGIN PGP SIGNATURE-----"));

        // verify detached signature
        OpenPGPSignature.OpenPGPDocumentSignature verifiedSig = api.verifyDetachedSignature()
                .addVerificationCertificate(aliceCert)
                .addSignature(signature.getSignature())
                .process(new ByteArrayInputStream(data))
                .get(0);
        assertTrue(verifiedSig.isValid());

BC-SOP

You might know of the Stateless OpenPGP Protocol, a specification for a common command line interface for OpenPGP backends. PGPainless provides an implementation (pgpainless-sop) already for some time (see e.g. this blog post). As a byproduct of the OpenPGP v6 work, I also created a vanilla Bouncy Castle implementation, called bc-sop.

This implementation makes use of the new high level API and can be used without additional dependencies. There is still some functionality missing, e.g. generation/processing of messages signed using the Cleartext Signature Framework (CSF), however the rest is already quite usable, although considered experimental.

bc-sop provides both a command line application (which is compiled to a native binary using graalvm, so its actually quite fast!), as well as a Java API, which is an implementation of sop-java, so it can act as a drop-in replacement for other sop-java implementations.

If you want to give it a try, check it out on codeberg.org.

Here is an example of the CLI tool:

PGPainless 2.0

So, PGPainless version 2.0 is on the horizon 🙂 I’m still waiting for some required changes to actually land in Bouncy Castle 1.82, but other than that, version 2.0 is pretty much ready.

I decided to take the opportunity of having a major release to fix some historic oversights and design flaws by reworking some important aspects of the library, which are considered breaking changes. Most importantly, the API entry point (the PGPainless class) is no longer merely a collection of static methods, but it actually holds state now. This way, you can have multiple instances of the API, each with a different configuration and algorithm policy. This is especially useful for testing.

As with Bouncy Castle, some changes were required to methods that identified keys via key-id. These methods were deprecated in favor of ones taking KeyIdentifier objects from Bouncy Castle.

In order to avoid having unnecessary code duplication, many core implementation aspects of PGPainless have been migrated over to use Bouncy Castles high-level API internally. This includes message encryption/decryption and signature verification, the latter being more efficient now thanks to the cached signature evaluations. If possible, the old API stayed mostly intact, so if you’ve been a user of PGPainless 1.X, you’ll find your way around in the 2.X release. I also wrote a migration guide that should help you with any issues that might pop up 🙂

I’m really looking forward to the release and am excited to learn what you think of the changes 🙂