Converting PEM certificates to DER for fun (and no profit)

Behrang Saeedzadeh 06 November 2018

In this article we will demistify PEM and DER encoding formats that are commonly used to store (and share) keys and certificates. We will do this by first using OpenSSL to generate an X509 certificate and its associated private key in PEM encoding and converting them to their corresponding DER encodings. Then we will write a Java program that can read PEM files that contain only one entry and convert them to their corresponding DER encodings. Finally, we verify the correctness of our program by comparing the DER files produced by it to the DER files produced by OpenSSL.

Generating a self-signed certificate

A popular tool for creating self-signed certificates is OpenSSL. Here’s a shell command that uses OpenSSL to generate one:

$ openssl req \
            -x509 \
            -sha256 \
            -nodes \
            -days 365 \
            -newkey rsa:2048 \
            -keyout private-key.pem \
            -out certificate.pem
Generating a 2048 bit RSA private key
....................................................+++
..........................+++
writing new private key to 'private-key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:
Email Address []:

First let’s consult the OpenSSL req man page and break down this command:

  • The req command primarily creates and processes certificate requests in PKCS#10 format. It can additionally create self signed certificates for use as root CAs for example.
  • -x509: This option outputs a self signed certificate instead of a certificate request.
  • -sha256: Use the SHA-256 hash algorithm.
  • -nodes (no DES): If this option is specified then if a private key is created it will not be encrypted.
  • -days 365: When the -x509 option is being used this specifies the number of days to certify the certificate for.
  • -newkey rsa:2048: This option creates a new certificate request and a new private key. When the argument takes the rsa:nbits form, it generates an RSA key nbits in size.
  • -keyout filename: This gives the filename to write the newly created private key to.
  • -out filename: This specifies the output filename to write to or standard output by default.

Our command produced a PEM encoded certificate and private key and we can use cat to view their contents:

$ cat certificate.pem
-----BEGIN CERTIFICATE-----
MIIDYDCCAkigAwIBAgIJAOWTgaVkV8c+MA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV
BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX
aWRnaXRzIFB0eSBMdGQwHhcNMTgxMTA1MTEwODQ2WhcNMTkxMTA1MTEwODQ2WjBF
MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50
ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEAsP4vPMK8EAnCHhFXcpg9CDsBKiyI/Z9EqaHcMVSITV3MEQHxs4ULuE0I
H1T5s61pMWw5LKpiT1sCaoLEOKI1gMgk1IWNaRat8pjGxfCNSedrUrdJqQlgi4je
7qq1viCX+sZAt29vprn1YlN3tPvZXknqCxdgBPnxDTUrTyrHRDrnyoyva5psZybD
qKMX9F75I4C4oWW5KfpTBwBcQVeoUC+7WOKhIm910xJ26Oe6aCaXOSvEO1aRcRj7
nVeZreiI9JuQql63LV0toxAeIUom7oX7tNF3ZV83fQFPd6GQZOLc2iFDAr5r6rYX
V39ArLTP/8OdpfyIvI7uPUDyfC9HuQIDAQABo1MwUTAdBgNVHQ4EFgQUadvDid7T
yA3HpdRJDTZSWksVcpIwHwYDVR0jBBgwFoAUadvDid7TyA3HpdRJDTZSWksVcpIw
DwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEASqTBlodVeIUTXOlZ
D9+PME/A9znf39QTjM8VA6PQcjw10DAGZxeLWYcN2xfa3Vk358nq/PIEYmwramYR
Ymys3g9UTT1rZw8ew+Pz0CRLJyF2UrINi9YqfauG6eKPgapeqBwK0A0cAmFs6q5T
wYQ0rJH9Vx21yveYotb0Ul3vGmfuUBCM7pz/qgtYAjbmN/yRs40aiXRYOvHLjewF
Vy/nzHMt/cyHgrETcYy8PLuIqo3IiabhKfoB5LUVAcs/V9ZVvkShtxG0Eaqa4VZr
Nu98oj+BmIpUKPGc5cOTXN3F98JoOVLgeauAsU7/Zi+1lL4x0cbkgyy99sisic/f
yb22Jg==
-----END CERTIFICATE-----

$ cat private-key.pem
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCw/i88wrwQCcIe
EVdymD0IOwEqLIj9n0SpodwxVIhNXcwRAfGzhQu4TQgfVPmzrWkxbDksqmJPWwJq
gsQ4ojWAyCTUhY1pFq3ymMbF8I1J52tSt0mpCWCLiN7uqrW+IJf6xkC3b2+mufVi
U3e0+9leSeoLF2AE+fENNStPKsdEOufKjK9rmmxnJsOooxf0XvkjgLihZbkp+lMH
AFxBV6hQL7tY4qEib3XTEnbo57poJpc5K8Q7VpFxGPudV5mt6Ij0m5CqXrctXS2j
EB4hSibuhfu00XdlXzd9AU93oZBk4tzaIUMCvmvqthdXf0CstM//w52l/Ii8ju49
QPJ8L0e5AgMBAAECggEADy1894Trg6SSlOl/yj/r2+zTaL5w8O7bgIB01XYeSiQk
/8Be7boXCzLv41/yGcphHu/c50ulKpDvYBreoMyoMeb34ZGiUsSzHf6j7Q130EKc
HXfCRVCcitqt42GYPDmnTsX5fsF7nZyMwmXN+iMuvLsx6smSXlrgpPVORXexptQR
0arFL8Ny3MR6EvAfjy7NwY2Vicpcb0sKt1i4IhAS+38UIPFE26e3WRXIEqfVl+Kv
VDfs793JTzkm8LOO5MJ6mSUoUGHZBw68sZtSTUG6s3CC+z35dDOpT6UZNSDlEbdc
BjTauU1njlcgelvRVrWV2uTkJtATiwJdtJAvTZ7wkQKBgQDXuvqAvYkm657inHqu
ZuCTgpNxiPovoJTvF1Kx2wiH0+V3zmwNTGsO8s+jdx7bBiXWUoReFJdgvTJzfLyr
SERjYK69VYhxwg6vJHBEAjLpRtEblWSOq+P8+E+RPjAmrsSA90qzG5fKqTiQYBbz
+Jdo/D1+jIP4p8+xCQdidT5C9wKBgQDSCBLfTgQIbv0XMH/B9FUs8jiSAr35s1PO
mbPiZHpDH9FLvTwwfc2aZ2ggIioo0yP0g+6Wz1iN8kkBwb563RsoJVZBrae+i5Fg
DO1LPssUmvzrP6HmQbiQL9vLPecEpm3nYoZZUzGF771acL5QmNU5E27XsCUv6u9P
G+fCDTduzwKBgQCXuqyhi61BMItpewFWVIkzdq1ZNywv/1OA7fh9mz5eugw0vXyF
xe+ubdzYbdnH1oUaPqNWwEjQA4HOdDcGiSeAdemwxM3YpNJLIOfNDXbdV6+PJP26
0u7hZ4we7YxUSKtwwX1Kh+e05FwiP1MlePyVHIPr08ymQPD05RLKjwehdQKBgDZs
bSd7+cONrvtRT+mcPG33Hf2HxF5tv2urnFcvnhbCqeMsV+pgzuQRo4OgFriZbZmw
jiR5+gJeI9jWsVr47E6Mzjsamnrfg34WG28CCHGAb8UtqfAWW5LRM+0IT1J/Jb9t
Yk7HJfAE3VYQHOCZmba+eFxfgmNWR0iCfJ1l+1U1AoGAMfwDipiy7KwR16B2aohj
vPDyMEzt7RUmNLNJ88z6vynTMDzIhzh8veeVySy6zNyBJo0WRX57dxRnwqd0ldDT
Aw7goO1rqyv9MfDGtlNVkjkGBAyATlis21E2Sbj8+tU6Xg8jGd16PLPURSZM29kq
S3uxXqoO7cuzEvfSqL+uQbQ=
-----END PRIVATE KEY-----

Now that we have or PEM files, we can use OpenSSL to convert them to their corresponding DER encodings:

# Convert PEM certificate to DER
$ openssl x509 \
    -inform pem \
    -outform der \
    -in certificate.pem \
    -out openssl-certificate.der

# Convert PEM private key to DER
$ openssl pkcs8 \
    -topk8 \
    -inform PEM \
    -outform DER \
    -in private-key.pem \
    -out openssl-private-key.der \
    -nocrypt

Unlike PEM, DER is a binary format and we cannot use cat to view the content of these .der files.

PemToDerConverter.java

In PEM files, we learned that:

Many cryptography standards use ASN.1 to define their data structures, and Distinguished Encoding Rules (DER) to serialize those structures. Because DER produces binary output, it can be challenging to transmit the resulting files through systems, like electronic mail, that only support ASCII.

The PEM format solves this problem by encoding the binary data using Base64. PEM also defines a one-line header, consisting of —–BEGIN , a label, and —–, and a one-line footer, consisting of —–END , a label, and —–. The label determines the type of message encoded. Common labels include CERTIFICATE, CERTIFICATE REQUEST, and PRIVATE KEY.

This seems to imply that we can convert a PEM encoded certificate to DER encoding by:

  • Reading all the lines
  • Dropping the first and last lines
  • Concatenating the remaining lines to form the Base64 encoded string
  • Convert the Base64 encoded string to binary

Let’s verify this by writing a Java program that performs these operations to covnert a PEM encoded key to its DER encoding:

package org.behrang.examples.pem2der;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Base64;
import java.util.stream.Collectors;

import static java.nio.charset.StandardCharsets.UTF_8;

public class PemToDerConverter {

    public static void main(String[] args) {
        try {
            doMain(args);
        } catch (Exception e) {
            System.err.println("Encountered an unknown error: " + e.getMessage());
            System.exit(1);
        }
    }

    public static void doMain(String[] args) throws IOException {
        if (args.length != 2) {
            showUsageAndExit();
        }

        Path sourceFile = Paths.get(args[0]);
        Path targetFile = Paths.get(args[1]);

        String base64 = Files.readAllLines(sourceFile, UTF_8)
                             .stream()
                             .filter(line -> !line.startsWith("-----BEGIN") && !line.startsWith("-----END"))
                             .collect(Collectors.joining());

        Base64.Decoder decoder = Base64.getDecoder();

        Files.write(targetFile, decoder.decode(base64));

        System.out.printf(
            "Converted %s into DER format and saved the results in %s\n",
            sourceFile,
            targetFile
        );
    }

    private static void showUsageAndExit() {
        System.out.println("Pass input and output filenames as arguments");
        System.exit(1);
    }
}

Let’s run the program and pass certificate.pem to convert it to DER and save it in java-certificate.der:

$ javac PemToDerConverter.java
$ java PemToDerConverter certificate.pem java-certificate.der

We can verify it is identical to the DER file produced by OpenSSL:

# diff outputs nothing if java-certificate.der and
# openssl-certificate.der are identical
$ diff java-certificate.der openssl-certificate.der

The DER file produced by our file is byte-for-byte identical with the DER file produced by OpenSSL. Sweet!

Can our program convert the PEM encoded private key to DER encoding too? Let’s see:

$ java PemToDerConverter private-key.pem java-private-key.der
$ diff java-private-key.der openssl-private-key.der

Ay!

Conclusion

  • A PEM encoded private key is nothing but a Base64 encoded DER private key wrapped between a header line and a footer line (i.e. -----BEGIN PRIVATE KEY-----/-----BEGIN PRIVATE KEY-----).
  • Similarly, A PEM encoded ceretificate is identical to a Base64 encoded DER certificate wrapped between a header line and a footer line (i.e. -----BEGIN CERTIFICATE-----/-----END CERTIFICATE-----).