Kubernetes is a powerful orchestration software that puts a lot of power into its CLI interface kubectl. However when it comes to credential storage, it’s rather mediocre. While it provides all you need, by default, it’ll store your access tokens, certificates or alike in your home directory in plaintext. This article should explain how to migrate them into the password manager pass, which can be secure using GnuPG and e.g. a YubiKey or smart card.

This article assumes that pass is already configured to work with a YubiKey.

Extracting the certificates

Before you can store your certificate, you have extract your public and private key from kubeconfig in PEM format. In order to do this for your current context, you can just run the following command:

kubectl config view --minify --raw --output 'jsonpath={..user.client-certificate-data}' | base64 -d | sed -e 's/$/\\n/g' | tr -d '\n'
kubectl config view --minify --raw --output 'jsonpath={..user.client-key-data}' | base64 -d | sed -e 's/$/\\n/g' | tr -d '\n'

This will return a single long string with multiple \n, This is intended, keep this output around.

Now create a new password entry in pass:

pass edit k8s.example.com/myuser

This will open your editor and you can paste the following template:

{
  "apiVersion": "client.authentication.k8s.io/v1",
  "kind": "ExecCredential",
  "status": {
    "clientCertificateData": "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----",
    "clientKeyData": "-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----"
  }
}

Replace -----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE----- with the output of kubectl config view --minify --raw --output 'jsonpath={..user.client-certificate-data}' | base64 -d | sed -e 's/$/\\n/g' | tr -d '\n' and -----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY----- with the output of kubectl config view --minify --raw --output 'jsonpath={..user.client-key-data}' | base64 -d | sed -e 's/$/\\n/g' | tr -d '\n'

Finally, condense the entire JSON into a single line. (E.g. in vim you can use :%s/\n//g.) Save the entry and exit your editor.

Setup your kubeconfig

In order to instruct your kubeconfig to read credentials from pass, there is no plugin required. Edit your kubeconfig (e.g. with vim ~/.kube/config) and adjust the user of the current context by removing the entries for client-certificate-data and client-key-data and replace them with the following:

- name: example-user
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1
      command: pass
      args:
        - show
        - k8s.example.com/myuser

The resulting kubeconfig should look like this:

apiVersion: v1
kind: Config
current-context: example # …
preferences: {}
clusters:
- cluster:
  name: example-cluster
  # …
contexts:
- context:
    cluster: example-cluster
    user: example-user
  name: example
users:
- name: example-user
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1
      command: pass
      args:
        - show
        - k8s.example.com/myuser
      interactiveMode: IfAvailable

Next time you use any kubectl command, it should automatically use pass to fetch the credentials.

Note: You might need to use client.authentication.k8s.io/v1beta1 instead of client.authentication.k8s.io/v1 if your kubectl is older.

How does this work?

Kubectl added the exec functionality for user-credentials a while ago. The process is called and is supposed to return a JSON object that fits the schema. pass is a simple CLI tool that allows to store and output text in a secure way. Therefore by storing the expected format in pass it can simple return it.

Details about this functionality can be found in the Kubernetes documentation. This can also be used for access tokens and alike, if you have own programs to generate these.

There are also krew plugins like passman that store credentials in your OS Credential store or alike.

Security hint

Be aware that the exec functionality also means that a harmless looking kubeconfig can run arbitrary code on your system. So be careful where you obtain your kubeconfig from.

Further keep in mind, that pass is only as secure as your GnuPG key. However, this should always be more secure than a plaintext file.

Conclusion

If used properly it’s easy to secure Kubernetes credentials very easily from being exposed on disk and instead residing in a secure password storage. Given the power these credentials yield it’s a really good idea, to store them securely.

There is nothing to loose but a lot to win.

PS: This article is mostly a note to myself, but I hope it’s not too confusing.