Best Practices for Encrypting Kubernetes Secrets

July 18, 2025
12:02 am

Best practices for encrypting Kubernetes Secrets include using encryption at rest, KMS integration, RBAC, and key rotation. Learn how to secure your cluster effectively.

🔈Introduction

Kubernetes Secrets are powerful yet potentially risky components — they store sensitive data such as tokens, keys, and passwords that, if exposed, can compromise an entire cluster. While Kubernetes offers default base64 encoding for Secrets, this is not strong protection. In this post, we explore proven best practices to encrypt Secrets at rest, control access, rotate credentials, and audit usage — ensuring your cluster remains secure and compliant.

✅ Use Encryption at Rest

Enable `EncryptionConfiguration`

Kubernetes supports encryption providers for Secrets at rest. To enable this, add an encryptionConfiguration file when starting the API server.

Example: /etc/kubernetes/encryption-config.yaml

				
					apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
  - resources:
    - secrets
    providers:
    - aescbc:
        keys:
        - name: key1
          secret: BASE64ENCODED32BYTESECRET==
    - identity: {}

Then start kube-apiserver with:

				
					--encryption-provider-config=/etc/kubernetes/encryption-config.yaml

The first provider listed (aescbc) will encrypt Secret data before writing to etcd. The fallback identity provider leaves other resources unencrypted.

Choose a Strong Encryption Provider

Provider	Description	Use Case
`aescbc`	AES-CBC with HMAC, key rotation capable	Strong, widely recommended for most clusters
`aesgcm`	AES-GCM, additional authenticity protections	Secure and modern alternative; faster for smaller Secrets
`kms`	Delegates encryption to external provider	Useful for centralized key management

💡Recommendation: Start with aescbc or aesgcm. Advance to kms when you need centralized control or key splitting.

✅ Use External KMS for Key Management

Centralizing encryption keys in an external Key Management Service (KMS) improves security by decoupling keys from etcd.

Example: AWS KMS provider in encryption-config.yaml

				
					providers:
- kms:
    name: aws
    endpoint: unix:///var/run/kms-provider/socket
    cachesize: 1000
- identity: {}

Here, Secrets are encrypted in etcd using keys managed by AWS KMS – ideal for team environments requiring tight credential control.

✅ Work with Secrets via CLI & Templates

Create a Secret

				
					kubectl create secret generic my-secret \
  --from-literal=username=admin \
  --from-literal=password='S3cur3P@ssw0rd'

View Encrypted Secret

You can view the encrypted blob directly:

				
					kubectl get secret my-secret -o yaml | grep '^data:' -A 3

Decode the Secret

Decode base64-encoded values:

				
					echo "YWRtaW4=" | base64 --decode

Adopt SealedSecrets

Use Bitnami’s kubeseal to encrypt Secrets as YAML, leaving only a sealed blob in repositories with decryption done at runtime:

				
					kubectl create secret generic my-secret \
  --dry-run=client -o yaml \
  --from-literal=username=admin \
  --from-literal=password='S3cur3P@ss' \
  | kubeseal --format yaml \
  > my-secret-sealed.yaml

Apply with:

				
					kubectl apply -f my-secret-sealed.yaml

✅ Rotate Keys Regularly & Migrate Secrets

Add New Encryption Key

Add a new block in encryption-config.yaml:

				
					- aescbc:
    keys:
    - name: key2
      secret: NEWBASE64KEYHERE==
    - name: key1
      secret: OLDBASE64KEYHERE==

Remove the Old Key After Rotation

After re-encryption:

				
					- aescbc:
    keys:
    - name: key2
      secret: NEWBASE64KEYHERE==

Re-encrypt Existing Secrets

Run:

				
					kubectl get secrets --all-namespaces -o name \
  | xargs -n1 kubectl get -o yaml \
  | kubectl replace --force -f -

This avoids having Secrets in plain text for more than one reconciliation cycle.

Photo by admingeek from Infotechys

✅ Enforce RBAC and Least Privilege

Limit who can view or modify Secrets:

				
					apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: prod
  name: read-secrets
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]

Then bind it:

				
					kubectl create rolebinding read-secrets-binding \
  --role=read-secrets \
  --user=alice@example.com \
  --namespace=prod

This ensures only intended users or service accounts have Secret access.

✅ Implement Auditing

Track access and changes with the audit log:

				
					apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
  verbs: ["get", "update", "delete", "create"]
  resources:
  - group: ""
    resources: ["secrets"]

Enable API server flags:

				
					--audit-log-path=/var/log/kubernetes/audit.log
--audit-policy-file=/etc/kubernetes/audit-policy.yaml
--audit-log-maxage=30
--audit-log-maxbackup=10
--audit-log-maxsize=100

Review logs to see who accessed secrets and when.

✅ Use Secrets Encryption in Transit and in CI/CD

Always use --tls flags and ensure the Kubernetes API server servers only over HTTPS.

Store only sealed versions or encrypted formats in your CI/CD pipelines.

Avoid exposing secrets in logs, CI systems, or error dumps.

Summary Table

Area	Best Practice	Tools/Commands
Encrypt at Rest	Enable `aescbc`, `aesgcm` or `kms` provider	`encryption-config.yaml`, API server flags
External Key Management	Use KMS (AWS, GCP, Azure)	KMS provider config
Access Management	RBAC with least privilege	`kubectl create role`, rolebinding
Key Rotation	Periodically rotate, re-encrypt data	`kubectl replace --force`
CI/CD Integration	Use sealed or encrypted artifacts	`kubeseal`, sealed secret workflows
Auditing & Monitoring	Enable audit logging of secrets	API server flags & policies

📌 Final Thoughts

Encrypting Kubernetes Secrets isn’t just about base64 encoding. By implementing encryption at rest, managing keys via KMS, enforcing RBAC, rotating credentials, and auditing access, you strengthen your cluster’s security posture and meet compliance needs.

These best practices ensure:

Data at rest is never stored in plaintext

Access is controlled and monitored

Your cluster remains robust and secure

Maintaining these workflows in CI/CD pipelines and automating secret rotation and auditing turns ad hoc security into sustainable operations excellence.

Did you find this article helpful? Your feedback is invaluable to us! Feel free to share this post with those who may benefit, and let us know your thoughts in the comments section below.

👉 Related Posts

Commands

Best Practices for Securing SSH Connections on Linux Servers

Discover the best practices for securing SSH connections on Linux servers. Learn essential security tips like disabling root login, using SSH keys, and more for