MOUNT AWS SECRETS in EKS
2 min readSep 29, 2023
Install CSI drivers
- Secrets Store CSI Secret driver.
helm repo add secrets-store-csi-driver \
https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
helm install -n kube-system csi-secrets-store \
--set syncSecret.enabled=true \
--set enableSecretRotation=true \
secrets-store-csi-driver/secrets-store-csi-driver
2. AWS Secrets and Configuration Provider
kubectl apply -f https://raw.githubusercontent.com/aws/secrets-store-csi-driver-provider-aws/main/deployment/aws-provider-installer.yaml
3. Verify the installation
kubectl get daemonsets -n kube-system -l app=csi-secrets-store-provider-aws
kubectl get daemonsets -n kube-system -l app.kubernetes.io/instance=csi-secrets-store
Prepare Secret and IAM Access Controls
- Create Policy
aws iam create-policy --policy-name {policy-name} --policy-document file://policy.json
Policy.json
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret"
],
"Resource": [
"arn:aws:secretsmanager:us-west-2:123456798012:secret:{secretname}-IKhKGX"
]
}]
}
2. Create Role with the above policy and trustrelationship should be according to trusy-policy.json file
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::1234567890:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/DEJHGKLJH4564sdkfh897897sdfGG"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"oidc.eks.us-west-2.amazonaws.com/id/DEJHGKLJH4564sdkfh897897sdfGG:aud": "sts.amazonaws.com",
"oidc.eks.us-west-2.amazonaws.com/id/DEJHGKLJH4564sdkfh897897sdfGG:sub": "system:serviceaccount:{namespace-name}:{service-account-name}"
}
}
}]
}
3. Create a Service Account with IAM role
eksctl utils associate-iam-oidc-provider \
--region="$AWS_REGION" --cluster="$EKS_CLUSTERNAME" \
--approve
# Service-account.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
annotationsya:
eks.amazonaws.com/role-arn: {role-arn}
labels:
app.kubernetes.io/managed-by: eksctl
name: {service-account-name}
namespace: {namespace-name}
# kubectl apply -f Service-account.yaml
# kubectl get sa -n {namespace-name}
Deploy Pods With Mounted Secrets
- Create Service Provider Class
# spc-k8s-secrets.yaml
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: deployment-spc-k8s-secrets
namespace: rds-app
spec:
provider: aws
parameters:
objects: |
- objectName: {"secret-name"}
objectType: "secretsmanager"
jmesPath:
- path: username
objectAlias: dbusername
- path: password
objectAlias: dbpassword
- path: engine
objectAlias: dbengine
- path: host
objectAlias: dbhost
- path: dbInstanceIdentifier
objectAlias: dbInstanceIdentifier
# Create k8s secret. It requires volume mount first in the pod and then sync.
secretObjects:
- secretName: {"secret-name"}
type: Opaque
data:
#- objectName: <objectName> or <objectAlias>
- objectName: dbusername
key: db_username_01
- objectName: dbpassword
key: db_password_01
- objectName: dbengine
key: db_engine
- objectName: dbhost
key: db_host
- objectName: dbInstanceIdentifier
key: dbInstanceIdentifier
# kubectl apply -f spc-k8s-secrets.yaml
# kubectl get SecretProviderClass
2. Create pod and mount secrets
# deployment.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: k8s-secrets
labels:
app: k8s-secrets
spec:
replicas: 1
selector:
matchLabels:
app: k8s-secrets
template:
metadata:
labels:
app: k8s-secrets
spec:
serviceAccountName: deployment-sa
containers:
- name: k8s-secrets
image: nginx
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
volumeMounts:
- name: secrets-store-inline
mountPath: "/mnt/secrets"
readOnly: true
env:
- name: DB_USERNAME_01
valueFrom:
secretKeyRef:
name: {secret-name}
key: db_username_01
- name: DB_PASSWORD_01
valueFrom:
secretKeyRef:
name: {secret-name}
key: db_password_01
volumes:
- name: secrets-store-inline
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: spc-k8s-secrets
# kubectl apply -f deployment.yaml
# kubectl get deployments -n {namespace}
# kubectl get po -n {namespace}
# kubectl describe secrets k8s-secrets
Verify Secrets
kubectl exec -it {pod name} - /bin/bash
echo $DB_PASSWORD_01