Este é o segundo post da série sobre o Vault. Clique aqui para ver o primeiro post e conhecer um pouco sobre essa ferramenta.
Obrigado Filipe Maia, André Luis Soares e Kleberson Sartori pelo apoio no dia a dia e compartilhamento do conhecimento com o Vault.
Requisitos
- O cluster Kubernetes já deve estar criado e você deve ter permissão para instalar aplicações. Neste caso, estou usando um cluster Google Kubernetes Engine (GKE) Standard. Você pode adaptar o procedimento, comandos e configurações de acordo com a sua necessidade. Neste caso, estou usando o Kuberentes na versão 1.30 (este passo não será mostrado no tutorial).
- O kubectl também precisa estar instalado no seu computador e conectado ao cluster Kubernetes. Neste caso, estou usando o kubectl na versão 1.30 (este passo não será mostrado no tutorial).
- O binário do helm também precisa estar instalado no seu computador. Neste caso, estou usando o helm 3.16 (este passo não será mostrado no tutorial).
- Crie um bucket GCS para armazenar os dados do vault (este passo não será mostrado no tutorial);
- Crie um KMS (Key Management Service) e uma chave simétrica para uso o vault criptografar/descriptografar os segredos armazenados no bucket (este passo não será mostrado no tutorial);
- Crie um role IAM customizada com as seguintes permissões (este passo não será mostrado no tutorial).
- cloudkms.cryptoKeyVersions.useToDecrypt
- cloudkms.cryptoKeyVersions.useToEncrypt
- cloudkms.cryptoKeys.get
- storage.objects.create
- storage.objects.delete
- storage.objects.get
- storage.objects.list
- storage.objects.update
- Crie uma service account na GCP associando a role IAM customizada (este passo não será mostrado no tutorial).
- Na service account, crie um chave JSON sem tempo de expiração (duração inifita) e salve o arquivo como credentials.json (este passo não será mostrado no tutorial).
Instalando o Vault no Kubernetes usando Helm
O Helm facilita muito a instalação de aplicações no cluster Kubernetes. Execute os passos a seguir.
Adicione o repositório Helm do HashiCorp.
helm repo add hashicorp https://helm.releases.hashicorp.com
Atualize todos os repositórios Helm para obter as versões mais recentes.
helm repo update
Verifique todas as versões disponíveis do helm chart do Vault
helm search repo vault --versions
Obtenha o helm values padrão para um arquivo e customize os valores de acordo com a necessidade. Tem muita opção!!! Após as referências deste tutorial, tem um arquivo de exemplo para um ambiente de dev/test. Mas lembre-se de mudar os valores que começam com a palavra change.
VAULT_CHART_VERSION=0.28.1
helm show values hashicorp/vault --version "$VAULT_CHART_VERSION" > values.yaml
Crie uma secret no kubernetes associando o arquivo credentials.json da service account a ser usada pelo vault para acessar o bucket GCS e o KMS.
kubectl create namespace vault
kubectl create secret generic vault-gcp-creds --from-file=credentials.json -n vault
Instale o Vault.
helm upgrade --install vault \
hashicorp/vault --version "$VAULT_CHART_VERSION" -f values.yaml \
--namespace vault --create-namespace --debug --timeout=900s
Visualize todos os objetos relacionados ao vault.
kubectl get all -n vault
Os pods vault-0
, vault-1
e vault-2
implantados executam um servidor Vault e relatam que estão em execução, mas não estão prontos (0/1). Isso ocorre porque a verificação de status definida em um readinessProbe retorna um código de saída diferente de zero.
O pod vault-agent-injector
implantado é um Kubernetes Mutation Webhook Controller. O controlador intercepta eventos de pod e aplica mutações ao pod se houver anotações específicas na solicitação.
Recupere o status do Vault no pod vault-0
.
kubectl exec vault-0 --namespace vault -- vault status
O comando status relata que o Vault não foi inicializado e que está lacrado. Para que o Vault seja autenticado com o Kubernetes e gerencie segredos, é necessário que ele seja inicializado e deslacrado.
Inicializando e unseal do Vault
Criar um port-forward do pod vault-0
para acessar a interface web do vault e inicializá-lo.
kubectl port-forward pod/vault-0 8200:8200 -n vault
Abra outro terminal e inicialize o Vault:
export VAULT_ADDR=http://127.0.0.1:8200
# O comando a seguir irá gerar o token root de acesso ao vault e 5 chaves de recuperação do Vault. Guarde em local seguro e vamos utilizar ao longo do tutorial e no dia a dia.
vault operator init
# Faça o login no vault
vault login
# Deslacre o vault
vault operator unseal
# Opcionalmente, se o token raiz inicial for perdido, você pode usar o comando a seguir para gerar um novo token raiz
# vault operator generate-root
Obtenha novamente o status do Vault no pod vault-0
.
kubectl exec vault-0 --namespace vault -- vault status
O servidor Vault é inicializado e deslacrado (unsealed).
Crie um token inicial k8s
Obtenha os detalhes da service account do vault no cluster Kubernetes.
kubectl describe serviceaccount vault -n vault
No fim do resultado do comando anterior deve ter algo semelhante:
Tokens: <nenhum>
Eventos: <nenhum>
ATENÇÃO!!! No Kubernetes 1.24+, o token não é criado automaticamente, e você deve criá-lo explicitamente.
Crie um arquivo de secret com os seguintes comandos.
cat > vault-secret.yaml <<EOF
apiVersion: v1
kind: Secret
metadata:
name: vault-token-g955r
annotations:
kubernetes.io/service-account.name: vault
type: kubernetes.io/service-account-token
EOF
Cria a secret com o seguinte comando.
kubectl apply -f vault-secret.yaml -n vault
Crie uma variável chamada VAULT_HELM_SECRET_NAME que armazena o token gerado pela secret (isso será usado adiante).
VAULT_HELM_SECRET_NAME=$(kubectl -n vault get secrets --output=json | jq -r '.items[].metadata | select(.name|startswith("vault-token-")).name')
Descreva novamente a conta de serviço e observe que ela foi atualizada com o segredo e o token.
kubectl describe serviceaccount vault -n vault
Configure o acesso do Kubernetes à API do Vault
O Vault fornece um método de autenticação do Kubernetes que permite que os microsserviços se autentiquem com um token da service account do Kubernetes.
Habilite o método de autenticação do Kubernetes.
export VAULT_ADDR=http://127.0.0.1:8200
vault login
vault auth enable kubernetes
O Vault aceita este token de serviço de qualquer pod dentro do cluster do Kubernetes. Durante a autenticação, o Vault verifica se o token da service account é válido. Para configurá-lo corretamente, é necessário capturar o token da web JSON (JWT) para a service account, o certificado da Autoridade Certificadora do cluster Kubernetes e a URL do host interno do GKE.
Obtenha o token da web JSON (JWT) do segredo.
export VAULT_ADDR=http://127.0.0.1:8200
vault login
TOKEN_REVIEW_JWT=$(kubectl -n vault get secret $VAULT_HELM_SECRET_NAME --output='go-template={{ .data.token }}' | base64 --decode)
Recupere o certificado da CA do Kubernetes.
KUBE_CA_CERT=$(kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.certificate-authority-data}' | base64 --decode)
Recupere o cluster interno da URL do host do GKE.
CLUSTER_NAME="change-gke-name"
GKE_REGION="change-gcp-region"
GKE_PROJECT="change-gcp-project"
KUBE_HOST=$(gcloud container clusters describe $CLUSTER_NAME --region $GKE_REGION --project $GKE_PROJECT --format="value(privateClusterConfig.privateEndpoint)")
Você pode cadastrar no vault o nome do emissor do cluster Kubernetes usando este método.
vault write auth/kubernetes/config \
token_reviewer_jwt="$TOKEN_REVIEW_JWT" \
kubernetes_host="https://${KUBE_HOST}" \
kubernetes_ca_cert="$KUBE_CA_CERT" \
issuer="https://kubernetes.default.svc.cluster.local"
Desistalando o Vault
Se precisar remover o vault, use o seguinte comando:
helm uninstall vault -n vault
Referências
- https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-google-cloud-gke
- https://developer.hashicorp.com/vault/docs
- https://developer.hashicorp.com/vault/docs/concepts/seal
- https://medium.com/@lingxiankong/vault-integration-with-kubernetes-unified-identity-management-service-36d9c8edddf0
- https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-external-vault?in=vault%2Fkubernetes#configure-kubernetes-authentication
- https://stackoverflow.com/questions/76533818/vault-init-container-cannot-authenticate-to-vault
- https://medium.com/adessoturkey/hashicorp-vault-authentication-options-465b5b8eb560
- https://support.hashicorp.com/hc/en-us/articles/4404389946387-Kubernetes-auth-method-Permission-Denied-error
- https://support.hashicorp.com/hc/en-us/articles/13772047777811-Getting-the-error-route-entry-is-tainted-when-performing-read-or-write-actions-on-a-custom-plugin
- https://discuss.hashicorp.com/t/info-agent-auth-handler-authenticating-url-put-http-x-x-svc-8200-v1-auth-kubernetes-login-code-403-permission-denied/61437
- https://support.hashicorp.com/hc/en-us/articles/4417287569555-Vault-Agent-injector-ERROR-auth-handler-error-authenticating-error-context-deadline-exceeded
Values.yaml de exemplo
# Vault Helm Chart Value Overrides
global:
enabled: true
injector:
enabled: true
replicas: 3
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- vault
topologyKey: "kubernetes.io/hostname"
# image sets the repo and tag of the vault-k8s image to use for the injector.
image:
repository: "hashicorp/vault-k8s"
tag: "1.4.2"
pullPolicy: IfNotPresent
# agentImage sets the repo and tag of the Vault image to use for the Vault Agent
# containers. This should be set to the official Vault image. Vault 1.3.1+ is
# required.
agentImage:
repository: "hashicorp/vault"
tag: "1.17.2"
# The default values for the injected Vault Agent containers.
agentDefaults:
# For more information on configuring resources, see the K8s documentation:
# https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
cpuLimit: "500m"
cpuRequest: "250m"
memLimit: "128Mi"
memRequest: "64Mi"
# Configures the log verbosity of the injector.
# Supported log levels include: trace, debug, info, warn, error
logLevel: "error"
# Configures the log format of the injector. Supported log formats: "standard", "json".
logFormat: "standard"
# Security context for the pod template and the injector container
# The default pod securityContext is:
# runAsNonRoot: true
# runAsGroup: {{ .Values.injector.gid | default 1000 }}
# runAsUser: {{ .Values.injector.uid | default 100 }}
# fsGroup: {{ .Values.injector.gid | default 1000 }}
# and for container is
# allowPrivilegeEscalation: false
# capabilities:
# drop:
# - ALL
securityContext:
pod: {}
container: {}
resources: {}
#resources:
# requests:
# memory: 256Mi
# cpu: 250m
# limits:
# memory: 256Mi
# cpu: 250m
# A disruption budget limits the number of pods of a replicated application
# that are down simultaneously from voluntary disruptions
# Disable in dev to cost optimization
podDisruptionBudget: {}
#podDisruptionBudget:
# podDisruptionBudget:
# maxUnavailable: 1
webhook:
# Fix race condition vault-injector-sidecar:
# https://discuss.hashicorp.com/t/vault-agent-injector-racing-condition-when-application-pods-utilizing-vault-sidecar-come-up-faster-then-vault/31297/7
# https://github.com/hashicorp/vault-k8s/issues/415
#
# Configures failurePolicy of the webhook. The "unspecified" default behaviour depends on the
# API Version of the WebHook.
# To block pod creation while the webhook is unavailable, set the policy to `Fail` below.
# See https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#failure-policy
#
failurePolicy: Fail
# namespaceSelector is the selector for restricting the webhook to only
# specific namespaces.
# See https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-namespaceselector
# for more details.
# Example:
# namespaceSelector:
# matchLabels:
# sidecar-injector: enabled
namespaceSelector:
matchExpressions:
- key: kubernetes.io/metadata.name
operator: In
values: ["change-app-namespace"]
# objectSelector is the selector for restricting the webhook to only
# specific labels.
# See https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#matching-requests-objectselector
# for more details.
# Example:
# objectSelector:
# matchLabels:
# vault-sidecar-injector: enabled
objectSelector: |
matchExpressions:
- key: app.kubernetes.io/name
operator: NotIn
values:
- {{ template "vault.name" . }}-agent-injector
server:
enabled: true
image:
repository: "hashicorp/vault"
tag: "1.17.2"
# Overrides the default Image Pull Policy
pullPolicy: IfNotPresent
# Configure the Update Strategy Type for the StatefulSet
# See https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#update-strategies
updateStrategyType: "OnDelete"
# Configure the logging verbosity for the Vault server.
# Supported log levels include: trace, debug, info, warn, error
logLevel: "error"
# Configure the logging format for the Vault server.
# Supported log formats include: standard, json
logFormat: ""
resources: {}
#resources:
# requests:
# memory: 256Mi
# cpu: 250m
# limits:
# memory: 256Mi
# cpu: 250m
# OpenShift only - create a route to expose the service
# By default the created route will be of type passthrough
route:
enabled: false
# When HA mode is enabled and K8s service registration is being used,
# configure the route to point to the Vault active service.
activeService: true
labels: {}
annotations: {}
host: chart-example.local
# tls will be passed directly to the route's TLS config, which
# can be used to configure other termination methods that terminate
# TLS at the router
tls:
termination: passthrough
# extraEnvironmentVars is a list of extra environment variables to set with the stateful set. These could be
# used to include variables required for auto-unseal.
extraEnvironmentVars:
#GOOGLE_REGION: change-gcp-region
#GOOGLE_PROJECT: change-gcp-project
GOOGLE_APPLICATION_CREDENTIALS: /vault/userconfig/gcp-creds/credentials.json
# Deprecated: please use 'volumes' instead.
# extraVolumes is a list of extra volumes to mount. These will be exposed
# to Vault in the path `/vault/userconfig/<name>/`. The value below is
# an array of objects, examples are shown below.
extraVolumes: []
# - type: secret (or "configMap")
# name: my-secret
# path: null # default is `/vault/userconfig`
# volumes is a list of volumes made available to all containers. These are rendered
# via toYaml rather than pre-processed like the extraVolumes value.
# The purpose is to make it easy to share volumes between containers.
volumes:
- name: gcp-creds
secret:
secretName: vault-gcp-creds
# volumeMounts is a list of volumeMounts for the main server container. These are rendered
# via toYaml rather than pre-processed like the extraVolumes value.
# The purpose is to make it easy to share volumes between containers.
volumeMounts:
- mountPath: /vault/userconfig/gcp-creds/
name: gcp-creds
readOnly: true
standalone:
enabled: false
# Run Vault in "HA" mode. There are no storage requirements unless the audit log
# persistence is required. In HA mode Vault will configure itself to use Consul
# for its storage backend. The default configuration provided will work the Consul
# Helm project by default. It is possible to manually configure Vault to use a
# different HA backend.
ha:
enabled: true
replicas: 3
raft:
# Enables Raft integrated storage
enabled: false
# config is a raw string of default configuration when using a Stateful
# deployment. Default is to use a Consul for its HA storage backend.
# This should be HCL.
# Note: Configuration files are stored in ConfigMaps so sensitive data
# such as passwords should be either mounted through extraSecretEnvironmentVars
# or through a Kube secret. For more information see:
# https://developer.hashicorp.com/vault/docs/platform/k8s/helm/run#protecting-sensitive-vault-configurations
config: |
ui = true
listener "tcp" {
tls_disable = "true"
address = "[::]:8200"
cluster_address = "[::]:8201"
}
seal "gcpckms" {
project = "change-gcp-project"
region = "change-gcp-region"
key_ring = "change-gcp-kms-ring"
crypto_key = "change-gcp-kms-key"
}
storage "gcs" {
bucket = "change-gcp-bucket"
ha_enabled = "true"
}
service_registration "kubernetes" {}
# A disruption budget limits the number of pods of a replicated application
# that are down simultaneously from voluntary disruptions
# Disable in dev to cost optimization
disruptionBudget:
enabled: false
securityContext:
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
add: ["SYS_TIME"]
# Affinity Settings
# Commenting out or setting as empty the affinity variable, will allow
# deployment to single node services such as Minikube
# This should be either a multi-line string or YAML matching the PodSpec's affinity field.
affinity: |
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app.kubernetes.io/name: {{ template "vault.name" . }}
app.kubernetes.io/instance: "{{ .Release.Name }}"
component: server
topologyKey: kubernetes.io/hostname
# Vault UI
ui:
# True if you want to create a Service entry for the Vault UI.
#
# serviceType can be used to control the type of service created. For
# example, setting this to "LoadBalancer" will create an external load
# balancer (for supported K8S installations) to access the UI.
enabled: false