Secrets Management

Overview of secrets management options for CRAFT, including ESO + GCP Secret Manager and Infisical as equal deployment alternatives.

Secrets Management

CRAFT supports two secrets management backends. Both operate through the same Governance Secrets API abstraction, applications access secrets through a unified interface regardless of which backend is deployed.

Secrets API Abstraction

The Governance service exposes a Secrets API that proxies secret read/write operations to the configured backend. This decouples application code from the underlying secrets provider.

Application → Governance Secrets API → [ ESO + GCP Secret Manager | Infisical ]

Option A: ESO + GCP Secret Manager

Recommended for cloud deployments on GCP. Uses the External Secrets Operator (ESO) to sync secrets from GCP Secret Manager into Kubernetes Secrets, with Stakater Reloader for automatic pod restarts on secret changes.

Architecture

Components

Component	Purpose
GCP Secret Manager	Cloud-native secret store, encrypted at rest, audit-logged, IAM-controlled
External Secrets Operator	Kubernetes operator that syncs GCP SM secrets into K8s Secrets on a schedule
ClusterSecretStore	Cluster-scoped ESO resource that configures the GCP SM connection
ExternalSecret	Namespace-scoped resource mapping a GCP SM secret to a K8s Secret
Stakater Reloader	Watches K8s Secrets for changes; triggers rolling restarts of annotated pods

Secret Naming Convention

Secrets in GCP Secret Manager follow a <service>-<environment> naming pattern. Specific names are environment-specific.

Authentication

ESO authenticates to GCP Secret Manager using Workload Identity, no service account keys are stored in the cluster.

Rotation

When a secret value is updated in GCP Secret Manager:

1. ESO detects the change on its next sync interval (configurable, default 1h)
2. The K8s Secret is updated
3. Stakater Reloader detects the change (via annotation reloader.stakater.com/auto: "true") and triggers a rolling restart

For immediate rotation, manually trigger an ESO sync or restart the affected deployment.

Secrets injected as environment variables require a pod restart to take effect, even after ESO syncs the updated value to the Kubernetes Secret. Stakater Reloader (included in reference infrastructure) automates this by triggering rolling restarts when annotated Secrets change. For security incidents, manually restart the affected deployment immediately after triggering an ESO sync. Consider reducing the default 1-hour sync interval for high-sensitivity secrets.

Deployment

See Infrastructure Secrets for the step-by-step deployment guide.

---

Option B: Infisical

Recommended for on-premises and cloud-agnostic deployments. Infisical provides application-level envelope encryption with a self-hostable server and native Kubernetes authentication.

Architecture

Infisical runs as a subchart within the em-runtime Helm release and stores secrets in its own PostgreSQL database (infisical). Services authenticate via Infisical’s Kubernetes machine identity.

Components

Component	Purpose
Infisical server	Self-hosted secret management platform (deployed as Helm subchart)
Infisical SDK	Python/TypeScript SDK used by em-runtime services at startup
Machine Identity	Kubernetes service account-based authentication (no static credentials)
Envelope encryption	Secrets encrypted at rest with a per-secret encryption key hierarchy

Configuration

Infisical is enabled by default in the em-runtime Helm chart:

infisical:
  enabled: true
  resources:
    requests:
      cpu: "250m"
      memory: "512Mi"

To use an external Infisical instance instead of the subchart:

infisical:
  enabled: false

governance:
  extraEnvVars:
    - name: INFISICAL_URL
      value: "https://infisical.example.com"
    - name: INFISICAL_PROJECT_ID
      value: "your-project-id"

Rotation

Infisical supports secret rotation via its dashboard or API. Service pods must be restarted to pick up new secret values (unlike ESO + Reloader which handles this automatically).

Infisical requires manual pod restarts to pick up rotated secrets. For regulated deployments, configure Stakater Reloader (included in reference infrastructure) to watch Infisical-managed Kubernetes Secrets and trigger automated rolling restarts on change. Document the rotation procedure including verification steps.

---

Comparison

Attribute	ESO + GCP Secret Manager	Infisical
Deployment model	Cloud-native (GCP)	Self-hosted or cloud-agnostic
Cloud dependency	Requires GCP	No cloud dependency
Encryption at rest	GCP KMS (managed)	Envelope encryption (self-managed)
Auto-rotation	Yes (Stakater Reloader)	Manual pod restart required
Audit logging	GCP Cloud Audit Logs	Infisical audit logs
On-prem support	No	Yes
Helm chart	ESO Helm chart (separate)	Infisical subchart (included)
Auth mechanism	GCP Workload Identity	Kubernetes machine identity

Choosing a Backend

• GKE (GCP cloud deployment): ESO + GCP Secret Manager is the default and is provisioned by Terraform
• On-premises or non-GCP cloud: Infisical (included in Helm chart as subchart)
• Multi-cloud (EKS/AKS): Either, ESO supports AWS Secrets Manager and Azure Key Vault; Infisical is cloud-agnostic

Both backends are supported and maintained. The choice depends on your deployment environment, not the platform version.

Related

Infrastructure Secrets

Step-by-step guide to deploying ESO or Infisical in your cluster.

Data Classification

How credentials are classified and protected within the platform.

Helm Configuration

Helm values for enabling and configuring secrets backends.

Network Security

Network-level controls that protect secret access paths.