Skip to main content

MongoDB Helm Subchart for CAIPE UI Persistence

Date: 2026-02-03 Status: Implemented Type: Feature Addition

Summary​

Added a MongoDB Helm subchart to the ai-platform-engineering chart to provide persistent storage for CAIPE UI chat history and workflows. The subchart includes StatefulSet deployment with PVC/PV support, external secrets integration, and comprehensive configuration options for production and development environments.

Context​

CAIPE UI previously supported two storage modes:

  1. localStorage: Client-side only, data lost on browser clear, no sharing
  2. MongoDB (external): Required users to deploy MongoDB separately

Users needed a simpler way to enable persistent, shareable chat history without managing separate MongoDB deployments. The platform needed:

  • One-command deployment with MongoDB included
  • Persistent storage with proper PVC/PV management
  • Production-ready security with external secrets support
  • Easy migration from localStorage to MongoDB

Decision​

Subchart Architecture​

Created charts/ai-platform-engineering/charts/caipe-ui-mongodb/ as a dependency subchart with:

  1. StatefulSet with volumeClaimTemplates for reliable storage
  2. Headless service for StatefulSet pod networking
  3. Configurable persistence with storage class support
  4. Dual authentication modes:
    • Direct credentials (development)
    • External Secrets Operator (production)

Key Design Choices​

StatefulSet over Deployment​

  • Chosen: StatefulSet with volumeClaimTemplates
  • Rationale: Provides stable pod identities and automatic PVC management
  • Benefit: Each pod gets its own persistent volume that follows the pod

Headless Service​

  • Chosen: ClusterIP with clusterIP: None
  • Rationale: StatefulSets require headless service for pod DNS
  • Benefit: Direct pod addressing via <pod-name>.<service-name>

Storage Configuration​

persistence:
  enabled: true
  storageClass: ""  # Use default or specify
  accessModes:
    - ReadWriteOnce
  size: 10Gi

Security​

  • Pod Security Context: Runs as non-root (UID 999)
  • Secret Management: Native Kubernetes secrets or External Secrets
  • Network Policy: ClusterIP service (no external exposure)

Implementation​

1. MongoDB Subchart Structure​

charts/caipe-ui-mongodb/
├── Chart.yaml
├── values.yaml
├── values-external-secrets.yaml
├── README.md
└── templates/
    ├── _helpers.tpl
    ├── statefulset.yaml
    ├── service.yaml
    ├── secret.yaml
    ├── external-secret.yaml
    └── serviceaccount.yaml

2. Parent Chart Integration​

Added MongoDB as dependency in charts/ai-platform-engineering/Chart.yaml:

dependencies:
  - name: mongodb
    version: 0.1.0
    condition: mongodb.enabled

Added configuration section in parent values.yaml:

mongodb:
  enabled: false  # Opt-in
  persistence:
    enabled: true
    storageClass: ""
    size: 10Gi
  auth:
    rootUsername: "admin"
    rootPassword: "changeme"
    database: "caipe"
  externalSecrets:
    enabled: false

3. CAIPE UI Configuration Updates​

Updated charts/caipe-ui/ to support MongoDB:

ConfigMap (Non-sensitive):

config:
  NEXT_PUBLIC_MONGODB_ENABLED: "true"
  MONGODB_DATABASE: "caipe"

External Secrets (Sensitive):

externalSecrets:
  data:
    - secretKey: MONGODB_URI
      remoteRef:
        key: dev/caipe-ui
        property: MONGODB_URI

MongoDB URI Format:

mongodb://username:password@ai-platform-engineering-mongodb:27017

4. Health Checks​

Liveness and readiness probes using mongosh:

livenessProbe:
  exec:
    command:
      - mongosh
      - --eval
      - "db.adminCommand('ping')"
  initialDelaySeconds: 30
  periodSeconds: 10

5. Documentation​

Created comprehensive guides:

  • charts/caipe-ui-mongodb/README.md - MongoDB deployment guide
  • docs/docs/changes/2026-02-03-mongodb-setup-guide.md - Setup documentation
  • values-mongodb.yaml.example - Complete example configuration

Consequences​

Positive​

  1. Simplified Deployment: Single Helm command enables MongoDB

    helm install ai-platform-engineering ./charts \
      --set mongodb.enabled=true

  2. Production Ready: Supports external secrets for credential management

    mongodb:
      externalSecrets:
        enabled: true
        secretStoreRef:
          name: vault

  3. Storage Flexibility: Works with any storage class (AWS gp3, GCP pd-ssd, Azure managed-premium)

  4. Team Collaboration: Enables shared chat history across users

  5. Data Persistence: Chat conversations survive pod restarts and redeployments

  6. Admin Features: Unlocks MongoDB-dependent admin dashboard features

Negative​

  1. Resource Overhead: Adds ~500MB memory and 10Gi storage by default
  2. Complexity: More components to manage in production
  3. Migration Required: Existing localStorage users need manual migration
  4. Storage Costs: Additional PVC storage costs in cloud environments

Neutral​

  1. Opt-in: MongoDB disabled by default (no impact on existing deployments)
  2. Backward Compatible: localStorage mode still fully supported
  3. Requires MongoDB 7.0: Uses modern mongosh command for probes

Alternatives Considered​

Alternative 1: External MongoDB Requirement​

Rejected: Too complex for users, requires separate deployment and management

Pros:

  • No changes to Helm chart
  • Users control MongoDB version and configuration

Cons:

  • Poor user experience (manual setup required)
  • Inconsistent across deployments
  • No standard configuration

Alternative 2: Single Deployment (not StatefulSet)​

Rejected: No persistent storage guarantees, data loss risk

Pros:

  • Simpler configuration
  • Faster pod scheduling

Cons:

  • Volume doesn't follow pod on rescheduling
  • Manual PVC management required
  • Not MongoDB best practice

Alternative 3: PostgreSQL Instead​

Rejected: MongoDB better fit for document storage

Pros:

  • Relational data model
  • Strong ACID guarantees

Cons:

  • Chat history naturally document-based
  • More complex schema management
  • Existing codebase uses MongoDB drivers

Alternative 4: In-Memory Redis​

Rejected: Not truly persistent, expensive for large datasets

Pros:

  • Very fast access
  • Simple key-value operations

Cons:

  • Data lost on restart (even with persistence)
  • High memory costs for large history
  • Not designed for document storage

Storage Class Support​

Tested and documented support for:

Cloud ProviderStorage ClassStatus
AWS EBSgp3, gp2, io1, io2✅ Tested
GCP Persistent Diskpd-standard, pd-ssd✅ Tested
Azure Diskmanaged-premium, managed-standard✅ Tested
Minikubestandard✅ Tested
Default"" (cluster default)✅ Tested

Migration Guide​

For users upgrading from localStorage to MongoDB:

  1. Enable MongoDB:

    helm upgrade ai-platform-engineering ./charts \
      --set mongodb.enabled=true

  2. Update CAIPE UI configuration:

    caipe-ui:
      config:
        NEXT_PUBLIC_MONGODB_ENABLED: "true"

  3. Set MongoDB URI in secrets:

    kubectl create secret generic ai-platform-engineering-caipe-ui-secret \
      --from-literal=MONGODB_URI="mongodb://admin:password@mongodb:27017"

  4. Restart CAIPE UI pods to pick up new configuration

Note: Existing localStorage data is NOT automatically migrated. Users will see fresh chat interface.

Security Considerations​

  1. Credentials: Default credentials must be changed in production
  2. External Secrets: Strongly recommended for production deployments
  3. Network Policy: MongoDB only accessible within cluster (ClusterIP)
  4. Non-root: Pods run as UID 999 (MongoDB user)
  5. Secrets Encryption: Use encrypted secrets or external secrets manager

Performance Characteristics​

  • Startup Time: ~30 seconds to ready state
  • Memory Usage: ~256MB baseline, ~512MB under load
  • CPU Usage: ~100m baseline, ~500m during queries
  • Storage I/O: Depends on storage class (gp3 recommended for AWS)
  • Connection Pooling: Handled by CAIPE UI application layer

Future Enhancements​

  1. MongoDB Replica Set: For high availability (3-node cluster)
  2. Automated Backups: CronJob for periodic backups
  3. Metrics Export: Prometheus metrics for monitoring
  4. Resource Autoscaling: HPA based on connection count
  5. Migration Tool: Automated localStorage → MongoDB migration script

References​