Upgrading Shoehorn

Upgrade Process

1. Review Release Notes

Before upgrading, check the release notes for:

• Breaking changes
• New required environment variables
• Database migration requirements
• Deprecated features

2. Backup Database

# Using pg_dump
pg_dump -h <host> -U shoehorn_user -d shoehorn > backup_$(date +%Y%m%d).sql

# Or using kubectl
kubectl exec -n shoehorn deploy/postgresql -- \
  pg_dump -U shoehorn_user shoehorn > backup.sql

3. Update Helm Values

Update the image tag in your values file:

image:
  tag: "0.7.0"  # New version

4. Run Upgrade

helm upgrade shoehorn oci://ghcr.io/shoehorn-dev/helm-charts/shoehorn \
  --namespace shoehorn \
  -f values.yaml

Helm will:

1. Apply new container images
2. Run database migrations automatically (init container)
3. Rolling-restart services with zero downtime

PostgreSQL is not restarted by helm upgrade. See PostgreSQL upgrades below.

5. Verify

# Check rollout status
kubectl rollout status -n shoehorn deploy/api

# Check API health
kubectl port-forward -n shoehorn svc/api 8080:8080
curl http://localhost:8080/health

# Check logs for migration errors
kubectl logs -n shoehorn -l app=api --tail=100

PostgreSQL upgrades

The postgres StatefulSet uses updateStrategy: OnDelete and pins its image in values.yaml (e.g. shoehorned/shoehorn-postgres:v18.3-pgaudit-1.0). helm upgrade won’t restart the pod even if the chart bumps the tag. Roll it deliberately:

kubectl get sts -n shoehorn shoehorn-postgresql -o jsonpath='{.spec.template.spec.containers[0].image}'
kubectl delete pod -n shoehorn shoehorn-postgresql-0

The PVC stays attached. Data is preserved.

The StatefulSet, PVC, and Service carry helm.sh/resource-policy: keep. helm uninstall leaves them in place. Delete them by hand for a clean slate.

For an external database, set postgresql.enabled: false and point DATABASE_URL and MIGRATION_DATABASE_URL at your server.

Rollback

If issues occur after upgrading:

# Rollback to previous Helm release
helm rollback shoehorn -n shoehorn

# Or rollback to a specific revision
helm history shoehorn -n shoehorn
helm rollback shoehorn <revision-number> -n shoehorn

Database migrations are forward-only. If a migration must be reverted, apply the corresponding down migration manually:

kubectl exec -n shoehorn deploy/api -- \
  ./migrate -path /migrations -database "$MIGRATION_DATABASE_URL" down 1

K8s Agent Upgrades

The K8s agent can be upgraded independently of the main platform:

helm upgrade shoehorn-k8s-agent oci://ghcr.io/shoehorn-dev/helm-charts/shoehorn-k8s-agent \
  --namespace shoehorn-agent \
  -f agent-values.yaml

The agent is backward-compatible with older API versions. Upgrade the platform first, then agents.

Things to watch during an agent upgrade:

• Leader re-election: The rolling restart triggers a new leader election. There is a brief gap (a few seconds) between the old leader stopping and the new leader starting. No data is lost — events are buffered in the Kubernetes informer cache.
• HA deployments: With a PodDisruptionBudget in place, at least one replica stays available throughout the upgrade. Followers become the new leader seamlessly.
• Configuration validation: If the new agent version introduces stricter configuration parsing, invalid environment variables that were previously silently accepted may cause startup failures. Check pod logs if a new version does not become ready.

See the shoehorn-dev/helm-charts repository for the chart changelog and values reference.