82 lines
2.7 KiB
Markdown
82 lines
2.7 KiB
Markdown
# GoChat Rolling Upgrade Strategy
|
|
# Reference: Chatwoot deployment uses zero-downtime upgrade pattern
|
|
|
|
## Overview
|
|
|
|
GoChat follows a blue-green deployment strategy for production upgrades,
|
|
ensuring zero downtime during version transitions.
|
|
|
|
## Upgrade Process
|
|
|
|
### Step 1: Pre-flight Checks
|
|
1. Verify new Docker image is built and pushed: `docker pull gochat/gochat:${NEW_VERSION}`
|
|
2. Run database migrations on a staging environment first
|
|
3. Verify backward compatibility of migrations (new code must work with old schema)
|
|
4. Check feature flags — new features should be disabled by default
|
|
|
|
### Step 2: Database Migration
|
|
```bash
|
|
# Run migrations BEFORE deploying new code
|
|
# Migrations must be backward-compatible
|
|
docker compose -f docker-compose.prod.yml exec gochat /app/gochat migrate up
|
|
```
|
|
|
|
### Step 3: Blue-Green Deployment (Docker Compose)
|
|
```bash
|
|
# 1. Deploy new version as "green" alongside "blue" (current)
|
|
docker compose -f docker-compose.prod.yml up -d --no-deps gochat-green
|
|
|
|
# 2. Wait for health check to pass
|
|
curl -f http://gochat-green:3000/health
|
|
|
|
# 3. Switch traffic (update nginx/upstream config)
|
|
# nginx: switch upstream from blue to green
|
|
|
|
# 4. Drain old connections on blue
|
|
# Wait 30s for in-flight requests to complete
|
|
|
|
# 5. Stop blue
|
|
docker compose -f docker-compose.prod.yml stop gochat
|
|
```
|
|
|
|
### Step 4: K8s Rolling Update
|
|
```bash
|
|
# Kubernetes handles rolling updates automatically via Deployment strategy
|
|
kubectl set image deployment/gochat gochat=gochat/gochat:${NEW_VERSION}
|
|
kubectl rollout status deployment/gochat
|
|
```
|
|
|
|
K8s config (in Helm chart):
|
|
- maxSurge: 1 (one extra pod during rollout)
|
|
- maxUnavailable: 0 (no pods allowed to be unavailable)
|
|
- readinessProbe: /ready endpoint
|
|
- livenessProbe: /live endpoint
|
|
|
|
### Step 5: Verification
|
|
1. Smoke test: hit /health, /ready endpoints
|
|
2. Check logs for errors: `docker compose logs gochat --since 5m`
|
|
3. Verify metrics: Prometheus dashboard should show normal traffic
|
|
4. Monitor for 15 minutes before finalizing
|
|
|
|
### Step 6: Rollback (if needed)
|
|
```bash
|
|
# Docker Compose rollback
|
|
docker compose -f docker-compose.prod.yml exec gochat /app/gochat migrate down ${N}
|
|
docker compose -f docker-compose.prod.yml up -d --no-deps gochat-${OLD_VERSION}
|
|
|
|
# K8s rollback
|
|
kubectl rollout undo deployment/gochat
|
|
```
|
|
|
|
## Migration Compatibility Rules
|
|
|
|
- Migrations MUST be additive only in production (add columns, never remove)
|
|
- Column removals require a 2-phase migration: soft-remove then hard-remove
|
|
- New columns should have defaults or be nullable
|
|
- Renames require a 3-phase migration: add new → copy data → remove old
|
|
|
|
## Worker Upgrade
|
|
|
|
Workers drain naturally: set a shutdown deadline, let in-flight jobs finish,
|
|
then stop. New workers pick up queued jobs from Redis.
|