Metis

Field note / fundamentals

publishedupdated 2026-05-01#deployment#blue-green#canary#feature-flags#devops

Deployment Strategies

Blue-green, canary, rolling, and feature flags — how to ship safely to production with zero downtime and controlled rollouts.

Overview

Zero-downtime deployment is a solved problem — but choosing the right strategy depends on your traffic patterns, rollback tolerance, and infrastructure. The four core strategies cover the full spectrum from "swap everything at once" to "one pod at a time."

Architecture

BLUE-GREEN DEPLOYMENT
════════════════════════════════════════════════════════
  Load Balancer
       │
       ├──▶ Blue (v1.0)  ← currently live, serving 100% traffic
       │
       └──▶ Green (v2.0) ← new version, idle

  DEPLOY STEP 1: Deploy v2.0 to Green while Blue still serves traffic
  DEPLOY STEP 2: Run smoke tests against Green
  DEPLOY STEP 3: Switch load balancer to send 100% → Green
  ROLLBACK:      Switch load balancer back to Blue (instant)

  Pros: instant rollback, full isolation, zero downtime
  Cons: 2× infrastructure cost, DB migrations must be backward-compatible

CANARY DEPLOYMENT
════════════════════════════════════════════════════════
  Load Balancer
       │
       ├──▶ Stable (v1.0)  ← 95% traffic
       │
       └──▶ Canary (v2.0)  ← 5% traffic ("canary in the coal mine")

  STEP 1: Deploy v2.0 to small subset (1% → 5% → 20% → 100%)
  STEP 2: Monitor error rates, latency, business metrics
  STEP 3: Promote or rollback based on telemetry
  STEP 4: Gradually shift all traffic to v2.0

  Used by: Netflix, Google, Amazon for every production deploy
  Pros: real-user validation, limited blast radius
  Cons: longer rollout, requires good observability

ROLLING DEPLOYMENT
════════════════════════════════════════════════════════
  Kubernetes: maxSurge=1, maxUnavailable=0

  Pod 1 (v1) → Pod 1 (v2) ✓
  Pod 2 (v1) → Pod 2 (v2) ✓
  Pod 3 (v1) → Pod 3 (v2) ✓  ... one at a time

  Pros: no extra infrastructure, built into k8s
  Cons: both versions run simultaneously (API compatibility required)
        rollback requires re-rolling backwards

FEATURE FLAGS (orthogonal to deployment strategy)
════════════════════════════════════════════════════════
  Code ships to production but feature is OFF:

  if (featureFlags.isEnabled('new-checkout', userId)) {
    return newCheckout();
  }
  return legacyCheckout();

  Separate DEPLOY (code to prod) from RELEASE (users see it)

  Kill switch: toggle off instantly without re-deploy
  A/B testing: different % see different variants
  Internal testing: enable only for employees (userId list)

COMPARISON:
════════════════════════════════════════════════════════
  Strategy      │ Rollback    │ Infra Cost │ Traffic Control
  ──────────────│─────────────│────────────│────────────────
  Blue-Green    │ Instant     │ 2×         │ Hard switch
  Canary        │ Redirect %  │ Small +    │ Gradual (%)
  Rolling       │ Re-roll     │ +1 pod     │ None
  Feature Flag  │ Toggle      │ 0          │ Per-user rules

Technical Implementation

Kubernetes Rolling Deployment

# deployment.yaml — rolling update strategy
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-service
spec:
  replicas: 6
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1         # allow 1 extra pod during update
      maxUnavailable: 0   # never go below desired replicas
  template:
    spec:
      containers:
        - name: api
          image: api-service:v2.0
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 10
            periodSeconds: 5
          # Kubernetes waits for readiness before terminating old pod

Kubernetes Blue-Green via Service Switch

# Two deployments, one service — switch selector to flip
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-blue
  labels:
    version: blue
spec:
  template:
    metadata:
      labels:
        app: api
        slot: blue
    spec:
      containers:
        - name: api
          image: api-service:v1.0

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-green
  labels:
    version: green
spec:
  template:
    metadata:
      labels:
        app: api
        slot: green
    spec:
      containers:
        - name: api
          image: api-service:v2.0

---
apiVersion: v1
kind: Service
metadata:
  name: api-service
spec:
  selector:
    app: api
    slot: green   # ← change this to 'blue' to switch traffic

Feature Flags with LaunchDarkly / Unleash

import { LDClient, LDUser } from '@launchdarkly/node-server-sdk';

const ldClient = await LDClient.init(process.env.LD_SDK_KEY!);

export async function getCheckoutExperience(userId: string): Promise<'new' | 'legacy'> {
  const user: LDUser = {
    key: userId,
    custom: {
      plan: await getUserPlan(userId),
      region: await getUserRegion(userId),
    }
  };

  const isNew = await ldClient.variation('new-checkout-v2', user, false);
  return isNew ? 'new' : 'legacy';
}

// Targeting rules in LaunchDarkly dashboard:
// - 100% of 'internal' users → new checkout (internal testing)
// - 10% of 'premium' users → new checkout (canary)
// - 0% everyone else → legacy

Canary with AWS CodeDeploy / ECS

// AWS CDK: ECS Blue/Green + canary traffic shifting
import { EcsDeploymentGroup } from 'aws-cdk-lib/aws-codedeploy';
import { DeploymentConfig } from 'aws-cdk-lib/aws-codedeploy';

const deploymentGroup = new EcsDeploymentGroup(this, 'ApiDeployment', {
  service: ecsService,
  blueGreenDeploymentConfig: {
    blueTargetGroup: blueTargetGroup,
    greenTargetGroup: greenTargetGroup,
    listener: albListener,
  },
  deploymentConfig: DeploymentConfig.CANARY_10_PERCENT_5_MINUTES,
  // 10% to green for 5 min, then 100% if no alarms fire
  autoRollback: {
    failedDeployment: true,
    deploymentInAlarm: true,  // rolls back if CloudWatch alarms fire
  },
  alarms: [latencyAlarm, errorRateAlarm],
});

Interview Preparation

Q: What is the difference between a rolling update and a canary deployment?

Rolling update is a pure infrastructure concern — Kubernetes updates pods one by one, but all users may hit either old or new pods with no traffic control. Canary is traffic-weighted — you explicitly send X% of users to the new version, monitor, then ramp up. Canary requires load balancer or service mesh support (Nginx, Istio, or ALB weighted target groups). Rolling is simpler; canary gives you real-world validation with limited blast radius.

Q: Why does blue-green require backward-compatible database migrations?

During a blue-green switch, you're running v2 code against the same database. If v2 added a NOT NULL column, v1 (still in blue) will fail writing rows because it doesn't know about the new column. The safe pattern: expand-contract (add nullable column in one deploy, backfill, add NOT NULL constraint in a later deploy after v1 is fully retired). Never add NOT NULL without default in a blue-green deploy.

Q: When would you use feature flags instead of canary?

Canary targets a random % of traffic by routing. Feature flags target specific users, cohorts, or conditions — "enable for the internal pilot group," "enable for paying customers in the US," "enable for 5% of iOS users only." Feature flags also separate deployment from release: code ships to prod dark (flag off) and releases weeks later. This is essential for trunk-based development where every commit goes to main and production daily.

Q: How do you handle a bad canary? What's the rollback?

Automated: set up CloudWatch alarms or Datadog monitors on error rate, p99 latency, and business metrics (add-to-cart rate). When alarm fires during canary window, CodeDeploy / Spinnaker automatically shifts 100% back to stable. Manual: shift traffic weight to 0% on canary target group. The key is defining "bad" with specific alarm thresholds before deploying — not reacting manually after user complaints.

Learning Resources