Metis

Field note / architecture

publishedupdated 2026-05-01#system-design#url-shortener#redis#cassandra#interview

System Design: URL Shortener

Design a URL shortening service like bit.ly — hash generation, redirect performance, analytics, and scaling to 100M URLs.

Overview

Design a URL shortener that:

  • Creates a short 7-character alias for any long URL
  • Redirects short.ly/abc1234 to the original URL in < 10ms p99
  • Handles 100M URLs, 10B redirects/month (~3,850 redirects/second)
  • Tracks click analytics (count, geo, device)
  • Custom aliases (user-defined short codes)

Read/write ratio: ~1000:1 (many redirects per creation). Optimize hard for reads.

Architecture

 ┌────────────────────────────────────────────────────────────┐
 │                   Clients                                   │
 └────────────┬─────────────────────────┬───────────────────┘
              │ POST /shorten           │ GET /abc1234
              ▼                         ▼
 ┌────────────────────────────────────────────────────────────┐
 │                    CDN (Cloudfront)                         │
 │           (cache 301/302 redirects at edge)                 │
 └────────────────────────────────────────────────────────────┘
              │                         │
              ▼                         ▼
 ┌────────────────────────────────────────────────────────────┐
 │              API Gateway + Load Balancer                    │
 └─────────┬──────────────────────┬────────────────────────┘
           │                      │
    ┌──────▼──────┐       ┌───────▼──────┐
    │  URL Write  │       │  URL Read    │
    │  Service    │       │  Service     │
    │  (Create)   │       │  (Redirect)  │
    └──────┬──────┘       └───────┬──────┘
           │                      │
           │               ┌──────▼──────┐
           │               │   Redis     │
           │               │   Cache     │
           │               │  (shortCode │
           │               │  → longUrl) │
           │               │  TTL: 24hr  │
           │               └──────┬──────┘
           │                      │ cache miss
           │                      ▼
           │        ┌─────────────────────────┐
           └───────▶│   Cassandra / DynamoDB  │
                    │   Primary store          │
                    │                          │
                    │  PK: short_code (text)   │
                    │  long_url                │
                    │  user_id                 │
                    │  created_at              │
                    │  expires_at              │
                    │  custom: bool            │
                    └──────────────────────────┘
                              │
                    ┌─────────▼────────────────┐
                    │   Analytics Pipeline      │
                    │                          │
                    │  Click events →          │
                    │  Kafka →                 │
                    │  Stream processor →      │
                    │  ClickHouse (OLAP)       │
                    └──────────────────────────┘

Technical Implementation

Short Code Generation — Two Approaches

Approach 1: Counter + Base62 (Recommended)

Global counter (auto-increment) → encode in base62

base62 alphabet: 0-9, a-z, A-Z (62 chars)
7 chars → 62^7 = 3.5 trillion unique codes

Counter 1000 → base62 → "g8" (short)
Counter 1000000 → base62 → "4c92"

Pros: predictable length, no collisions
Cons: sequential codes reveal scale (enumerable)

Approach 2: Hash (MD5/SHA256) + truncate

MD5("https://example.com/products/abc-123")
→ "5d41402abc4b2a76b9719d911017c592"
→ take first 7 chars → "5d41402"
→ base62 encode → "abc1234"

Pros: not guessable
Cons: collisions possible (check DB, retry with different offset)

// src/services/url-shortener.service.ts
import { createHash } from 'crypto';
import { DynamoDBDocumentClient, PutCommand, GetCommand } from '@aws-sdk/lib-dynamodb';

const BASE62 = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';

export class UrlShortenerService {
  async shorten(longUrl: string, customCode?: string, userId?: string): Promise<string> {
    const shortCode = customCode ?? (await this.generateUniqueCode(longUrl));

    await this.db.send(new PutCommand({
      TableName: 'urls',
      Item: {
        shortCode,
        longUrl,
        userId: userId ?? 'anonymous',
        createdAt: new Date().toISOString(),
        expiresAt: null,
      },
      ConditionExpression: 'attribute_not_exists(shortCode)',  // prevent overwrite
    }));

    return shortCode;
  }

  async resolve(shortCode: string): Promise<string | null> {
    // 1. Check Redis cache first
    const cached = await this.redis.get(`url:${shortCode}`);
    if (cached) return cached;

    // 2. Fall back to DynamoDB
    const result = await this.db.send(new GetCommand({
      TableName: 'urls',
      Key: { shortCode },
    }));

    if (!result.Item) return null;

    // 3. Cache for 24 hours
    await this.redis.setex(`url:${shortCode}`, 86400, result.Item.longUrl);
    return result.Item.longUrl;
  }

  private async generateUniqueCode(longUrl: string): Promise<string> {
    const hash = createHash('md5').update(longUrl).digest('hex');

    for (let offset = 0; offset <= hash.length - 7; offset++) {
      const candidate = this.base62Encode(parseInt(hash.slice(offset, offset + 7), 16));
      const exists = await this.codeExists(candidate);
      if (!exists) return candidate;
    }
    // Fallback: random code
    return this.base62Encode(Math.floor(Math.random() * 62 ** 7));
  }

  private base62Encode(num: number): string {
    let result = '';
    while (num > 0) {
      result = BASE62[num % 62] + result;
      num = Math.floor(num / 62);
    }
    return result.padStart(7, '0');
  }
}

Redirect Handler — Optimize for Speed

// app/api/[shortCode]/route.ts — Next.js handler
export async function GET(req: Request, { params }: { params: { shortCode: string } }) {
  const longUrl = await urlService.resolve(params.shortCode);

  if (!longUrl) {
    return Response.redirect('https://short.ly/not-found', 302);
  }

  // Track analytics fire-and-forget (never block the redirect)
  trackClick(params.shortCode, req).catch(console.error);

  // 301 = permanent (browsers cache, reduces load — but can't change destination)
  // 302 = temporary (no client cache — use for analytics-tracked short URLs)
  return Response.redirect(longUrl, 302);
}

async function trackClick(shortCode: string, req: Request): Promise<void> {
  await kafka.emit('url.clicks', {
    shortCode,
    timestamp: Date.now(),
    userAgent: req.headers.get('user-agent'),
    referer: req.headers.get('referer'),
    // IP geolookup done downstream
  });
}

Interview Preparation

Q: 301 vs 302 redirect — which do you use?

302 (temporary) for short URL services. 301 (permanent) tells browsers to cache the redirect permanently — they stop hitting your server, you lose analytics. 302 ensures every click goes through your server, enabling click counting and A/B experiments.

Q: How do you scale to 100M URLs?

Cassandra or DynamoDB — both partition by short_code, scale horizontally, handle millions of writes/sec. The read path is almost entirely from Redis cache (10B/month = 3850 RPS with ~99% cache hit rate → real DB load is ~40 RPS). Redis cluster with read replicas handles this trivially.

Q: How do you prevent enumeration attacks (guessing short codes)?

Use hash-based codes (not sequential counters). Add a random salt to the hash input. Implement rate limiting on the redirect endpoint per IP. Monitor for sequential access patterns. Add CAPTCHA for suspicious IPs.

Q: Custom aliases — how do you prevent squatting on reserved words?

Block list of reserved slugs (api, admin, login, about, health). Min length 4 characters for custom aliases. Rate limit custom alias creation per user account. Periodic audit for offensive/trademark terms.

Q: How does the analytics pipeline scale?

Click events emit to Kafka. Stream processors (Kafka Streams / Flink) aggregate counts in 1-minute windows. Write aggregated counts to ClickHouse (columnar OLAP DB, excellent for time-series queries). API queries ClickHouse for per-URL analytics dashboards — can handle billions of rows with sub-second queries.

Learning Resources