Overview
At scale, you cannot use a single auto-increment database column for IDs — it creates a single-point bottleneck and leaks business metrics (order #12345 tells competitors your volume). Requirements for production ID generation:
- Globally unique (no collisions across services/DBs)
- Roughly sorted by time (enables range queries, pagination)
- 64-bit integers (efficient storage, DB indexing)
- High throughput (millions per second)
- No single point of failure
Architecture
TWITTER SNOWFLAKE (64-bit ID)
════════════════════════════════════════════════════════
Bit layout:
┌──────────────────────┬──────────────┬─────────────────┐
│ 41 bits │ 10 bits │ 12 bits │
│ Timestamp (ms) │ Machine ID │ Sequence # │
│ since custom epoch │ (datacenter │ (per machine, │
│ │ + worker) │ per ms) │
└──────────────────────┴──────────────┴─────────────────┘
41 bits timestamp = 2^41 ms = 69 years from epoch
10 bits machine ID = 1024 machines
12 bits sequence = 4096 IDs per millisecond per machine
Total throughput = 1024 × 4096 = 4M IDs/ms = 4B IDs/sec
Example:
timestamp = 1714604400000 ms (from 2010-01-01 epoch)
machine_id = 42
sequence = 1
→ ID = (timestamp << 22) | (machine_id << 12) | sequence
= 7241839723069513729
Why sortable? Higher timestamp → higher ID value.
Sort by ID ≈ sort by creation time.
COMPARISON OF ID STRATEGIES:
════════════════════════════════════════════════════════
Strategy │ Size │ Sortable │ Decentralized │ Readable
──────────────│───────│──────────│───────────────│─────────
Auto-increment│ 4/8B │ ✓ │ ✗ (DB lock) │ ✓
UUID v4 │ 128b │ ✗ (random)│ ✓ │ ✗
UUID v7 │ 128b │ ✓ (time) │ ✓ │ ✗
Snowflake │ 64b │ ✓ │ ✓ │ ✓ (numeric)
ULID │ 128b │ ✓ │ ✓ │ ✓ (Base32)
NanoID │ var │ ✗ │ ✓ │ ✓ (URL-safe)
CLOCK SKEW PROBLEM:
════════════════════════════════════════════════════════
Machine A: clock at T=100ms, generates ID
Machine A: NTP sync adjusts clock back to T=95ms
Machine A: generates ID at T=95ms — LOWER than previous!
→ Duplicate or out-of-order IDs
Solutions:
1. Refuse to generate IDs if clock goes backward (wait until caught up)
2. Use logical clocks (not wall clock)
3. Use monotonic clock source
Technical Implementation
Snowflake ID Generator (TypeScript)
export class SnowflakeIdGenerator {
private readonly EPOCH = 1577836800000n; // 2020-01-01 as bigint
private readonly MACHINE_ID_BITS = 10n;
private readonly SEQUENCE_BITS = 12n;
private readonly MAX_SEQUENCE = (1n << this.SEQUENCE_BITS) - 1n; // 4095
private readonly MAX_MACHINE_ID = (1n << this.MACHINE_ID_BITS) - 1n; // 1023
private sequence = 0n;
private lastTimestamp = -1n;
private readonly machineId: bigint;
constructor(machineId: number) {
if (machineId > 1023 || machineId < 0) throw new Error('Invalid machine ID');
this.machineId = BigInt(machineId);
}
generate(): bigint {
let timestamp = BigInt(Date.now()) - this.EPOCH;
if (timestamp < this.lastTimestamp) {
throw new Error(`Clock moved backwards — refusing to generate ID`);
}
if (timestamp === this.lastTimestamp) {
this.sequence = (this.sequence + 1n) & this.MAX_SEQUENCE;
if (this.sequence === 0n) {
// Sequence exhausted in this ms — wait for next ms
while (timestamp <= this.lastTimestamp) {
timestamp = BigInt(Date.now()) - this.EPOCH;
}
}
} else {
this.sequence = 0n;
}
this.lastTimestamp = timestamp;
return (timestamp << (this.MACHINE_ID_BITS + this.SEQUENCE_BITS))
| (this.machineId << this.SEQUENCE_BITS)
| this.sequence;
}
// Extract timestamp from ID (for debugging)
extractTimestamp(id: bigint): Date {
const timestamp = id >> (this.MACHINE_ID_BITS + this.SEQUENCE_BITS);
return new Date(Number(timestamp + this.EPOCH));
}
}
// Usage
const generator = new SnowflakeIdGenerator(parseInt(process.env.MACHINE_ID!));
const id = generator.generate(); // 7241839723069513729n
Snowflake in Java (Twitter's original approach)
public class SnowflakeIdGenerator {
private static final long EPOCH = 1577836800000L; // 2020-01-01
private static final long MACHINE_ID_BITS = 10L;
private static final long SEQUENCE_BITS = 12L;
private static final long MAX_MACHINE_ID = ~(-1L << MACHINE_ID_BITS); // 1023
private static final long MAX_SEQUENCE = ~(-1L << SEQUENCE_BITS); // 4095
private final long machineId;
private long sequence = 0L;
private long lastTimestamp = -1L;
public SnowflakeIdGenerator(long machineId) {
if (machineId < 0 || machineId > MAX_MACHINE_ID)
throw new IllegalArgumentException("Invalid machine ID");
this.machineId = machineId;
}
public synchronized long generate() {
long timestamp = System.currentTimeMillis() - EPOCH;
if (timestamp < lastTimestamp) {
throw new RuntimeException("Clock moved backwards");
}
if (timestamp == lastTimestamp) {
sequence = (sequence + 1) & MAX_SEQUENCE;
if (sequence == 0) {
while (timestamp <= lastTimestamp) {
timestamp = System.currentTimeMillis() - EPOCH;
}
}
} else {
sequence = 0;
}
lastTimestamp = timestamp;
return (timestamp << (MACHINE_ID_BITS + SEQUENCE_BITS))
| (machineId << SEQUENCE_BITS)
| sequence;
}
}
UUID v7 (Modern Alternative — Sortable UUID)
// UUID v7: time-ordered, RFC 9562
// Format: 48-bit timestamp | 4-bit version | 12-bit random | 2-bit variant | 62-bit random
import { v7 as uuidv7 } from 'uuid';
const id = uuidv7(); // "0190a5d8-f0a3-7000-b123-456789abcdef"
// First 12 hex chars = timestamp in milliseconds
// Sortable AND human-readable
Interview Preparation
Q: Why not use UUID v4 as IDs?
UUID v4 is random — no time ordering. Using random UUIDs as primary keys in B-tree indexes (MySQL, PostgreSQL) causes page splits on every insert because the new UUID randomly falls somewhere in the existing index, not at the end. Snowflake and UUID v7 append to the end of the index (time-ordered) → sequential writes → dramatically better insert performance.
Q: How does Snowflake handle multiple machines without coordination?
Each machine has a unique machine ID (0–1023), typically assigned via ZooKeeper, a Kubernetes pod index, or a DB sequence at startup. IDs from different machines have different machine ID bits, so they never collide even if generated at the exact same millisecond.
Q: What happens when two requests arrive in the same millisecond?
The 12-bit sequence counter handles up to 4096 IDs per millisecond per machine. When the sequence exhausts in the current millisecond, the generator spin-waits until the next millisecond. In practice, 4096/ms per machine is almost never the bottleneck.
Q: How do you extract the creation timestamp from a Snowflake ID?
Right-shift the ID by 22 bits (10 + 12) to get the timestamp offset, then add your epoch. Useful for debugging and for building time-range queries on ID (e.g., find all IDs from the last hour without a timestamp column).
Learning Resources
- Twitter Snowflake GitHub — original implementation
- UUID v7 RFC 9562 — new sortable UUID standard
- ULID Spec — Universally Unique Lexicographically Sortable Identifier
- System Design Interview Vol. 1 — Alex Xu, Chapter 7