ποΈ Cab Booking Service (Uber) β High-Level Design¶
Target Level: Staff/Principal Engineer | Focus: Geo-spatial indexing, event-driven architecture, real-time stream processing, distributed systems
1. SYSTEM OVERVIEW¶
Purpose: On-demand cab booking connecting riders with drivers in real-time with dynamic pricing.
Scale: 50 cities, 1M rides/day, 100K active drivers peak, 500 concurrent rides/minute per city
Users: Riders, Drivers, Operations team, Analytics
Use Cases: Request ride, Driver matching, Real-time tracking, Surge pricing, Payment & billing, Driver dispatch optimization
Constraints: <1s driver matching latency, 99.99% uptime, GPS accuracy <10m, sub-100ms ETA calculation
2. HIGH-LEVEL ARCHITECTURE¶
ββββββββββββββββ ββββββββββββββββ
β Rider App β β Driver App β
β (React β β (React β
β Native) β β Native) β
ββββββββ¬ββββββββ ββββββββ¬ββββββββ
β β
ββββββββββ¬ββββββββββββ
β WebSocket / HTTPS
ββββββββββΌββββββββββββ
β API Gateway β
β (Kong, SSL term) β
ββββββββββ¬ββββββββββββ
β
βββββββββββββΌββββββββββββββββ
β β β
βββββΌββββ ββββββΌβββββ ββββββββΌβββββ
β Rider β β Driver β β Trip β
β Svc β β Svc β β Svc β
β (Go) β β (Go) β β (Python) β
βββββ¬ββββ ββββββ¬βββββ ββββββββ¬βββββ
β β β
βββββββββββββΌβββββββββββββββ
β
βββββββββΌββββββββ
β Kafka Bus β
β(GPS events, β
β trip updates, β
β zone analytics)β
βββββββββ¬ββββββββ
β
βββββββββββββΌββββββββββββββββ
β β β
βββββΌββββ ββββββΌβββββ ββββββββΌβββββ
β Redis β β Post- β β Cassandra β
β Geo β β greSQL β β (Location β
β β β + PostGISβ β history) β
βββββββββ βββββββββββ βββββββββββββ
Figure: apps, services, Kafka bus and stores.
flowchart TB
R["Rider app"] --> G["API gateway"]
D["Driver app"] --> G
G --> RS["Rider svc (Go)"]
G --> DS["Driver svc (Go)"]
G --> TS["Trip svc (Python)"]
RS --> K["Kafka bus"]
DS --> K
TS --> K
K --> RG[("Redis GEO")]
K --> PG[("PostgreSQL + PostGIS")]
K --> CA[("Cassandra: location history")]
Data Flow for a Ride Request¶
1. Rider opens app β sends pickup location
2. API Gateway routes to Rider Service
3. Rider Service calls Driver Service with pickup coords
4. Driver Service queries Redis GEO: GEOSEARCH pickup BYRADIUS 2 km (expand to 5, 10 km on miss)
5. Redis returns nearby drivers (sorted by distance); the matcher ranks them
6. Matcher claims the best one atomically (SET NX PX / conditional UPDATE); on failure, the next
7. Trip Service creates the trip in REQUESTED (+ outbox row) and sends the offer to the driver
8. Driver accepts β ACCEPTED; declines or times out β claim released, re-match excluding them
9. Trip events relayed from the outbox to Kafka 'trip.events'; Payment pre-authorises asynchronously
Figure: ride request flow, from pickup to driver acceptance.
sequenceDiagram
participant R as Rider
participant RS as Rider svc
participant DS as Driver svc
participant G as Redis GEO
participant T as Trip svc
participant D as Driver
R->>RS: Pickup location
RS->>DS: Find driver
DS->>G: GEOSEARCH radius (expand on miss)
G-->>DS: Nearby drivers
DS->>DS: Rank, claim best atomically
DS->>T: Create trip REQUESTED
T->>D: Offer
D-->>T: Accept (ACCEPTED) or decline (re-match)
π¬ Animated Sequence Diagram¶
π¬ Animated Cab Booking Sequence β Rider Request β Geo Matching β Driver Dispatch β Trip Creation β Kafka Event Pipeline. Click βΆ to play/pause. Created with Remotion.
3. KAFKA EVENT-DRIVEN ARCHITECTURE¶
Kafka Topics¶
| Topic | Partitions | Retention | Producers | Consumers | Message Schema |
|---|---|---|---|---|---|
gps.raw.updates |
5 per city | 7 days | Driver WS Gateway | GPS Stream Processor | {driver_id, lat, lng, speed, heading, timestamp} |
gps.enriched.locations |
5 per city | 3 days | GPS Stream Processor | Zone Analytics, ETL | {driver_id, location, zone_id, speed, heading, timestamp} |
gps.zone.driver_counts |
3 per city | 1 day | Zone Analytics | Surge Engine, Dashboard | {zone_id, driver_count, surge_multiplier, timestamp} |
trip.events |
5 | 14 days | Trip Service | Payment, Notification, Analytics | {trip_id, status, rider_id, driver_id, fare, ...} |
gps.dlq |
1 | 30 days | Failed messages | Dead-letter handler | {error, original_message} |
GPS Location Update Pipeline¶
βββββββββββββββ ββββββββββββββ βββββββββββββββ ββββββββββββββββ
β Driver App ββββββΆβ WebSocket ββββββΆβ Kafka ββββββΆβ GPS Stream β
β (GPS 3s) β β Gateway β β Raw Updates β β Processor β
βββββββββββββββ ββββββββββββββ ββββββββ¬βββββββ ββββββββ¬ββββββββ
β β
β βββββββββββΌβββββββββ
β β GeoIndex Update β
β β (Redis GEOADD) β
β βββββββββββ¬βββββββββ
β β
β βββββββββββΌβββββββββ
β β Zone Lookup β
β β (H3, in-process)β
β βββββββββββ¬βββββββββ
β β
β βββββββββββΌβββββββββ
β β Kafka: Enriched β
βββββββββββ Locations Topic β
βββββββββββββββββββ
Staff-level considerations:
- Partition key: driver_id β ensures same driver's events go to same partition for ordered processing
- Delivery semantics: at-least-once. Commit offsets after the Redis write; the consumer drops any ping whose timestamp is not newer than the stored one, so replays are harmless. (An idempotent producer only removes duplicate appends from producer retries; it does not make the Redis write exactly-once.)
- Dead letter queue: Messages that fail enrichment (e.g., malformed GPS) go to gps.dlq
- Backpressure: If Redis is slow, pause partitions (Kafka buffers durably). After a long lag, skip to the newest pings per driver: a stale position is worth less than a fresh one
- Scaling: Partition by driver_id, add partitions/consumers per region. Isolate blast radius per region (cluster per region), not per city
Figure: GPS ingestion pipeline, partitioned by driver_id.
flowchart LR
A["Driver app (GPS every 3 s)"] --> W[WebSocket gateway]
W --> K1["Kafka: raw updates"]
K1 --> P[GPS stream processor]
P --> G["Redis GEOADD"]
P --> Z["Zone lookup (H3)"]
Z --> K2["Kafka: enriched locations"]
P -. malformed .-> DLQ[gps.dlq]
4. GEORADIUS DRIVER MATCHING¶
Redis GEO Implementation¶
# Production: Redis GEO (GEOSEARCH, Redis >= 6.2; GEORADIUS is deprecated)
def find_nearest_drivers(city, cab_type, pickup_lat, pickup_lng, radius_km=2, limit=10):
return redis.geosearch(
f"drivers:available:{city}:{cab_type}",
longitude=pickup_lng, latitude=pickup_lat, # Redis takes (lng, lat)
radius=radius_km, unit="km",
withdist=True, sort="ASC", count=limit,
)
def claim(driver_id, trip_id):
# The only authoritative step: atomic, self-expiring if we crash.
return redis.set(f"driver:{driver_id}:assignment", trip_id, nx=True, px=15000)
Progressive Radius Expansion¶
Pickup Location
β
βββ Query: GEORADIUS pickup 2km
β βββ Found 3 drivers β pick closest
β
βββ Query: GEORADIUS pickup 2km β No results
β βββ Query: GEORADIUS pickup 5km
β βββ Found 1 driver β dispatch
β
βββ Query: GEORADIUS pickup 2km β No results
β βββ Query: GEORADIUS pickup 5km β No results
β βββ Query: GEORADIUS pickup 10km
β βββ No results β "No cabs available"
Staff-level considerations:
- Query cost: GEOSEARCH is O(N + log M): N = points in the 9 bounding cells, M = points in the result shape. Dense downtown cells make N large; keep radii small there and expand only on miss
- Neighbour cells: a geohash prefix search must include the 8 neighbouring cells or drivers just across an edge are missed (Redis does this internally)
- Read-replicas: replicas lag the primary (async replication), so a replica may still list a just-booked driver. Fine, because the claim below is the authority
- Race condition: the search result is stale the moment it returns. Reserve with an atomic claim (SET driver:{id}:assignment {trip} NX PX 15000 or UPDATE drivers SET status='BOOKED' WHERE id=? AND status='AVAILABLE'); on failure try the next candidate
- Key layout: shard by city and cab type β drivers:available:{city}:mini. One GEO key lives on one Redis shard, so this is also how you spread load; remove a driver from the key when they are booked
Figure: progressive radius expansion for driver search.
flowchart TD
A[Pickup] --> B{"Drivers within 2 km?"}
B -- Yes --> Z[Pick closest, claim atomically]
B -- No --> C{"Within 5 km?"}
C -- Yes --> Z
C -- No --> D{"Within 10 km?"}
D -- Yes --> Z
D -- No --> N["No cabs available"]
5. ZONE MANAGEMENT & SURGE PRICING¶
Hexagonal Grid Partitioning¶
City partitioned into ~500m hexagonal zones:
___
___/ Z \___
/ Z \___/ Z \
\___/ Z \___/
/ Z \___/ Z \
\___/ \___/
Benefits over square grids:
- All neighbors are equidistant
- Better circular radius approximation
- Minimizes zone-transition edge effects
Zone Analytics Pipeline¶
Every 30 seconds (tumbling window):
βββββββββββββββββββ
β Zone Analytics β
β Aggregator β
β β
β For each zone: β
β drivers = β
β COUNT(DISTINCT driver_id) β
β WHERE zone_id = Z AND β
β status = AVAILABLE β
β β
β requests = β
β COUNT(ride_requests) β
β WHERE zone_id = Z β
β IN THIS 30 s WINDOW β
β β
β surge = f(drivers, requests) β
β β
β Publish to β
β Kafka topic β
βββββββββββββββββββ
Surge Multiplier Calculation¶
def compute_surge(demand: int, supply: int) -> Decimal:
if demand == 0:
return Decimal("1.0") # no demand, no surge, even with zero drivers
if supply == 0:
return Decimal("2.5") # cap
ratio = demand / supply
if ratio > 3.0: return Decimal("2.0")
if ratio > 2.0: return Decimal("1.5")
if ratio > 1.5: return Decimal("1.25")
return Decimal("1.0")
# Production adds: smoothing across neighbouring cells, hysteresis/decay so the
# multiplier doesn't flicker, and the multiplier is locked into the quote.
6. DATA MODEL¶
PostgreSQL + PostGIS Tables¶
See DB_SCHEMA.md for the complete DDL.
Key tables:
- riders β user accounts, ratings, payment methods (JSONB)
- drivers β license, cab type, availability status, GEOGRAPHY(Point) for geo-radius queries
- trips β full trip state machine, fare breakdown, GEOGRAPHY pickup/dropoff
- zones β zone (H3 cell) definitions with GIST spatial index
- driver_location_history β partitioned by month for query performance
- payments β with idempotency key for exactly-once processing
- surge_pricing_log β time-series of zone supply/demand
Redis Keys¶
drivers:available:{city}:{type} β GEO (sorted set of available drivers)
driver:{id}:assignment β STRING trip_id, SET NX PX 15000 (the claim)
driver:{id}:status β STRING (AVAILABLE/BOOKED/ON_TRIP/OFFLINE)
trip:{id}:state β HASH (current trip state machine)
zone:{id}:surge β STRING (current surge multiplier)
zone:{id}:drivers β ZSET driver_id scored by last-seen ts (ZCARD = supply)
7. STAFF-LEVEL INTERVIEW QUESTIONS & ANSWERS¶
Q1: "How would you design the GPS location ingestion pipeline to handle 100K drivers updating every 3 seconds?"¶
Expected Answer:
Scale calculation: 100K drivers Γ 1 update/3s = ~33K writes/second. This is significant but manageable with proper partitioning.
Architecture:
Driver App ββ(WebSocket, binary protobuf)βββΆ WS Gateway (Nginx/HAProxy)
β
βββ Kafka producer (async, batching)
β βββ Topic: gps.raw.updates (5 partitions per city)
β
βββ Stream processor β Redis GEO (every update, ts-guarded)
β βββ GEOADD drivers:available:{city}:{type} <lng> <lat> <driver_id>
β
βββ Batch write to Cassandra (30s cadence)
βββ driver_location_history (TTL: 90 days)
Key decisions:
1. WebSocket with binary protocol (protobuf) β 60% less bandwidth than JSON
2. Adaptive polling β 3s when moving, 30s when stationary (detect via accelerometer)
3. Dead reckoning β if GPS drops, estimate position from last known speed/direction
4. Kalman filter β smooth GPS noise at the app level before sending
5. Rebuildable index β Redis GEO is the source of truth for current position only; every driver re-pings within ~3 s, so a lost index heals in one interval. Cassandra holds history
6. Kafka partitioning β driver_id as partition key ensures ordered processing per driver
Failure modes:
- Redis GEO shard down: fail over to its replica (seconds); the index refills from the next round of pings. PostGIS ST_DWithin on the last persisted positions is a degraded fallback
- Kafka broker failure: Rebalance partitions across remaining brokers; drivers re-publish on reconnect
- GPS data corruption: Schema validation at Kafka producer; DLQ for bad messages
Q2: "How would you implement zone-based surge pricing at city scale?"¶
Expected Answer:
Geo-fencing architecture:
1. Partition the city into H3 resolution-8 cells (β0.74 kmΒ² each; a 10 km-radius city β 314 kmΒ² β 425 cells)
2. Maintain zone state in Redis:
ZADD zone:{id}:drivers <ts> <driver_id> (on each ping; ZREM from the old zone when the cell changes)
ZREMRANGEBYSCORE zone:{id}:drivers -inf <now-30s>; ZCARD β supply
HINCRBY zone:{id}:stats requests 1 (when ride requested in zone)
3. Run aggregation worker every 30 seconds:
For each zone:
supply = ZCARD zone:{id}:drivers (after trimming stale entries)
demand = MULTI; HGET requests; HSET requests 0; EXEC (read-and-reset = tumbling window)
surge = f(demand, supply)
HSET zone:{id}:stats surge {surge}
4. Publish zone stats to Kafka topic 'gps.zone.driver_counts'
5. Rider app reads surge via GET /api/zones/{zone_id}/surge
(cached for 5 seconds at API Gateway)
Sticky surge decay: Surge doesn't drop instantly. If ratio drops below threshold, surge decays linearly over 5 minutes. This prevents drivers from gaming the system by waiting for surge to hit then immediately leaving.
Leading indicators: - Airport flight arrivals β pre-load surge prediction - Concert/sports event schedules β pre-load surge zones - Weather data (rain = higher demand) β adjust base multiplier
Q3: "Design a system to handle driver location updates during GPS signal loss in tunnels."¶
Expected Answer:
Dead Reckoning System:
βββββββββββββββββββββββββββββββββββββββββββββββ
β Mobile SDK (on-device) β
β β
β GPS available: β
β β Send precise location every 3s β
β β Record: speed, heading, timestamp β
β β
β GPS lost (tunnel, parking garage): β
β β Use last known speed + heading β
β β Estimate position: β
β new_lat = last_lat + (speed * time * cos(heading)) / earth_radius
β new_lng = last_lng + (speed * time * sin(heading)) / earth_radius
β β Tag message: {dead_reckoning: true} β
β β Update every 1s (more frequent since β
β error accumulates faster) β
β β
β GPS restored: β
β β Send actual location with tag β
β β {corrected: true, drift_meters: X} β
β β Server applies correction to estimate β
βββββββββββββββββββββββββββββββββββββββββββββββ
Server-side handling: - Mark dead-reckoned locations with lower confidence score - Don't use dead-reckoned locations for surge/gamification calculations - Widen geo-radius search radius for dead-reckoned drivers - On GPS re-acquisition, calculate drift and update ML models
Q4: "How would you prevent driver fraud in a geo-spatial cab booking system?"¶
Expected Answer:
Fraud vectors and mitigations:
| Fraud Type | Detection | Mitigation |
|---|---|---|
| GPS spoofing (fake location) | Compare GPS with cell tower triangulation + WiFi BSSID fingerprint | Two-factor location verification; flag discrepancies > 100m |
| Route manipulation (taking longer route) | ML model predicts expected route/duration per segment | Auto-adjust fare; flag outliers > 2Ο from expected |
| Fake ride requests (driver self-booking) | Device fingerprinting; payment auth before dispatch | Require payment method for all rides; rate-limit per device |
| Collusion (rider and driver gaming surge) | Check if rider and driver share IP, device, or payment method | Ban both accounts on detection |
| Bait-and-switch (different car than registered) | Photo verification on trip start; AI model compares car photos | Penalize driver rating; suspend after N violations |
GPS spoofing detection algorithm:
def detect_gps_spoofing(gps_location, cell_towers, wifi_bssids):
# Cell tower triangulation
cell_location = trilaterate(cell_towers)
cell_distance = haversine(gps_location, cell_location)
# WiFi fingerprint lookup (pre-mapped)
wifi_location = lookup_wifi_bssid_db(wifi_bssids)
wifi_distance = haversine(gps_location, wifi_location)
# Combined score
if cell_distance > 200 or wifi_distance > 100:
return FRAUD_FLAG # Likely spoofed
# Speed check: impossible speeds = spoofing
speed = calculate_speed(consecutive_updates)
if speed > 250: # km/h
return SPOOF_FLAG
return LEGITIMATE
Q5: "Design an RTA (Real-Time Agreement) system for Uber pool / shared rides."¶
Expected Answer:
Core challenge: Match multiple riders going in the same direction, splitting fare, with minimum time penalty.
Key components: - Route similarity index: Compare pickup/dropoff pairs using road network (not straight-line) distance - Time penalty budget: Max 5 minutes extra per rider per shared trip - Dynamic pricing: Pool rides are 30-50% cheaper than individual - Matching window: 15-30 second batch window for pool requests in same zone
Algorithm:
def match_pool_riders(pending_pool_requests, available_drivers):
# 1. Cluster pending requests by pickup proximity
clusters = dbscan_cluster(
[r.pickup for r in pending_pool_requests],
eps=500, # 500m cluster radius
min_samples=2
)
# 2. For each cluster, compute optimal sequence
matched_pairs = []
for cluster in clusters:
# Compute pairwise route efficiency
for r1, r2 in combinations(cluster, 2):
# Optimized route: pickup1 β pickup2 β dropoff1 β dropoff2
efficiency = compute_route_efficiency(r1, r2)
if efficiency.detour_minutes <= 5: # Within budget
matched_pairs.append((r1, r2, efficiency))
# 3. Assign to nearest driver with seat capacity
return assign_to_drivers(matched_pairs, available_drivers)
Q6: "How would you handle cross-city (inter-city) trips?"¶
For long trips spanning cities, the system needs: - Cross-cluster trip coordinator β manages the trip state across city boundaries - ETA calculation using inter-city driving time (not city-level model) - Pricing that accounts for return trip deadhead (driver won't get a return fare) - Driver incentive β premium payout for inter-city trips to compensate for return deadhead
Architecture:
Rider requests Mumbai β Pune (150km)
β
βββ Mumbai cluster evaluates: available drivers
βββ Inter-city trip coordinator created
βββ Fare: normal rate Γ 1.5 (deadhead adjustment)
βββ Driver notified with deadhead compensation
β
βββ Trip starts in Mumbai cluster
βββ On city boundary β trip state transfers to Pune cluster
βββ Trip completes in Pune cluster β deadhead payout processed
7a. CONSISTENCY, IDEMPOTENCY & FAILURE MODES¶
Where strong consistency is required, and where it isn't:
| Data | Consistency | Why |
|---|---|---|
| Driver assignment (claim) | Linearizable per driver (Redis SET NX on the primary, or a conditional UPDATE) |
Double-booking is the one unacceptable failure |
| Trip status | Linearizable per trip (conditional UPDATE on status/version) |
Cancel-vs-start must have one winner |
| Driver positions | Eventual, last-writer-wins by device timestamp | Stale by β€ 3 s anyway; rebuildable |
| Surge multipliers | Eventual (30 s windows) | Quoted and locked into the trip at request time |
| Payments | Exactly-once effect via idempotency keys | Money |
Idempotency:
- POST /rides takes an Idempotency-Key; trips.idempotency_key is UNIQUE, so a retried request returns the original trip.
- GPS consumer is idempotent by timestamp; trip-event consumers dedupe on (trip_id, version).
- Payment capture uses payments.idempotency_key; retries never double-charge.
Failure modes:
| Failure | Effect | Mitigation |
|---|---|---|
| Matcher dies after claim, before trip insert | Driver stuck BOOKED | Claim TTL (PX 15000) or reaper for claims with no trip |
| Trip committed but event not published | Payment/notifications never hear about it | Transactional outbox + relay |
| Driver app disconnects mid-offer | Rider waits forever | Offer timeout β decline path β re-match |
| Redis primary failover | Last ~1 s of positions/claims may be lost (async replication) | Positions heal on next ping; re-check claims against the trip DB on accept |
| Kafka consumer lag | Positions stale, matches worse | Alert on lag; skip to latest offsets; widen first search radius |
| Hot zone (stadium exit) | Thousands of requests on one geo cell | Batch matching per cell every 1β2 s; per-cell single-writer dispatch |
Capacity (per the scale in Β§1): 100K drivers / 3 s β 33K position writes/s (~3.3 MB/s); a GEO member is ~50β100 bytes, so 100K drivers β 10 MB of index β memory is not the constraint, write rate and per-key hotness are. Peak 500 ride requests/min/city β 8/s/city, each β 1β3 GEOSEARCH calls + 1 claim.
8. TRADE-OFF ANALYSIS¶
| Decision | Choice | Rationale | Alternative |
|---|---|---|---|
| Geo-index | Redis GEO | Sub-ms server time, O(N + log M) per search, built-in geo commands | PostGIS (durable, polygons; 33K updates/s causes MVCC/vacuum churn) |
| Event bus | Kafka | Durable, replayable, ordered per partition | RabbitMQ (simpler, lower throughput) |
| Zone shape | H3 hexagons (res 8) | All 6 neighbours equidistant, O(1) pointβcell, hierarchical | Geohash cells (simpler, but rectangles distort with latitude) |
| Driver matching | Progressive radius | Simple, predictable, easy to debug | ML-based (optimal but opaque) |
| Surge calculation | Fixed thresholds | Transparent, easier to tune | ML pricing (optimal but hard to explain) |
| Location DB | Cassandra | Writes optimized, TTL support, horizontal scaling | TimescaleDB (SQL interface, but more overhead) |
9. COST (Monthly)¶
| Component | Configuration | Cost |
|---|---|---|
| Compute (Go services) | ECS Fargate, 10 tasks Γ 2 vCPU | $5,000 |
| Redis Geo cluster | ElastiCache, r6g.xlarge, cluster mode (3 shards) | $1,500 |
| Kafka + Stream processing | MSK, 5 brokers, m5.large | $3,500 |
| PostgreSQL + PostGIS | RDS, db.r6g.large, Multi-AZ, 500GB | $3,000 |
| Cassandra (location history) | 5 nodes, i3.large | $2,000 |
| WebSocket Gateway | ALB + Nginx, auto-scaling | $1,500 |
| Maps API (Google) | Pay-per-use | $2,000 |
| Monitoring (CloudWatch, Datadog) | Metrics + traces | $1,500 |
| Total | $20,000 |