Skip to content

🦦 Go — Staff-Level Interview Questions & Answers

Interviewer Persona: Principal Software Engineer, 15+ years in distributed systems and infrastructure \ Target Level: Senior / Staff Engineer \ Evaluation Focus: Go runtime internals, CSP concurrency, memory model, production system design \ Current as of: Go 1.27 (released August 2026). Version-specific behaviour is tagged inline, e.g. (Go 1.22+).

Runtime source snippets below are simplified sketches of runtime/*.go, written to show the algorithm. They are not the literal source and will not compile.


Question 1: The GMP Scheduler — How Goroutines Actually Run

Interviewer: "Explain how Go schedules goroutines. What happens when a goroutine blocks on a syscall or channel operation?"

🎯 Expected Answer (Staff Level)

30-second answer: Go multiplexes many goroutines (G) onto a few OS threads (M). To run Go code, a thread must hold a P (processor), and there are GOMAXPROCS Ps, so at most GOMAXPROCS goroutines execute Go code in parallel. Each P has a local run queue. Idle Ps steal work from busy ones. A goroutine blocked on a channel, mutex or network read is parked: it costs no thread, and its M moves on to the next G. A goroutine in a blocking syscall does pin its thread, so the runtime hands that thread's P to another M. Blocked syscalls therefore don't reduce parallelism, but each one still occupies an OS thread.

The three abstractions:

What it is Key contents
G A goroutine Its own stack (starts at 2 KB, grows by copying; since Go 1.19 the starting size adapts to the average stack use seen so far), saved PC/SP, status
M An OS thread g0 (a scheduler stack), signal stack, the current G, the P it holds
P A scheduling context; GOMAXPROCS of them Local run queue (256-slot ring), a runnext slot, an mcache for allocation, timers, GC work buffers
G states (simplified):

            newproc
   _Gidle ──────────► _Grunnable ◄──────────────────────┐
                          │ schedule()                    │ ready() / goready
                          ▼                               │
                      _Grunning ──── gopark (chan, mutex, │
                       │   │   │      netpoll, sleep) ──► _Gwaiting
          entersyscall │   │   └─ preempted / Gosched ──► _Grunnable
                       ▼   │
                   _Gsyscall ── exitsyscall: got a P ──► _Grunning
                               no P: G to global queue ─► _Grunnable
                           │
                      goexit ──► _Gdead (G struct is cached for reuse)

Finding the next goroutine (schedule → findRunnable, simplified):

// Runs on the M's g0 stack whenever the current G blocks, yields, is
// preempted, or exits.
func findRunnable(pp *p) *g {
    // 0. GC: if a mark phase needs a dedicated worker on this P, run it.
    // 1. Fairness: every 61st schedule on this P, take one G from the
    //    GLOBAL queue first, so the global queue cannot be starved.
    if pp.schedtick%61 == 0 && sched.runqsize > 0 {
        if gp := globrunqget(); gp != nil { return gp }
    }
    // 2. Local queue. runnext (a single slot holding the G most recently
    //    readied by this P, e.g. a channel peer) is checked first: this is
    //    what makes producer/consumer ping-pong cheap.
    if gp := runqget(pp); gp != nil { return gp }
    // 3. Global queue (takes a batch, moves some to the local queue).
    if gp := globrunqget(); gp != nil { return gp }
    // 4. Non-blocking netpoll: goroutines whose sockets are now ready.
    if gp := netpoll(0); gp != nil { return gp }
    // 5. Work stealing: up to 4 passes over all Ps in random order,
    //    stealing HALF of a victim's local queue (runnext only on the
    //    last pass). Also runs any due timers on the victim.
    if gp := stealWork(pp); gp != nil { return gp }
    // 6. Nothing to do: release the P, maybe block in netpoll, park the M.
    stopm()
    return nil
}

A limited number of Ms spin (look for work without sleeping) so that newly readied goroutines are picked up without a thread wake-up. The runtime caps spinning Ms at about half the number of busy Ps, so idle threads don't burn CPU.

Blocking syscalls vs network I/O. This is the heart of the question:

Blocking syscall (read on a file, cgo call, getaddrinfo) Network I/O on a socket
What happens entersyscall: the G keeps its M, and the P is marked _Psyscall fd is non-blocking; read returns EAGAIN, the G parks on the fd's pollDesc
Thread cost 1 OS thread per concurrent blocking syscall Zero: the M runs other goroutines
P handoff sysmon retakes the P if the syscall runs longer than one sysmon tick (~20 µs) and there is other work. Calls known to block (entersyscallblock) hand off immediately Not needed
Wake-up exitsyscall: take the old P back if free, else any idle P, else put G on the global queue and park the M The scheduler's netpoll (epoll / kqueue / IOCP) returns ready Gs, which go back on run queues

So 100K goroutines blocked on sockets cost only memory. 10K goroutines blocked in file syscalls or cgo cost about 10K OS threads, and the default thread limit is 10,000 (debug.SetMaxThreads). Past that, the program crashes. This is why disk-heavy Go services cap file-I/O concurrency with a semaphore.

Preemption (Go 1.14+: asynchronous, signal-based):

  • Before 1.14: cooperative only. A goroutine could be preempted only at a function prologue, where the stack-bound check doubles as a preemption check. for { i++ } with no calls could stall GC's stop-the-world forever.
  • Since 1.14: sysmon is a dedicated thread that runs without a P. It wakes every 20 µs to 10 ms, backing off when idle. If a G has run for more than 10 ms, sysmon sets its preempt flag (cooperative path). It also sends SIGURG to its M. The signal handler checks whether the interrupted PC is at an async safe point. If it is, the handler rewrites the context so the thread calls asyncPreempt. That function saves all registers and yields to the scheduler. No function call in the user code is needed.
  • Some code is not async-preemptible: runtime code, nosplit functions, and some assembly. A loop inside those can still delay preemption.
  • The GC uses the same mechanism to stop goroutines for stack scanning and for stop-the-world.

GOMAXPROCS (container-aware since Go 1.25): On Linux the default is the smaller of the CPU count and the cgroup CPU limit (rounded up, minimum 2 when a limit is set). The runtime also re-checks periodically, so a changed limit takes effect. Setting GOMAXPROCS explicitly, or calling runtime.GOMAXPROCS(n), turns both behaviours off. GODEBUG containermaxprocs=0 and updatemaxprocs=0 turn them off individually. Before 1.25, a pod limited to 2 CPUs on a 64-core node ran with GOMAXPROCS=64. The result was CFS throttling and tail-latency spikes, which is why uber-go/automaxprocs existed. CPU requests (shares) are not considered, only limits.

What they probe next: - "Why have P at all? Why not just M:G?" P holds the per-CPU state: run queue, mcache and GC buffers. If an M blocks in a syscall, that state can be handed to another M without being torn down. Before P was added in Go 1.1, a single global run queue and lock was the bottleneck. - "How do you see this?" GODEBUG=schedtrace=1000,scheddetail=1, the execution tracer (go tool trace; see also runtime/trace.FlightRecorder in Go 1.25), and the /sched/latencies:seconds metric in runtime/metrics. - "runtime.LockOSThread?" It pins a G to its M. You need it for thread-local OS state: some C libraries, Linux namespaces, OpenGL. Its cost is that the M cannot run other goroutines while the G is blocked. - "Is scheduling fair?" Mostly FIFO with runnext LIFO-ish, with the time slice enforced by preemption. There are no priorities. If you need priorities, build them yourself with queues.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
GMP understanding Knows G, M, P are separate, why P exists, local/global queues, work stealing
Blocking Distinguishes parked goroutines (no thread) from blocking syscalls (thread per call, P handoff via sysmon)
Preemption Knows async preemption (1.14, SIGURG, safe points) vs the old cooperative prologue check
Containers Knows the Go 1.25 cgroup-aware GOMAXPROCS default and the pre-1.25 throttling problem

GMP model: a thread (M) must hold a P to run Go code, each P has a local run queue, and idle Ps steal from busy ones or take from the global queue.

flowchart TD
    GQ["Global run queue"]
    subgraph P1["P1 (mcache, local queue)"]
        direction TB
        RN1["runnext slot"]
        LQ1["Local queue: G, G, G"]
    end
    subgraph P2["P2 (mcache, local queue)"]
        direction TB
        LQ2["Local queue: G"]
    end
    M1["M1: OS thread"] ---|holds| P1
    M2["M2: OS thread"] ---|holds| P2
    M1 --> R1["Runs current G"]
    M2 --> R2["Runs current G"]
    GQ -.->|"batch pulled"| P1
    LQ1 -.->|"P2 steals half"| LQ2

Order in which a P looks for its next goroutine (findRunnable, simplified).

flowchart TD
    A["Current G blocks, yields, is preempted or exits"] --> B{"Every 61st tick and global queue non-empty?"}
    B -->|yes| G1["Take one G from the global queue"]
    B -->|no| C{"Local queue (runnext first) has a G?"}
    C -->|yes| L["Run it"]
    C -->|no| D{"Global queue has Gs?"}
    D -->|yes| G2["Take a batch, run one"]
    D -->|no| E{"netpoll: ready sockets?"}
    E -->|yes| N["Run ready G"]
    E -->|no| F{"Steal half of another P's queue?"}
    F -->|yes| S["Run stolen G"]
    F -->|no| Z["Release P, park the M"]

Question 2: Channels — CSP, Internal Structure, and Patterns

Interviewer: "Implement a channel from scratch in Go. Then explain fan-in, fan-out, and how you'd build a pipeline with proper cancellation."

🎯 Expected Answer

30-second answer: A channel is a runtime struct (hchan). It holds a mutex, an optional ring buffer, and two FIFO queues of parked goroutines (senders and receivers). A send first hands the value directly to a waiting receiver. If there is none, it goes into the buffer. If the buffer is full, the sender parks. Closing wakes every waiter. Pipelines are chains of stages, and each stage owns and closes its output channel. Every blocking send also selects on ctx.Done(), so cancellation can never strand a goroutine.

Channel internal structure (runtime/chan.go, simplified):

type hchan struct {
    qcount   uint           // items currently in the buffer
    dataqsiz uint           // buffer capacity (0 = unbuffered)
    buf      unsafe.Pointer // ring buffer of dataqsiz elements
    elemsize uint16
    closed   uint32
    timer    *timer         // set for time package channels (Go 1.23+)
    elemtype *_type
    sendx    uint           // ring indexes
    recvx    uint
    recvq    waitq          // parked receivers (FIFO of sudog)
    sendq    waitq          // parked senders
    lock     mutex          // runtime mutex (spins briefly, then futex-sleeps)
}

// Simplified chansend: the order of the checks is the interesting part.
func chansend(c *hchan, ep unsafe.Pointer, block bool) bool {
    if c == nil {                 // nil channel: block forever (checked BEFORE locking)
        if !block { return false }
        gopark(nil, nil, waitReasonChanSendNilChan)
        throw("unreachable")
    }
    lock(&c.lock)
    if c.closed != 0 {
        unlock(&c.lock)
        panic("send on closed channel")
    }
    if sg := c.recvq.dequeue(); sg != nil {
        // A receiver is parked: copy straight onto ITS stack slot and wake it.
        // The buffer is bypassed. This is why unbuffered channels work at all.
        send(c, sg, ep)           // copies, then goready(sg.g)
        unlock(&c.lock)
        return true
    }
    if c.qcount < c.dataqsiz {   // room in the ring buffer
        typedmemmove(c.elemtype, chanbuf(c, c.sendx), ep)
        c.sendx = (c.sendx + 1) % c.dataqsiz
        c.qcount++
        unlock(&c.lock)
        return true
    }
    if !block { unlock(&c.lock); return false } // select with default
    // Park: enqueue a sudog on sendq; gopark releases c.lock atomically.
    mysg := acquireSudog()
    mysg.elem, mysg.g, mysg.c = ep, getg(), c
    c.sendq.enqueue(mysg)
    gopark(chanparkcommit, unsafe.Pointer(&c.lock), waitReasonChanSend)
    // Woken by a receiver (value already taken) or by close (→ panic).
    return true
}

To "implement a channel from scratch" in user code, use a sync.Mutex plus two sync.Conds (notFull, notEmpty) around a ring buffer. Then point out what you lose. A Cond cannot take part in select, so you get no timeouts, no cancellation and no multiplexing. You also have to rebuild close semantics yourself. Those are the parts the runtime gives you for free.

Patterns:

// ── Pipeline stage: owns and closes its output; every send can be cancelled ──
func Square(ctx context.Context, in <-chan int) <-chan int {
    out := make(chan int)
    go func() {
        defer close(out)                 // the sender closes, never the receiver
        for n := range in {
            select {
            case out <- n * n:
            case <-ctx.Done():
                return                   // downstream gave up: don't block forever
            }
        }
    }()
    return out
}

// ── Fan-out: N workers read the SAME input channel (Go's channels already
//    load-balance; no dispatcher needed). Fan-in: merge their outputs. ──
func FanOut[T, U any](ctx context.Context, in <-chan T, n int, f func(T) U) <-chan U {
    out := make(chan U)
    var wg sync.WaitGroup
    for range n {
        wg.Go(func() {                   // Go 1.25+; else wg.Add(1) + go + defer wg.Done()
            for v := range in {
                select {
                case out <- f(v):
                case <-ctx.Done():
                    return
                }
            }
        })
    }
    go func() { wg.Wait(); close(out) }() // close only after ALL senders finish
    return out
}

// ── Or-done: wrap a channel you don't control so ranging over it
//    also stops on cancellation ──
func OrDone[T any](ctx context.Context, in <-chan T) <-chan T {
    out := make(chan T)
    go func() {
        defer close(out)
        for {
            select {
            case <-ctx.Done():
                return
            case v, ok := <-in:
                if !ok {
                    return
                }
                select {
                case out <- v:
                case <-ctx.Done():
                    return
                }
            }
        }
    }()
    return out
}

// ── Tee: duplicate a stream; both consumers must keep up (lockstep) ──
func Tee[T any](ctx context.Context, in <-chan T) (<-chan T, <-chan T) {
    out1, out2 := make(chan T), make(chan T)
    go func() {
        defer close(out1)
        defer close(out2)
        for v := range OrDone(ctx, in) {
            o1, o2 := out1, out2           // local copies we can nil out
            for range 2 {
                select {
                case o1 <- v:
                    o1 = nil               // sent: disable this case
                case o2 <- v:
                    o2 = nil
                case <-ctx.Done():
                    return
                }
            }
        }
    }()
    return out1, out2
}

// Usage
func main() {
    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()
    src := make(chan int)
    go func() {
        defer close(src)
        for i := range 5 { // range-over-int: Go 1.22+
            select {
            case src <- i:
            case <-ctx.Done(): // a bare send would leak this goroutine on timeout
                return
            }
        }
    }()
    for v := range Square(ctx, Square(ctx, src)) {
        fmt.Println(v) // 0 1 16 81 256
    }
}

Trade-offs and failure modes: - Unbuffered vs buffered: unbuffered gives a hand-off guarantee: the receiver has the value when the send returns. A buffer absorbs bursts, but it hides backpressure until it fills. Pick a buffer size from a measured burst, not by guessing. - Channels are not free: each operation takes a lock, and at high rates (millions of ops per second across cores) that becomes contention. Batch items, or use a mutex-protected slice for hot paths. - Deadlock detection is weak. fatal error: all goroutines are asleep fires only when every goroutine is blocked. One stuck worker in a live server goes unnoticed. Use the goroutine profile, or the goroutineleak profile (GA in Go 1.27), which uses GC reachability to find goroutines blocked on channels nobody else can reach.

What they probe next: what close does to parked receivers (they all wake with the zero value and ok == false) and to parked senders (they panic). Why can't a receiver safely close? Because another sender may still be sending, and a send on a closed channel panics. How does select choose? It shuffles the cases into a random order to prevent starvation, and it locks the channels in address order to avoid deadlock.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Internal structure Knows hchan, sudog, recvq/sendq, ring buffer, direct hand-off to a parked receiver
Patterns Implements fan-out, fan-in, pipeline, tee, or-done naturally
Cancellation Every blocking send also selects on ctx.Done(); receives are guarded by a ctx-aware upstream that closes on cancel (or by OrDone). No goroutine leaks
Deadlock detection Knows the runtime detector only catches global deadlock; knows nil-channel and closed-channel behaviour

Channel send: hand off directly to a parked receiver, else buffer, else park the sender on sendq; a close wakes all waiters.

flowchart TD
    S["chansend"] --> NL{"c is nil?"}
    NL -->|yes| PF["Park forever"]
    NL -->|no| LK["Lock channel"]
    LK --> CL{"closed?"}
    CL -->|yes| PN["Panic: send on closed channel"]
    CL -->|no| RQ{"recvq has a parked receiver?"}
    RQ -->|yes| DH["Copy value to receiver's stack slot, goready"]
    RQ -->|no| BF{"qcount < dataqsiz?"}
    BF -->|yes| BUF["Copy into ring buffer at sendx"]
    BF -->|no| PK["Enqueue sudog on sendq, gopark"]

Question 3: Interface Satisfaction — The Type System's Secret Weapon

Interviewer: "Explain how Go's interface satisfaction works at runtime. How does an interface value get stored in memory? What's the difference between io.Reader and any?"

🎯 Expected Answer

30-second answer: An interface value is two words. For a non-empty interface like io.Reader they are (itab, data), where the itab holds the dynamic type plus a table of method pointers. For any they are (type, data), with no methods to look up. Satisfaction is structural and checked at compile time when assigning a concrete type. A method call through an interface is an indirect call through the itab. A type assertion to a concrete type is a single pointer comparison. The famous trap is that an interface holding a nil *T is not itself nil.

Layout (runtime/runtime2.go, internal/abi):

type iface struct {           // non-empty interface, e.g. io.Reader
    tab  *itab
    data unsafe.Pointer       // pointer to the value (or the value itself if pointer-shaped)
}
type eface struct {           // any / interface{}
    _type *_type
    data  unsafe.Pointer
}
type itab struct {            // one per (interface type, concrete type) pair
    inter *interfacetype
    _type *_type
    hash  uint32              // copy of _type.hash, used by type switches
    fun   [1]uintptr          // method table; variable length; fun[0]==0 means "doesn't implement"
}
  • Where itabs come from: for a conversion the compiler can see, like var r io.Reader = f with f *os.File, the compiler emits the itab statically into the binary. Only dynamic conversions need the runtime. Those are assertions from one interface to another (x.(io.Writer)) and reflection. They go through getitab, which caches results in a global hash table. Since Go 1.22 each call site also has its own small cache.
  • Boxing: storing a non-pointer value in an interface usually heap-allocates a copy (data must be a pointer). The runtime avoids the allocation for zero-sized values, single-byte values, small integers 0–255, and constants.
// Structural typing: no "implements" keyword. Define the interface where it's CONSUMED.
type Store interface {
    Get(ctx context.Context, key string) ([]byte, error)
}

// Any type with this method set satisfies Store, even one written before Store existed.
func CacheMiddleware(store Store) func(http.Handler) http.Handler { /* ... */ }

// Compile-time assertion that a type satisfies an interface (costs nothing at runtime):
var _ Store = (*RedisStore)(nil)

Type assertions and switches:

Operation Cost
x.(*os.File): assert to a concrete type Compare x's itab (or type) pointer with the known one. One comparison
x.(io.Writer): assert to an interface itab lookup: per-call-site cache, then the global itab table, then building it from method sets
switch x.(type) with concrete cases Compiler compares type hashes (binary search or jump table), then confirms by pointer
Method call r.Read(p) Load tab.fun[i], indirect call. Not inlinable unless the compiler can devirtualize (PGO helps here)

The nil-interface trap:

func find() error {
    var err *MyError          // nil pointer
    return err                // returns error{tab: (*MyError, error), data: nil}
}

fmt.Println(find() == nil)   // false: the interface has a type, so it is not nil

// Fix: return the untyped nil literal on the success path.
func findFixed() error {
    var err *MyError
    if err == nil {
        return nil            // a truly nil interface
    }
    return err
}

go vet does not catch this. The nilness analyzer and staticcheck catch some cases. The real defence is a convention: functions return error, never a concrete error pointer type.

Generics vs interfaces (how Go actually implements generics):

Go does not fully monomorphize. It uses GC-shape stenciling with dictionaries. One copy of the function body is compiled per GC shape: all pointer types share one shape, and each distinct underlying non-pointer type gets its own. A hidden dictionary argument supplies type information and method pointers.

Interfaces Generics
Dispatch itab indirect call Direct for operators on value types. For method calls on a type parameter, an indirect call through the dictionary, often no faster than an interface
Allocation Boxing non-pointer values may allocate No boxing. []T stays a flat slice
Code size One copy One copy per GC shape: moderate, not C++-style bloat
Heterogeneous collections Yes ([]io.Reader) No: []T holds one T
Best for Behavioural abstraction at boundaries (storage, transport) Type-safe containers and algorithms (slices, maps, sync-style wrappers)

Generics features by version: type parameters (1.18). Generic type aliases type Set[T comparable] = map[T]struct{} (1.24). Constraints that refer to the type being constrained, type Adder[A Adder[A]] interface{ Add(A) A } (1.26). Generic methods: a method may declare its own type parameters, func (l *List[T]) Map[U any](f func(T) U) []U (1.27), but interface methods still cannot, and a generic method cannot satisfy an interface method.

What they probe next: "Why can't a value of type T with pointer-receiver methods satisfy the interface?" The method set of T excludes *T methods. The value inside an interface isn't addressable, so the runtime couldn't take its address safely. "Cost of any in hot paths?" Boxing allocations and lost inlining; measure with -gcflags=-m and benchmarks.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Memory layout Knows iface/eface/itab; static vs runtime itab creation; boxing
Structural typing Consumer-defined interfaces; var _ I = (*T)(nil) assertions
Nil interface trap Explains it from the layout and gives the convention that prevents it
Generics vs interfaces Knows Go uses GC-shape stenciling + dictionaries, not full monomorphization; knows 1.24–1.27 additions

Question 4: The Memory Model — Happens-Before, Data Races, and sync/atomic

Interviewer: "Without using sync.Mutex, implement a thread-safe counter. Explain every memory ordering guarantee you're relying on."

🎯 Expected Answer

30-second answer: Use atomic.Int64.Add. It is a single atomic read-modify-write (LOCK XADD on x86, LDADDAL or an LL/SC loop on ARM64). Since the 2022 memory-model revision (Go 1.19), all sync/atomic operations behave as sequentially consistent. If an atomic load observes the value written by an atomic store, the store is synchronized before the load. Everything the writer did before the store therefore happens before everything the reader does after the load. That is what makes "publish with an atomic flag" safe. A program with a data race on a word-sized value is not undefined behaviour as in C++. Each read sees some value that was actually written. But races on multi-word values (strings, slices, interfaces, maps) can corrupt memory or crash.

The rules you rely on (from go.dev/ref/mem):

Synchronization Guarantee ("A is synchronized before B")
Goroutine start The go statement is synchronized before the new goroutine starts running. Goroutine exit synchronizes with nothing: you must use a channel or WaitGroup
Channel send A send is synchronized before the completion of the matching receive
Unbuffered receive A receive is synchronized before the completion of the matching send. Both sides know the other got there
Buffered channel, cap C The k-th receive is synchronized before the (k+C)-th send completes. This is why a buffered channel works as a counting semaphore
close(ch) Synchronized before a receive that returns because the channel is closed
sync.Mutex / RWMutex For n < m, the n-th Unlock is synchronized before the m-th Lock returns
sync.Once The completion of f in once.Do(f) is synchronized before any Do returns
sync/atomic If atomic B observes the effect of atomic A, A is synchronized before B. All atomics act as if executed in one global sequentially consistent order
// ── Counter: one atomic RMW per increment ──
type Counter struct{ n atomic.Int64 } // atomic.Int64 is 8-byte aligned even on 32-bit platforms

func (c *Counter) Inc() int64  { return c.n.Add(1) } // NOT load+add+store: a single atomic instruction
func (c *Counter) Load() int64 { return c.n.Load() }

// ── Publication through an atomic flag IS safe ──
type Service struct {
    ready atomic.Bool
    cache map[string]Result // written once, before ready.Store(true); never mutated after
}

func (s *Service) Initialize() {
    s.cache = buildCache() // (1) plain writes
    s.ready.Store(true)    // (2) atomic store "publishes" (1)
}

func (s *Service) Get(key string) (Result, bool) {
    if !s.ready.Load() {   // (3) if this observes true, (2) is synchronized before (3)…
        return Result{}, false
    }
    r, ok := s.cache[key]  // (4) …so (1) happens before (4). No race (verified with -race).
    return r, ok
}

// For a cache that is REBUILT periodically, swap an immutable snapshot instead:
type SnapshotCache struct{ m atomic.Pointer[map[string]Result] }

func (c *SnapshotCache) Reload(fresh map[string]Result) { c.m.Store(&fresh) } // never mutate after Store
func (c *SnapshotCache) Get(k string) (Result, bool) {
    if m := c.m.Load(); m != nil {
        r, ok := (*m)[k]
        return r, ok
    }
    return Result{}, false
}

// ── Sharded counter for very hot paths: avoid ONE contended cache line ──
type paddedInt64 struct {
    n atomic.Int64
    _ [56]byte // pad to 64 bytes so shards don't share a cache line (false sharing).
               // Use 128 on CPUs with 128-byte lines or adjacent-line prefetch.
}

type ShardedCounter struct{ shards [64]paddedInt64 }

func (c *ShardedCounter) Inc() { c.shards[rand.Uint32()%64].n.Add(1) } // math/rand/v2: per-thread, lock-free

func (c *ShardedCounter) Total() (t int64) { // not an atomic snapshot: fine for metrics
    for i := range c.shards {
        t += c.shards[i].n.Load()
    }
    return t
}

// ── CAS spin lock: to explain CAS, NOT to use ──
type SpinLock struct{ locked atomic.Bool }

func (s *SpinLock) Lock() {
    for !s.locked.CompareAndSwap(false, true) {
        runtime.Gosched() // sync.Mutex already spins briefly, then parks: use it
    }
}
func (s *SpinLock) Unlock() { s.locked.Store(false) }

How sequential consistency maps to hardware: on x86-64, atomic loads are plain MOVs, because x86's TSO model already gives acquire semantics. Stores use XCHG, which acts as a full fence. On ARM64, loads are LDAR and stores STLR, and Read-Modify-Write uses LSE atomics (LDADDAL, CASAL) where available. Go offers no weaker "relaxed" atomics on purpose.

The race detector (-race): - It uses ThreadSanitizer: the compiler instruments every memory access, and the runtime keeps vector clocks per goroutine plus shadow memory per 8-byte word. Two accesses to the same location, at least one a write, with no happens-before order between them, produce a race report. - It has no false positives, but it only sees races that actually execute during the run. Run it in CI with real concurrency (go test -race, integration tests, load tests). - Cost, per the Go docs: memory use grows 5–10× and execution time 2–20×. Most teams don't run it in production. Some run a race-enabled canary on a small slice of traffic. - It tracks up to 8,128 simultaneously alive goroutines. Unsupported or slow on some platforms.

What they probe next: "Is a racy bool read OK if you don't care about staleness?" No. It's a data race, and the compiler may hoist the load out of a loop and spin forever. Use atomic.Bool. "Double-checked locking in Go?" Use sync.Once / sync.OnceValue (Go 1.21), which does exactly this correctly. "Why does fatal error: concurrent map writes exist?" Maps have a cheap best-effort write flag. When it trips, the runtime throws an unrecoverable fatal error rather than corrupt memory.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Happens-before States the channel/mutex/atomic rules precisely, including that goroutine exit synchronizes with nothing
Atomic memory ordering Knows Go atomics are sequentially consistent (2022 model) and that atomic publication of plain writes is safe
Race detector Knows TSan/vector clocks, "only races that execute", 2–20× time / 5–10× memory
Pointer swap / sharding Uses atomic.Pointer snapshots for read-mostly data, padding against false sharing

Question 5: GC — The Go Garbage Collector

Interviewer: "Walk me through a full GC cycle in Go. How does it decide when to start? How does it minimize STW? What triggers a GC?"

🎯 Expected Answer

30-second answer: Go's GC is a concurrent, tri-color, mark-sweep collector. It is non-generational, non-moving and non-compacting. Each cycle has two short stop-the-world pauses, typically tens to a few hundred microseconds, around a concurrent mark phase. The mark phase uses about 25 % of GOMAXPROCS plus mark assists charged to goroutines that allocate fast. Sweeping is lazy and concurrent. The pacer starts a cycle so that marking finishes just as the heap reaches its goal: live heap × (1 + GOGC/100) plus roots, capped by GOMEMLIMIT. Since Go 1.26 the default marker is "Green Tea". It scans small objects span by span, a page at a time, instead of chasing one object at a time, for better cache locality. The Go team reports 10–40 % less GC CPU on GC-heavy programs.

One cycle:

 STW #1: sweep termination + mark setup  (~10–100 µs)
   finish any unswept spans; enable the write barrier; queue root-marking jobs
        │
 CONCURRENT MARK  (application keeps running)
   • dedicated mark workers ≈ 25% of GOMAXPROCS (plus idle-P workers)
   • mark assists: a goroutine that allocates during marking must do
     proportional mark work first, which is how the GC keeps up with fast allocators
   • roots: goroutine stacks (each stack scanned at a brief per-goroutine stop),
     globals, runtime structures
   • write barrier shades pointers that the program overwrites/installs
        │
 STW #2: mark termination  (~10–100s of µs)
   drain remaining work, disable the write barrier, compute the next goal
        │
 CONCURRENT SWEEP
   spans with no marked objects are freed; others are swept lazily on the
   next allocation from them or by a background sweeper.
   A separate background SCAVENGER returns unused pages to the OS (madvise).

When a cycle starts (the pacer):

heap goal = live_heap + (live_heap + GC roots) × GOGC/100        (roots = stacks + globals; Go 1.18+)

GOGC=100 (default): next GC around 2× the live heap
GOGC=50:            around 1.5× (less memory, about twice as many cycles)
GOGC=off:           never, UNLESS GOMEMLIMIT is set

GOMEMLIMIT (Go 1.19+): a SOFT limit on total Go-managed memory (heap + stacks + runtime).
  As memory approaches it, GC runs more often regardless of GOGC.
  "Soft": the runtime caps GC CPU at about 50% (over a short window) to avoid a death
  spiral, so the limit CAN be exceeded when the live heap genuinely doesn't fit.

The pacer triggers before the goal (trigger < goal) so concurrent marking can finish in time. Other triggers: runtime.GC(), and a forced GC if none has run for 2 minutes.

Tri-color invariant and the write barrier:

  • White = not yet seen. Grey = seen, fields not yet scanned. Black = scanned. At the end of mark, white objects are garbage.
  • The danger: the program stores a pointer to a white object into an already-black object, then deletes the only other path to it. The GC never revisits the black object, so it frees a live object.
  • Go 1.5–1.7 used a Dijkstra insertion barrier (shade the new pointee). Because stacks had no barrier, every stack had to be re-scanned during STW, and that caused pauses of tens of milliseconds with many goroutines.
  • Go 1.8 introduced the hybrid barrier: Yuasa deletion plus Dijkstra insertion. It shades both the overwritten pointer and the new one. Stacks no longer need re-scanning, and typical STW dropped well under 100 µs.
  • Allocation during marking is allocate-black: new objects are already marked.

Tuning in production:

Situation Lever
Container with a hard memory limit Set GOMEMLIMIT to about 90 % of the container limit. Optionally also GOGC=off, or a large GOGC, for steady-state services, which then use memory up to the limit instead of collecting at 2× live heap
GC CPU too high (gctrace, /gc/ metrics, the CPU profile shows gcBgMarkWorker / mallocgc) Allocate less first: preallocate, reuse buffers (sync.Pool), avoid []byte↔string round-trips, avoid boxing into any. Raise GOGC only if you have spare memory
Memory too high Lower GOGC, or set GOMEMLIMIT. Check for retained references (heap profile inuse_space)
Pause-time concerns Pauses barely depend on GOGC or heap size. Long ones usually come from huge numbers of goroutines, large stack scans or non-preemptible loops. Look at the execution trace

GOGC does not trade pause length for memory, as older notes claimed. It trades GC CPU for memory.

Observing it (prefer runtime/metrics to ReadMemStats, which stops the world):

import "runtime/metrics"

func gcSnapshot() {
    samples := []metrics.Sample{
        {Name: "/gc/heap/live:bytes"},       // live heap after the last mark
        {Name: "/gc/heap/goal:bytes"},       // current heap goal
        {Name: "/gc/cycles/total:gc-cycles"},
        {Name: "/gc/gogc:percent"},          // read GOGC without changing it
        {Name: "/gc/gomemlimit:bytes"},
        {Name: "/cpu/classes/gc/total:cpu-seconds"},
    }
    metrics.Read(samples)
    for _, s := range samples {
        switch s.Value.Kind() {
        case metrics.KindUint64:
            fmt.Printf("%-36s %d\n", s.Name, s.Value.Uint64())
        case metrics.KindFloat64:
            fmt.Printf("%-36s %.3f\n", s.Name, s.Value.Float64())
        }
    }
}
// Fresh process output (Go 1.27): heap goal 4194304 (the 4 MiB minimum), gogc 100,
// gomemlimit 9223372036854775807 (math.MaxInt64 = "no limit").
// GODEBUG=gctrace=1 prints one line per cycle: heap sizes, pause times, CPU%.

Never read GOGC with debug.SetGCPercent(-1). That call disables the GC and returns the old value.

Why no generational or compacting GC? Go has value types and escape analysis, so many short-lived objects never reach the heap. That weakens the generational hypothesis, and a generational design needs a write barrier that is always on. A non-moving heap keeps cgo and unsafe simple and makes interior pointers cheap. The cost is fragmentation, which the size-class allocator limits.

What they probe next: finalizers vs runtime.AddCleanup (1.24; supports several cleanups per object and doesn't resurrect objects). Weak pointers (weak.Pointer, 1.24) for canonicalizing caches. How does sync.Pool interact with GC? Pools are cleared each cycle, and a victim cache keeps objects for one more cycle. How is GOMEMLIMIT different from a cgroup limit? The cgroup limit is hard, so you get OOM-killed. GOMEMLIMIT is soft and covers Go-managed memory only, not cgo or mmap.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Phase knowledge Two short STW pauses, concurrent mark with assists, lazy sweep, separate scavenger
Tri-color + barrier Explains the invariant, why stacks were rescanned pre-1.8, hybrid barrier
GC pacing Heap-goal formula, GOGC trades CPU for memory, GOMEMLIMIT is soft with a CPU cap
Currency Knows Green Tea is default since 1.26; uses runtime/metrics; knows AddCleanup/weak

One GC cycle: two short stop-the-world pauses around a concurrent mark, followed by lazy concurrent sweep and a background scavenger.

flowchart TD
    A["Pacer trigger (before the heap goal)"] --> B["STW 1: sweep termination, enable write barrier"]
    B --> C["Concurrent mark: workers at about 25% of GOMAXPROCS plus mark assists"]
    C --> D["STW 2: mark termination, disable write barrier, compute next goal"]
    D --> E["Concurrent sweep: lazy, spans freed or reused"]
    D --> F["Background scavenger returns pages to the OS"]
    E --> A

Tri-color marking: white objects are unseen, grey are seen but unscanned, black are scanned; whatever is still white at the end is garbage, and the write barrier shades pointers the program changes.

stateDiagram-v2
    [*] --> White: allocated before mark
    [*] --> Black: allocated during mark (allocate-black)
    White --> Grey: reachable, queued for scan
    Grey --> Black: all fields scanned
    White --> Grey: shaded by write barrier
    Black --> [*]: survives
    White --> [*]: unmarked at end of mark, freed by sweep

Question 6: sync Package Deep Dive — Beyond Mutexes

Interviewer: "Design a connection pool for a database using only the standard library's sync package. Then explain the internals of sync.Map."

🎯 Expected Answer

30-second answer: Bound live connections with a semaphore: a buffered channel of size maxOpen, where acquiring blocks with ctx for backpressure. Keep idle connections in a second buffered channel. Health-check a connection when you take it, and release the semaphore slot on every exit path. That last rule is where most hand-written pools are buggy. In production you rarely write this: database/sql and pgx already pool. sync.Pool is not a connection pool, because it drops objects at every GC. sync.Map was reimplemented in Go 1.24 as a concurrent hash-trie, so the old read/dirty two-map design is history.

Connection pool (compiles; verified with go test -race):

var ErrPoolClosed = errors.New("pool closed")

type Pool[T any] struct {
    sem     chan struct{} // one token per allowed LIVE connection (in use + idle)
    idle    chan T        // idle connections
    factory func(context.Context) (T, error)
    closeFn func(T)
    healthy func(T) bool  // cheap check; a real driver pings only if idle for a while

    mu     sync.Mutex
    closed bool
}

func NewPool[T any](maxOpen, maxIdle int, factory func(context.Context) (T, error),
    closeFn func(T), healthy func(T) bool) *Pool[T] {
    return &Pool[T]{
        sem:  make(chan struct{}, maxOpen),
        idle: make(chan T, maxIdle),
        factory: factory, closeFn: closeFn, healthy: healthy,
    }
}

// Acquire blocks until a slot is free or ctx ends (backpressure, not unbounded queuing).
func (p *Pool[T]) Acquire(ctx context.Context) (T, error) {
    var zero T
    select {
    case p.sem <- struct{}{}:
    case <-ctx.Done():
        return zero, ctx.Err()
    }
    if p.isClosed() {
        <-p.sem
        return zero, ErrPoolClosed
    }
    for { // prefer an idle connection
        select {
        case c := <-p.idle:
            if p.healthy(c) {
                return c, nil
            }
            p.closeFn(c) // stale: discard and look again
            continue
        default:
        }
        break
    }
    c, err := p.factory(ctx)
    if err != nil {
        <-p.sem // give the slot back, or the pool slowly "leaks" capacity
        return zero, err
    }
    return c, nil
}

// Release returns a connection; broken=true if the caller saw an I/O error on it.
func (p *Pool[T]) Release(c T, broken bool) {
    defer func() { <-p.sem }() // ALWAYS free the slot
    p.mu.Lock() // hold across the closed check AND the send: Close sets closed under this lock
    if !broken && !p.closed { // before draining, so no connection can slip in after the drain
        select {
        case p.idle <- c:
            p.mu.Unlock()
            return
        default: // more idle than maxIdle: shrink
        }
    }
    p.mu.Unlock()
    p.closeFn(c)
}

func (p *Pool[T]) Close() {
    p.mu.Lock()
    p.closed = true
    p.mu.Unlock()
    for {
        select {
        case c := <-p.idle:
            p.closeFn(c)
        default:
            return
        }
    }
}

func (p *Pool[T]) isClosed() bool { p.mu.Lock(); defer p.mu.Unlock(); return p.closed }

Design points to say out loud: - Why a semaphore, not a counter plus "wait on the idle channel": a waiter blocked only on idle is never woken when a broken connection is discarded. The capacity was freed but nobody was told, so the waiter hangs until its ctx times out. A semaphore makes "capacity freed" and "waiter wakes" the same event. - What database/sql adds: SetMaxOpenConns, SetMaxIdleConns, SetConnMaxLifetime (rotate connections, so DNS or failover changes are picked up and server-side state is released), SetConnMaxIdleTime, and driver.ErrBadConn retry. Mention ConnMaxLifetime below the load balancer's or DB's idle timeout. - Failure modes: pool exhaustion under slow queries, which shows up as the WaitCount/WaitDuration stats in db.Stats(). Leaked connections from a forgotten rows.Close(). A thundering herd of reconnects after a DB failover: jitter and limit the factory.

sync.Map internals:

Go ≤ 1.23 Go 1.24+ (current)
Structure read (atomic, read-only map) + dirty (mutex-protected map) + misses counter; promote dirty→read after enough misses; "expunged" entries Concurrent hash-trie (internal/sync.HashTrieMap): a tree of fixed-fan-out nodes indexed by hash bits; lookups are lock-free atomic loads; writes lock only the affected node
Weak spot Writes of new keys took the global mutex; re-promotion after a write burst copied the whole map Much less contention for disjoint-key writes; no promotion "ramp-up"
Opt-out – GOEXPERIMENT=nosynchashtriemap (build time)

The API is unchanged: Load, Store, LoadOrStore, LoadAndDelete, Delete, Range, Swap / CompareAndSwap / CompareAndDelete (1.20), Clear (1.23). It is still any-typed. Most teams wrap it in a small generic type.

When to use what:

Need Use
Typical shared map, mixed reads/writes map + sync.Mutex (or RWMutex if reads dominate and critical sections aren't tiny)
Read-mostly cache, keys added over time, many cores sync.Map
Read-mostly, whole thing rebuilt periodically atomic.Pointer[map[K]V] snapshot swap
Very hot, many writers Sharded map (N × mutex+map, shard by maphash)

RWMutex is not free for readers. Every RLock does an atomic add on a shared counter, so with many cores and tiny critical sections a plain Mutex can win. Benchmark both.

The rest of sync, briefly: - Mutex: spins briefly, then parks. Starvation mode kicks in when a waiter has waited more than 1 ms. Ownership is then handed directly FIFO, trading throughput for bounded tail latency. Not reentrant. Must not be copied after first use (go vet copylocks). - Once, OnceFunc/OnceValue/OnceValues (1.21): if f panics, Once treats it as done. OnceFunc re-panics on every call. - WaitGroup.Go(f) (1.25) replaces the Add(1) / go / defer Done() boilerplate. The waitgroup vet check flags Add inside the goroutine. - Cond: rarely the right tool, because it can't be used in select or with ctx. Prefer channels. - Pool: per-P caches. Contents are dropped across GCs (with a one-cycle victim cache). Use it for scratch buffers, and cap the size of objects you return to it.

What they probe next: "How would you add max-lifetime or idle eviction?" Store createdAt and lastUsed with the connection, and check them in Acquire. "How do you test the pool?" Use testing/synctest (Go 1.25) for timeouts without real sleeping, -race, and a fault-injecting factory.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Pool design Bounded live connections, ctx-aware waiting, slot released on every path, health checks, shutdown
sync.Map internals Knows the Go 1.24 HashTrieMap rewrite (and the old read/dirty design as history)
sync patterns Mutex vs RWMutex trade-off, starvation mode, OnceValue, WaitGroup.Go, why not Cond
Production awareness Knows database/sql pool knobs and failure modes (exhaustion, leaks, failover herds)

Question 7: Context — Cancellation Propagation and Request-Scoped Values

Interviewer: "The standard library's context package — implement a custom context that logs every cancellation. Then explain how to propagate values through the context tree safely."

🎯 Expected Answer

30-second answer: Don't implement the Context interface yourself unless you must. Wrap the standard one. context.WithCancelCause plus context.AfterFunc (both Go 1.20/1.21) give you "log every cancellation, with the reason" in a few lines, with no extra goroutine. Cancellation flows down the tree: cancelling a parent cancels all its children, never the reverse. Values are for request-scoped metadata (trace ID, auth principal, logger). Use unexported key types so no other package can collide with or overwrite your keys, and expose typed accessors.

// Logs when ctx ends and WHY: deadline, parent cancellation, or an explicit cause.
func WithCancelLogging(parent context.Context, log *slog.Logger) (context.Context, context.CancelCauseFunc) {
    ctx, cancel := context.WithCancelCause(parent)
    context.AfterFunc(ctx, func() { // runs once, in its own goroutine, after ctx is done
        log.Info("context cancelled", "err", ctx.Err(), "cause", context.Cause(ctx))
    })
    return ctx, cancel
}

// ctx, cancel := WithCancelLogging(r.Context(), log)
// defer cancel(nil)
// cancel(errors.New("client went away"))
// → msg="context cancelled" err="context canceled" cause="client went away"
// Parent deadline → err="context deadline exceeded" cause="context deadline exceeded"

If you really implement Context yourself, the contract is strict: - Done() must return the same channel every time, closed exactly once. Use sync.Once, or close under a mutex with an "already closed" check. Two paths that both close it (your Cancel and a "parent cancelled" watcher) will panic with close of closed channel. - Err() must be nil until Done is closed, and non-nil afterwards: Canceled or DeadlineExceeded. - Children derived from your type: WithCancel(yourCtx) cannot see your internals. It falls back to starting a goroutine per child to watch Done(), unless your type implements AfterFunc(func()) (stop func() bool) (Go 1.21 optimization). Custom contexts quietly cost goroutines.

Values, safely:

type ctxKey int // unexported type: no other package can construct this key

const (
    traceIDKey ctxKey = iota
    principalKey
)

func WithTraceID(ctx context.Context, id string) context.Context {
    return context.WithValue(ctx, traceIDKey, id)
}

func TraceID(ctx context.Context) (string, bool) {
    id, ok := ctx.Value(traceIDKey).(string) // comma-ok: absent or wrong type → ok=false
    return id, ok
}
  • string or int keys collide across packages (staticcheck SA1029 flags built-in key types). An exported key variable lets other packages overwrite your value. Keep the key unexported and export functions.
  • Lookup is a linked-list walk up the parent chain, O(depth). Fine for a handful of values, not for a per-request map of 50 things. Bundle related values into one struct.
  • Do put: trace/span IDs (OpenTelemetry does this), the authenticated principal, a request-scoped logger, deadline-sensitive metadata.
  • Don't put: DB handles, config, feature flags, optional function parameters. These are hidden dependencies that the compiler can't check.

Rules that matter in production:

func HandleRequest(w http.ResponseWriter, r *http.Request) {
    // r.Context() is already cancelled when the client disconnects (HTTP/1.1 and
    // HTTP/2) or when ServeHTTP returns. You don't need a goroutine to watch for it.
    ctx, cancel := context.WithTimeout(r.Context(), 2*time.Second)
    defer cancel() // ALWAYS: releases the timer and the parent's reference to the child.
                   // go vet's lostcancel check flags a missing cancel.

    // Work that must outlive the request (audit log, async publish) but keep its values:
    bg := context.WithoutCancel(ctx) // Go 1.21
    go audit(bg, r)                  // give it its OWN timeout inside audit()

    if err := process(ctx); errors.Is(err, context.DeadlineExceeded) {
        http.Error(w, "timeout", http.StatusGatewayTimeout)
    }
}
  • Don't store a Context in a struct. Pass it as the first parameter. A struct-held context ties every method call to one request's lifetime. Exceptions are types that are request-scoped, like http.Request itself.
  • Deadlines propagate across services only if you send them. gRPC does so automatically (grpc-timeout). For HTTP, pass a header and subtract a safety margin.
  • WithTimeoutCause / WithDeadlineCause (1.21) let you record which timeout fired. This is very useful when there are three nested ones.

What they probe next: "What happens to a goroutine that ignores ctx?" Nothing. Cancellation is cooperative, so blocking calls must take ctx or have their own timeout. "How does net/http cancel a slow client request?" Request.WithContext / NewRequestWithContext. Cancelling closes the connection, or resets the stream in HTTP/2.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Custom Done channel Knows the Context contract (one channel, closed once, Err non-nil after) and prefers wrapping via AfterFunc / Cause
Value propagation Unexported key types, typed accessors, O(depth) lookup, what not to store
defer cancel() Always cancels; knows WithoutCancel for detached work
Context in structs Knows the anti-pattern and the exceptions

Question 8: Error Handling — Errors as Values, Wrapping, and Production Patterns

Interviewer: "Implement an error handling strategy for a gRPC microservice. Handle different error types (validation, not-found, internal), propagate them across service boundaries, and ensure PII isn't leaked in error messages."

🎯 Expected Answer

30-second answer: Inside the service, wrap errors with context (fmt.Errorf("load user %d: %w", id, err)). Classify them with errors.Is / errors.As (or errors.AsType, Go 1.26). At the edge, in one interceptor, translate to a gRPC status. The client gets a stable code, a public message and a machine-readable reason. The full cause chain goes only into the server log, keyed by request ID. Internal error text routinely contains SQL, hostnames, emails or tokens, so it never crosses the boundary.

type Kind int

const (
    KindInternal Kind = iota // zero value = the safest default
    KindValidation
    KindNotFound
    KindConflict
    KindUnauthenticated
    KindPermissionDenied
    KindUnavailable
)

func (k Kind) String() string { // or generate with `stringer -type=Kind`
    return [...]string{"internal", "validation", "not_found", "conflict",
        "unauthenticated", "permission_denied", "unavailable"}[k]
}

// Error separates what the client may see (Public, Reason) from what only logs see (Err).
type Error struct {
    Kind      Kind
    Public    string // safe for clients: no PII, no SQL, no hostnames
    Reason    string // stable code for programs, e.g. "USER_NOT_FOUND"
    Retryable bool
    Err       error  // internal cause: logged, never sent
}

func (e *Error) Error() string {
    if e.Err != nil {
        return e.Public + ": " + e.Err.Error()
    }
    return e.Public
}
func (e *Error) Unwrap() error { return e.Err }

// Is lets errors.Is(err, ErrNotFound) match ANY *Error of that Kind,
// not only the one sentinel pointer (errors.Is compares with == by default).
func (e *Error) Is(target error) bool {
    t, ok := target.(*Error)
    return ok && t.Err == nil && t.Kind == e.Kind
}

var ErrNotFound = &Error{Kind: KindNotFound, Public: "not found"} // sentinel, not "Sentry"

func classify(err error) *Error {
    if e, ok := errors.AsType[*Error](err); ok { // Go 1.26; before: var e *Error; errors.As(err, &e)
        return e
    }
    switch {
    case errors.Is(err, sql.ErrNoRows), errors.Is(err, fs.ErrNotExist): // not os.IsNotExist: it doesn't unwrap
        return &Error{Kind: KindNotFound, Public: "not found", Err: err}
    case errors.Is(err, context.DeadlineExceeded):
        return &Error{Kind: KindUnavailable, Public: "timed out", Retryable: true, Err: err}
    }
    return &Error{Kind: KindInternal, Public: "internal error", Err: err} // default: reveal nothing
}

var grpcCode = map[Kind]codes.Code{
    KindValidation:       codes.InvalidArgument,
    KindNotFound:         codes.NotFound,
    KindConflict:         codes.AlreadyExists, // or Aborted for optimistic-concurrency retries
    KindUnauthenticated:  codes.Unauthenticated,
    KindPermissionDenied: codes.PermissionDenied,
    KindUnavailable:      codes.Unavailable,
    KindInternal:         codes.Internal,
}

// ToGRPC runs once, at the boundary (a unary/stream server interceptor).
func ToGRPC(ctx context.Context, log *slog.Logger, err error) error {
    if err == nil {
        return nil
    }
    e := classify(err)
    level := slog.LevelWarn
    if e.Kind == KindInternal || e.Kind == KindUnavailable {
        level = slog.LevelError
    }
    log.LogAttrs(ctx, level, "request failed",
        slog.String("kind", e.Kind.String()), slog.Any("err", err)) // full chain: server-side only

    st := status.New(grpcCode[e.Kind], e.Public)
    if e.Reason != "" { // structured details the CLIENT may act on, never internal text
        if d, derr := st.WithDetails(&errdetails.ErrorInfo{
            Reason: e.Reason, Domain: "users.example.com",
        }); derr == nil {
            st = d
        }
    }
    return st.Err()
}

Verified behaviour: wrapping &Error{Kind: KindNotFound, Public: "user not found"} gives rpc error: code = NotFound desc = user not found. An unclassified errors.New("pq: password=hunter2 failed") gives code = Internal desc = internal error, and the secret stays in the log.

Practices, with the reasons:

Practice Why
Wrap with %w and a short, lowercase prefix: "load user %d: %w" Builds a readable causal chain; keeps Is/As working. Use %v when you deliberately want to hide the cause type from callers (API boundary)
Compare with errors.Is / As, never err.Error() == "..." Messages change; wrapping changes the string
Handle once: log or return Logging at every layer produces N copies of one failure. Log at the boundary where you stop propagating
errors.Join and multiple %w (1.20) Aggregate errors (validation, cleanup). Is/As search all branches
Return error, not *MyError Avoids the typed-nil-interface trap (Question 3)
Map context errors deliberately context.Canceled usually means the client left: don't page anyone. DeadlineExceeded → Unavailable/DeadlineExceeded, retryable
Panics are for programmer bugs Recover in the interceptor, log the stack, return Internal

Across services, propagate the code and reason, not the message. Clients retry on Unavailable and sometimes Aborted, with backoff. They never retry InvalidArgument. Mention go vet's new check (Go 1.27): it flags fmt.Errorf("...: %w", p) where p is a *E but E itself implements error, which is usually a latent bug.

What they probe next: "Stack traces?" Go errors carry none by default. Capture runtime.Callers once, at the point where an error is created inside your code, or rely on tracing spans. Don't capture at every wrap. "Sentinel vs typed errors?" Sentinels (io.EOF) are for simple conditions. Typed errors carry data. Behaviour interfaces (interface{ Temporary() bool }) are mostly out of favour.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Error wrapping %w vs %v deliberately, Is/As/AsType, errors.Join, custom Is methods
Error types A small kind taxonomy mapped once at the boundary, safe default = internal
PII safety Public message + machine reason to the client; cause chain only in logs
Once handling Logs OR returns; context errors and panics handled deliberately

Question 9: io.Reader/Writer — The Composition Pattern

Interviewer: "Design a streaming data transformation pipeline using io.Reader and io.Writer. Show how to compose readers for encryption, compression, and buffering."

🎯 Expected Answer

30-second answer: io.Reader and io.Writer are one-method interfaces, so every transform (gzip, cipher stream, base64, hashing, buffering) is a wrapper that is a Reader or Writer around another one. Data streams through in bounded memory. Three things decide correctness: layer order (compress before encrypt, because ciphertext doesn't compress), close order (outermost first, so every layer flushes its tail), and checking the errors returned by Close and Flush. For writers those errors are the real write errors.

// Encrypt streams: src → gzip → AES-CTR → base64 → bufio → dst
func Encrypt(dst io.Writer, src io.Reader, key []byte) error {
    block, err := aes.NewCipher(key) // 16/24/32-byte key; load it from a KMS, never a literal
    if err != nil {
        return err
    }
    iv := make([]byte, aes.BlockSize)
    if _, err := rand.Read(iv); err != nil { // crypto/rand
        return err
    }

    bw := bufio.NewWriterSize(dst, 32<<10)
    b64 := base64.NewEncoder(base64.StdEncoding, bw)
    if _, err := b64.Write(iv); err != nil { // the decryptor needs the IV: write it first
        return err
    }
    enc := cipher.StreamWriter{S: cipher.NewCTR(block, iv), W: b64}
    gz := gzip.NewWriter(enc)

    if _, err := io.Copy(gz, src); err != nil {
        return fmt.Errorf("copy: %w", err)
    }
    // Close OUTERMOST first: gzip writes its footer into enc → b64; base64
    // flushes its final partial block; then bufio pushes everything to dst.
    return errors.Join(gz.Close(), b64.Close(), bw.Flush())
}

// Decrypt mirrors it with readers: dst ← gzip ← AES-CTR ← base64 ← src
func Decrypt(dst io.Writer, src io.Reader, key []byte) error {
    block, err := aes.NewCipher(key)
    if err != nil {
        return err
    }
    b64 := base64.NewDecoder(base64.StdEncoding, src)
    iv := make([]byte, aes.BlockSize)
    if _, err := io.ReadFull(b64, iv); err != nil {
        return err
    }
    gz, err := gzip.NewReader(cipher.StreamReader{S: cipher.NewCTR(block, iv), R: b64})
    if err != nil {
        return err
    }
    defer gz.Close()
    _, err = io.Copy(dst, gz)
    return err
}

Verified: a 120 KB input round-trips byte-for-byte. Say the caveat: CTR mode is unauthenticated, so an attacker can flip ciphertext bits undetected. Production code uses an AEAD. AES-GCM over the whole stream would need the entire message in memory, so streaming systems use a chunked AEAD format: age, Tink's streaming AEAD, or per-chunk GCM with a counter nonce and a final-chunk flag.

The other building blocks:

// Transform reader: only touch p[:n], and process n bytes BEFORE looking at err
type upperReader struct{ r io.Reader }

func (u upperReader) Read(p []byte) (int, error) {
    n, err := u.r.Read(p)
    for i, c := range p[:n] {
        if 'a' <= c && c <= 'z' {
            p[i] = c - ('a' - 'A')
        }
    }
    return n, err // a Reader may return n > 0 AND io.EOF together
}

// Hash while copying: MultiWriter fans each write out; Sum is valid only AFTER the copy
func CopyAndHash(dst io.Writer, parts ...io.Reader) ([]byte, error) {
    h := sha256.New()
    if _, err := io.Copy(io.MultiWriter(dst, h), io.MultiReader(parts...)); err != nil {
        return nil, err
    }
    return h.Sum(nil), nil
}

// Bounded read: ask for max+1 bytes so "exactly max" and "too big" are distinguishable
func ReadAtMost(r io.Reader, max int64) ([]byte, error) {
    data, err := io.ReadAll(io.LimitReader(r, max+1))
    if err != nil {
        return nil, err
    }
    if int64(len(data)) > max {
        return nil, fmt.Errorf("input exceeds %d bytes", max)
    }
    return data, nil
}
// In HTTP handlers use http.MaxBytesReader(w, r.Body, max): it also stops the client early.

// io.Pipe: connect a writer-shaped producer to a reader-shaped consumer, no buffering
func Produce(write func(io.Writer) error) io.ReadCloser {
    pr, pw := io.Pipe()
    go func() {
        pw.CloseWithError(write(pw)) // nil → reader sees io.EOF; else reader sees the error
    }()
    return pr // the consumer MUST Close it: that makes a stuck producer's Write fail and exit
}
// Classic use: stream a multipart or gzip body into http.Post without building it in memory.

// Streaming JSON: decode array elements one at a time
func ProcessJSONStream(r io.Reader, handle func(Item) error) error {
    dec := json.NewDecoder(r)
    if _, err := dec.Token(); err != nil { // consume '['
        return err
    }
    for dec.More() {
        var it Item
        if err := dec.Decode(&it); err != nil {
            return fmt.Errorf("decode item: %w", err)
        }
        if err := handle(it); err != nil {
            return err
        }
    }
    _, err := dec.Token() // consume ']' (catches a stream cut off between elements)
    return err
}

Performance and failure modes: - io.Copy uses WriterTo / ReaderFrom when a side implements them. *os.File → *net.TCPConn can then use sendfile/splice: zero-copy in the kernel. Wrapping the file in a custom Reader silently loses this optimization. - bufio.Writer without Flush means truncated output with no error. It is the most common bug in this area. - Calling Close() from a defer drops its error. On write paths, return it (see errors.Join above), or use a named-result defer that records it. - JSON in Go 1.27: encoding/json/v2 and encoding/json/jsontext are now standard packages, and encoding/json itself is backed by the v2 engine with v1 behaviour preserved. Unmarshalling is significantly faster. v2 defaults are stricter: it rejects invalid UTF-8 and duplicate object keys. jsontext.Decoder gives true token-level streaming. The opt-out is GOEXPERIMENT=nojsonv2.

What they probe next: "Why does Read return (n, err) instead of just err?" Partial reads are normal: sockets return what has arrived. "How do you apply backpressure?" The pipeline is pull-based. A slow writer blocks io.Copy, which stops reading. "Where would you add a checksum?" Use io.TeeReader(r, hasher) on the read path. With an AEAD the authentication tag already covers it.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Reader/Writer composition Correct layer order (compress → encrypt), symmetric decode chain
io.Pipe Uses CloseWithError; knows the consumer must close to unblock the producer
Production patterns Bounded reads done correctly, streaming JSON, sendfile/ReaderFrom awareness
Cleanup Close order, flushes, and Close/Flush errors not silently dropped; knows CTR is unauthenticated

Question 10: reflect — When and Why (And When to Avoid It)

Interviewer: "Implement a generic validator that validates struct fields based on tags. Then explain the performance implications and when you'd avoid reflection."

🎯 Expected Answer

30-second answer: Walk the struct with reflect, parse each field's validate tag, and check its value. The staff-level part is cost. Parsing tags and resolving fields is the expensive bit, so do it once per type and cache a "plan" keyed by reflect.Type. Per-call work is then just field reads. Avoid reflection where the shape is known at compile time: generics, code generation, or hand-written Validate() methods. Also avoid it where its failures would surface as runtime panics instead of compile errors.

type ValidationError struct{ Field, Rule, Msg string }

func (e ValidationError) Error() string { return e.Field + ": " + e.Rule + ": " + e.Msg }

type ValidationErrors []ValidationError

func (v ValidationErrors) Error() string {
    msgs := make([]string, len(v))
    for i, e := range v {
        msgs[i] = e.Error()
    }
    return "validation failed: " + strings.Join(msgs, "; ")
}

type rule struct {
    name, arg string
    num       float64
}
type fieldPlan struct {
    index int
    name  string
    rules []rule
}

var plans sync.Map // reflect.Type → []fieldPlan, built once per type

func planFor(t reflect.Type) []fieldPlan {
    if p, ok := plans.Load(t); ok {
        return p.([]fieldPlan)
    }
    var fp []fieldPlan
    for i := 0; i < t.NumField(); i++ { // Go 1.26+: for f := range t.Fields()
        f := t.Field(i)
        tag := f.Tag.Get("validate")
        if !f.IsExported() || tag == "" {
            continue
        }
        var rules []rule
        for _, part := range strings.Split(tag, ",") {
            name, arg, _ := strings.Cut(part, "=")
            r := rule{name: name, arg: arg}
            if arg != "" {
                r.num, _ = strconv.ParseFloat(arg, 64) // real code: fail fast on bad tags at startup
            }
            rules = append(rules, r)
        }
        fp = append(fp, fieldPlan{index: i, name: f.Name, rules: rules})
    }
    plans.Store(t, fp) // benign race: two goroutines may build the same plan once
    return fp
}

// Validate returns `error` and a literal nil on success. Returning ValidationErrors(nil)
// as an error would be the typed-nil trap: a non-nil error holding a nil slice.
func Validate(v any) error {
    rv := reflect.ValueOf(v)
    if rv.Kind() == reflect.Pointer {
        rv = rv.Elem()
    }
    if rv.Kind() != reflect.Struct {
        return fmt.Errorf("validate: want struct, got %s", rv.Kind())
    }
    var errs ValidationErrors
    for _, f := range planFor(rv.Type()) {
        fv := rv.Field(f.index)
        for _, r := range f.rules {
            if msg := check(r, fv); msg != "" {
                errs = append(errs, ValidationError{Field: f.name, Rule: r.name, Msg: msg})
            }
        }
    }
    if len(errs) == 0 {
        return nil
    }
    return errs
}

func check(r rule, v reflect.Value) string {
    switch r.name {
    case "required":
        if v.IsZero() {
            return "is required"
        }
    case "min", "max": // numbers compare by value; strings/slices/maps by length
        var x float64
        switch {
        case v.CanInt():
            x = float64(v.Int())
        case v.CanUint():
            x = float64(v.Uint())
        case v.CanFloat():
            x = v.Float()
        case v.Kind() == reflect.String, v.Kind() == reflect.Slice, v.Kind() == reflect.Map:
            x = float64(v.Len())
        default:
            return "unsupported kind " + v.Kind().String()
        }
        if r.name == "min" && x < r.num {
            return "must be >= " + r.arg
        }
        if r.name == "max" && x > r.num {
            return "must be <= " + r.arg
        }
    }
    return ""
}

type User struct {
    Name  string `validate:"required,max=50"`
    Age   int    `validate:"min=0,max=150"`
    Email string `validate:"required"`
}
// Validate(&User{Age: 200}) →
// validation failed: Name: required: is required; Age: max: must be <= 150; Email: required: is required

Performance, measured rather than guessed. Go 1.27, Apple M5, for b.Loop() benchmark:

Approach ns/op allocs/op
Reflection validator above, plan cached ~36 0
Hand-written / generated Validate() ~1.6 0

Without the plan cache, re-parsing tags every call costs several times more and allocates. A regex rule that calls regexp.MatchString per call recompiles the pattern each time, which is far worse: compile it into the plan. The lesson: reflection is about 10–30× slower than direct code. That rarely matters next to a network call. It matters inside tight loops, serializers and per-row processing.

Why reflection is slow and fragile: - Every field access is a dynamic kind check plus an indirect read, and nothing can be inlined. - Value.Interface() and reflect.New box values, so they allocate. - Mistakes surface as panics at runtime (Elem on a non-pointer, Set on an unexported field, Int() on a string), not as compile errors.

Alternatives: - Code generation (go generate): protobuf/protoc-gen-go, stringer, sqlc, easyjson, and validator generators such as protoc-gen-validate / protovalidate. Fast and type-checked, at the cost of a build step. - Generics for algorithms over known shapes. They can't introspect struct fields, so they don't replace tag-driven validation. - Explicit Validate() error methods behind an interface: boring, fast, greppable.

unsafe field access (unsafe.Add(ptr, field.Offset)) is what fast serializers do internally. It is rarely justified in application code: it bypasses type safety and is hard to review.

What they probe next: "How does encoding/json avoid paying reflection costs per call?" It caches per-type encoders and decoders, the same plan idea. "Can reflection set unexported fields?" No: CanSet() is false. You would need unsafe, which is a code-review red flag. Go 1.26 added iterator forms: Type.Fields(), Type.Methods(), Value.Fields().

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
reflect mastery Navigates fields, tags, kinds; handles pointer vs value; exported-only
Performance awareness Caches per-type plans; quotes measured numbers, not folklore
Tag parsing Handles key=value lists; validates tags early
Production judgment Knows when codegen/explicit methods beat reflection; avoids typed-nil errors

Question 11: Testing — From Unit to Integration to E2E

Interviewer: "Design a testing strategy for a Go microservice. Cover unit tests with mocking, integration tests with testcontainers, and E2E tests. Show me the patterns you use."

🎯 Expected Answer

30-second answer: Most tests should be fast, table-driven unit tests against small interfaces. Use hand-written fakes, not mock frameworks, for anything stateful. Integration tests run against real dependencies in containers (Postgres, Kafka, Redis) behind a build tag, so the SQL and the drivers get tested for real. A few E2E tests cover the critical user journeys. Run everything with -race. Use testing/synctest (Go 1.25) to test time-dependent concurrent code without real sleeps.

// ── 1. Unit: consumer-defined interface + in-memory fake ──
type UserStore interface {
    GetUser(ctx context.Context, id string) (*User, error)
    CreateUser(ctx context.Context, u *User) error
}

type fakeStore struct {
    mu    sync.Mutex
    users map[string]*User
}

func newFakeStore() *fakeStore { return &fakeStore{users: map[string]*User{}} }

func (s *fakeStore) GetUser(_ context.Context, id string) (*User, error) {
    s.mu.Lock()
    defer s.mu.Unlock()
    if u, ok := s.users[id]; ok {
        return u, nil
    }
    return nil, ErrNotFound // the same sentinel the real store returns
}

func (s *fakeStore) CreateUser(_ context.Context, u *User) error {
    s.mu.Lock()
    defer s.mu.Unlock()
    s.users[u.ID] = u
    return nil
}

func TestUserService_CreateUser(t *testing.T) {
    tests := []struct {
        name    string
        user    *User
        wantErr error
    }{
        {"valid user", &User{Name: "Alice", Email: "alice@example.com"}, nil},
        {"missing name", &User{Email: "alice@example.com"}, ErrValidation},
        {"invalid email", &User{Name: "Alice", Email: "not-an-email"}, ErrValidation},
    }
    for _, tt := range tests { // Go 1.22+: no `tt := tt` needed; each iteration has its own tt
        t.Run(tt.name, func(t *testing.T) {
            t.Parallel()
            svc := NewUserService(newFakeStore()) // fresh state per subtest: no ordering coupling
            err := svc.CreateUser(t.Context(), tt.user) // t.Context(): Go 1.24, cancelled at test end
            if !errors.Is(err, tt.wantErr) {
                t.Fatalf("CreateUser() error = %v, want %v", err, tt.wantErr)
            }
        })
    }
}
//go:build integration

// ── 2. Integration: real Postgres via testcontainers-go's postgres module ──
// (the //go:build constraint must come before the package clause, preceded only by blank lines and line comments; run with `go test -tags integration`)
package store_test

func TestPostgresStore(t *testing.T) {
    ctx := t.Context()
    pg, err := postgres.Run(ctx, "postgres:18-alpine",
        postgres.WithDatabase("testdb"),
        postgres.WithUsername("test"),
        postgres.WithPassword("test"),
        postgres.BasicWaitStrategies(), // waits for the "ready" log line TWICE (initdb restarts
                                        // the server once) plus the port; waiting once is a classic flake
    )
    testcontainers.CleanupContainer(t, pg) // terminate even if the test fails
    if err != nil {
        t.Fatal(err)
    }
    dsn, err := pg.ConnectionString(ctx, "sslmode=disable")
    if err != nil {
        t.Fatal(err)
    }
    db, err := sql.Open("pgx", dsn)
    if err != nil {
        t.Fatal(err)
    }
    t.Cleanup(func() { db.Close() })

    runMigrations(t, db) // the SAME migrations production uses, not an inline copy

    store := NewPostgresUserStore(db)
    u := &User{Name: "Alice", Email: "alice@example.com"}
    if err := store.CreateUser(ctx, u); err != nil {
        t.Fatal(err)
    }
    got, err := store.GetUser(ctx, u.ID)
    if err != nil || got.Name != u.Name {
        t.Fatalf("got %+v, %v", got, err)
    }
    if err := store.CreateUser(ctx, u); !errors.Is(err, ErrConflict) {
        t.Fatalf("duplicate email: got %v, want ErrConflict", err) // the constraint mapping is what you're testing
    }
}

Start one container per package (TestMain) and isolate tests with a schema or transaction each. A container per test is correct but slow.

// ── 3. HTTP handlers: httptest, no network needed ──
func TestCreateUserHandler(t *testing.T) {
    h := NewUserHandler(NewUserService(newFakeStore()))
    req := httptest.NewRequest(http.MethodPost, "/users",
        strings.NewReader(`{"name":"Alice","email":"alice@example.com"}`))
    rec := httptest.NewRecorder()
    h.ServeHTTP(rec, req)
    if rec.Code != http.StatusCreated {
        t.Fatalf("status = %d, body = %s", rec.Code, rec.Body)
    }
    var got User
    if err := json.NewDecoder(rec.Body).Decode(&got); err != nil { // check decode errors too
        t.Fatal(err)
    }
}

// ── 4. Time and concurrency: testing/synctest (Go 1.25) ──
// Inside the bubble the clock is fake: time advances only when every goroutine
// in the bubble is blocked, so a 5-second timeout test runs in microseconds.
func TestTimeoutFires(t *testing.T) {
    synctest.Test(t, func(t *testing.T) {
        ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second)
        defer cancel()

        time.Sleep(4 * time.Second)
        synctest.Wait() // wait until all bubble goroutines are blocked
        if ctx.Err() != nil {
            t.Fatal("cancelled too early")
        }
        time.Sleep(time.Second)
        synctest.Wait()
        if ctx.Err() != context.DeadlineExceeded {
            t.Fatalf("got %v, want DeadlineExceeded", ctx.Err())
        }
    })
}

// ── 5. Fuzzing: properties, not examples ──
func FuzzParsePhone(f *testing.F) {
    for _, s := range []string{"+1-555-123-4567", "5551234567", "(555) 123-4567", "invalid"} {
        f.Add(s)
    }
    f.Fuzz(func(t *testing.T, in string) {
        p, err := ParsePhone(in)
        if err != nil {
            return
        }
        // Round-trip property: formatting then parsing gives the same number.
        p2, err := ParsePhone(p.Format())
        if err != nil || p2 != p {
            t.Fatalf("round trip %q → %q → %v, %v", in, p.Format(), p2, err)
        }
    })
}
// go test -fuzz=FuzzParsePhone -fuzztime=30s ; failures are saved to testdata/fuzz/ as regressions

Modern toolbox (all standard library):

Tool Since Use
t.Context(), t.Chdir() 1.24 Context cancelled at test end; temporary working dir
for b.Loop() { … } 1.24 Benchmarks that can't be optimized away and need no ResetTimer. Since 1.26 it no longer blocks inlining in the loop body
testing/synctest 1.25 (GA) Deterministic tests of timeouts, retries, tickers. synctest.Sleep helper in 1.27
httptest.NewTestServer 1.27 In-memory fake-network HTTP server that works inside a synctest bubble
T.ArtifactDir() 1.26 Directory for test output files (-artifacts flag)
Fuzzing 1.18 Parsers, decoders, anything taking untrusted input
go test -race, -shuffle=on, -count=1 – Catch races, order dependence, cache-masked flakes

Trade-offs: mocks generated from interfaces (gomock, mockery) test interactions and break on refactors. Fakes test behaviour. Use mocks only to assert calls that are important on their own ("we must not call the payment API twice"). E2E tests are slow and flaky: keep them few, and make them hermetic, with seeded data and no shared environment.

What they probe next: "How do you detect goroutine leaks in tests?" go.uber.org/goleak in TestMain. A synctest bubble fails if goroutines are still blocked when it ends. In production, the goroutineleak pprof profile (Go 1.27). "Flaky test policy?" Quarantine with an owner and a deadline, and fix the root cause. Usually it is real sleeps, shared state or port collisions.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Table-driven tests Subtests, error matching with errors.Is, isolated state per case
Interface-based mocking Prefers fakes; knows when interaction mocks are warranted
Testcontainers Real dependencies, module API, readiness done right, shared container per package
Parallel / time t.Parallel() without shared mutable state; synctest for time; -race, -shuffle

Question 12: Production Patterns — Graceful Shutdown, Middleware, and Observability

Interviewer: "Design a production HTTP service in Go with graceful shutdown, middleware chaining, structured logging, and metrics. Handle SIGTERM properly."

🎯 Expected Answer

30-second answer: Use signal.NotifyContext for SIGTERM/SIGINT. On signal, fail readiness first (not liveness), keep serving while the load balancer removes the pod, then call server.Shutdown(ctx) with a deadline shorter than the platform's grace period. Shutdown stops accepting connections and waits for in-flight requests. Then flush telemetry and exit. Middleware is func(http.Handler) http.Handler, composed with recovery outermost. Always set ReadHeaderTimeout. Use slog for logs and Prometheus or OpenTelemetry for metrics. Never use a hand-rolled map of durations.

type Middleware func(http.Handler) http.Handler

// Chain(a, b, c)(h) == a(b(c(h))): the first middleware is the outermost.
func Chain(mws ...Middleware) Middleware {
    return func(h http.Handler) http.Handler {
        for i := len(mws) - 1; i >= 0; i-- {
            h = mws[i](h)
        }
        return h
    }
}

type statusRecorder struct {
    http.ResponseWriter
    status int
}

func (r *statusRecorder) WriteHeader(code int) { r.status = code; r.ResponseWriter.WriteHeader(code) }

// Unwrap lets http.ResponseController (Go 1.20) reach Flush/Hijack/deadlines
// on the real writer: wrappers otherwise silently break streaming and SSE.
func (r *statusRecorder) Unwrap() http.ResponseWriter { return r.ResponseWriter }

func Logging(log *slog.Logger) Middleware {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            start := time.Now()
            rec := &statusRecorder{ResponseWriter: w, status: http.StatusOK}
            next.ServeHTTP(rec, r)
            log.LogAttrs(r.Context(), slog.LevelInfo, "request",
                slog.String("method", r.Method),
                slog.String("route", r.Pattern), // Go 1.23: matched pattern, e.g. "GET /hello/{name}". Low cardinality
                slog.Int("status", rec.status),
                slog.Duration("dur", time.Since(start)))
        })
    }
}

func Recover(log *slog.Logger) Middleware {
    return func(next http.Handler) http.Handler {
        return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
            defer func() {
                if v := recover(); v != nil {
                    if v == http.ErrAbortHandler { // deliberate abort: let net/http handle it
                        panic(v)
                    }
                    log.Error("panic", "value", v, "stack", string(debug.Stack()))
                    http.Error(w, "internal error", http.StatusInternalServerError) // no-op if headers already sent
                }
            }()
            next.ServeHTTP(w, r)
        })
    }
}

func run(ctx context.Context, log *slog.Logger, addr string, drainDelay time.Duration) error {
    var ready atomic.Bool
    mux := http.NewServeMux()
    // Liveness: "is the process wedged?" Keep it trivial; failing it makes the kubelet RESTART you.
    mux.HandleFunc("GET /livez", func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusOK) })
    // Readiness: "send me traffic?" This is the one to fail during shutdown or overload.
    mux.HandleFunc("GET /readyz", func(w http.ResponseWriter, _ *http.Request) {
        if !ready.Load() {
            http.Error(w, "draining", http.StatusServiceUnavailable)
            return
        }
        w.WriteHeader(http.StatusOK)
    })
    mux.HandleFunc("GET /hello/{name}", func(w http.ResponseWriter, r *http.Request) { // method+wildcard patterns: Go 1.22
        fmt.Fprintf(w, "hello %s", r.PathValue("name"))
    })

    srv := &http.Server{
        Addr:              addr,
        Handler:           Chain(Logging(log), Recover(log))(mux), // Logging outermost: a panic becomes a logged 500, not a missing log line
        ReadHeaderTimeout: 5 * time.Second,  // Slowloris protection: the one timeout you must set
        ReadTimeout:       15 * time.Second,
        WriteTimeout:      30 * time.Second, // per response; streaming endpoints extend it via ResponseController
        IdleTimeout:       120 * time.Second,
        // Don't set BaseContext to the signal ctx: in-flight requests would be
        // cancelled the instant SIGTERM arrives, which defeats draining.
    }

    errCh := make(chan error, 1)
    go func() { errCh <- srv.ListenAndServe() }()
    ready.Store(true)

    select {
    case err := <-errCh:
        return err // failed to bind, etc.
    case <-ctx.Done():
    }

    ready.Store(false)     // 1. fail readiness → endpoints controller removes the pod
    time.Sleep(drainDelay) // 2. keep serving while that propagates to kube-proxy / LB (or use a preStop sleep)
    shutdownCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 20*time.Second)
    defer cancel()
    if err := srv.Shutdown(shutdownCtx); err != nil { // 3. close listeners, finish in-flight requests
        return errors.Join(err, srv.Close())          //    deadline hit: force-close what's left
    }
    if err := <-errCh; !errors.Is(err, http.ErrServerClosed) {
        return err
    }
    return nil // 4. caller flushes traces/metrics, closes DB pools, exits 0
}

func main() {
    log := slog.New(slog.NewJSONHandler(os.Stdout, nil))
    ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
    defer stop()
    if err := run(ctx, log, ":8080", 5*time.Second); err != nil {
        log.Error("server exited", "err", err)
        os.Exit(1)
    }
}

This compiles and passes a test that checks /readyz returns 503 during the drain window and that run returns nil after a clean shutdown.

The shutdown timeline (Kubernetes): SIGTERM and endpoint removal happen concurrently. Requests can therefore still arrive for a few seconds after SIGTERM. Shutting the listener immediately turns them into connection-refused errors. Budget drainDelay + Shutdown timeout + cleanup below terminationGracePeriodSeconds, which defaults to 30 s. Shutdown does not wait for hijacked connections (WebSockets) or background goroutines. Use srv.RegisterOnShutdown and your own WaitGroup for those. Don't catch SIGQUIT: its default action dumps all goroutine stacks, which you want when a process hangs.

Observability: - Logs: log/slog with JSON output and request or trace IDs taken from ctx. slog.NewMultiHandler (1.26) writes to several handlers at once. - Metrics: Prometheus client_golang or the OpenTelemetry SDK. Use histograms with fixed buckets for latency. Label by route pattern, never raw path (unbounded cardinality). An in-process map that appends every duration grows without bound: it is a memory leak, not a metric. - Profiling: net/http/pprof on a separate admin port, never the public one. Continuous profiling (Pyroscope, Parca, cloud profilers) feeds PGO (default.pgo, GA since 1.21), for which the Go team reports around 2–14 % better performance on representative programs. - Rate limiting: per client key, with golang.org/x/time/rate (token bucket, Wait(ctx) or Allow()). Return 429 with Retry-After. A single global limiter lets one noisy client starve everyone.

What they probe next: "Request timeouts vs server timeouts?" http.TimeoutHandler or a per-request ctx deadline for handler time. WriteTimeout is for the connection. "Load shedding?" Use a concurrency limit (semaphore) and fail fast with 503 rather than queuing. "Zero-downtime deploys?" Readiness gates, maxUnavailable: 0, and connection draining as above.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Graceful shutdown NotifyContext, readiness-before-shutdown, drain delay, bounded Shutdown, grace-period budget
Middleware chain Correct order, Unwrap/ResponseController, ErrAbortHandler re-panic
Observability slog, histograms by route pattern, pprof on admin port, tracing context
Production readiness ReadHeaderTimeout, liveness vs readiness, per-client rate limits, load shedding

Graceful shutdown on Kubernetes: fail readiness first, keep serving while endpoints update, then Shutdown with a bounded deadline.

sequenceDiagram
    participant K as Kubernetes
    participant S as Service process
    participant LB as Load balancer
    K->>S: SIGTERM
    K->>LB: remove pod from endpoints (concurrent)
    S->>S: ready = false, /readyz returns 503
    Note over S,LB: drainDelay: still serving while removal propagates
    S->>S: srv.Shutdown(ctx with deadline)
    Note over S: stop accepting, wait for in-flight requests
    alt deadline hit
        S->>S: srv.Close() force-closes the rest
    end
    S->>S: flush telemetry, close DB pools
    S-->>K: exit 0 before terminationGracePeriodSeconds

Question 13: Modern Go (1.21–1.27) — Loop Variables, Iterators, and What Changed

Interviewer: "Old Go advice says to write v := v inside loops. Is that still true? Then show me a custom iterator and tell me what else changed recently that affects production code."

🎯 Expected Answer

30-second answer: Since Go 1.22, every for loop iteration creates new loop variables. This applies to range loops and three-clause loops alike, so the closure-captures-the-loop-variable bug is gone and v := v is dead code. The change is gated by the go line in go.mod: a module that declares go 1.21 or lower still gets the old shared-variable semantics, even when built with Go 1.27. Since Go 1.23, range also accepts iterator functions (iter.Seq), which lets custom collections plug into for … range.

Loop variables, precisely:

var prints []func()
for i := 0; i < 3; i++ {
    prints = append(prints, func() { fmt.Print(i, " ") })
}
for _, p := range prints {
    p()
}
// go.mod says go 1.22+ : 0 1 2
// go.mod says go 1.21  : 3 3 3   (one i shared by all closures; verified with Go 1.27 toolchain)
  • The semantics follow the module's go version, not the toolchain version. Per-file //go:build go1.22 lines can also opt in.
  • go fix (rewritten in 1.26 around "modernizers") removes now-redundant v := v copies. go vet's loopclosure check only fires for pre-1.22 modules.
  • Performance: the compiler gives each iteration a fresh variable only when a variable actually escapes (captured, address taken). Otherwise nothing changes.
  • An old snippet whose answer was "this prints 3 3 3" (or "all goroutines see the last value") is wrong for modern modules. Say "it did before Go 1.22".

Range-over-func iterators (Go 1.23):

// iter.Seq[V] is just func(yield func(V) bool). The loop body becomes yield;
// yield returns false when the caller breaks, and the iterator MUST then stop.
func Filter[T any](seq iter.Seq[T], keep func(T) bool) iter.Seq[T] {
    return func(yield func(T) bool) {
        for v := range seq {
            if keep(v) && !yield(v) {
                return
            }
        }
    }
}

type Tree[T any] struct {
    Left, Right *Tree[T]
    Val         T
}

// In-order traversal as an iterator: recursion is easy because yield is a callback.
func (t *Tree[T]) All() iter.Seq[T] {
    return func(yield func(T) bool) { t.push(yield) }
}
func (t *Tree[T]) push(yield func(T) bool) bool {
    if t == nil {
        return true
    }
    return t.Left.push(yield) && yield(t.Val) && t.Right.push(yield)
}

func main() {
    evens := Filter(slices.Values([]int{1, 2, 3, 4, 5, 6}), func(n int) bool { return n%2 == 0 })
    for n := range evens {
        if n > 4 {
            break // yield returns false; Filter stops its upstream loop
        }
        fmt.Print(n, " ") // 2 4
    }
    fmt.Println()
    tr := &Tree[int]{Val: 2, Left: &Tree[int]{Val: 1}, Right: &Tree[int]{Val: 3}}
    fmt.Println(slices.Collect(tr.All()))       // [1 2 3]
    m := map[string]int{"b": 2, "a": 1}
    fmt.Println(slices.Sorted(maps.Keys(m)))    // [a b]: deterministic order from a map

    next, stop := iter.Pull(tr.All())            // convert push → pull (e.g. to zip two sequences)
    defer stop()                                 // ALWAYS stop, or the iterator's goroutine-like state leaks
    v, ok := next()
    fmt.Println(v, ok)                           // 1 true
}
  • Push iterators (iter.Seq, iter.Seq2[K,V]) are the default: cheap, and the compiler can inline them. Pull (iter.Pull) is for consuming two sequences in lockstep. It costs more (a coroutine switch per element).
  • If an iterator keeps calling yield after it returned false, the program panics. Defers inside the iterator run when the loop exits. A panic in the loop body propagates through the iterator.
  • Standard library: slices.All/Values/Backward/Collect/Sorted/Chunk, maps.Keys/Values/All/Collect, strings.Lines/SplitSeq/FieldsSeq (1.24), reflect.Type.Fields (1.26).

Other changes that affect production code:

Version Change Why it matters
1.21 min, max, clear built-ins; slices, maps, log/slog; context.AfterFunc/WithoutCancel; PGO GA Standard library replaces many helper packages
1.22 Per-iteration loop vars; range over ints; math/rand/v2; ServeMux method + wildcard patterns Fewer third-party routers; v2 rand has rand.N, ChaCha8/PCG, no Seed
1.23 Iterators; unique (interning); timers: unstopped, unreferenced time.Timer/Ticker are now GC-collectable, and their channels are unbuffered so Reset/Stop never deliver stale values time.After in a loop no longer leaks until it fires
1.24 Generic type aliases; Swiss-table map; sync.Map → hash-trie; weak, runtime.AddCleanup; os.Root; testing.B.Loop; go.mod tool directives; encoding/json omitzero Faster maps (lower CPU, better memory use at large sizes), safer file access within a directory
1.25 Container-aware GOMAXPROCS; sync.WaitGroup.Go; testing/synctest GA; runtime/trace.FlightRecorder; Green Tea GC experiment; a compiler fix for nil checks wrongly delayed since 1.21 Fixes CPU throttling in containers. Code that used a result before checking err now panics correctly
1.26 Green Tea GC default; new(expr) (new(42), new(f())); self-referential generic constraints; errors.AsType; cgo calls ~30 % cheaper; go fix modernizers; goroutine-leak profile (experiment) 10–40 % less GC CPU on GC-heavy workloads
1.27 Generic methods; encoding/json/v2 + jsontext standard, with encoding/json running on the v2 engine; goroutineleak profile GA; size-specialized small allocations; asynctimerchan GODEBUG removed; new uuid package JSON unmarshal significantly faster; leaks detectable in production

Swiss-table maps (1.24): the map is now a set of open-addressing tables. Each table is split into groups of 8 slots plus 8 one-byte control words, holding 7 bits of hash each. A lookup compares all 8 control bytes at once (SIMD on amd64), then checks only the candidate slots. Tables grow independently, so one huge map never pays a single giant rehash. What doesn't change: iteration order is still randomized, maps are still not safe for concurrent writes, and &m[k] is still illegal.

What they probe next: "How do you upgrade safely?" Bump the go line deliberately, run go fix ./..., and run tests with -race. Check the GODEBUG defaults for that version: the godebug block in go.mod can pin old behaviour for one setting while you migrate. Read the release notes' "Ports" and "Removed GODEBUG" sections.

🔍 Staff-Level Evaluation

Criterion What I'm Looking For
Loop variables Knows the 1.22 change, that it's gated by the go.mod version, and which old answers are now wrong
Iterators Writes a correct iter.Seq that honours yield's return; knows push vs pull
Currency Can name the production-relevant changes from 1.21–1.27 and why they matter
Upgrade discipline go.mod version, GODEBUG pinning, go fix, release-note reading

📊 Staff-Level Evaluation Rubric

Score What It Looks Like
5 — Exceptional Cites Go source code (runtime/proc.go, runtime/chan.go), references commits/versions, has shipped production workarounds for GC/goroutine issues. Discusses trade-offs without prompting.
4 — Strong Deep understanding of GMP, GC phases, channel internals, memory model. Can implement lock-free patterns, design production services. Knows the tooling (pprof, trace, race detector).
3 — Competent Good Go programmer. Knows goroutines, channels, interfaces, error handling. But doesn't understand scheduling internals or GC pacing deeply.
2 — Developing Proficient with Go syntax but doesn't understand why things work. No production experience at scale.
1 — Needs Growth Can write basic Go but doesn't understand concurrency patterns, interface satisfaction, or the toolchain.

Built for experienced Go engineers targeting Staff/Principal roles at top-tier companies