Skip to content
samit
Interviews/Resume Deep Dive

Resume Deep Dive

16 questions

16 questions
Beltech AI - Production CV System
You replaced anchor-based YOLO with NMS-free YOLOv10 and cut latency 54%. What is NMS-free detection and why is it faster?frontier▶
Beltech AI

Traditional YOLO (v3-v8): each anchor at each scale predicts a box - one object produces 3-9 redundant candidates. NMS (Non-Maximum Suppression) de-duplicates post-inference: keep max-confidence box, suppress overlapping boxes with IoU > threshold. NMS is sequential, CPU-bound, ~15-30ms on edge hardware.

YOLOv10 - dual head during training:

  • One-to-many head: standard multi-anchor head used only during training for rich supervision signal.
  • One-to-one head: trained simultaneously via Hungarian matching + consistency distillation from the one-to-many head. Learns to predict exactly you confident box per object.

Inference: discard the one-to-many head. The one-to-one head outputs one box per object - no NMS needed. Post-processing is O(1).

Why 54%: NMS on CPU with 50+ candidates takes 20-40ms on Jetson-class hardware. Removing it was the bottleneck. Model FLOP count is essentially unchanged.

Follow-up interviewers ask: How does the one-to-one head avoid predicting multiple overlapping boxes without NMS? Hungarian assignment during training forces strict one-to-one matching between predictions and GT boxes - the model learns to commit to one prediction rather than hedge across anchors.

How does TensorRT optimize inference? Walk through the full pipeline.frontier▶
Beltech AI

TensorRT takes ONNX → hardware-optimized.engine file via 4 steps:

  1. Layer fusion: Conv + BatchNorm + ReLU → single CUDA kernel (1 launch instead of 3). Each kernel launch costs ~5-20μs. Fusion also eliminates intermediate tensor writes to slow HBM.
  2. Kernel autotuning: benchmarks multiple CUDA implementations (cuBLAS, cutlass, custom GEMM) for each op on target GPU. Selects fastest. Engine is GPU-model specific - an A100 engine won't run on RTX 3090.
  3. Precision calibration: INT8 calibration runs representative data, finds per-tensor scale factors minimizing quantization error. Calibrators: MinMax, Entropy, Percentile.
  4. Memory planning: analyzes tensor lifetimes, reuses buffers for tensors that don't overlap in time. Reduces peak VRAM 20-40%.

Beltech pipeline: RTSP frame → GPU memcpy → CUDA preprocess (resize/normalize) → TensorRT engine → CUDA decode (box conversion) → results. No CPU in the hot path after initial upload. This is why it hit ≤10ms.

Critical gotcha:.engine files are GPU-model specific. Always build on the target hardware at deployment time.

How does Triton Inference Server handle concurrent requests? How does it compare to Flask/FastAPI?frontier▶
Beltech AI

Triton's key capabilities over Flask:

  • Dynamic batching: accumulates requests within a configurable window (max_queue_delay_microseconds), batches them → 1 GPU forward pass instead of N sequential. Critical for throughput.
  • Instance groups: count: N in config.pbtxt → N model replicas share GPU. Each handles one request concurrently.
  • Model ensemble: preprocessing → model → postprocessing all run GPU-side via shared memory. Zero CPU overhead between stages.
  • Backend abstraction: TensorRT, ONNX, PyTorch TorchScript, OpenVINO - same gRPC/HTTP API.
  • Built-in perf_analyzer: measure optimal concurrency and batch size before deploying.

Flask/FastAPI limitation: one request per worker, no native GPU batching, no queue management. At 100 RPS with 10ms inference, Flask needs 100 workers - can't manage GPU contention efficiently.

Zero-downtime swap: Triton supports loading a new model version while old version serves traffic, then atomic swap. This is how the continuous learning pipeline deployed with zero downtime.

Walk through the continuous learning pipeline. How do you handle data quality and catastrophic forgetting?▶
Beltech AI
  1. Data capture: inference service logs low-confidence predictions (<0.4) and temporal inconsistencies (box disappears mid-scene). Frames + metadata → object storage staging bucket.
  2. Annotation (2-week cadence): high-confidence captures → auto-labeled by teacher model. Uncertain examples → active learning queue → human reviewers.
  3. Dataset merge: new annotations + curated training set. Dedup by perceptual hash. Rebalance class distribution.
  4. Retrain from checkpoint - not from scratch, preserves prior class knowledge. 5-10 epochs on combined dataset.
  5. Quality gate: eval on fixed held-out validation set. mAP50 ≥ previous_best × 0.98 → proceed. Else → reject, alert team.
  6. Blue-green deploy: load candidate as model-B in Triton while model-A serves traffic. Route 5% → monitor 30 min → promote to 100% → unload A.

Hard problems: (1) Failure distribution ≠ class distribution - failures cluster in edge cases, biasing retraining. Fix: replay buffer with 20% random old data mixed in. (2) Catastrophic forgetting on head classes when fine-tuning on tail-heavy new data. Fix: EWC regularization or replay. (3) Annotation drift between annotators/rounds. Fix: inter-annotator agreement threshold + calibration sessions.

Explain DeepSORT. What does the "deep" add over SORT? What is the Mahalanobis distance term?▶

SORT (baseline): detections → Kalman filter prediction → Hungarian algorithm assigns detections to tracks by IoU. Fast but loses tracks during occlusion (IoU drops to 0).

DeepSORT adds:

  • Appearance re-ID CNN: lightweight network extracts 128-dim L2-normalized embedding per detection crop.
  • Combined association metric: cost = λ·d_motion + (1-λ)·d_appearance

Mahalanobis distance (motion): d_motion = (d - ŷ)^T · S^{-1} · (d - ŷ) where d = detection, ŷ = Kalman predicted state, S = prediction covariance. Unlike Euclidean, Mahalanobis accounts for tracker uncertainty - when Kalman is uncertain (wide covariance), it tolerates more position mismatch. Gating: reject associations where d_motion > χ²-threshold.

Cosine distance (appearance): minimum cosine distance to any of the last 100 embeddings stored per track. Allows re-identification after occlusion.

Production optimization: re-ID CNN costs ~5ms/frame. Run only when IoU-based matching fails. For vehicles (less diverse appearance than pedestrians), reduce embedding dim to 64 and only use appearance when IoU < 0.3.

mAP50 vs mAP50-95 - when does each matter? Which did you optimize at Beltech and why?▶

IoU(pred, gt) measures box overlap. IoU threshold determines TP/FP: if IoU > threshold → TP, else FP.

  • mAP50: mean AP at IoU=0.50. Lenient - rough localization OK. Pascal VOC standard. Well-trained YOLO typically 0.85-0.95.
  • mAP50-95: mean AP averaged over IoU = {0.50, 0.55,..., 0.95}. COCO standard. Penalizes poor box quality. Same model might be mAP50=0.90 but mAP50-95=0.55 if boxes are loose.

Beltech use case (surveillance): mAP50 is appropriate. LPR/counting/wrong-way detection needs "is there a vehicle approximately here?" Pixel-perfect boundaries don't change system outcome. Optimizing mAP50-95 adds training cost with no production benefit.

When mAP50-95 matters: autonomous driving (pedestrian safety margins), medical imaging (tumor volume), satellite imagery (building footprint area).

Resume note: the 0.9+ F1 likely refers to classification F1 (is_phone / is_helmet), not box-level mAP. These are different metrics on different tasks - be ready to clarify this distinction in interviews.

MakeMyTrip - Recommendation & Bandits
Compare epsilon-greedy, UCB, and Thompson Sampling. Which suits high-traffic personalization?▶

Epsilon-greedy: explore with prob ε (random arm), exploit with 1-ε (best arm). Simple. Decaying ε is common. Problem: treats all non-optimal arms identically in exploration - wastes pulls on clearly bad arms.

UCB1 (Upper Confidence Bound): i* = argmax [μ̂_i + √(2 ln t / n_i)]. Exploration bonus = confidence interval width, shrinks as arm is pulled more. O(log T) regret. Deterministic, no tuning, principled.

Thompson Sampling: maintain Beta(α_i, β_i) per arm. Sample θ_i ~ Beta, pick argmax. Update: reward=1 → α_i++, reward=0 → β_i++.

  • Matches UCB regret bounds empirically, often outperforms in practice
  • Handles reward variance naturally - uncertain arms get wider posteriors → more exploration
  • Extends to contextual bandits (LinTS) for user-feature-dependent personalization
  • Non-stationarity: add temporal decay to beta parameters for concept drift

For MakeMyTrip (high traffic, rich user context): Contextual Thompson Sampling (LinTS). User features (destination, device, trip type, season) feed into a linear reward model with Bayesian posterior. Each user is different - simple bandit ignores this and leaves money on the table.

What is the exploration-exploitation tradeoff? How does it compare to bias-variance?▶

Explore-exploit: at each step choose between learning more about uncertain options (explore) vs. using the currently best-known option (exploit). Regret = Σ(μ* - μ_{a_t}).

Too much exploration: low immediate reward, better long-term decisions. Too much exploitation: stuck with suboptimal choices forever. UCB and Thompson Sampling balance this via principled uncertainty quantification.

Connection to bias-variance:

  • High exploitation = biased estimator (biased toward incumbent best arm)
  • High exploration = high variance in decisions but unbiased estimates of all arms

Key structural difference: bias-variance is a one-shot estimation problem. Explore-exploit is a sequential decision problem where each choice changes what you know, which changes future choices. You can't decouple exploration from exploitation without losing regret guarantees.

Non-stationarity: real recommendation has concept drift - trending destinations change daily. Fix: sliding window UCB or discounted Thompson Sampling (multiply α, β by γ < 1 per round to forget old data).

Walk through how you trained the RLHF reward model at MakeMyTrip. What data, what loss, what did it output?startup▶

The goal was to rank itinerary recommendations - so the reward model needed to capture human preference, not a proxy like click-through rate.

Data: pairs of itineraries annotated with user preference labels (A preferred over B). Collected from user interaction logs and manual annotation.

Architecture: a pretrained encoder (text representation of itinerary features) with a linear head producing a scalar reward score. Trained with Bradley-Terry pairwise ranking loss: for a preferred item A and rejected item B, maximise log σ(r_A − r_B).

Output: a scalar score for any itinerary - used downstream as the reward signal to rank candidates. Not a language model RM (no token-level reward), just preference ranking over structured travel items.

Note: connect this to modern RLHF - the same Bradley-Terry pairwise loss is used in LLM reward models (InstructGPT, Claude). The difference: LLM RMs operate on token sequences, yours operated on structured item features. Same principle, different domain.

How did you build the NLP-based resume screening pipeline at MakeMyTrip? What signals did you use?startup▶

The goal was to help the Data team filter engineering resumes at volume - identifying candidates with relevant ML/data skills.

Text similarity: TF-IDF or dense embeddings (sentence-transformers) to compute cosine similarity between a resume and a target job description. Candidates above a threshold pass the first stage.

Keyword extraction: frequency-based or RAKE/KeyBERT to identify the most salient technical terms (frameworks, methods, tools). Matched against a curated skills dictionary for the role.

Pipeline: parse resume (PDF → text via pdfplumber/pdfminer), preprocess (lemmatise, remove stopwords), embed or TF-IDF, score against JD, rank candidates.

Gotcha to prepare for: interviewers often ask about bias and fairness here - a keyword-matching system can penalise non-standard resume formats, unconventional career paths, or different naming conventions for the same skill. Have a sentence ready on limitations.

MiniTorch & MathLM - Build From Scratch
Explain reverse-mode autodiff. How does topological sort guarantee correct gradient computation?frontier▶

Reverse-mode AD computes ∂L/∂x_i for ALL parameters in ONE backward pass. Cost = O(forward pass) regardless of number of parameters. This is what makes training 175B parameter models tractable.

Forward pass builds a computation DAG: each op creates a node storing (1) output value, (2) references to input nodes, (3) the local VJP (vector-Jacobian product) - the backward rule for that op.

Why topological sort: node A's gradient = Σ_{children C} (upstream gradient from C × local Jacobian A→C). You can ONLY compute A's gradient AFTER all its children have computed their gradients and accumulated into A.grad. Topological sort on the DAG ensures children appear after parents. Reverse topological order → process children before parents → every accumulation is done before the node's _backward is called.

Forward vs reverse mode: forward computes one column of the Jacobian per pass (one input's sensitivity). Reverse computes one row per pass (all sensitivities for one output). For scalar loss and millions of params: reverse is O(n_params × forward) cheaper than forward.

python
# Minimal reverse-mode autodiff (micrograd-style)
class Value:
    def __init__(self, data, _prev=()):
        self.data = float(data)
        self.grad = 0.0
        self._backward = lambda: None
        self._prev = set(_prev)

    def __mul__(self, other):
        other = other if isinstance(other, Value) else Value(other)
        out = Value(self.data * other.data, (self, other))
        def _bwd():
            self.grad += other.data * out.grad  # product rule
            other.grad += self.data * out.grad  # += because diamond paths in DAG
        out._backward = _bwd
        return out

    def __add__(self, other):
        other = other if isinstance(other, Value) else Value(other)
        out = Value(self.data + other.data, (self, other))
        def _bwd(): self.grad += out.grad; other.grad += out.grad
        out._backward = _bwd
        return out

    def relu(self):
        out = Value(max(0, self.data), (self,))
        def _bwd(): self.grad += (out.data > 0) * out.grad
        out._backward = _bwd
        return out

    def backward(self):
        topo, seen = [], set()
        def build(v):
            if v not in seen:
                seen.add(v)
                for c in v._prev: build(c)
                topo.append(v)  # post-order: children before parents
        build(self)
        self.grad = 1.0
        for v in reversed(topo): v._backward()  # reversed: parents after children

    __rmul__ = __mul__; __radd__ = __add__
Walk through the BPE tokenization algorithm. How does it differ from WordPiece?▶

BPE (Byte Pair Encoding):

  1. Initialize vocabulary = all unique bytes in corpus (256 for byte-level BPE)
  2. Count all adjacent token pairs across entire corpus
  3. Find most frequent pair (A, B)
  4. Create new token "AB", replace all (A, B) in corpus with "AB", add to vocab
  5. Repeat until vocab reaches size V (e.g., 50257 for GPT-2, 32000 for LLaMA-2)

Common words become single tokens. Rare words split into subwords. Byte-level BPE never fails on OOV - every input maps to valid bytes.

WordPiece (BERT): merge criterion is likelihood-based. Merge (A,B) that maximizes P(corpus | vocabulary after merge). Continuation tokens prefixed with ##. Used in BERT, DistilBERT, multilingual models.

SentencePiece (LLaMA): language-agnostic, treats raw bytes without pre-tokenization. Handles CJK, Arabic, Devanagari without word boundary assumptions. Word-initial subwords get ▁ prefix. LLaMA-3 uses SentencePiece BPE with 128K vocab.

python
# Complete BPE training from scratch
def get_stats(ids):
    counts = {}
    for pair in zip(ids, ids[1:]):
        counts[pair] = counts.get(pair, 0) + 1
    return counts

def merge(ids, pair, new_id):
    out, i = [], 0
    while i < len(ids):
        if i < len(ids)-1 and ids[i]==pair[0] and ids[i+1]==pair[1]:
            out.append(new_id); i += 2
        else:
            out.append(ids[i]); i += 1
    return out

def train_bpe(text, vocab_size=500):
    ids = list(text.encode("utf-8"))   # byte-level init: 256 tokens
    merges = {}
    for i in range(vocab_size - 256):
        stats = get_stats(ids)
        if not stats: break
        pair = max(stats, key=stats.get)
        new_id = 256 + i
        ids = merge(ids, pair, new_id)
        merges[pair] = new_id
    return merges

# Karpathy's minBPE uses exactly this structure
GQA vs MHA vs MQA - tradeoffs and KV cache memory math for LLaMA-3 70B.frontier▶

MHA: H query heads, H key heads, H value heads. Full expressiveness. KV cache: 2 × L × H × d_head × dtype_bytes per layer.

MQA: H queries, 1 shared K, 1 shared V. 1/H the KV cache. Quality degrades on complex reasoning (too aggressive sharing). Used in PaLM.

GQA (LLaMA-3, Mistral): H queries, G key/value heads (G divides H). Groups of H/G queries share one K,V pair. Near-MHA quality at 1/G the KV cache.

LLaMA-3 70B numbers:

  • H=64 query heads, G=8 KV heads, d_head=128, L=8192, batch=1, bf16 (2 bytes)
  • KV cache per layer = 2 × 8192 × 8 × 128 × 2 = 33.6 MB
  • 80 layers total = 2.7 GB for a single 8K-context sequence
  • With MHA (H=64): 2.7 × 8 = 21.5 GB - fills an entire A100-80GB just for KV cache!

Implementation: Q: (B,H,L,dk). K,V: (B,G,L,dk). Expand K,V by repeating each group H/G times before standard attention.

python
# GQA in PyTorch
import torch

def gqa(Q, K, V, mask=None):
    """
    Q: (B, H, L, dk)   # H query heads
    K: (B, G, L, dk)   # G < H key/value heads
    V: (B, G, L, dv)
    """
    B, H, L, dk = Q.shape
    G = K.shape[1]
    n_rep = H // G  # queries per KV group

    # Expand K and V to match query head count
    K = K.unsqueeze(2).expand(B, G, n_rep, L, dk).reshape(B, H, L, dk)
    V = V.unsqueeze(2).expand(B, G, n_rep, L, dk).reshape(B, H, L, dk)

    # Standard scaled dot-product attention
    scores = Q @ K.transpose(-2, -1) / dk**0.5
    if mask is not None:
        scores = scores.masked_fill(mask == 0, float('-inf'))
    return torch.softmax(scores, dim=-1) @ V
What is SwiGLU? Why do modern LLMs prefer it over ReLU for FFN blocks?frontier▶

Standard FFN: FFN(x) = max(0, xW₁)W₂. ReLU activation. Two weight matrices.

GLU (Gated Linear Unit, 2016): FFN(x) = (xW) ⊙ σ(xV). Sigmoid gate filters the linear projection.

SwiGLU: replace sigmoid gate with Swish(x) = x·σ(x):

FFN(x) = (xW1 * swish(xW2)) @ W3

Why better than ReLU:

  • No dead neurons: Swish is smooth, non-zero gradient everywhere. ReLU has zero gradient for x<0.
  • Adaptive gating: gate (Swish term) learns per-input, per-feature control of information flow. Each forward pass has different gating - dynamic computation vs. fixed topology of ReLU.
  • Empirical wins: PaLM paper (2022) tested SwiGLU vs ReLU, GELU, GLU across all scales - SwiGLU won consistently. Now standard in LLaMA 1/2/3, Mistral, PaLM, Gemma, MathLM.

Parameter overhead: 3 weight matrices instead of 2. Fix: set d_ff = (8/3)×d_model instead of 4×d_model. Same total params, different allocation. LLaMA-3 8B: d_model=4096, d_ff=14336 ≈ (8/3)×4096.

SFT loss masking on assistant tokens - what is it and what breaks without it?▶

Problem: an SFT training sample contains system prompt + user message + assistant response. Cross-entropy over the entire sequence penalizes the model for not predicting the user's question - wrong objective.

Implementation:

labels = input_ids.clone
labels[:, :response_start_idx] = -100 # mask system + user tokens
loss = F.cross_entropy( logits.view(-1, vocab_size), labels.view(-1), ignore_index=-100 # PyTorch skips this index in gradient computation
)

Without masking:

  • Gradient signal dominated by system/user token prediction (often longer)
  • Model learns to predict human-written prompts, not generate responses
  • Training loss appears artificially low (easy-to-predict template text dominates)
  • Model generates text that looks like conversation starters, not completions

Gotcha in MathLM: response_start_idx varies per example in a batch. You need dynamic per-sample masking - find the special assistant-start token position per example, create a mask, then apply it before computing loss.

Explain Sparse MoE routing: Top-K selection, load balancing loss, expert capacity.frontier▶

Architecture: replace each FFN block with E expert FFNs. Router selects K experts per token. Output = weighted sum of selected expert outputs. Active params per token ≈ K/E × total FFN params, but total model capacity = E × a dense model.

Router: g(x) = softmax(xW_r) ∈ ℝ^E. Select Top-K indices by value. Route token to K experts; output = Σ g_i(x) · Expert_i(x) for selected i (normalized).

Load collapse problem: without regularization, router routes all tokens to 1-2 "popular" experts. Other experts never train → wasted parameters.

Auxiliary load balancing loss (Switch Transformer / GShard):

# f_i = fraction of tokens to expert i (stop_gradient)
# P_i = mean router prob for expert i (differentiable)
L_aux = alpha * sum(f_i * P_i for i in range(E))
# Minimized when f_i = P_i = 1/E (uniform dispatch)
total_loss = task_loss + alpha * L_aux # alpha = 0.01-0.1

Expert capacity: each expert processes at most capacity_factor × (batch_tokens/E) tokens. Overflow tokens are dropped or passed through unchanged (no expert). CF=1.25 is typical.

DeepSeek fine-grained MoE: E=64, K=6 instead of E=8, K=2. Finer granularity → better expert specialization. Same active compute (≈6× dense FFN) but much larger model capacity. the MathLM likely used E=8, K=2.

← Previous

Reading List

Next →

Karpathy Questions