Speech AI Interview Sprint

Split real-time captioning from async summarization. Use VAD, streaming ASR, diarization, punctuation, chunk storage, redaction, retrieval over sanitized meeting chunks, and an answer service with citations. Track first partial latency, final WER, speaker-attributed WER, summary factuality, action-item precision/recall, retrieval recall, p95 answer latency, tenant isolation, retention policy, and deletion propagation. Roll out by tenant and feature flag; rollback ASR, summarizer, retriever, and answer prompt independently.

Use cascaded ASR-LLM-RAG-TTS unless direct speech-to-speech is needed for a narrow domain and has stronger safety gates. Budget VAD, partial ASR, intent confidence, retrieval, tool calls, first token, first audio byte, and playback. Add barge-in events, echo cancellation, cancellation tokens, tool confirmation, handoff, and transcript repair. Evaluate task success, containment, groundedness, unsafe tool refusal, interruption recovery, first response latency, and cost per resolved call.

Isolate real-time pools from batch pools. Use workload classes, admission control, model registry, canary routing, autoscaling, queue-age SLOs, accelerator utilization, KV-cache or decoder memory budgets, warm pools for TTS, and backpressure. Track per-tenant cost, p95/p99 latency, queue age, error budget burn, saturation, retry storms, cold starts, model version, and rollback readiness. Never let eval backfills starve live speech traffic.

Do not launch globally from aggregate WER. Define slice gates for language, code-switching, SNR, microphone type, duration, region, entity error rate, confidence calibration, and first partial latency. Use traffic routing to keep regressed slices on the old model while canarying improved slices. Add a model card, known limitations, rollback trigger, shadow comparison, and post-launch drift watch.

CTC is simple and efficient but often needs decoding support and can be weaker for long context. RNN-T is strong for streaming partials and production dictation, with more complex training and decoding. Encoder-decoder models can use richer context but may be harder to stream tightly. Whisper-style weak supervision is robust offline and multilingual but may be costly and not ideal for ultra-low-latency partials. Match the family to latency, domain, data, decoding, and controllability constraints.

Cascades are easier to inspect, moderate, localize, retrieve over, log with privacy controls, and roll back by component. Direct models may reduce latency and preserve prosody but are harder to debug, evaluate, and constrain. A strong answer proposes an experiment with matched tasks, latency budgets, safety checks, grounding, turn-taking recovery, controllability, and a fallback path before replacement.

Start with routing and workload isolation before model surgery: smaller model for easy turns, cache retrieval and prompts, trim context, tune batching, use quantization where eval passes, and consider speculative decoding, GQA, FlashAttention, distillation, or LoRA only with slice-level quality gates. Watch p95 latency, first token latency, task success, hallucination, tool error, and cost per successful task.

Inspect partial churn, first stable token latency, endpointing, punctuation rewrites, timestamp jitter, client debounce behavior, decoder rescoring, and noisy or long-utterance slices. Mitigate with rollback, committed-prefix stabilization, or endpointing changes. Add streaming UX metrics to release gates because final WER hides partial instability.

Check autoscaler floor, warm replicas, batch jobs on live pools, shadow traffic, retry loops, longer prompts, disabled quantization, cache miss rate, tenant mix, and per-model utilization. Stabilize by isolating batch, restoring utilization targets, capping shadow traffic, and preserving SLOs. Add cost-per-success and utilization alerts, not just latency alerts.

Pin the previous index or filter stale namespaces. Compare document lineage, chunker version, embedding model, top-k overlap, freshness metadata, ASR query variants, and answer-grounding checks. Add stale document fixtures, freshness-aware ranking, citation validation, and index-release rollback to CI/CD.

Invariant: every slice is judged against its own configured budgets, and severity is normalized so different metric units can be sorted together. Test missing metrics, exactly-on-threshold values, lower-is-better metrics, higher-is-better metrics, and multiple failures in the same slice.

def rank_canary_risks(slices, gates):
    failures = []
    for slice_name, metrics in slices.items():
        for metric, gate in gates.items():
            if metric not in metrics:
                failures.append((float("inf"), slice_name, metric, "missing"))
                continue

            value = metrics[metric]
            direction = gate["direction"]
            threshold = gate["threshold"]

            if direction == "max":
                over = value - threshold
                if over > 0:
                    severity = over / max(abs(threshold), 1e-9)
                    failures.append((severity, slice_name, metric, value))
            elif direction == "min":
                under = threshold - value
                if under > 0:
                    severity = under / max(abs(threshold), 1e-9)
                    failures.append((severity, slice_name, metric, value))
            else:
                raise ValueError(f"unknown direction: {direction}")

    failures.sort(reverse=True)
    return [
        {"slice": s, "metric": m, "value": v, "severity": round(sev, 4)}
        for sev, s, m, v in failures
    ]

In the coding problem, sort intervals by start time and keep a min-heap of end times; the heap size is the number of rooms needed. In serving, requests or streams occupy accelerator memory and decode slots for an interval. The same idea estimates peak concurrent sessions, but production also needs variable token rates, queueing, batching, priority classes, and safety margin.

Word Search II uses a trie to reject impossible prefixes early and avoid exploring every path. Speech systems use similar prefix constraints for command grammars, contact names, entity biasing, or safety-sensitive tool names. The mistake is over-constraining so the decoder cannot recover from ASR noise or user paraphrases.