Speech ML System Design Interviews

Compare correction rate by entity-heavy turns, noise level, language, device, utterance length, consented or approved aggregate cohorts, and confidence. Inspect WER, entity error, timestamp drift, decoder beam settings, punctuation, and postprocessing. Keep the latency win only if the regression is within the launch budget or can be routed away from product-critical slices that are safe and approved to detect operationally. Otherwise pause or roll back.

Look beyond the voice model: sentence segmentation, LLM response length, first audio byte, playback buffer, client audio focus, echo cancellation, barge-in detector threshold, turn manager state, and cached responses. Rollback may need feature flags, cache invalidation, queue draining, and client config restoration.

Clarify latency and accuracy SLOs, supported devices, vocabulary update path, retention rules, and offline fallback. Propose client capture plus VAD, streaming ASR with partial stabilization, custom vocabulary or contextual biasing, punctuation restoration, edit history, privacy-safe traces, model registry, eval slices for domain terms, canary rollout, rollback, and dashboards for partial latency, finalization, correction rate, entity error, and cost per audio minute.

Use ASR, query rewrite, retrieval, reranking, grounded generation, citation policy, TTS, and feedback collection. Evaluate clean text queries, ASR hypotheses, noisy ASR hypotheses, retrieval recall, answer groundedness, refusal correctness, first audio byte, and end-to-end task success. Monitor ASR entity substitutions because one wrong name can break retrieval even when transcript WER looks acceptable.

Compare latency, natural prosody, controllability, transcript auditability, safety filters, retrieval integration, tool use, observability, eval maturity, data requirements, rollback, and cost. A good recommendation may be hybrid: use direct speech for low-risk conversational responses while keeping cascaded paths for tool use, regulated content, debugging, and high-precision tasks.

Invariant: each reported trace has at least one named stage whose observed latency is greater than the allowed budget. Missing stages are ignored, but each record still needs a trace ID and a mapping of stage latencies. Reject malformed records, negative or non-finite latencies, and non-positive or non-finite budgets before making SLO decisions.

import math


def budget_violations(records, budgets_ms):
    if not isinstance(budgets_ms, dict):
        raise ValueError("budgets_ms must be a mapping")
    for name, limit in budgets_ms.items():
        if not isinstance(limit, (int, float)) or not math.isfinite(limit) or limit <= 0:
            raise ValueError(f"budget for {name} must be a positive finite number")

    violations = []
    for record in records:
        if not isinstance(record, dict):
            raise ValueError("each record must be a mapping")
        trace_id = record.get("trace_id")
        if not trace_id:
            raise ValueError("record is missing trace_id")
        stages = record.get("stages_ms", {})
        if not isinstance(stages, dict):
            raise ValueError(f"stages_ms for {trace_id} must be a mapping")
        for name, value in stages.items():
            if not isinstance(value, (int, float)) or not math.isfinite(value) or value < 0:
                raise ValueError(f"latency for {name} must be a non-negative finite number")
        bad = {
            name: value
            for name, value in stages.items()
            if name in budgets_ms and value > budgets_ms[name]
        }
        if bad:
            violations.append({
                "trace_id": trace_id,
                "violations": bad,
            })
    return violations