Advanced Audio ML Interview And Production Exam

Prefer a cascaded VAD, streaming ASR, retrieval, LLM policy engine, tool router, and streaming TTS design unless the product requires direct speech-to-speech research. Separate interactive capacity from batch evaluation. Version ASR, retrieval index, prompt, tools, LLM, TTS voice, and client config. Gate launches by ASR slice WER, retrieval grounding, task success, unsafe action rate, first audio byte, end-to-end turn latency, interruption recovery, escalation precision, cost per resolved ticket, and rollback time.

Start with measurement by stage and tenant before changing models. Consider VAD tuning, early partial ASR, streaming retrieval, smaller rerankers, prompt trimming, KV-cache reuse where safe, continuous batching with deadlines, speculative decoding only if quality holds, TTS first-segment streaming, warm pools, request admission for noninteractive work, quantization, and fallback model tiers. Do not blend batch and interactive queues if it harms tail latency. Every savings proposal needs quality, safety, and rollback gates.

Freeze promotion, route affected slices back to the prior model, preserve aggregate-only metrics, and compare model version, region, client, device, VAD, language ID, noise level, and contextual bias features. Check whether eval coverage missed those accents or whether serving config differs from offline evaluation. Communicate blast radius and rollback status. Prevention should add slice gates, consented eval fixtures, labeling rubric updates, and canary alerts on correction rate, not only WER.

Split latency into request queue, text normalization, segmentation, acoustic model, vocoder, streaming chunk assembly, network, and client playback. Compare cold versus warm paths, voice version, sentence length, language, punctuation, cache hit rate, and GPU saturation. Mitigate by rolling back the voice, sending shorter first segments, warming voice pools, or routing long noninteractive reads to a cheaper path. Add release gates for first audio byte and chunk cadence, not only MOS.

Invariant: after advancing left, every record in the active window is within window_seconds of the current right edge. Test empty input, all success, all failure, same timestamp, and records exactly on the boundary.

from collections import deque


def max_window_error_rate(records, window_seconds):
    window = deque()
    failures = 0
    best = 0.0

    for ts, ok in records:
        window.append((ts, ok))
        if not ok:
            failures += 1

        while window and ts - window[0][0] > window_seconds:
            _, old_ok = window.popleft()
            if not old_ok:
                failures -= 1

        best = max(best, failures / len(window))

    return best

Invariant: each candidate slice is compared only if the baseline exists, and positive delta means worse. Common mistakes include sorting in the wrong direction, hiding missing baselines, and using aggregate averages that mask slice failures.

import heapq


def top_k_regressed_slices(baseline, candidate, k):
    heap = []
    missing = []
    for name, new_value in candidate.items():
        if name not in baseline:
            missing.append(name)
            continue
        delta = new_value - baseline[name]
        if delta > 0:
            heapq.heappush(heap, (-delta, name, baseline[name], new_value))

    worst = []
    while heap and len(worst) < k:
        neg_delta, name, old, new = heapq.heappop(heap)
        worst.append({
            "slice": name,
            "delta": -neg_delta,
            "baseline": old,
            "candidate": new,
        })
    return worst, missing

Invariant: the first matching healthy record is the safest rollback target because records are already sorted newest first. Test no match, tenant-specific block, model-family mismatch, and a global release that applies to all tenants.

def resolve_rollback_target(releases, tenant, family):
    for release in releases:
        tenants = release.get("tenants", ["*"])
        if family != release["family"]:
            continue
        if tenant not in tenants and "*" not in tenants:
            continue
        if release.get("healthy") and not release.get("blocked"):
            return release["version"]
    return None