AI Voice Calling Platform

A configurable AI voice platform for insurance operations, designed to support 10,000+ daily calls across multiple concurrent bots while giving staff a dashboard for monitoring and control.

Sub-second turn latency on real telephony audio leaves very little room for a slow model call, at a call volume where any systemic slowdown compounds fast. On top of that: keeping the conversation flowing over a lossy phone audio path in noisy real-world environments, correctly pronouncing a wide range of Indian names across regional languages, staying compliant with do-not-call and calling-hours regulations, and integrating enough independent providers (telephony, speech-to-text, text-to-speech, and multiple LLMs) that any single one failing should not take calls down at that scale.

The call audio stream flows from Twilio through Silero VAD and DeepFilterNet noise suppression, streaming speech-to-text, a conversation layer that resolves most turns deterministically and escalates the rest to an LLM (backed by a per-bot RAG pipeline over PostgreSQL), then text-to-speech back to the caller. The voice engine is a separate service from the multi-tenant control-plane dashboard — the dashboard never talks to telephony or AI providers directly, only through the engine — and pushes live monitoring updates to connected browsers in real time.

A deterministic state machine resolves roughly 80% of conversation turns without touching an LLM at all, at under 50ms — which is what makes the sub-second latency target achievable at this call volume; only genuinely off-script utterances escalate to the LLM. The LLM layer itself is provider-agnostic across OpenAI, Anthropic Claude, Google Gemini, Groq, Mistral, and DeepSeek, with automatic health-based failover and the ability to switch providers mid-call. The RAG pipeline uses OpenAI embeddings with cosine-similarity retrieval over PostgreSQL-backed knowledge bases, with source documents stored in S3.

Production resilience is built in at every layer: async circuit breakers around each external provider, an answering-machine-detection gate, TRAI-compliant do-not-call enforcement, and a custom phonetics module covering 500+ names for correct pronunciation across Indian regional languages. Prompts are structured as an explicit call state machine plus a Hindi objection-handling playbook and anti-hallucination guardrails, refined iteratively from real call recordings. The control-plane dashboard is a multi-tenant Angular + .NET application with role-based access control, audit trails, and a function-calling tool framework, deployed via Docker with multi-worker scaling.

The first few seconds of a call are the highest-risk window — if the pipeline needs time to initialize, the caller hears dead air, which reads as a dropped call. A misbehaving provider or a quietly degrading fallback path can also affect multiple concurrent bots before anyone notices.

A short greeting is pre-synthesized and played immediately on connect, so the caller hears something natural while the rest of the pipeline finishes initializing in the background. Every external provider call is wrapped in a circuit breaker so a struggling STT, TTS, or LLM provider degrades gracefully — failing over automatically — instead of taking calls down, and explicit startup validation checks catch misconfiguration before it can reach a live call.

An automated test suite covers the voice engine, with component health checks exposed per provider and structured logging throughout. Prompt behavior is refined iteratively against real call recordings rather than tuned once and left alone.

The deterministic fast-path and phonetics module are both designed to keep improving with more real-call data rather than being tuned once and left static.