AI Chatbot with Voice & WhatsApp

This system brings AI-powered conversation directly into WhatsApp — the messaging channel users already live in. Incoming text messages are processed by AWS Bedrock and replied to in natural language. Incoming voice notes are transcribed via a speech-to-text pipeline, fed into the same AI model, and the response is synthesized back into an audio file sent as a voice reply. The result is a fully conversational AI assistant accessible through a phone number.

What it does

• 01Natural language conversation powered by AWS Bedrock foundation models
• 02Incoming voice note transcription via speech-to-text pipeline
• 03AI-generated responses synthesized to audio and sent as voice replies
• 04Webhook-based WhatsApp Business API integration for real-time message handling
• 05Conversation context management across multi-turn sessions
• 06Graceful fallback to text reply when audio synthesis is unavailable

What made it hard

1. 01

   WhatsApp voice notes use Opus/OGG encoding, while most TTS services produce MP3 or WAV output. Bridging these formats required a server-side audio transcoding step to keep the round-trip within WhatsApp's delivery time expectations.

2. 02

   AWS Bedrock model inference latency needed to fit within the user-perceived response window for a messaging app, requiring careful prompt design and streaming response handling to avoid timeouts.

3. 03

   Maintaining coherent multi-turn conversation context across stateless webhook invocations required a session store keyed on the sender's phone number with appropriate expiry logic.