WhatsApp Document Verification Bot
A privacy-first WhatsApp bot that extracts and validates fields from photographed ID documents, stores them encrypted, and answers follow-up questions through a local vector search layer that never sends document content to an external LLM.
- 0document content sent to external LLMs
- Encryptedfiles and sensitive fields at rest
Tech stack
- Python
- FastAPI
- PostgreSQL
- Tesseract OCR
- OpenCV
- sentence-transformers
- FAISS
- cryptography (Fernet/AES)
- Docker
How it works
- 1Receive document via WhatsApp
- 2Preprocess image
- 3OCR + field extraction
- 4Validate + mask PII
- 5Encrypt + store
- 6Answer questions via local vector search
Problem
Customers send ID documents as photos over WhatsApp for verification. The bot extracts the relevant fields, validates them, and can answer follow-up questions from the extracted data — without the document or any extracted personal information ever leaving the system through an external API.
Why it was difficult
Photos taken by customers are inconsistent — skewed, poorly lit, low resolution — which makes OCR unreliable without real preprocessing. Because the documents are government ID, encryption and PII masking had to be first-class design decisions from the start, not added later. Answering questions from the extracted data without calling an external LLM also meant building retrieval entirely locally.
Architecture
A WhatsApp Cloud API webhook receives incoming documents, downloads the media, and hands it to an image-preprocessing step (grayscale, denoise, deskew, threshold) before OCR. Extracted text runs through field extraction and validation, then the original file is encrypted and the plaintext copy is deleted. A separate local embedding index lets the bot answer questions against stored knowledge without any external LLM call.
Technical approach
Every stored file and sensitive field is encrypted at rest, with masking functions that only ever echo back a redacted form of sensitive data in bot replies. Because the question-answering embeddings are generated and searched locally, no document content needs to leave the server to answer a follow-up question.
Implementation
A FastAPI service backed by PostgreSQL, containerized with Docker Compose, with a separate lightweight admin interface for reviewing flagged extractions.
Challenges
Balancing OCR reliability against processing time on modest hardware — heavier preprocessing improves accuracy but costs latency.
Solution
Preprocessing is tuned to the minimum needed for consistent extraction rather than maximizing accuracy at any cost, and a configurable confidence threshold routes low-confidence extractions to manual review instead of auto-accepting them.
Evaluation
Test coverage for this project is still being built out — validated so far through manual review of extracted fields rather than a full automated suite.
Results
Not documented — ask Suraj for real figures if he wants OCR accuracy or volume numbers included.
Future improvements
Growing out the automated test suite is the clearest next step before this handles a larger volume of real documents.
Want something like this built?
Let’s talk