AI-Powered Insurance Policy Intelligence Platform
An intelligent document processing system built with Python, FastAPI, MongoDB and Azure OpenAI. It ingests policy documents, extracts the information that matters, identifies and classifies benefits, and exposes the result as structured data for downstream systems.
- 5,000+
- policies processed per month
- 4 h → 15 min
- extraction time per policy (about 93% less)
- ~95%
- accuracy in benefit identification and classification
- 80%+
- less manual review effort
Problem statement
Problem statement
Insurance policy documents are long, inconsistent and written for lawyers rather than databases. Analysts were reading each policy by hand to find benefits, limits and conditions, then re-keying them into spreadsheets.
Business challenge
Business challenge
Manual extraction took roughly four hours per policy, did not scale with volume, and produced inconsistent results between analysts. Any automated replacement had to be accurate enough to trust and transparent enough to review.
Solution architecture
- 01
A FastAPI service accepts policy documents and runs them through an asynchronous processing pipeline.
- 02
Documents are parsed and split into sections so that each LLM call sees a focused, relevant slice of the policy rather than the whole file.
- 03
Azure OpenAI performs extraction against a strict output schema; a second pass classifies every identified benefit into the target taxonomy.
- 04
Results are validated, stored in MongoDB with full provenance, and served through REST endpoints to downstream systems.
01Ingestion
Policy documents
PDF / DOCX uploads
FastAPI gateway
Auth, validation, job creation
02Processing
Parser & sectioner
Text, tables, layout-aware chunks
Extraction prompts
Schema-constrained LLM calls
Benefit classifier
Maps to target taxonomy
03Intelligence
Azure OpenAI
Extraction & classification models
04Quality & storage
Validation rules
Schema, ranges, completeness
MongoDB
Structured results + provenance
05Delivery
REST API
Status, results, exports
Downstream systems
Review tools & integrations
Workflow
- 1
Upload
A policy document is submitted to the API and a processing job is created.
- 2
Parse
Text and tables are extracted and the document is split into logical sections.
- 3
Extract
Each section is sent to the model with a strict schema for the fields to return.
- 4
Identify & classify
Benefits are identified and classified into the standard benefit taxonomy.
- 5
Validate
Rule-based checks flag missing, conflicting or low-confidence values for review.
- 6
Store & serve
Structured results are stored with source references and exposed over REST.
Technology stack
- Python
- FastAPI
- Azure OpenAI
- LangChain
- Azure AI Search
- MongoDB
- Pydantic
- Docker
- Kubernetes
- REST APIs
Implementation details
- Pydantic models define every extraction schema, so a malformed model response fails fast instead of polluting the data store.
- Long policies are processed section by section with bounded concurrency, which keeps latency and token cost predictable.
- Every extracted value keeps a reference to the source section it came from, making reviewer spot-checks quick.
- Prompts are versioned alongside code, so a change in output quality can be traced to a specific prompt revision.
Engineering challenges
Inconsistent document structure
No two insurers lay out a policy the same way. Sectioning had to be driven by content cues rather than fixed page positions.
Benefits described in many ways
The same benefit appears under different names and wordings. Classification needed clear taxonomy definitions and examples in the prompt, plus validation after the fact.
Hallucinated schedule fields
Some benefits-schedule fields were being invented when a single retrieval query missed the right passage. Moving to multi-query retrieval combined with hybrid search fixed it.
Business value and results
Business value
- Analysts review structured output instead of reading every policy end to end.
- Consistent benefit classification across documents and reviewers.
- An API other systems can call, rather than a one-off script.
Measured results
- 5,000+policies processed per month
- 4 h → 15 minextraction time per policy (about 93% less)
- ~95%accuracy in benefit identification and classification
- 80%+less manual review effort
Figures are approximate and were measured on the project's own review process.
Screenshots, code and demo
Screenshots
Not published — the interface belongs to a client engagement.
Source code
Private — client-owned code.
Live demo
No public demo. Ask for a walkthrough.
Have an AI or Data Engineering challenge? Let's turn it into a working product.
Taking on a limited number of AI and data engineering projects. Typical reply within 2 business days.