Small, focused experiments — and what each one taught me.

15 experiments

• Generative AIImplemented

  Schema-constrained document extraction

  Extracting structured fields from long documents by pairing section-level prompts with strict Pydantic output schemas, so malformed responses fail fast.

  Architecture

  1. Document
  2. Sectioner
  3. LLM + schema
  4. Validator
  5. Structured JSON

  Learning outcomes

  • Smaller, focused prompts beat one large prompt on both accuracy and cost.
  • Validation errors are the most useful signal for improving prompts.
  • Python
  • Azure OpenAI
  • Pydantic
  • FastAPI

  Source: not yet public
• RAG SystemsImplemented

  Hybrid retrieval: vectors plus fuzzy matching

  Combining embedding similarity with lexical and fuzzy string scores to match short, domain-specific phrases where either signal alone is unreliable.

  Architecture

  1. Query
  2. Embedding search
  3. Fuzzy scorer
  4. Score fusion
  5. Ranked matches

  Learning outcomes

  • Semantic search alone confidently matches look-alike terms; a lexical signal corrects this.
  • Thresholds must be calibrated on labelled examples, not intuition.
  • LangChain
  • ChromaDB
  • Sentence Transformers
  • RapidFuzz

  Source: not yet public
• RAG SystemsIn progress

  Chunking strategies compared

  Measuring how fixed-size, structure-aware and semantic chunking affect retrieval quality on the same document set and question set.

  Architecture

  1. Corpus
  2. Chunkers (×3)
  3. Index
  4. Retrieval eval
  5. Comparison

  Learning outcomes

  • Structure-aware chunking helps most on documents with clear headings and tables.
  • Python
  • FAISS
  • Hugging Face

  Source: not yet public
• MCPImplemented

  A read-only MCP server for a data source

  Exposing a database to an LLM through the Model Context Protocol with read-only tools, parameter validation and row limits.

  Architecture

  1. LLM client
  2. MCP client
  3. MCP server
  4. Query guard
  5. Database

  Learning outcomes

  • Tool descriptions matter as much as tool code — the model only knows what the schema tells it.
  • Separate read and write capabilities from the first version.
  • MCP
  • Python
  • PostgreSQL

  Source: not yet public
• Agentic AIIn progress

  Recoverable tool-calling errors

  Designing tool error messages so that a model can correct its own arguments and retry, instead of the whole run failing.

  Architecture

  1. Agent
  2. Tool router
  3. Validator
  4. Tool
  5. Structured error → Agent

  Learning outcomes

  • A precise error message is often enough for the model to fix the call on the next step.
  • Python
  • Pydantic
  • Tool calling

  Source: not yet public
• Agentic AIIn progress

  Orchestrator and specialist agents

  A planner that assigns scoped tasks to specialist agents and merges their output, with budgets per agent to prevent loops and runaway cost.

  Architecture

  1. Goal
  2. Orchestrator
  3. Specialists (parallel)
  4. Merge & verify
  5. Result

  Learning outcomes

  • Most multi-agent problems are context-sharing problems; a shared, factual index reduces contradictions.
  • Python
  • LangChain
  • Async

  Source: not yet public
• LLM EvaluationImplemented

  Three-layer evaluation for LLM and agent outputs

  An evaluation framework on self-hosted Langfuse that scores agent, RAG and extraction outputs in three layers: programmatic evaluators first, then an LLM judge, then human calibration of the judge.

  Architecture

  1. Traces
  2. Programmatic checks
  3. LLM-as-judge
  4. Human calibration
  5. Scores & dashboards

  Learning outcomes

  • Cheap programmatic checks catch most failures before a judge model is needed.
  • An LLM judge is only trustworthy once its scores are calibrated against human reviewers.
  • Langfuse
  • Python
  • LLM-as-judge
  • AWS

  Source: not yet public
• LLM EvaluationImplemented

  Semantic caching for repeated LLM calls

  Caching responses by embedding similarity so near-identical requests reuse an earlier answer, with a similarity threshold that protects correctness.

  Architecture

  1. Request
  2. Embed
  3. Cache lookup
  4. LLM (on miss)
  5. Response

  Learning outcomes

  • The threshold is a correctness setting, not a performance setting — tune it conservatively.
  • Python
  • Vector store
  • Azure OpenAI

  Source: not yet public
• Data EngineeringImplemented

  Column standardization as a service

  A REST service that manages column-naming standards and exports them as configuration consumed directly by Spark transformation jobs.

  Architecture

  1. Spreadsheet upload
  2. Validation gates
  3. Standards store
  4. YAML export
  5. Glue jobs

  Learning outcomes

  • Keeping standards in one service removes a whole class of mismatch bugs between pipelines.
  • FastAPI
  • PostgreSQL
  • AWS Glue
  • PySpark

  Source: not yet public
• Data EngineeringImplemented

  Data quality gates in PySpark

  Reusable validation steps — completeness, uniqueness, referential checks — that stop a batch before bad data reaches the target.

  Architecture

  1. Source batch
  2. Rules
  3. Gate
  4. Quarantine / proceed
  5. Target

  Learning outcomes

  • Quarantining failed rows with a reason is more useful than failing the whole batch.
  • PySpark
  • AWS Glue

  Source: not yet public
• Data EngineeringIn progress

  Medallion layout on a Lakehouse

  A reference layout for bronze, silver and gold layers, with notes on where validation and deduplication belong.

  Architecture

  1. Sources
  2. Bronze (raw)
  3. Silver (clean)
  4. Gold (serving)
  5. Consumers

  Learning outcomes

  • Deciding what 'clean' means for the silver layer is a business conversation, not a technical one.
  • Databricks
  • Delta Lake
  • PySpark

  Source: not yet public
• Cloud EngineeringImplemented

  Containerized FastAPI service template

  A starting point for Python services: multi-stage Docker build, health checks, structured logging and a CI pipeline.

  Architecture

  1. Code
  2. CI build & test
  3. Image registry
  4. Kubernetes
  5. Observability

  Learning outcomes

  • A consistent template makes every new service cheaper to operate.
  • Docker
  • Kubernetes
  • Azure DevOps

  Source: not yet public
• Cloud EngineeringPlanned

  Event-driven pipelines with serverless components

  Orchestrating small ingestion and transformation steps with managed serverless services instead of an always-on cluster.

  Architecture

  1. Event
  2. Function
  3. Workflow
  4. Storage
  5. Notification
  • AWS Lambda
  • Step Functions
  • S3

  Source: not yet public
• Python EngineeringImplemented

  Bounded concurrency for LLM calls in FastAPI

  Using asyncio semaphores and retries with backoff so a service can call an LLM API in parallel without hitting rate limits.

  Architecture

  1. Request
  2. Task queue
  3. Semaphore
  4. LLM calls
  5. Aggregated result

  Learning outcomes

  • Unbounded concurrency looks fast in testing and fails under real rate limits.
  • Python
  • asyncio
  • FastAPI

  Source: not yet public
• Python EngineeringImplemented

  Dynamic column detection for messy spreadsheets

  Locating the real header row and mapping columns to a target schema from names and sample values, then writing template-accurate Excel output.

  Architecture

  1. Raw file
  2. Header locator
  3. Column mapper
  4. Transform
  5. Formatted workbook

  Learning outcomes

  • Flagging what could not be mapped is safer than guessing.
  • Pandas
  • OpenPyXL
  • Regex

  Source: not yet public