Evidence-Grounded Technical Knowledge
A reviewable document-knowledge extraction and retrieval system that extracts procedures, rules, and issues from visual engineering documents, grounding every synthesized claim in traceable source evidence.
Evidence-Grounded RAG · Knowledge Pipeline
- 01
Deck Ingestion
PPTX / DOCX / PDF Render
- 02
Section Split
Dividers + Fallback Rules
- 03
VLM Analysis
Diagrams & Prior Context
- 04
Typed Synthesis
Pydantic & Quote Check
- Human Gate05
Human Review
Streamlit Approval Gate
- 06
Chroma QA
SHA-256 IDs & Page Citations
No unverified claim enters Chroma—every synthesized rule links back to its exact source slide and native quote.
Before
Critical engineering troubleshooting rules and design procedures were buried in hundreds of slide decks and PDFs, making keyword search ineffective and knowledge transfer slow and error-prone.
After
Delivered a traceable knowledge pipeline where VLM-extracted claims are validated, human-reviewed via Streamlit, and indexed into Chroma, providing exact page-level citations for every retrieved answer.
Context & Background
- Engineering teams accumulate vital operational rules, debugging flows, and issue workarounds in visual PPTX presentations and PDF reports.
- Traditional RAG workflows chunk documents blindly by character counts, breaking logical procedure steps and losing diagrams.
- Engineers need answers they can verify instantly against original document pages before taking action on sensitive equipment.
Constraints
- Verifiable evidence required: every published claim must cite explicit page evidence; unsupported model inferences must be flagged for review.
- Human-in-the-loop: no claim enters the production vector database without explicit human inspection and approval in the review UI.
- Resilient to model variability: system must recover gracefully from JSON formatting errors and model API hiccups through schema validation.
Problem Solved
- Rescued trapped tribal knowledge and debugging steps from fragmented presentation slides and technical memos.
- Eliminated blind trust in generative AI by providing transparent, verifiable citations for every technical answer.
- Prevented context fragmentation common in naive chunking by structuring knowledge at the logical section level.
My Role
- Designed the document-scoped Pydantic domain models (
SectionMap,PageEvidence,SectionKnowledge,EvidencedClaim). - Implemented multi-format document conversion, page rendering, and sequential context injection for VLM interpretation.
- Built the Streamlit review application, grounding validation rules, deterministic claim hashing, Chroma indexing, and provenance-aware QA retrieval.
System Design
- Ingestion & Evidence: PPTX/DOC/PDF files are converted to PDF; pages are rendered to high-res images while native text is preserved as ground truth.
- Section Inference: Deterministic candidate detection (divider slides, headers) is reconciled by the model with fallback safety.
- Knowledge Synthesis: VLM analyzes page images with prior-page context; synthesizes typed rules, procedures, and issues with explicit page citations.
- Human Review & Vector Storage: Streamlit displays audit artifacts and unverified claims; upon explicit approval, claims are upserted into Chroma with deterministic IDs for cited QA retrieval.
Implementation Highlights
- Established dual-evidence baseline: native text guarantees lexical accuracy while VLM interpretation captures flowchart semantics and visual callouts.
- Engineered Pydantic validation and JSON repair heuristics to guarantee structured claim schema adherence despite LLM stochasticity.
- Built grounding audit logic that rejects synthesized claims whose cited quotes cannot be matched back to retained source page text.
- Developed a deterministic SHA-256 claim ID generation scheme to prevent duplicate or conflicting embeddings during re-indexing.
Outcome & Impact
- Transformed static, unsearchable engineering presentation decks into an interactive, high-precision semantic knowledge base.
- Achieved 100% citation traceability, allowing engineers to click directly through to the original slide image and source paragraph.
- Eliminated unverified hallucinations through strict human-in-the-loop review gating before vector store persistence.