Multimodal Technical Document Review
A specialized LLM and vision-assisted document review service that turns tedious datasheet proofreading into an automated, high-signal technical verification workflow across Web, CLI, and MCP interfaces.
Datasheet Review · Multimodal Pipeline
- 01
DOCX Ingest
Tables & EMF/WMF Vectors
- 02
Token Chunker
Header & Path Preserved
- Multimodal03
Dual LLM / VLM
Specs, Units & Pinouts
- 04
Global Pass
Terminology & Noise Filter
- 05
Location Map
Section, Cell & Fig Ref
- 06
Multi-Surface
Web / CLI / MCP & XLSX
Shared grammar_checker core pairs token-bounded chunking with headless EMF conversion and traceable coordinates.
Before
Hardware and firmware teams spent days manually reviewing 50+ page technical datasheets for typographical, register table, and diagram consistency errors, while general tools generated overwhelming false alarms.
After
Unified datasheet review into a shared multimodal pipeline supporting Web, CLI, and MCP, eliminating formatting false positives and delivering localized section-by-section reports in minutes.
Context & Background
- Technical semiconductor datasheets require meticulous review across prose, register maps, electrical characteristics tables, pinout figures, and terminology.
- Teams needed an automated tool that understands hardware document conventions rather than generic English grammar rules.
- Review tools had to be accessible across different developer workflows: web portal for non-technical users, CLI for CI scripts, and MCP for agentic IDEs.
Constraints
- Must run in private intranet environments without exposing proprietary chip designs to public third-party APIs.
- Token consumption must be strictly bounded, preventing runaway context window usage on 100-page specification documents.
- Vector graphics (EMF/WMF) commonly embedded in Word documents must be converted reliably with timeout fallbacks.
Problem Solved
- Eliminated repetitive manual datasheet inspection bottlenecks prior to customer release.
- Prevented embarrassing documentation bugs in published pin definitions, electrical units, and register naming.
- Provided an extensible foundation for multimodal document intelligence across engineering departments.
My Role
- Architected the core
grammar_checkerpackage and unified its execution across FastAPI web service, CLI, and MCP protocol. - Engineered domain-tailored prompt templates, few-shot exclusion rules, and two-pass review (chunk-level + global consistency).
- Developed the multimodal ingestion pipeline: table matrix extraction, image filtering, vector graphic conversion, and VLM dispatch.
System Design
- Ingestion & Parsing: python-docx extracts paragraph hierarchy, tables with pipe-preservation, and embedded images/vector assets.
- Chunking Engine: Token-bounded segmentation splits oversized tables and sections while retaining descriptive path metadata.
- Dual-Model Review: Parallel text workers inspect technical phrasing and units; bounded vision workers evaluate pinout diagrams; global consistency pass checks terminology uniformity.
- Location Mapping & Export: Deduplicates findings, resolves exact section headings and table cells, and renders styled Excel sheets and Markdown summaries.
Implementation Highlights
- Created token-aware splitting for large table grids, ensuring headers and cell relations remain intact during LLM evaluation.
- Integrated headless vector graphic conversion for EMF/WMF formats with process timeout management.
- Built an asynchronous experiment runner in FastAPI allowing side-by-side comparison of prompt variants and model checkpoints.
- Implemented an MCP tool interface enabling AI assistants to invoke datasheet reviews directly within development tools.
Outcome & Impact
- Drastically reduced human proofreading time on technical datasheets while suppressing trivial false positives.
- Delivered actionable, auditable reports pinpointing exact page, section, and table coordinates for every defect found.
- Successfully enabled multi-interface usage across web, automated scripts, and AI coding agents.