Multimodal Competitor Analysis with a Local LLM

The problem

Strategy teams had stacks of scanned, handwritten field notes about competitors. The information was valuable but sensitive, so sending it to an external AI API was not an option.

The approach

1. Digitise scanned pages, then use a locally deployed multimodal LLM (NVIDIA Nemotron 3 Super) to extract structure: the client, each competitor's customer-facing products, features, pricing and positioning.
2. Normalise the output into a comparison matrix and a gap summary that strategy teams can act on.
3. OCR fallback for pages the model struggles with, schema-validated JSON output, and human review of low-confidence extractions.
4. Run fully on-premises, with no external API calls.

Architecture

How a request flows through the system, top to bottom.

1. Input
   • Scanned handwritten pages
2. Extraction
   • Nemotron 3 Super on vLLM / NVIDIA NIM
   • OCR fallback
3. Validation
   • Schema-validated JSON
   • Confidence scoring
   • Human review queue
4. Output
   • Comparison matrix
   • Gap summary
   • PostgreSQL

Results

sensitive business data never leaves the network

On-prem

comparison matrix and gap summary from handwriting

Structured

low-confidence extractions routed to a human

Reviewed

Stack

Python · Nemotron 3 Super · vLLM · NVIDIA NIM · FastAPI · Docker · PostgreSQL