Performance vs size
What is the minimal model size that meets task accuracy? Benchmark vs GPT-class and Mixtral on domain tasks.
Prove small private LLMs can deliver enterprise intelligence
Design and validate on-device language model systems that meet accuracy, latency, memory, and safety constraints under real-world edge deployment. This is an active, early-stage program — see the roadmap below for what's shipped vs. planned, and our benchmarks methodology for exactly how current performance figures are derived.
Success KPIs
<200msFirst-token latency target
<8GBMemory footprint on edge
≥85%Domain task accuracy goal
100%Offline core flow capability
Key research questions
Optimization
Quantization (4/8-bit), LoRA/QLoRA fine-tuning, sparse and MoE-lite architectures for edge efficiency.
On-device constraints
Latency, memory, and energy efficiency — critical for mobile and automotive deployment paths.
Robustness & safety
Hallucination rate measurement, deterministic fallbacks, guardrails for sensitive actions.
RAG effectiveness
Performance gain from private data augmentation vs model-only inference; retrieval accuracy studies.
Future: vision AI
Lightweight architectures for automotive damage detection, cabin monitoring, and industrial inspection.
Roadmap
Research milestones feeding product configuration and guardrails.
Q3 2026
Edge benchmark suite
Latency and accuracy matrix across Pi, Jetson, and mini PC SKUs — published into hardware configurator.
Q4 2026
RAG & citation quality
Retrieval eval harness, reranker selection, and citation UX patterns for regulated domains.
2027
Command safety & automotive packs
Intent validation library, red-team results, and OEM knowledge base templates.
Research → implementation bridge
| Research output | Implementation impact |
|---|---|
| Model benchmarks | Hardware SKU mapping & model selection |
| Latency measurements | Device requirements in configurator |
| Safety evaluation | Guardrails & UX constraints |
| RAG experiments | Production retrieval pipeline |
| Optimization techniques | Final inference configuration |
Publications
We haven't published a formal paper or whitepaper yet — the program is still early. As the Q3 2026 edge benchmark suite and Q4 2026 RAG evaluation harness (above) produce results, we'll publish methodology and findings here rather than only marketing claims.