Skip to content
Yoshio NomuraContact

All work · Gated inference

Develarper

LLM task routing

  • Team of 4
  • July 2026
  • AMD Developer Hackathon, Act II

What it is

A two-tier LLM router built by team Develarper, four people, for the AMD Developer Hackathon (Act II). A local Qwen 2.5 3B classifier scores each task's difficulty and dispatches the hard ones to a cloud API while the easy ones stay on the local model.

Evaluation

Correct tier routing
711 of 800
tasks (88.9%), team-labelled routing evaluation, 10 Jul 2026
Answer accuracy
186 of 200
100 factual and 100 summarisation tasks, scored by similarity and Jaccard overlap — not human review, not an LLM judge; 505.1 ms mean, run sequentially

How it works

Two-tier router with a local difficulty classifier

A local Qwen 2.5 3B Instruct model served through Ollama classifies incoming task difficulty; the service layer is FastAPI, and the cloud tier is Fireworks, with per-call usage telemetry. A dedicated BERT encoder classifier was considered and rejected: holding a second resident model would not fit under the harness's memory ceiling, so the router reuses the generative model already loaded rather than paying for a specialist one.