General Reasoner: The smarter Local Agent
🔧 Summary
The General Reasoner paper shows how we can train LLMs to reason across domains using diverse data and a generative verifier. In this post, I walk through our open-source implementation showing how we built a modular reasoning agent capable of generating multiple hypotheses, evaluating them with an LLM-based judge, and selecting the best answer.
🧠What We Built
We built a GeneralReasonerAgent that:
- Dynamically generates multiple hypotheses using different reasoning strategies (e.g.,
cot,debate,verify_then_answer, etc.) - Evaluates each pair of hypotheses using either a local LLM judge or our custom MR.Q evaluator
- Classifies the winning hypothesis using rubric dimensions
- Logs structured results to a PostgreSQL-backed system
All of this was integrated with our existing co_ai framework, which includes: