Open Reference Stack
I built the open reference stack for getting mid-market AI agents safely into production. It is the method behind every Assessment, and it is public so you can see exactly how the work is done.
Most AI pilots fail on the way to production. The agent works in the demo, then fails silently on a real customer, exposes data it should never have touched, or leaves you unable to answer an auditor asking who approved it. The risk is the gap between a pilot and a system in durable production.
GRADE is a structured way to close that gap: the set of checks that turn a promising prototype into something you can put your name on in front of a board.
The Framework
Five pillars. Each one has an answer a board can act on and an implementation a practitioner can build.
You can show a board, an auditor, or a regulator who owns the agent, what it is allowed to do, and how those limits are enforced.
In practice Policy and guardrails at an LLM gateway, and a mapping to the NIST AI RMF and the EU AI Act deployer timeline.
The agent behaves consistently under real conditions and volume, beyond the one path that looked good in the demo.
In practice A planner, executor, and validator pattern with retries and fallbacks, and cost-per-success tracked as a first-class metric.
The agent has exactly the autonomy and tool access its job requires, and nothing more, so it cannot take an action nobody signed off on.
In practice MCP for tool integration with least-privilege, scoped access per tool, checks against the OWASP Agentic Top 10, and a human approval gate on high-impact actions.
The system ships onto infrastructure a mid-market firm can run and operate, with no dedicated ML platform team required.
In practice OpenTelemetry tracing wired through every run, so the deployed system is observable rather than a black box.
Quality is measured before and after every change, so you know the agent is improving and can prove it is not quietly regressing.
In practice An evals harness on DeepEval, promptfoo, or Phoenix, with trace-level scoring and CI gating on every change to the system.
The Deliverable
The Scorecard is the board-level output of the framework: a single production-readiness document that grades an AI system across all five pillars in language a non-technical board and an auditor can both read.
It covers where the system stands on each of the five pillars, calls out the gaps that carry real production or audit risk, and puts them in priority order. It is the executive translation of the technical work, and it is what anchors the Assessment.
Walkthrough
The method end to end: what it takes to move a stalled pilot through the five pillars and into production.
Walkthrough Video · Coming Soon
A short walkthrough of the GRADE Stack: how a pilot moves into production, governed and audit-ready.
Why It Is Open
A firm cannot open-source its method without giving away the product. A solo senior operator can, and that is the point. The stack is public because the fastest way to prove I can get your agent into production is to show the working code and the checks that get it there.
Read the repo, fork it, or hand it to your own engineers. The open artifact is proof of the work. The Assessment is that same rigor, pointed at your system.
The stack is the finished standard. The experiments and works in progress built around it live in Labs.
The repo shows how GRADE works. The Assessment applies it to your AI and gives your board the Scorecard.