matchspec Tutorials
Step-by-step tutorials for matchspec. Learn how to write your first eval suite, gate deployments in CI, and build custom graders for domain-specific correctness.
Run Your First Eval Suite
10 minWrite a dataset for a summarization task, wire up an exact_match grader, run matchspec run, and interpret the results. Full working Go code included.
Gate Deployments with matchspec
20 minSet up a full GitHub Actions workflow that runs your eval suite on every pull request, fails the build when pass rate drops below threshold, and posts results as a PR comment.
Write a Custom Grader
15 minImplement the Grader interface to score model outputs against domain-specific rules. Learn how to test graders in isolation and register them for use in YAML config.