matchspec Tutorials

Step-by-step tutorials for matchspec. Learn how to write your first eval suite, gate deployments in CI, and build custom graders for domain-specific correctness.

Run Your First Eval Suite

10 min

beginner datasets graders

Write a dataset for a summarization task, wire up an exact_match grader, run matchspec run, and interpret the results. Full working Go code included.

Gate Deployments with matchspec

20 min

intermediate ci/cd github actions

Set up a full GitHub Actions workflow that runs your eval suite on every pull request, fails the build when pass rate drops below threshold, and posts results as a PR comment.

Write a Custom Grader

15 min

intermediate graders go api

Implement the Grader interface to score model outputs against domain-specific rules. Learn how to test graders in isolation and register them for use in YAML config.