Use code, rubrics, and model judges appropriately
Different claims need different graders
Use code to check totals, allowed transitions, and whether an unavailable item slipped through. Use a rubric for a subjective property such as whether an explanation makes the tradeoff understandable.
A grader is the mechanism that assesses a result. Human review can establish example judgments. A model judge can help assess open-ended outputs, but its decisions also need checking. Do not replace exact arithmetic with another model’s opinion.
Check your understanding
Which grader should verify the sum of integer price components?
A rubric needs observable anchors
For explanation quality, define 0: unsupported or misleading; 1: accurate but misses the main tradeoff; 2: accurate, sourced, and explains the relevant tradeoff. Supply examples at each level so graders share an interpretation.
If you use a model judge, compare its judgments with human-labeled cases and inspect disagreements. Watch for irrelevant preferences such as rewarding verbosity. Evaluate the judge on the task rather than assuming a model name makes it reliable.
Check your understanding
Match the criterion to a suitable grading approach.
Keep critical failures separate from style scores
An elegant explanation cannot compensate for submitting without approval. Report critical gates separately from average quality. The lab shows a gate failure explicitly instead of hiding it inside a composite score.
In your reflection, grade two fictional recommendations using the rubric. Record the evidence for each score. If your interpretation changes, revise the rubric and regrade the same examples.
Check your understanding
Why should approval failures stay outside an average style score?
Build checkpoint
Open the Evaluation Lab and follow the practice above. Each lab explains its inputs, its actual logic, and its limits. Save your observations below; your check-ins and reflection remain in Product Academy.
For an AI-assisted code change, download the lab bundle from the lab page. Ask your coding assistant to explain the relevant function first, change one behavior, and add one test that could fail. Compare the result with your prediction.
Save your experiment and check-in
Record your prediction, what you tried, what happened, and the evidence for your conclusion. Name one thing you can now do independently and one uncertainty to revisit.
Key takeaways
- Use exact checks for exact requirements.
- Calibrate subjective graders with examples.
- Do not average away permission or constraint failures.