Free lessonsEvaluating LLMs and Agents
Evaluating LLMs and Agents
45 lessons
Lesson 26 · 5:33
Graders and pitfalls in long-horizon Agent evaluation
Experiment：Compare code, model, and human graders, then examine noise, gaming, and premature completion in long tasks.
Open this lesson on Bilibiliwanzij177Bilibili seriesThe lesson is in Chinese and plays from Bilibili.
Not translated yet — showing the Chinese originals.
About the series
How AI evaluation differs from software testing, how to score and grade defects, then how to run batch evals with EvalScope and Langfuse.
- Who it is for
- Testers and engineers who need to accept a model or an agent, not just demo it.
- What you can do afterwards
- You can name the object, the metric, and the defect grade, then reproduce a batch run.
Training a whole team? It is arranged by scope. Book an evaluation