Аудит оценок в Inspect AI

testing

eval-validity-review is a Claude Code skill that audits the validity of Inspect AI evaluations across four dimensions: claims coherence, name accuracy, dataset validity (whether models can both succeed and fail on each sample), and scoring validity (whether the scorer measures ground truth rather than a proxy). The skill reads task definitions, dataset loading code, scorer implementation, README, and eval.yaml, then produces a findings report with recommendations in agent_artefacts/<artefact_name>/validity/ — without modifying any evaluation code. It is intended for reviewing existing evals or as a step in the Master Checklist workflow, not for code quality checks or test coverage. Special attention is paid to dataset quality signals such as placeholder content, unsubstantiated statistics, and missing or inconsistent sample fields.