Agent Operations
Held-Out Set
Also: private test set, holdout
Evaluation tasks kept private until testing, so a model cannot have learned them from public data.
In more detail
Public benchmark scenarios can end up in training data, so a static public score is weak long-term assurance. A held-out set is created, dated and used privately, then rotated. It is one reason to treat any single public score modestly.