app + benchmark
LM Evaluation Harness
A framework for evaluating language models across a broad collection of academic tasks.
What it is
The harness standardizes task definitions, model adapters, few-shot prompting, and metric calculation for model evaluation.