agent + benchmark
SWE-agent
A research system that enables language models to resolve software engineering tasks in repositories.
Benchmark evidence
Results need context.
This catalog links methodologies, evaluated targets, metrics, and limitations. It does not invent a universal ranking.
15 records
benchmark
SWE-bench
A benchmark for evaluating language models on real software engineering issues from open-source repositories.
benchmark
Terminal-Bench
A benchmark for measuring AI agents on practical tasks completed in terminal environments.
benchmark
AgentBench
A benchmark suite designed to evaluate language models acting as agents across varied environments.
benchmark + agent
BrowserGym
A framework and benchmark family for evaluating web agents in browser-based environments.
benchmark
WebArena
A realistic web environment for evaluating autonomous agents on reproducible tasks.
benchmark
OSWorld
A benchmark for multimodal agents that operate full computer environments through visual interaction.
app + skill + benchmark
Inspect AI
An open-source framework for building and running language-model evaluations.
app + benchmark
OpenAI Evals
An open-source framework and registry for evaluating language models and model systems.
app + benchmark
LM Evaluation Harness
A framework for evaluating language models across a broad collection of academic tasks.
app + benchmark
Langfuse
An open-source platform for traces, prompt management, evaluations, metrics, and datasets for LLM applications.
app + benchmark
Arize Phoenix
An open-source observability and evaluation platform for language-model and machine-learning applications.
app + benchmark
Promptfoo
An open-source command-line and web toolkit for testing prompts, models, agents, and red-team scenarios.
skill + benchmark
Ragas
An open-source framework for evaluating retrieval-augmented generation systems and AI workflows.
skill + benchmark
DeepEval
An open-source evaluation framework for testing language-model systems with metrics and datasets.
No matching records
Clear one or more filters to broaden the catalog.