SWE-bench

SWE-bench's repositories

experiments

Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.

Language:Shell7400

swe-bench.github.io

Landing page + leaderboard for SWE-Bench benchmark

Language:HTML500

humanevalfix-results

Evaluation data + results for SWE-agent inference on HumanEvalFix task

Language:Jupyter Notebook000

swe-bench__humaneval

Language:Python000

swe-bench__humanevalfix-go

Language:Go000

swe-bench__humanevalfix-js

Language:JavaScript000

swe-bench__humanevalfix-python

Language:Python000

swe-bench__humanevalfix-java

Language:Java000

swe-bench__livecodebench-selfrepair

Language:Python000

pytest-dev__pytest

Language:PythonMIT000

pyvista__pyvista

Language:PythonMIT000

pydicom__pydicom

Language:PythonNOASSERTION000

pvlib__pvlib-python

Language:PythonBSD-3-Clause000

marshmallow-code__marshmallow

Language:PythonMIT000

pylint-dev__astroid

Language:PythonLGPL-2.1000

mwaskom__seaborn

Language:PythonBSD-3-Clause000

sqlfluff__sqlfluff

Language:PythonMIT000

psf__requests

Language:PythonApache-2.0000

pydata__xarray

Language:PythonApache-2.0000

pylint-dev__pylint

Language:PythonGPL-2.0000

Qiskit__qiskit

Language:PythonApache-2.0000

sympy__sympy

Language:PythonNOASSERTION000

astropy__astropy

Language:PythonBSD-3-Clause000

matplotlib__matplotlib

Language:Python000

pallets__flask

Language:PythonBSD-3-Clause000

sphinx-doc__sphinx

Language:PythonNOASSERTION000

django__django

Language:PythonBSD-3-Clause000

scikit-learn__scikit-learn

Language:PythonBSD-3-Clause000