SWE-bench

SWE-bench's repositories

SWE-bench

[ICLR 2024] SWE-bench: Can Language Models Resolve Real-world Github Issues?

Language:PythonMIT2360 30 187

experiments

Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.

Language:Shell138 8 32

sb-cli

Run SWE-bench evaluations remotely

Language:PythonMIT100

humanevalfix-results

Evaluation data + results for SWE-agent inference on HumanEvalFix task

Language:Jupyter Notebook010

pydicom__pydicom

Language:PythonNOASSERTION010

pyvista__pyvista

Language:PythonMIT010

swe-bench.github.io

Landing page + leaderboard for SWE-Bench benchmark

Language:HTML01 1

swe-bench__humaneval

Language:Python010

swe-bench__humanevalfix-go

Language:Go010

swe-bench__humanevalfix-java

Language:Java010

swe-bench__humanevalfix-python

Language:Python010

swe-bench__livecodebench-selfrepair

Language:Python010

.github

000

alibaba-fusion__next

Language:TypeScriptMIT000

Automattic__wp-calypso

Language:TypeScriptGPL-2.0000

bpmn-io__bpmn-js

Language:JavaScriptNOASSERTION000

chartjs__Chart.js

Language:JavaScriptMIT000

diegomura__react-pdf

Language:JavaScriptMIT000

eslint__eslint

Language:JavaScriptMIT000

GoogleChrome__lighthouse

Language:JavaScriptApache-2.0000

grommet__grommet

Language:JavaScriptApache-2.0000

markedjs__marked

Language:JavaScriptNOASSERTION000

marshmallow-code__marshmallow

Language:PythonMIT020

openlayers__openlayers

Language:JavaScriptBSD-2-Clause000

prettier__prettier

Language:JavaScriptMIT000

PrismJS__prism

Language:JavaScriptMIT000

processing__p5.js

Language:JavaScriptLGPL-2.1000

pvlib__pvlib-python

Language:PythonBSD-3-Clause010

pytest-dev__pytest

Language:PythonMIT010

swe-bench__humanevalfix-js

Language:JavaScript000