SWE-bench

SWE-bench

SWE-bench: Can Language Models Resolve Real-world Github Issues?

experiments

Open sourced predictions, execution logs, trajectories, and results from model inference + evaluation runs on the SWE-bench task.

sb-cli

Run SWE-bench evaluations remotely

swe-bench.github.io

Landing page + leaderboard for SWE-Bench benchmark

.github

humanevalfix-results

Evaluation data + results for SWE-agent inference on HumanEvalFix task