We evaluate DeepCode on the PaperBench benchmark (released by OpenAI), a rigorous testbed requiring AI agents to independently reproduce 20 ICML 2024 papers from scratch. The benchmark comprises 8,316 ...

Aider is a “pair-programming” tool that can use various providers as the AI back end, including a locally running instance of ...