AI coding agents can attempt real compiler work, but they stumble on implementing optimizations: asked to add a rewrite rule to LLVM's InstCombine pass, they often produce patches that miscompile programs, break tests, or land in the wrong place, and our benchmarking shows agents fail many such tasks. The open question is what feedback closes the gap: when the agent is handed a correctness counterexample, a profitability estimate, or a regression result, does its success rate improve, and which helps most? This project answers that on a fixed open model in a fully observable loop.
Tags: Compilers, Artificial Intelligence, Python, Command Line, C/C++, 2nd Year +, Experienced 1st Years