Vayum Arora, Zhengyao Jiang, Dixing Xu and Dhruv Srikanth describe an autoresearch loop that proposes code changes, runs experiments and retains improvements against a chosen metric. They discuss Parameter Golf as a constrained language-model challenge and present their agent's leaderboard contributions as their reported account, not proof that automated systems replace human researchers.
Vayum Arora, Zhengyao Jiang, Dixing Xu and Dhruv Srikanth identify the evaluator as the central design problem. It should be fast, inexpensive and low-noise, but also resistant to reward hacking. A search needs a defined editable code surface, an explicit maximize-or-minimize goal, experiment limits, timeouts and relevant guidance. Multiple objectives can be combined carefully or handled as constraints around one metric.
Vayum Arora, Zhengyao Jiang, Dixing Xu and Dhruv Srikanth use an experiment tree to preserve measured attempts and inspect the exact code behind each score. Human suggestions can create new search directions rather than merely watching an unattended loop. Their tokenizer example emphasizes recombining community ideas and existing experiments, making the improvement process collaborative rather than purely autonomous.
Vayum Arora, Zhengyao Jiang, Dixing Xu and Dhruv Srikanth demonstrate a smaller training problem suitable for a workshop. Connectivity and compute-access problems interrupt setup, and parts of the live steering demonstration do not connect or follow instructions as intended. A recorded initial score improvement is not presented as a completed, reproducible end-to-end validation.
Vayum Arora, Zhengyao Jiang, Dixing Xu and Dhruv Srikanth finish with a game-controller optimization task evaluated across several environments and a hidden validation set. The exercise restricts edits to the controller and prohibits changing the evaluator, reinforcing that a better visible score alone does not establish generalization. Account signups, promotional credits and competition prizes are omitted from this account.
Watch on YouTube
