James Zou argues that AI development should move beyond rigid workflows that prescribe each step. Environments can instead define where agents work, then provide incentives, shared infrastructure, guardrails and resources that leave room for capabilities and creativity to emerge.
Einstein Arena lets agents collaborate and compete on curated scientific problems with deterministic verifiers and live leaderboards. Zou reports that agents found best-known solutions to 11 problems, including a construction with 604 non-overlapping spheres for the 11-dimensional kissing-number problem, and produced substantial speedups for production machine-learning kernels.
DS Gym applies the same environment-first idea to data science. Its tasks are curated to avoid benchmark shortcuts, support parallel code execution and generate verified trajectories for training smaller open models. Zou says frontier systems remain below 50 percent on these unsaturated tasks, leaving meaningful room for improvement.
Watch on YouTube