Bijan Bowen tests Step 5 Preview through OpenRouter using the Pi harness at high reasoning effort. He distinguishes vendor benchmarks and announced open-weight plans from the hands-on results, noting that the eventual license is not yet confirmed in his account. The video is a particular preview-model and harness evaluation, not a controlled ranking of every model or deployment setup.
Bijan Bowen's browser-desktop test produces a detailed interface, file browser and some useful visual touches, but window interactions behave erratically and embedded games exhibit spatial inversion. A native C++ skating task also creates recognizable scene elements while failing important movement and geometry requirements. Attractive assets do not establish a working application.
Bijan Bowen's subway game initially leaves the player beneath the floor. A later repair improves access to the scene but still fails the train-boarding progression. Further Blender/Godot and watch-site tasks repeat geometric problems, while a simpler two-dimensional game has relevant details but missing interface features and crashes. The repaired run is not represented as a clean first-attempt success.
Bijan Bowen reports a total test cost slightly above $22 and judges the demonstrated results disappointing relative to alternatives he has previously tried. Those costs and comparisons are his reported observations, not a standardized cost-adjusted benchmark. He leaves open the possibility of useful niches or later improvements while separating that possibility from the failures in this test suite.
Watch on YouTube
