Coding Agents Can Now Drive Full Desktop Apps: The Blender Test

In a short TIL post, Simon Willison describes pointing ChatGPT Codex at a fully installed copy of Blender on his Mac and asking it to render a scene of "a pelican riding a bicycle." No plugin, no custom integration — just the standard Blender app from blender.org and a plain-English instruction telling the agent where to find it.

What actually happened

Willison's prompt told Codex to "use the already install /Applications/Blender" to render the scene. He followed up with two more casual instructions — "OK add a background and a lot of flair," then "OK make it a whole lot better" — and got back a finished image: a pelican on a bicycle at sunset, complete with a boater hat, scarf, flowers, balloons, beach huts, and palm trees. The image was produced by the agent generating and running Blender's Python API calls directly, the same scripting interface human 3D artists use to automate scenes, rather than through any special agent-Blender bridge.

The pelican itself isn't incidental — it's Willison's long-running, informal benchmark image for probing what a new model or agent setup can actually do, used across many of his posts comparing model capability over time. Using it here signals he's treating "can it drive Blender" as a genuine capability test, not a party trick.

Why this matters beyond the demo

For anyone building with agentic dev tools, the interesting part isn't the pelican — it's the surface area. A coding agent that can read a prompt, figure out which Blender Python calls produce a given visual result, execute them, inspect the output, and iterate on feedback ("more flair") is doing the same loop it uses on a codebase: understand intent, write code, run it, check the result, refine. Blender just happens to be the target instead of a test suite.

• Any application with a scriptable API — Blender, Excel, Figma, Photoshop, GIS tools, CAD software — becomes something an agent can plausibly automate end to end.
• The interaction pattern is identical to good vibecoding practice: short, specific, iterative prompts rather than one giant instruction.
• No custom integration layer was needed; the agent worked with Blender's existing, publicly documented Python API.

The catch for anyone shipping this pattern

A fun render is low-stakes: if the agent's Blender script is wrong, you get an ugly pelican, not a production incident. The same pattern pointed at a scriptable app that touches real data — a spreadsheet with financial models, a CAD file feeding a manufacturing pipeline, a design file with client assets — inherits all the usual agentic-coding risks: an agent executing code it wrote against a live application, with real files, and no reviewer in the loop. Teams experimenting with this kind of automation should treat any app-scripting agent the way they'd treat an agent with shell access — sandbox it, review what it executes, and don't hand it write access to anything you can't afford to regenerate.

The vibecoding angle

This is a clean, low-risk illustration of where agentic development is headed: the agent's target isn't always a repo. It's whatever has an API. That's worth internalizing for anyone building automation pipelines around Claude Code, Codex, or similar tools — the discipline (clear prompts, small iterative steps, sandboxed execution) is the same whether the agent is editing a Python file or scripting a 3D renderer.