AI speaks code, not clicks

AI models are made of text and trained on text, and code is text. That's why they are so good at it. Ask them to use software the way we do, by looking at a screen and clicking, and they get slower, more expensive and less reliable, the longer the task runs.

My argument: software whose output is code gets the full benefit of AI first. Give the model a way to see what its code produces, and it can do remarkable work. I expect more creative tools to become code underneath for exactly that reason.

Why is AI so good at code?

Three reasons, and none of them is magic.

• It is what the models learned from. Language models are trained on enormous amounts of text, and a large part of the public internet's text is code, documentation and discussions about code.
• Code gives feedback. Code runs or it doesn't. Tests pass or fail. A model can try, read the error and try again, a loop that's much harder with a vague goal like "make this slide look nicer."
• Code is exact and reproducible. A change to code is a diff you can read, review and undo. Run the same code twice and you get the same result.

How well can AI use a computer today?

Much better than two years ago, and this is where I had to correct my own assumption.

When the OSWorld benchmark for computer use came out in 2024, the best model completed 12% of its desktop tasks, against 72% for humans. The paper said models were "primarily struggling with GUI grounding and operational knowledge," meaning finding the right thing on the screen and knowing what to do with it. By late 2026, the top self-reported scores on the cleaned-up OSWorld-Verified are between 83% and 86%, above that human baseline. For short tasks like "change this setting" or "rename these files," clicking is no longer the weak spot.

Long tasks are another story. OSWorld 2.0, published in June 2026, uses 108 tasks that take a skilled person a median of about 1.6 hours. The agents need about 318 actions per task, and the best model fully completed just 20.6% of them. The paper describes agents that lose track of constraints, guess instead of asking, and skip checking their work.

Anthropic's own documentation for its computer-use tool is frank about why. It is slow compared to a person, the model "might make mistakes or hallucinate when outputting specific coordinates," dropdowns and scrollbars are "tricky," and every screenshot is billed as image input.

The underlying problem is simple arithmetic. If each step works 99% of the time, a task of 300 steps succeeds about 5% of the time (0.99 to the power of 300). Clicking through a long workflow turns small error rates into big failures.

Isn't MCP the fix?

The Model Context Protocol, introduced by Anthropic in November 2024, lets AI tools talk to apps through defined functions instead of the screen. It helps a lot, but it has limits of its own.

First, an agent can only do what the app chooses to expose. That's my own reading, not a vendor quote, but it follows from how MCP works. Second, tools are expensive to carry around. In November 2025, Anthropic wrote that 58 tools from five MCP servers took up about 55,000 tokens "before the conversation even starts," and that "the most common failures are wrong tool selection and incorrect parameters."

The most telling part is Anthropic's own fix. In a post on "code execution with MCP," it showed an agent writing code that calls the tools, instead of calling them one by one. In their example, a Google Drive to Salesforce task went from 150,000 tokens to 2,000, a saving of 98.7%. The best way for an AI to use tools turned out to be writing code.

What happens when the output is code?

Motion graphics are the clearest example I know. Frameworks like Remotion ("Create videos & motion graphics with React") and HyperFrames ("Write HTML. Render video. Built for agents.") describe a video as a web page with a timeline. For an AI, that is home ground.

The 15-second clip below was made that way for this post. Every frame is calculated from time in code: 500 particles that form a word, 336 tiles moving in two interfering waves, a shape that morphs from circle to star, and bars that count up to the benchmark figures from this article. An AI wrote the HTML composition, and automatic checks plus snapshots of key frames confirmed that the text was readable and nothing overlapped before it was rendered. Nobody keyframes 500 particles by hand.

The prompt, word for word:

Make a 15-second video that shows what code-driven animation can do, something you could only make with code. A line of code bursts into hundreds of particles that become the word MOTION, a grid of 336 tiles ripples with two interfering waves, a shape morphs from circle to star to blob, bars count up to real benchmark figures, and the title slams in letter by letter. Every frame is calculated from time, so nothing is hand-keyed.

The point is that I can change any part of it by changing a line of text, and the AI can too.

Why does AI make such good Canva presentations?

I build my own presentations in Canva, and this is where the idea becomes practical. An AI can write a whole deck as code, HTML with exact colours, fonts and layout, then render it, look at it and fix it before you ever see it. That loop of writing, looking and fixing only works because the output is code.

The tools are now built around exactly that. Decks made with Anthropic's Claude Design can be sent straight to Canva, where, according to Anthropic, they are "fully editable and collaborative." And since April 2026, Canva can import HTML that Claude generates and turn it into a design you edit like any other.

So the AI does what it's good at, writing and checking code, and you get a normal Canva presentation that you and your colleagues can adjust by hand. That's the pattern I expect to spread: the AI works in code, and the tool turns the result into something people can edit.

What do I expect?

• Creative software grows a code layer. Import and export as HTML, scripting interfaces, text-based project files. The tools that have one will get the AI features first.
• Office formats get wrapped, not replaced. People will keep presenting in PowerPoint, but more decks will be generated from code or an AI layer and exported to .pptx at the end.
• "Eyes" matter as much as "hands." The jump in quality comes when the AI can render what it made, look at it and fix it. Tools that give AI a fast way to see its own output will feel far smarter than tools that only let it type.

For anyone choosing software at work: if you want AI to help with something, ask whether that thing can be written as text. If it can, you're in luck.

This also changes what people need to know, which I wrote about in AI replaces junior tasks, not junior people.