When the canvas starts acting, who’s really in control?

How agentic interfaces are changing what designers control, from interactions to agency, orchestration and attention.

For most of the history of software, the interface has been the place where people tell a system what to do. We click, drag, type and select, and the system responds. A lot of interaction design has been about making those actions easier to understand and making the next step obvious.

The terminal has always been a different kind of interface. It gives you a lot of power, but most of that power is hidden behind commands. You need to know what the system can do and how to ask it to do it. This is one reason designers have always struggled with terminal-based tools. They expose the language of the machine more than the possibilities of the tool.

AI coding agents are bringing some of that interaction model back. You can describe an outcome and let the agent decide which commands to run, which files to change and which tests to execute. You no longer have to operate the software step by step, but a familiar design problem comes back in another form: What did the system understand? What is it doing? What has it changed? What can I stop? What should I review?

The interface is no longer just something we operate; it can act.

The interface that gives nothing away

The first generation of generative AI made this problem particularly visible. We were presented with an empty field and asked to describe what we wanted. This might seem like another version of the familiar blank page problem: when there is nothing to react to, it can be difficult to know where to begin.

Takuma Kakehi’s article, « Nobody wants a blank page. Nobody touches a perfect one », explores this tension in creative tools, arguing for a space between having nothing to work with and being handed something that already feels finished. But AI interfaces introduce a different problem.

The blank prompt is not simply difficult because it asks us to start from nothing. It is difficult because it gives us no indication of what is possible. The system’s capabilities remain hidden behind language, meaning that users must already know what to ask before they can discover what the tool can do.

The interface therefore places the burden of exploration on the user. Instead of revealing its possibilities and inviting experimentation, it asks the user to imagine them first.

This is where some of the principles we have been using in interaction design for years become relevant again. Good interfaces have always helped people understand what is possible without requiring them to already understand the system. They provide affordances, show what is happening and give people enough context to decide what to do next. The blank prompt puts much of that responsibility back on the person.

The heuristics still matter

Nielsen’s heuristics give us a useful way to think about this new relationship. Visibility of system status means that people need to understand what an agent has understood, what it is doing and where it is in the process. Recognition rather than recall becomes a question of making the agent’s capabilities visible through the context and material already in front of us, rather than expecting people to remember the right command or formulate the perfect prompt. User control and freedom now includes the ability to interrupt, redirect or undo an action. Error prevention means defining what the agent should be allowed to do on its own and what should require a person’s involvement. The principles are still there, but the relationship they describe has changed.

We used to design software that waited for instructions. Now we are designing software that can interpret an intention, take action and decide when it needs us.

When the canvas starts acting

This is where the canvas becomes autonomous. Miro’s agentic canvas brings context from tools such as Slack, GitHub, Amplitude and Granola, as well as PDFs, decks, spreadsheets and screenshots, into a shared space. Instead of pages, slides or chat histories, the canvas becomes a spatial environment where you can zoom, pan, group ideas and connect different pieces of work. Its Sidekicks can make sense of that context and create documents, diagrams, prototypes or boards, while Flows turn those interactions into workflows the team can see, run and reuse.

The important shift is that the AI is no longer working from an isolated prompt. It can read the canvas, reference upstream assets and pass outputs from one step to the next. The value is not only in what the AI creates, but in where that work lands. People can inspect the inputs, change the outputs and build on top of them.

Figma is moving in a similar direction. Its design agent works directly on the canvas, while Figma Make enables users to move from ideas and existing designs to working prototypes and applications. The company has also opened its canvas to third party agents and introduced skills that provide agents with context about design decisions and intent, mirroring the way skills work in Anthropic’s ecosystem.

A Miro board contains the context of a project. A Figma file contains components, layouts and relationships. A codebase contains files, dependencies, tests and errors. The agent has something to work with, and that changes the nature of the interaction. But there is another shift here: agents are beginning not only to consume context, but to decide which context they need.

Google DeepMind’s work on agentic video understanding is a useful example. Instead of processing a video as a fixed stream of frames, Gemini can determine which part of the video it needs to inspect, at what speed, and whether it needs frames, audio or a transcript. It can retrieve the moments and signals relevant to the task through an agentic loop. This may sound like a technical optimisation, but it points to a bigger change in the interface. The system is no longer simply waiting for us to point it at the right information. It can determine what information is relevant to the task and retrieve it for itself.

The same pattern is emerging in agentic canvases. The canvas gives the agent an environment to work from, but the agent is also navigating that environment. It can find relevant context, connect information across tools, decide what to act on and carry the result forward. Context is no longer just something the system receives, but something the system can seek.

And that means attention becomes part of the design problem.

Human in the loop

We have spent a lot of time talking about keeping humans in the loop. In his Stanford essay Humans in the Loop: The Design of Interactive AI Systems, Ge Wang makes a useful distinction between automating a task and designing a system where people can still shape what happens.

Human involvement does not have to mean doing the work manually. It can mean deciding where the agent should act on its own, where it should check in, and where a person needs to make the call. The goal is not an all or nothing choice between doing everything yourself and handing everything over to AI. It is a more flexible division of responsibility.

This feels particularly relevant to agents. Imagine asking an agent to plan a trip. I might give it my budget, preferences and a rough idea of where I want to go, then let it search for options and build an itinerary. I would not want to approve every search or booking step because that would leave me doing much of the work myself. But I also would not want the agent to make every decision without me. If two hotels are similar but one is in a neighbourhood I know I dislike, or if the cheapest flight involves an inconvenient overnight layover, those are decisions where context and personal preference matter. The agent can handle the routine work and bring those decisions back to me. The work moves between us depending on who is better placed to handle it.

AI in the loop, humans in charge captures what seems most important here. Human involvement should not be measured by how many times a person has to click approve. What matters is whether people remain in control of the decisions that depend on judgment, context or preference. Designing that boundary, deciding what the agent should handle, what it should surface and when it should hand control back, is an orchestration problem.

Designing the handoff

Once the system can act, the handoff becomes part of the interface. We need to decide when the agent should continue without us, when it should ask, and when it should return the work for a decision. That means designing more than screens and interactions.

We are designing the distribution of responsibility between a person and a system.

Once we start designing for systems that can act on their own, decisions that used to sit outside the interface become part of the design:

• What can the agent do without asking?
• When should it request confirmation?
• Which actions should be reversible?
• Which decisions should always remain with a person?

These choices may not look like interface design in the traditional sense, but they shape the experience just as much as a screen or interaction does. This is where agency becomes a design material.

So what are designers designing?

If humans stop operating interfaces directly, it is tempting to think that the role of the interface designer becomes less important. I think the opposite is happening, but the object of design is moving.

We still design screens, components and interactions, but we are also designing behaviour, constraints, defaults, permissions and outcomes. We are also deciding what the system can do without asking, what it needs permission to do and which situations should bring a person back into the process. Much of this will never appear as a traditional interface element. It lives in the rules and assumptions that determine how the system behaves.

This is the policy layer of the product.

The designer is no longer only defining the interface through which people operate the system. They are defining the conditions under which the system operates itself. This is already starting to change how we think about design systems and design knowledge. Figma’s work around agents and skills points towards a future where the decisions and conventions that once lived primarily in designers’ heads or in design documentation can become part of the context through which an agent works.

Design systems can become part of the instructions and constraints that shape how a machine operates, rather than remaining only a reference for human designers.

This also changes what taste means.

When AI can generate possibilities, taste helps us decide which ones are worth pursuing. When AI can execute decisions, taste also helps us decide which decisions should belong to the human and which can be delegated. The designer is shaping not only the outcome, but the distribution of agency that produces it.

What deserves our attention?

There is another consequence to this shift. If agents can gather context, generate options, make changes and carry work forward, execution becomes less scarce. Attention becomes more important. But there are now two kinds of attention to think about: There is what the system pays attention to, and there is what it asks the human to pay attention to.

The designer does not need to make every decision, they need to recognise which decisions are worth making. This changes the meaning of human in the loop design. The goal is not to keep a person involved at every step. It is to make sure that the right things reach them.

An agent might inspect hundreds of files, compare dozens of options or search through hours of video without involving us. The design problem is deciding what it should surface when it does need us, and what it should be trusted to handle without us. Human attention becomes a resource that the system can help allocate, and where intelligent interfaces, orchestration and taste come together.

The role of the designer is moving from specifying every interaction to deciding how responsibility and attention should move between people and systems.

We are designing what the system can see, what it can do, what it should leave alone and when it should ask for us. When the canvas starts acting, we are still designing interfaces. But now we’re also designing the behaviour behind them, the boundaries around them and the moments when control moves back to a person.

Perhaps the question is no longer simply what the interface should look like. It is who should decide what happens next, and who should be paying attention when it does.

From AI × Taste

A monthly series exploring the economics of taste in the age of AI:

• AI made everyone a creator, not a designer
  How AI made generation abundant, making design judgment and taste the new competitive advantage.
• The organizational costs of low taste
  How weak design judgment compounds across products, teams and organizations.
• Taste cannot be delegated
  Why AI can generate endless possibilities, but deciding what should exist remains fundamentally human.
• Designing for the Proxy
  Why AI is changing who reads first, and what happens when we optimize for the intermediary instead of the person.
• When the canvas starts acting, who’s really in control?
  How interfaces are changing what designers control, from interactions to agency, orchestration and attention.

When the canvas starts acting, who’s really in control? was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.