There are two quite different things people mean when they say this, and it’s worth sorting them out before you touch anything. One is an agent, something that clicks and types on your behalf while you watch. The other is screen-sharing, where the AI can see what’s on your screen and talks you through it while you stay in charge of the mouse. Both exist right now in mainstream tools. Neither is magic, and both need a bit of care.

What you’ll be able to do

With an agent, you can hand over a low-stakes task, something like “find three flights and put the options in a table”, and watch it open a browser, click around, and come back with an answer. ChatGPT’s agent mode does this, operating a browser while you supervise. Anthropic’s computer-use capability lets Claude do something similar, though it’s aimed more at developers building on the API than at someone opening an app. Perplexity’s Comet browser automates web tasks in its own way too.

With screen-sharing, you point the camera or share your screen and ask “what am I looking at” or “why won’t this button work”. Gemini Live can see your phone screen or your camera and talk you through what it sees, in real time. Microsoft Copilot Vision does the same on Windows or in Edge, once you share your screen with it.

Both of these are useful. Neither replaces you doing the actual thinking. You’re still the one who decides whether the flight it found is any good.

Before you start

Set your expectations properly here, because this is where most of the disappointment happens. These tools are good at small, reversible jobs. Booking is not always reversible, so be careful there. Filling in a form that doesn’t submit yet, that’s reversible. Clicking “confirm purchase”, that’s not.

Agents can and do click the wrong thing. Not often, but often enough that you shouldn’t leave one unsupervised on anything that matters. Screen-sharing is lower risk in one sense, because the AI isn’t actually doing anything, it’s just watching and talking. But watching means it can see everything on your screen, not just the bit you meant to show it. That includes tabs, notifications, and whatever’s open behind the window you care about.

So, before you start: close your banking tab. Close your email if you can. Log out of anything with money or personal medical information attached to it. If you’re about to share a screen or fire up an agent, treat it like you’re about to screen-record a tutorial, because in a sense, you are.

Get set up

Right, the actual setup. This bit’s quick.

For agent mode in ChatGPT, you need a ChatGPT account with agent mode available to you, and you start a chat and ask it to complete a task using the browser. It’ll show you what it’s doing step by step, and it pauses for approval at points that matter.

For Claude’s computer-use capability, this one’s more for people comfortable with the API or using an app that’s built it in, it isn’t a simple toggle in the everyday Claude app the way agent mode is in ChatGPT. If you’re not writing code or using a tool that wraps this for you, it’s less likely to be your starting point.

For Comet, Perplexity’s own browser, you install the browser itself and use its built-in automation from there, rather than adding it to a browser you already use.

For Gemini Live, you open the Gemini app on your phone, start a Live session, and choose to share your camera or your screen. It’ll talk to you about whatever it can see, live, as you move the camera or scroll.

For Copilot Vision, you’re on Windows or in Edge, and you share your screen with Copilot when prompted. Once shared, it can see what’s there and answer questions about it.

None of this needs anything unusual on your device beyond having the app or browser installed and being logged in. The setup is genuinely the easy part.

Try it yourself

Here are three prompts that do different jobs. Copy them, adjust the details, and see what happens.

An agent doing a simple, reversible task:

Open a browser and search for three train times from Leeds to
Manchester tomorrow morning before 9am. Don't book anything.
Just bring back the options with prices and times in a table
so I can pick one myself.

Screen-sharing to get help with what’s in front of you:

I'm sharing my screen. I'm stuck on a settings page and I can't
find where to turn off notifications for this app. Can you look
at what's on screen and tell me where to click?

A mixed workflow, agent plus your own check:

I want to compare prices for the same pair of trainers across
three different retailer websites. Open each one, find the
product, and note the price and stock status. Don't add
anything to a basket or enter any payment details. Just report
back what you find.

Notice the pattern. Every one of these tells the tool what not to do as clearly as what to do. That’s not paranoia, it’s just how you keep an agent inside the boundary you want.

Check the result

Once it’s done something, don’t just accept it. Check it.

If an agent booked or found something, open the same page yourself and compare. Did it actually find the flights it says it found, or has it summarised something slightly wrong? This happens more than you’d think, small numbers get transposed, dates get misread.

If it clicked through several steps, ask it to show you the steps it took, or scroll back through the agent’s own log if the tool keeps one. ChatGPT’s agent mode shows its actions as it goes, so you can watch in real time rather than just trusting the summary at the end.

For screen-sharing sessions, the test is simpler: did the instructions actually work? If Copilot Vision told you to click a setting that wasn’t there, that’s your answer. Try again, describe it differently, or just do it your own way.

And check nothing happened that you didn’t ask for. Look at your basket. Look at your recent orders. Look at whether a form got submitted when you only meant it to be filled in. This takes thirty seconds and it’s worth doing every single time, not just the first time.

If it doesn’t work

Most failures fall into a few categories, and most of them are about privacy, not competence.

If you shared your screen and it saw something you didn’t want it to see, that’s not really a bug, that’s how screen-sharing works. A shared screen shares everything on it, notifications and all. The fix isn’t clever prompting, it’s closing things before you start.

Never hand an agent access to a logged-in banking session or your email inbox to act on your behalf. Keep those out of its reach entirely. If a task needs your bank details, do that bit yourself.

If an agent clicked the wrong thing, stop it immediately rather than letting it carry on and hoping it corrects itself. Most agent modes have a pause or stop control, use it. Then check what state things were left in, whether an item’s in a basket, whether a form half-submitted, before you trust it to carry on.

If Gemini Live or Copilot Vision seem to be struggling to understand what they’re looking at, that’s often just a camera angle or lighting problem rather than anything deeper. Move the camera, try better light, or switch to sharing your screen directly instead of pointing a phone at a monitor.

And a general rule: if a task would embarrass you or cost you money when it goes wrong, don’t give it to an agent unsupervised. Give it to yourself, with the AI as an assistant rather than the one holding the mouse.

Keep exploring

If you liked the automation side of this, have a look at “How to get AI to automate repetitive tasks”, which goes further into the kind of jobs worth handing over. If forms were the bit that interested you, “How to get AI to fill in forms” covers that specifically. And if you’d rather none of this ran through someone else’s servers at all, “How to get AI to run on my own computer” is the place to start.

Sources and review notes

Review date: 15 September 2026 — every source above was opened and checked against the text on that date.