Computer Use
If a human can click it, Claude can click it.
TUTOR WITH THEFOCUS.AI
Copy this prompt into Claude, ChatGPT, or any external AI assistant. It points the assistant to the course instructions and links it to your student profile to track your progress and customize observations.
You are not enrolled yet. Enroll to generate a Student ID to track lesson completions and store learning notes.
Have Claude operate the screen
Computer Use is the capability underneath all of this: Claude controlling a computer the way you do — through the screen, keyboard, and mouse. If a human can click it, Claude can click it. You meet it in two forms today. For anything with a web interface, Claude in Chrome is Computer Use in its most convenient packaging — Claude operating a real browser next to you. For everything else — desktop applications, legacy tools — the Computer Use API lets developers give Claude a full (sandboxed) screen to work with.
Cornwall Market: This is the Fountain Creek pattern. Invoices arrive as PDFs — different formats from different suppliers. Someone has to open each one, read the line items, then open the accounting software and type in the data. Ten to fifteen minutes per invoice. Errors are common — wrong account codes, transposed numbers, missed line items.
With Cowork and the browser, the workflow changes:
1. Cowork reads the invoice PDF directly — no clicking required; files are native territory
2. Extracts the vendor name, date, line items, and totals
3. Applies the categorization skill — Chen's Produce items go to 5100, bakery supplies to 5120
4. Opens QuickBooks Online in Chrome via the extension
5. Navigates to the invoice entry screen
6. Enters each line item — vendor, amount, date, account code
7. Verifies the total matches the PDF
8. Saves and moves to the next invoice
Notice the hierarchy at work even inside one task: the PDF is read as a file (native), the data entry happens in the browser (Claude in Chrome), and only a truly GUI-bound legacy app would need the screen-level fallback. The same work that took Sarah ten minutes per invoice takes Claude about two — and the error rate drops because Claude is not guessing account codes or transposing numbers. It is applying the categorization skill systematically.
Install the Claude in Chrome extension and give it a real errand in a web app you use — "fill out the reorder form on the supplier portal with these five items," "download last month's statements from the bank portal." Watch it work in the side panel; you can take over at any point. Then try a two-app version: have Cowork read a PDF or spreadsheet from a folder and enter the data into a web tool. The goal is to experience what it feels like when the application no longer needs an API to be automatable.
What people are actually doing with it
The early use cases cluster around a common theme: tasks that were too tedious to do manually but impossible to automate because the application had no API.
A photographer pointed Claude at a folder of 150 photos and told it to open each one in Photoshop, apply specific adjustments (exposure +0.5, crop to 16:9, export as WebP), and save. Claude clicked through the menus, applied the settings, and processed the entire batch. Photoshop has scripting, but most people do not know ExtendScript — they know how to click menus. Now Claude clicks for them.
Take a complex Excel workbook with pivot tables, conditional formatting, and macros. Claude can navigate sheets, enter formulas, adjust formatting, and build charts — the same way a human would. No VBA required.
Every large corporation has those 20-year-old internal tools with no API. The ones built on ancient Java Swing or Windows Forms. The ones that cost millions to replace but run critical workflows. Computer Use does not care how old the UI is. If a human can click it, Claude can click it.
Copy data from one app, paste into another, switch tabs, fill forms, submit. The kind of work that administrative staff spend hours on — moving information between systems that were never designed to talk to each other.
Check your understanding
Answer in your own words — write it down before opening the key. Your tutor grades against the criteria and generates fresh variants on retries.
Q1. The invoice task: read the PDF, apply categorization rules, enter data into a web accounting app. Assign each step its level in the hierarchy and explain why mixing levels inside one task beats picking one tool for everything.
Q2. “We can’t automate that, there’s no API” is now a dead excuse — but not a free lunch. What are the honest costs of screen-level automation, and how do you design around them?
Answer key — attempt every question first
Answer key
Q1
Model answer: Reading the PDF: native file access (top of hierarchy — no clicking). Applying rules: the skill, pure knowledge. Data entry: browser level via Claude in Chrome, because the accounting app has no suitable API. Mixing wins because each step runs at the highest level available to it — one-tool thinking forces every step down to the worst step’s level.
Pass criteria: sensible per-step assignment; states the use-highest-level-per-step principle
Q2
Model answer: It is slower (per-action interaction vs API calls), it should be watched or checkpointed (misreads are possible), and it is brittler when UIs change. Design around it: reserve it for the last mile only, verify outcomes (totals match the PDF), and keep approval gates on consequential steps.
Pass criteria: at least two honest costs (speed, supervision/brittleness); mitigation includes verification or gating