Developers can now direct Claude to use computers the way people do—by looking at a screen, moving a cursor, clicking, and typing text.
Claude can use other tools as well.
Introducing an upgraded Claude 3.5 Sonnet, and a new model, Claude 3.5 Haiku. We’re also introducing a new capability in beta: computer use.
Developers can now direct Claude to use computers the way people do—by looking at a screen, moving a cursor, clicking, and typing text. pic.twitter.com/ZlywNPVIJP
— Anthropic (@AnthropicAI) October 22, 2024
Brian Wang is a Futurist Thought Leader and a popular Science blogger with 1 million readers per month. His blog Nextbigfuture.com is ranked #1 Science News Blog. It covers many disruptive technology and trends including Space, Robotics, Artificial Intelligence, Medicine, Anti-aging Biotechnology, and Nanotechnology.
Known for identifying cutting edge technologies, he is currently a Co-Founder of a startup and fundraiser for high potential early-stage companies. He is the Head of Research for Allocations for deep technology investments and an Angel Investor at Space Angels.
A frequent speaker at corporations, he has been a TEDx speaker, a Singularity University speaker and guest at numerous interviews for radio and podcasts. He is open to public speaking and advising engagements.
Progress is good. According to my tests, Chat GPT gives me the best answers. I don’t have any doubt about that. Grok 2 is not bad but ranked about 4-10th place. I mean very generic in some cases. Similar to Gemini or Llama, or a bit worse.
Sonnet is okay. I still think that it is not as good as Chat GPT for my line of work.
Jaron Lanier advanced some time ago the idea of phenotropic computing. Computers and programs using computers and programs through their UIs.
Akin to Wall-E’s robots using the buttons and switches of other robots to accomplish something.
It seems silly at first glance, but the advantage is universal accessibility. Basically is the computer being the user.
At the time it required some foolish ideas of UI standardization, to cover the immense variety of methods and nuances of computer interfaces. Ofc, it would never work in real life.
But now we are teaching AIs to see screens and use OSes and programs, and generalize with common sense. Now it really has a chance.