Anthropic's Best AI Tried to Fly a Drone. It Flew Straight Into a Wall.

That is Anthropic's best model, confidently flying a drone directly into a wall. It is the third story in a pattern this channel has been tracking: a model that had to be paused, a safety report card where nobody passed, and now what happens when an AI is physically in control of something. Anthropic published the research in a blog post, which is why almost nobody is talking about it.

The experiment

Anthropic partnered with Andon Labs, a company that specializes in embodied, real-world AI evaluation, to test one specific question: can an AI model autonomously fly a real drone to find and follow a person? Bringing in a specialist for the physical side while Anthropic focused on the model is a meaningful methodology choice, not just a partnership announcement.

It was not a simulation from start to finish, either. They flew a real $129 drone in a real office. Simulations are cheap and repeatable, but they can hide failure modes that only show up with real sensor noise, real lighting, and a real physical room. If you want to know whether an AI can fly something, eventually you have to let it fly something.

DroneBench: five subtasks

They built a formal benchmark called DroneBench, breaking the mission into five subtasks the AI has to chain together, each graded on its own:

• Reconstruct: turn video of the space into a 3D map, essentially building a mental floor plan from what the camera has seen.
• Localize: figure out where the drone actually is on that map, the equivalent of finding the blue dot on your phone.
• Navigate: plan and fly a path between rooms without hitting anything.
• Detect: find a specific person using a reference photo.
• Follow: keep that person in frame as they move, continuously.

This is not a cute robotics demo. A locate-and-follow drone is the same capability used in search and rescue and disaster response on the legitimate end, and in unaccountable tracking and targeting on the other. Anthropic chose the task specifically because it is dual use, to get real signal on how close AI is to piloting hardware autonomously.

What the results showed

They tested 15 models from three companies, from GPT-4 up through the newest releases, scoring each against a human-built reference solution. The pattern was consistent: newer models get further on every subtask. Claude Fable 5, Anthropic's own frontier model, was the best performer in the entire test. It beat the human-built baseline on four of the five subtasks: detect, follow, navigate, and localize.

That baseline is a specific bar. It consists of hand-coded reference algorithms, tuned by people who know exactly what they are doing. On the follow task, Fable 5 did not just clear the bar; it tracked the target more tightly than the reference algorithm.

The wall

The one task Fable 5 could not do was reconstruct: building an accurate 3D map of the room from video. Because every later step depends on that map, the single failure cascaded through the whole mission. Fable 5 flew confidently toward what it believed, based on its own flawed reconstruction, was an open doorway. It was not confused about the mission. It was wrong about the room.

And here is the detail that makes the story more interesting. While working on a different subtask, Fable 5 independently worked out the drone's camera angle by analyzing the grout lines on the floor in a video frame, tracing them to a vanishing point, and calculating the tilt to within 4 degrees of the true value. Nobody told it to. That is a genuine computer-vision technique, and it reached for it because the problem called for it. So you have a model smart enough to derive camera geometry from floor tiles, and still confident enough to fly into a wall minutes later. Both are true of the same model in the same experiment.

One more number to sit with: Anthropic ran 10 runs per model. Even Fable 5 only hit the human baseline on three of the five tasks on average across those runs. The frontier and the average are two different things.

Why this is bigger than a funny crash

Anthropic's researchers made a point worth repeating in spirit: at low capability, keeping a human in the loop is easy and obviously worth the time. But once a model crosses a real reliability threshold, there will be genuine pressure to treat human oversight as a cost to cut rather than a safeguard to keep. This experiment is that threshold, mid-crossing, in public. Four of five subtasks cleared, and the fifth compounds into a wall. That is not "AI can't do this." It is "AI can almost do this," which is a different and more interesting problem, because almost is exactly the zone where people are tempted to remove the human anyway.

This is a familiar pattern. Whenever a technology moves from obviously needs supervision to usually works fine, the economic pressure to remove supervision shows up fast, long before the technology is flawless. The system gets good enough that watching it constantly feels like wasted effort, right up until the one time it wasn't.

Dual use, for the record, means the civilian and harmful applications are close enough together that you cannot build one without most of the other. A drone that can locate and track a person is a rescue tool and a surveillance tool using the same underlying capability. The difference is entirely in who holds the controller and what they point it at.

What to watch

Keep an eye on reconstruct specifically. It is the one bottleneck standing between an impressive lab result and an AI that can fly itself somewhere on its own. Once that piece clicks, and the trend line suggests it is a matter of when, not if, the rest of the chain is already sitting there.