Andrej Karpathy describes AI’s ‘sharp funnel’ of access

OpenAI cofounder Andrej Karpathy has revisited his April argument about how people judge AI, describing a “sharp funnel” of access to its most capable systems. He argues that people asking chatbots quick questions and researchers inside AI labs can encounter very different levels of capability.

Karpathy traces that gap, covered in The Rundown, to access tiers, the scale of the task, and model strengths in areas where answers are checkable and commercially valuable, such as code and math.

Different tasks reveal different capabilities

In his April 30 account of a Sequoia talk, Karpathy argued that model strengths reflect both the tasks labs emphasize during training and whether results can be checked.

An OpenAI study published September 15, 2025 found that practical guidance, information seeking and writing made up nearly 80% of consumer conversations. Programming accounted for 4.2% of messages. The research examined ChatGPT activity in 2024 and 2025 across Free, Plus and Pro accounts, excluding business and enterprise plans.

In a February 11 engineering account, OpenAI said Codex generated the code for an internal software beta, with engineers building the tools, working environment and feedback loops around it. Some individual agent runs exceeded six hours, the company reported.

Why it matters

Karpathy’s observation helps explain a familiar disconnect in conversations about AI. A person judging a chatbot by quick answers may have little exposure to the sustained technical work some agents already handle. Their expectations can lag behind capabilities that someone working closely with those systems sees firsthand.

For workers and managers, the choice of test matters. A trial on a familiar task with a checkable result can reveal capabilities that casual chat misses. Its value still depends on the time spent reviewing, correcting and fitting the output into the job. Comparisons need to account for the model, its tools and the time it had to work.

A September 25 Anthropic account illustrates how a checked result can change expectations. It describes Claude Science completing a difficult theoretical physics calculation with limited guidance. Lance Dixon checked the result. The work applied established methods. Anthropic paid author Matt von Hippel for the article, and Dixon received Claude usage credits.

Checking those results still takes work. OpenAI’s coding account describes human review capacity becoming a constraint. More generated output can increase the burden on the people who must check it. Reliability on other tasks still needs testing.

Internal systems add another gap. On October 6, OpenAI reported mathematical results from an internal model and said it was working toward releasing it. The announcement gave outsiders selected outputs to inspect before they could test the system themselves.

For the public and regulators, internal models can make the disconnect harder to resolve. Insiders may experience rapid progress before outsiders can examine the same systems. The size of that gap in understanding remains unclear. Evaluation access, with tools and operating conditions documented, would help regulators assess those capabilities.