Let’s be real: Building LLM applications today feels like purgatory. Someone hacks together a quick demo with ChatGPT and LlamaIndex. Leadership gets excited. “We can answer any question about our docs!” But then…reality hits. The system is inconsistent, slow, hallucinating—and that amazing demo starts collecting digital dust. We call this “POC purgatory”—that frustrating limbo where…
Thomas Wolf’s blog post “The Einstein AI Model” is a must-read. He contrasts his thinking about what we need from AI with another must-read, Dario Amodei’s “Machines of Loving Grace.”1 Wolf’s argument is that our most advanced language models aren’t creating anything new; they’re just combining old ideas, old phrases, old words according to probabilistic…
On April 22, 2022, I received an out-of-the-blue text from Sam Altman inquiring about the possibility of training GPT-4 on O’Reilly books. We had a call a few days later to discuss the possibility. As I recall our conversation, I told Sam I was intrigued, but with reservations. I explained to him that we could…
It almost sounds pejorative, doesn’t it? But the distinction between senior and junior software developers is built into our jobs and job titles. Whether we call it entry-level or something else, we distinguish between people who are just starting their careers and those who have been around for a while. We’re all still learning (one…
Most AI teams focus on the wrong things. Here’s a common scene from my consulting work: AI TEAMHere’s our agent architecture—we’ve got RAG here, a router there, and we’re using this new framework for… ME[Holding up my hand to pause the enthusiastic tech lead]Can you show me how you’re measuring if any of this actually…
For the past decade and a half, I’ve been exploring the intersection of technology, education, and design as a professor of cognitive science and design at UC San Diego. Some of you might have read my recent piece for O’Reilly Radar where I detailed my journey adding AI chat capabilities to Python Tutor, the free…
In a recent edition of The Sequence Engineering newsletter, “Why Did MCP Win?,” the authors point to context serialization and exchange as a reason—perhaps the most important reason—why everyone’s talking about the Model Context Protocol. I was puzzled by this—I’ve read a lot of technical and semitechnical posts about MCP and haven’t seen context serialization…
Most companies find that the biggest challenge to AI is taking a promising experiment, demo, or proof-of-concept and bringing it to market. McKinsey Digital Analyst Rodney Zemmel sums this up: It’s “so easy to fire up a pilot that you can get stuck in this ‘death by 1,000 pilots’ approach.” It’s easy to see AI’s…
Last November, Bradford G. Smith got a brain implant from Elon Musk’s company Neuralink. The device, a set of thin wires attached to a computer about the thickness of a few quarters that sits in his skull, lets him use his thoughts to move a computer pointer on a screen. And by last week he…
What would a behind-the-scenes look at a video generated by an artificial intelligence model be like? You might think the process is similar to stop-motion animation, where many images are created and stitched together, but that’s not quite the case for “diffusion models” like OpenAl’s SORA and Google’s VEO 2. Instead of producing a video…