When testing an AI model, it’s hard to tell if it is reasoning or just regurgitating answers from its training data. Xbench, a new benchmark developed by the Chinese venture capital firm HSG, or HongShan Capital Group, might help to sidestep that issue. That’s thanks to the way it evaluates models not only on the…
A large language model (LLM) deployed to make treatment recommendations can be tripped up by nonclinical information in patient messages, like typos, extra white space, missing gender markers, or the use of uncertain, dramatic, and informal language, according to a study by MIT researchers. They found that making stylistic or grammatical changes to messages increases…
May 8 AI Codecon was a huge success. We had amazing speakers and content. We also had over 9,000 live attendees and another 12,000 who signed up to be able to view the content later on the O’Reilly learning platform. (Here’s a post with video excerpts and some of my takeaways.) So we’re doing it…
Launched in February of this year, the MIT Generative AI Impact Consortium (MGAIC), a presidential initiative led by MIT’s Office of Innovation and Strategy and administered by the MIT Stephen A. Schwarzman College of Computing, issued a call for proposals, inviting researchers from across MIT to submit ideas for innovative projects studying high-impact uses of…
This is the last of four parts in this series. Part 1 can be found here, Part 2 here, and Part 3 here. 9. Future Directions and Wishlist for MCP The trajectory of MCP and AI tool integration is exciting, and there are clear areas where the community and companies are pushing things forward. Here…
It feels like every other AI announcement lately mentions “agents.” And already, the AI community has 2025 pegged as “the year of AI agents,” sometimes without much more detail than “They’ll be amazing!” Often forgotten in this hype are the fundamentals. Everybody is dreaming of armies of agents, booking hotels and flights, researching complex topics,…
AI companions like Replika are designed to engage in intimate exchanges, but people use general-purpose chatbots for sex talk too, despite their stricter content moderation policies. Now new research shows that not all chatbots are equally willing to talk dirty: DeepSeek is the easiest to convince. But other AI chatbots can be enticed too, if…
A new paper from OpenAI released today has shown why a little bit of bad training can make AI models go rogue but also demonstrates that this problem is generally pretty easy to fix. Back in February, a group of researchers discovered that fine-tuning an AI model (in their case, OpenAI’s GPT-4o) by training it…
When OpenAI acquired Io to create “the coolest piece of tech that the world will have ever seen,” it confirmed what industry experts have long been saying: Hardware is the new frontier for AI. AI will no longer just be an abstract thing in the cloud far away. It’s coming for our homes, our rooms,…
During his first year at MIT in 2021, Matthew Caren ’25 received an intriguing email inviting students to apply to become members of the MIT Schwarzman College of Computing’s (SCC) Undergraduate Advisory Group (UAG). He immediately shot off an application. Caren is a jazz musician who majored in computer science and engineering, and minored in music…