Quick author’s note: Sorry I’ve been off for a while! For those who don’t know me, I spent the last year as an investor at an awesome Seed Stage venture fund, Unusual Ventures. I ended up leaving last month — all of this writing about LLMs has given me a pretty strong urge to get back to hands-on work. I’ve spent the last month taking a short break from the AI rush, and then doing a deep dive on a few different topics.
I have some longer articles in the works I’m excited to share with you — I’ve been spending a lot of time on fine-tuning in particular and expect to publish a few thoughts on that adventure in the next few weeks.
For now, let’s chat about OpenAI’s latest announcements!
My OpenAI Dev Day Takeaways
If you follow my newsletter, you’re probably aware that OpenAI announced a lot of new stuff at their Developer Day on Monday. Rather than restate their announcement, I thought I’d share a few thoughts in list format:
Most Exciting Announcements, ranked
Cheaper, faster GPT-4
OpenAI continues to drive down its pricing rapidly, cutting GPT-4-Turbo’s cost ~2.5x from GPT-4. Early testing shows it’s about ~2x faster as well (and they said that they will continue to optimize in the coming weeks). I’ve written about cheaper inference before, and the point stands — faster and cheaper models are necessary for a wide range of potential applications.
Vision via API
I’ve played with various multimodal language/vision models; they’ve been incredibly impressive. We all know what having generalist language models available via API has done to NLP, and I think generalist computer vision models will have a similar impact on computer vision.
Text-to-Speech API
OpenAI announced their new TTS offering, and it’s cheap (~10-20x cheaper than Elevenlabs). I don’t have strong thoughts on this market yet, but it’s interesting that OpenAI is planting its flag here. OpenAI is clearly playing to be the singular provider of API-hosted ML models, and it’s hard to argue that they’re in a great position to succeed in a variety of domains.
Logprobs
This may be niche, but OpenAI announced they plan to re-add log probabilities to their API in the next few weeks. Logprobs are very useful for understanding the output of LLMs — they tell you how “strongly” a language model believes something and can enable much more powerful systems.
New Knowledge Cutoff
Sam Altman implied that OpenAI will push to have their models nearly always up to date — effectively eliminating their “Knowledge Cutoff.” Models that know more about the world are probably better, and that’s nice!
Announcement I’m most skeptical of
Assistant API
I think the Assistant API is useful, but I’m pretty skeptical that the current iteration of AI “assistants” will be around for the long run. The Assistant API makes it easy to bolt AI onto an existing application; I think that applications and AI need to evolve together, and I’m not sure we’ve quite figured out what that will look like well enough for an Assistant API to be a big deal yet.
Biggest ¯\_(ツ)_/¯
Custom GPTs
Custom GPTs are fascinating to me. They are clearly an evolution of plugins (which, as far as I can tell, are a flop). They’re setting up for Custom GPTs to be “Apps” in the OpenAI “App Store” — for this model to work, you have to assume that ChatGPT will become a platform roughly as ubiquitous as the smartphone or desktop computer before it. ChatGPT certainly seems to have enough users to support an App Store, but they certainly spend less time on ChatGPT than on their browser or smartphone. Maybe some killer apps are the first step to changing that? We’ll see.