Chip Huyen
Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix), covering AI product work, product design, and engineering tradeoffs.
Episode
Al Engineering 101 with Chip Huyen (Nvidia, Stanford, Netflix)
Summary
Author of 'AI Engineering' gives a ground-up explanation of how AI systems actually work — pre-training, post-training, fine-tuning, RAG, reinforcement learning — and why most companies that try to build AI products fail because they don't understand the problem they're solving. She argues the industry is in an 'idea crisis' where people are enamored with tools but have lost sight of user problems.
Key Takeaways
Stop keeping up with every new AI release. Companies that improve AI products fastest obsessively read user feedback rather than immediately upgrading to the newest model.
Most companies don't need to fine-tune. RAG solves the majority of cases where you need the model to know about proprietary data. Only fine-tune when you need to change model behavior, not knowledge.
Don't over-invest in evals at early stages. If traffic is growing and customers are happy, a vibe check may be sufficient. Invest in formal evals at scale.
RLHF is what made ChatGPT feel different from raw GPT-3. Understanding this helps you reason about why models behave the way they do.
The AI industry is in an 'idea crisis,' not a technology crisis. User empathy and problem definition are the scarce resource, not model capability.
Notable Quotes
“I'm not sure if I'm bearish on it. I think I'm curious because I think things has a way of work out in ways that I don't expect. So I think that maybe these companies, they have a lot of data, maybe they wouldn't be able to use that to have some insight that helps them stay ahead of the curve. So I don't know.”
“I'm shocked that we don't have better voice assistant at home yet. I think I have been testing out a bunch, honestly. I keep hoping, oh my God, that could be the one and then I know how many of them I just had to give away because they're not that good.”
“So I watched a lot of movie and TV shows as a research because I working on my first novel and I recently sold it. So I'm interested what makes, it's a drama. It's not a science fiction or anything that tech people usually read. So it very, I know it's a very out of left field and very, so it's almost like reading, watching TV to see what kind of stories become popular, trying to understand the trope and stuff like that. So I'm not sure if the audience will like...”