Gemini Robotics – AI for the Physical World, with Keerthana and Ted of Google DeepMind
In this engaging episode of the Cognitive Revolution, host Nathan Labenz welcomes guests Keerthana Gopalakrishnan and Ted Xiao to revisit significant advancements in robotics over the past year. Key themes discussed include the proliferation of new robotics companies, the emergence of humanoid robots, and the development of sophisticated foundation models. The conversation highlights the transformative potential of imitation learning, the evolution from simple lab-based tasks to complex, real-world deployments, and the critical role of hardware in pushing the boundaries of AI capabilities. With insights on various aspects of robot dexterity, safety, and the practical steps towards deploying robots in everyday environments, the episode provides a comprehensive overview of the current state and future directions of the robotics landscape. Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive The AGNTCY: The AGNTCY is an open-source collective dedicated to building the Internet of Agents, enabling AI agents to communicate and collaborate seamlessly across frameworks. Join a community of engineers focused on high-quality multi-agent software and support the initiative at https://agntcy.org/ Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) About the Episode (04:05) Intro (04:48) Imitation Learning and Humanoid Advancements (06:21) Commercialization and Community Growth (06:41) Comparing Robotics to Language Models (08:11) Scaling and Fine-Tuning in Robotics (12:30) Embodied Reasoning and ERQA Benchmark (13:59) Gemini Robotics: Technical Insights (Part 1) (19:07) Sponsors: ElevenLabs | Oracle Cloud Infrastructure (OCI) (21:34) Gemini Robotics: Technical Insights (Part 2) (22:39) Model Architecture and Distributed Systems (Part 1) (30:45) Sponsors: The AGNTCY | Shopify | NetSuite (35:07) Model Architecture and Distributed Systems (Part 2) (37:17) Real-World Applications and Safety (52:52) Failures and Safety Measures (59:35) Current State of Robot Safety (01:02:05) Deployment Challenges and Strategies (01:04:44) Data Collection and Scaling in Robotics (01:08:00) Synthetic vs. Real-World Data (01:11:58) Future of Robotics and AI Integration (01:28:05) Fine-Tuning and Task-Specific Performance (01:34:26) Embodiments and Hardware Interplay (01:37:44) Humanoids: The Next Frontier (01:40:01) Future Prospects and Challenges (01:47:52) Outro SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Video lezen · Transcriptie en inzichten
Deze aflevering heeft een volledige transcriptie en AI-inzichten
Gratis account · geen creditcard nodig · 150 credits bij aanmelding, genoeg om deze aflevering te ontgrendelen
- 📄 Volledige transcriptie met tijdcodes
- ✨ AI-samenvatting, trefwoorden en mindmap
- 💡 Conclusies en citaten
Tijdlijn van de aflevering
Introduction to the episode and the Gemini Robotics release from Google DeepMind
- The host frames robotics as a major blind spot in AI impact forecasts, noting that robots are not yet widely available for people to experiment with directly.
- Returning guests Keerthana Gopalakrishnan and Ted Xiao of Google DeepMind discuss their recent Gemini Robotics technical report, which brings AI into the physical world.
- Keerthana previously described robotics as being in its GPT-2 era; she now places it somewhere between GPT-3 and GPT-3.5.
- The team created two distilled models: a cloud-based Gemini Robotics embodied reasoning model updating plans every 250 milliseconds, and a smaller vision-language-action model running partly on-device at 50 cycles per second.
- The host's bottom line is that robotics trends suggest it consistently lags the LLM wave by three to four years, implying a possible GPT-4 moment for robotics within one to two years.
What has changed in robotics over the past year
- Keerthana notes that imitation learning now demonstrably works, humanoid walking is working for many groups, and VLAs are proliferating alongside very cheap hardware.
- Ted argues the biggest game-changer is that the community advanced its goalposts beyond academic tabletop pick-and-place toward realistic deployment settings and commercialization.
- The same canned in-lab demos that would have amazed people a year ago are now mundane, and any new release today is expected to feature humanoids, dexterous hands, or in-the-wild settings.
- VCs now understand the space, bringing more funding and companies, which changes how players act and how open the state of the art becomes.
Where robotics sits on the GPT timeline and why the ChatGPT moment metaphor breaks down
- Keerthana argues robotics has not reached its ChatGPT moment because that moment depended on ubiquitous consumer hardware like phones and laptops, whereas experiencing a robot brain requires owning a robot.
- She describes a chicken-and-egg problem: people need to know models are capable to buy robots, but robots must exist first to generate the data that makes models capable.
- Ted places robotics technically between GPT-3 and GPT-3.5, because that was when language models first worked out of the box across varied settings rather than requiring fine-tuning.
- Ted notes that scaling laws and predictable engineering of scale, which turned language modeling into a science around the GPT-3 era, are only now on the horizon for robotics.
- Keerthana observes that actions are progressing more slowly than reasoning, since reasoning can borrow from general vision research while action scaling laws remain poorly understood.
Kernbegrippen
- Gemini Robotics— Google DeepMind's new robotics foundation model family that brings AI into the physical world.
- embodied reasoning— The high-level spatial and physical understanding capability that Gemini Robotics ER is specifically trained for.
- vision-language-action model— The VLA model that outputs low-level motor commands and runs distributed between cloud and robot.
Opvallende citaten
I don't think we have gotten to chad GPT yet and in fact a lot of people keep saying a chad GPT moment for robotics but I personally think it's or the proliferation of robots is not going to look like that just because for charge GPT everyone could experience it because consumer hardware was everywhere.
💡— It overturns the common assumption that robotics will have a single ChatGPT-like moment, because robots require physical hardware that most people do not own.
I think maybe the biggest gamecher in my mind is that the community broadly has really advanced the goalposts beyond I think the academic lab setting simple tabletop pick and place from a decade ago and really everyone kind of transitioned last year I would say to more advanced embodiment more realistic deployment settings a lot of players started thinking about commercialization so very high bars for robustness and performance in generality
🤯— It reveals that the field's baseline has shifted so fast that demos which would have amazed people a year ago are now considered mundane.
Praktische conclusies
🤖AI and Robotics Strategy
Robotics is consistently three to four years behind the LLM wave, and the GPT-4 moment may arrive in one to two years.
Map your current AI product roadmap against robotics timelines and identify one area where a physical-world pilot could be relevant within 24 months.
The field has moved beyond simple tabletop pick-and-place to realistic deployment settings with high bars for robustness and generality.
Audit your assumptions about robot capabilities by watching the Gemini Robotics demo videos and noting which tasks now look feasible.
⚙️Technical Architecture
The winning architecture splits high-level reasoning in the cloud from fast on-device action decoding, with modifiable interfaces between them.
Sketch a similar split for any real-time AI system you are building, separating slow reasoning from fast control loops.
The action model is distributed, not just on-device, and can be composed with keypoint-based methods or run end-to-end.
Experiment with a small local model for low-latency output while keeping a larger model in the cloud for planning.
Transcripties en inzichten zijn door AI gegenereerd en kunnen fouten bevatten. De nauwkeurigheid hangt af van de audiokwaliteit en verstaanbaarheid van sprekers; bij twijfel is de oorspronkelijke audio altijd leidend.
Afleveringen en video's klaar om te lezen
Podcastafleveringen
Wisdom for Traffic Jams - Ecclesiastes 6:10-7:14
Willingdon Church Podcast
19 jul 202633:48EN
How gratitude rewires your brain | Christina Costa
TED Talks Daily
6 aug 202612:55EN
PA Replay: GOING ALL IN....What You Need To Know
The Pure Athlete Podcast
14 jul 202652:14EN
The Open Special met JulieLikesGolf | DeChambeau zorgt voor ophef & Ryan Fox verrast | 18 holes Oostende ⛳
PAMPAS Golfpodcast
20 jul 20261:08:22NL
No.209 为什么西游记是一本商业名著？
三五环
18 nov 202544:23ZH
Jejum e Autoridade Espiritual com Teo Hayashi e Todd White | EP 89
Dunamis Hangout
6 feb 20251:07:04PT
Video's
Anthropic CEO Dario Amodei: AI's Potential, OpenAI Rivalry, GenAI Business, Doomerism
Alex Kantrowitz
30 jul 20251:08:37EN
GPT-6 Astra: How I’d Make Money With It
Greg Isenberg
10 sep 202622:52EN
Uber Sells the Dream, Waymo Logs the Autonomous Miles
The Road to Autonomy
28 feb 202653:54EN
#40 Karl IV. und sein goldenes Prag
99 mal Geschichte
8 jan 20261:03:51DE
Firma bez šéfov: Funguje to? - Money Talk 113 s Ferom Baníkom
Milan Dubec
4 aug 202654:10SK