Gemini Robotics – AI for the Physical World, with Keerthana and Ted of Google DeepMind
In this engaging episode of the Cognitive Revolution, host Nathan Labenz welcomes guests Keerthana Gopalakrishnan and Ted Xiao to revisit significant advancements in robotics over the past year. Key themes discussed include the proliferation of new robotics companies, the emergence of humanoid robots, and the development of sophisticated foundation models. The conversation highlights the transformative potential of imitation learning, the evolution from simple lab-based tasks to complex, real-world deployments, and the critical role of hardware in pushing the boundaries of AI capabilities. With insights on various aspects of robot dexterity, safety, and the practical steps towards deploying robots in everyday environments, the episode provides a comprehensive overview of the current state and future directions of the robotics landscape. Upcoming Major AI Events Featuring Nathan Labenz as a Keynote Speaker https://www.imagineai.live/ https://adapta.org/adapta-summit https://itrevolution.com/product/enterprise-tech-leadership-summit-las-vegas/ SPONSORS: ElevenLabs: ElevenLabs gives your app a natural voice. Pick from 5,000+ voices in 31 languages, or clone your own, and launch lifelike agents for support, scheduling, learning, and games. Full server and client SDKs, dynamic tools, and monitoring keep you in control. Start free at https://elevenlabs.io/cognitive-revolution Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive The AGNTCY: The AGNTCY is an open-source collective dedicated to building the Internet of Agents, enabling AI agents to communicate and collaborate seamlessly across frameworks. Join a community of engineers focused on high-quality multi-agent software and support the initiative at https://agntcy.org/ Shopify: Shopify powers millions of businesses worldwide, handling 10% of U.S. e-commerce. With hundreds of templates, AI tools for product descriptions, and seamless marketing campaign creation, it's like having a design studio and marketing team in one. Start your $1/month trial today at https://shopify.com/cognitive NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive PRODUCED BY: https://aipodcast.ing CHAPTERS: (00:00) About the Episode (04:05) Intro (04:48) Imitation Learning and Humanoid Advancements (06:21) Commercialization and Community Growth (06:41) Comparing Robotics to Language Models (08:11) Scaling and Fine-Tuning in Robotics (12:30) Embodied Reasoning and ERQA Benchmark (13:59) Gemini Robotics: Technical Insights (Part 1) (19:07) Sponsors: ElevenLabs | Oracle Cloud Infrastructure (OCI) (21:34) Gemini Robotics: Technical Insights (Part 2) (22:39) Model Architecture and Distributed Systems (Part 1) (30:45) Sponsors: The AGNTCY | Shopify | NetSuite (35:07) Model Architecture and Distributed Systems (Part 2) (37:17) Real-World Applications and Safety (52:52) Failures and Safety Measures (59:35) Current State of Robot Safety (01:02:05) Deployment Challenges and Strategies (01:04:44) Data Collection and Scaling in Robotics (01:08:00) Synthetic vs. Real-World Data (01:11:58) Future of Robotics and AI Integration (01:28:05) Fine-Tuning and Task-Specific Performance (01:34:26) Embodiments and Hardware Interplay (01:37:44) Humanoids: The Next Frontier (01:40:01) Future Prospects and Challenges (01:47:52) Outro SOCIAL LINKS: Website: https://www.cognitiverevolution.ai Twitter (Podcast): https://x.com/cogrev_podcast Twitter (Nathan): https://x.com/labenz LinkedIn: https://linkedin.com/in/nathanlabenz/ Youtube: https://youtube.com/@CognitiveRevolutionPodcast Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431 Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk
Leggi video · Trascrizione e analisi
Questo episodio ha una trascrizione completa e analisi AI
Account gratuito · nessuna carta necessaria · 150 crediti all'iscrizione, sufficienti per sbloccare questo episodio
- 📄 Trascrizione completa con timestamp
- ✨ Riepilogo AI, parole chiave e mappa mentale
- 💡 Conclusioni e citazioni chiave
Cronologia dell'episodio
Introduction to the episode and the Gemini Robotics release from Google DeepMind
- The host frames robotics as a major blind spot in AI impact forecasts, noting that robots are not yet widely available for people to experiment with directly.
- Returning guests Keerthana Gopalakrishnan and Ted Xiao of Google DeepMind discuss their recent Gemini Robotics technical report, which brings AI into the physical world.
- Keerthana previously described robotics as being in its GPT-2 era; she now places it somewhere between GPT-3 and GPT-3.5.
- The team created two distilled models: a cloud-based Gemini Robotics embodied reasoning model updating plans every 250 milliseconds, and a smaller vision-language-action model running partly on-device at 50 cycles per second.
- The host's bottom line is that robotics trends suggest it consistently lags the LLM wave by three to four years, implying a possible GPT-4 moment for robotics within one to two years.
What has changed in robotics over the past year
- Keerthana notes that imitation learning now demonstrably works, humanoid walking is working for many groups, and VLAs are proliferating alongside very cheap hardware.
- Ted argues the biggest game-changer is that the community advanced its goalposts beyond academic tabletop pick-and-place toward realistic deployment settings and commercialization.
- The same canned in-lab demos that would have amazed people a year ago are now mundane, and any new release today is expected to feature humanoids, dexterous hands, or in-the-wild settings.
- VCs now understand the space, bringing more funding and companies, which changes how players act and how open the state of the art becomes.
Where robotics sits on the GPT timeline and why the ChatGPT moment metaphor breaks down
- Keerthana argues robotics has not reached its ChatGPT moment because that moment depended on ubiquitous consumer hardware like phones and laptops, whereas experiencing a robot brain requires owning a robot.
- She describes a chicken-and-egg problem: people need to know models are capable to buy robots, but robots must exist first to generate the data that makes models capable.
- Ted places robotics technically between GPT-3 and GPT-3.5, because that was when language models first worked out of the box across varied settings rather than requiring fine-tuning.
- Ted notes that scaling laws and predictable engineering of scale, which turned language modeling into a science around the GPT-3 era, are only now on the horizon for robotics.
- Keerthana observes that actions are progressing more slowly than reasoning, since reasoning can borrow from general vision research while action scaling laws remain poorly understood.
Concetti chiave
- Gemini Robotics— Google DeepMind's new robotics foundation model family that brings AI into the physical world.
- embodied reasoning— The high-level spatial and physical understanding capability that Gemini Robotics ER is specifically trained for.
- vision-language-action model— The VLA model that outputs low-level motor commands and runs distributed between cloud and robot.
Citazioni rilevanti
I don't think we have gotten to chad GPT yet and in fact a lot of people keep saying a chad GPT moment for robotics but I personally think it's or the proliferation of robots is not going to look like that just because for charge GPT everyone could experience it because consumer hardware was everywhere.
💡— It overturns the common assumption that robotics will have a single ChatGPT-like moment, because robots require physical hardware that most people do not own.
I think maybe the biggest gamecher in my mind is that the community broadly has really advanced the goalposts beyond I think the academic lab setting simple tabletop pick and place from a decade ago and really everyone kind of transitioned last year I would say to more advanced embodiment more realistic deployment settings a lot of players started thinking about commercialization so very high bars for robustness and performance in generality
🤯— It reveals that the field's baseline has shifted so fast that demos which would have amazed people a year ago are now considered mundane.
Conclusioni applicabili
🤖AI and Robotics Strategy
Robotics is consistently three to four years behind the LLM wave, and the GPT-4 moment may arrive in one to two years.
Map your current AI product roadmap against robotics timelines and identify one area where a physical-world pilot could be relevant within 24 months.
The field has moved beyond simple tabletop pick-and-place to realistic deployment settings with high bars for robustness and generality.
Audit your assumptions about robot capabilities by watching the Gemini Robotics demo videos and noting which tasks now look feasible.
⚙️Technical Architecture
The winning architecture splits high-level reasoning in the cloud from fast on-device action decoding, with modifiable interfaces between them.
Sketch a similar split for any real-time AI system you are building, separating slow reasoning from fast control loops.
The action model is distributed, not just on-device, and can be composed with keypoint-based methods or run end-to-end.
Experiment with a small local model for low-latency output while keeping a larger model in the cloud for planning.
Trascrizione e analisi sono generate dall'AI e possono contenere errori. L'accuratezza dipende dalla qualità dell'audio e dalla chiarezza dei parlanti: in caso di dubbi, l'audio originale resta la fonte attendibile.
Episodi e video pronti da leggere
Episodi podcast
9 Rules for a Happier Vacation
Office Hours with Arthur Brooks
15 giu 202642:24EN
Raising a Dog & Mastering Calm Assertive Energy | Cesar Millan
Huberman Lab
6 lug 20262:38:23EN
The Total Christian Man - 1 - Major Thomas
Torchbearers Archives
3 set 202555:23EN
EP79 你不是不自律，你只是被劫持了注意力
纵横四海
30 mar 20264:28:52ZH
Zweifel-Los! 15 - Das Leben wie ein Profi leben - Peter Ballmer
Zweifel-Los!
24 lug 202424:33DE
Café com Deus Pai | 24 de maio
Café Com Deus Pai | Podcast oficial
24 mag 20264:44PT
Video
Finally, some truth about loop engineering
NeetCode
28 lug 202617:18EN
Tom Holland and Jon Bernthal Team Up While Eating Spicy Wings | Hot Ones
First We Feast
23 lug 202629:21EN
Dario Amodei — “We are near the end of the exponential”
Dwarkesh Patel
13 feb 20262:22:01EN
#32 Die Schlacht von Worringen - Der Freiheitskampf der Kölner
99 mal Geschichte
12 nov 202553:29DE
«Современный урок по ФГОС: требования, этапы, цифровые решения»
ЯКласс
11 apr 20231:39:24RU