Advancing Robotics Technology

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data.  2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro  -> RoboCasa produces N (varying visuals)  -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are creating tools to enable everyone in the ecosystem to scale up with us: - RoboCasa: our generative simulation framework (Yuke Zhu). It's fully open-source! Here you go: http://robocasa.ai - MimicGen: our generative action framework (Ajay Mandlekar). The code is open-source for robot arms, but we will have another version for humanoid and 5-finger hands: https://lnkd.in/gsRArQXy - We are building a state-of-the-art Apple Vision Pro -> humanoid robot "Avatar" stack. Xiaolong Wang group’s open-source libraries laid the foundation: https://lnkd.in/gUYye7yt - Watch Jensen's keynote yesterday. He cannot hide his excitement about Project GR00T and robot foundation models! https://lnkd.in/g3hZteCG Finally, GEAR lab is hiring! We want the best roboticists in the world to join us on this moon-landing mission to solve physical AGI: https://lnkd.in/gTancpNK

What if a machine could balance a ball better than a human ever could using only intelligence and motion? This robotic platform does exactly that. With a seamless blend of precision motors and intelligent sensors, it keeps a ball perfectly centered on its surface. The moment the ball begins to move, the platform senses the shift and instantly adjusts its tilt to bring it back to balance. The sensors act like the eyes of the system, constantly watching every tiny motion of the ball. They feed this information to the motors, which respond with exact movements in real time. There is no delay and no visible effort, just smooth and continuous correction. This technology is more than a clever trick. It represents a growing field where machines are able to respond to their environment with speed and accuracy that rivals natural reflexes. From robotics research to future applications in automation and control systems, this platform shows how far intelligent motion has come and how much further it can go.

🚀𝐖𝐞 𝐚𝐫𝐞 𝐢𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐢𝐧𝐠 𝐒𝐦𝐨𝐥𝐕𝐋𝐀-𝟒𝟓𝟎𝐌, 𝐚𝐧 𝐨𝐩𝐞𝐧-𝐬𝐨𝐮𝐫𝐜𝐞 𝐕𝐢𝐬𝐢𝐨𝐧-𝐋𝐚𝐧𝐠𝐮𝐚𝐠𝐞-𝐀𝐜𝐭𝐢𝐨𝐧 𝐦𝐨𝐝𝐞𝐥 𝐟𝐨𝐫 𝐫𝐨𝐛𝐨𝐭𝐢𝐜𝐬! SmolVLA achieves best-in-class performance and inference speed, and the best part? It’s trained entirely on open-source datasets from the 🤖 LeRobot project hosted on the Hugging Face Hub. 🔍 Why is SmolVLA so good? Turns out that pretraining on a large, diverse and noisy collection of real-world community robotics data leads to better generalization and control. We saw a 26% boost in task success rate simply from adding community dataset pretraining! ⚡ Why is SmolVLA so fast? 1. We halved the size of SmolVLM and extract intermediate representations 2. Introduced interleaved cross- and self-attention layers in the action expert 3. Enabled asynchronous inference so the robot acts and reacts simultaneously 💡 Unlike most academic datasets, these community-contributed datasets are naturally diverse: ✅ Multiple robots, camera angles, and manipulation tasks ✅ Real-world messiness and complexity ✅ Crowd-sourced and community-cleaned using Qwen2.5-VL for high-quality task descriptions 🌍 SmolVLA is a step toward making robotics research more affordable, reproducible, and collaborative. 📖 Want to dive deeper? Check out our blog post & start using it today: https://lnkd.in/e3Gmy8gT Huge thanks to the team who made this possible: @Mustafa Shukor Francesco Capuano Remi Cadene, and the entire Lerobot team, amazing HF team Andrés Marafioti Merve Noyan Aritra Roy Gosthipaty Pedro Cuenca Loubna Ben Allal, Thomas Wolf  and to the amazing contributors to the LeRobot community: Ville Kuosmanen, Alexandre Chapin, Marina Barannikov, and more!

Can robots feel the real world? Will we ever truly feel the digital world? Meta just announced three innovations in robotics and touch technology that might bridge this gap: 1. Meta Sparsh – A versatile sensor that can “sense” a variety of textures and objects, thanks to a massive database of 460K+ images. 2. Meta Digit 360 – A fingertip sensor providing detailed, human-like touch data. 3. Meta Digit Plexus – A unified platform making touch-sensing integration smoother for robots. It's amazing how we can build robots with immense strength and precision, machines capable of lifting tons, working tirelessly, and performing with superhuman precision. But despite their physical power, they lack something profoundly human: the sensation of touch. This “small” gap in sensation is actually a massive hurdle. Touch isn’t just a physical feeling; it’s how we interpret the world and make delicate decisions in real time. This leap could revolutionize healthcare, manufacturing, and beyond by enabling robots to work in delicate, precise environments. Kudos to the Meta team for pushing the boundaries! Follow me Endrit Restelica for more.

What does 2025 mean for robots? I recently interviewed Matha Chen, Head of Global Marketing at KEENON Robotics, a cutting-edge robotics company and market leader in service robots, to explore how AI and robotics are transforming our world. Matha shared how Keenon’s service robots - dining, delivery, cleaning, and even a butler bot - are already making life easier in hotels and restaurants around the globe, each uniquely adapted to local needs. My own experience with Keenon’s dining bot in Singapore showed just how seamlessly functional robots fit into our daily routines, navigating obstacles and delivering orders with remarkable precision. While humanoids often capture the spotlight, Matha emphasized that robots come in all shapes and sizes, each serving a specific purpose. And that’s the real takeaway: service robots are already here, quietly reshaping industries without us even noticing. Curious about the next wave of humanoids and special-purpose bots? Check out my full interview with Matha to learn more!

Singapore Hotel Association learning journey in Shenzhen and Hong Kong: on physical robots. I just came back from an excellent journey on the topic of innovation and sustainability in hospitality. Shenzhen, in China, is just opposite to Hong Kong and is part of what is called "the Greater Bay Area" ecosystem, hosting thousands of manufacturing companies there. But don't be mistaken: we are not talking about 20th century old plants. We are talking about high tech manufacturing. The city has much more modern infrastructures than most cities in the world. The robotics fields is doing impressive progress. For hotels, security and cleaning robots for public spaces are completely common, delivery robots from reception to guest rooms as well (especially for economy to midscale hotels) - some solutions start to emerge to allow for brownfield integration without full change of lifts. Housekeeping robots - the dream of hoteliers! - are not ready yet but it is coming fast, see the video in this post: the startup currently showcasing cleaning robots for bathrooms and toilets was just launched 9 months ago! It shows the speed at how these companies move. Human like robots are now capable of impressive balance and movements, even better than most of us, but they are not yet autonomous to recognize the world around them: this is the next wave, enabled with GenAI embedded software which will act as the brain of the robots, able to converse with us and manipulate the physical world. This is already the case in warehouse, logistics where the environment is simple, and I expect that in 2-3 years in China we will see this in hotels - A lot of inspiration to get from hotels in Singapore and elsewhere. #Hospitality #Innovation #China #Technology #Robotics #TheWayForward

Here's why local manufacturing is important for tech innovation. The conventional wisdom says innovation happens in Silicon Valley and manufacturing in Shenzhen. But after three decades in tech, I've learned that separating thinking from making is innovation's biggest bottleneck. When design teams sit continents away from production lines, products get optimised for boardrooms, not reality. The feedback loop stretches from days to quarters. Market insights get lost in translation. By the time products reach end users, the world has moved on. Local manufacturing compresses this cycle dramatically. Engineers can walk the factory floor in the morning and redesign by afternoon. Quality issues become innovation opportunities in real-time. More importantly, proximity to actual users sparks insights that distant R&D centres might miss entirely. Consider India's unique challenges - extreme temperatures, voltage fluctuations, dust, humidity variations. Products designed for controlled environments fail spectacularly here. But when manufacturing happens locally, these constraints become innovation drivers. Suddenly, products emerge that work not just in ideal conditions but in real-world chaos. The ecosystem effect multiplies this impact. Local suppliers stop being just vendors - they become innovation partners. Educational institutions align with industry needs. Startups emerge to solve niche problems. The entire value chain starts thinking, not just executing. Critics point to global supply chain efficiencies. True, but efficiency without relevance is meaningless. The technology that transforms lives in Tier 3 cities needs fundamentally different innovation than what works in Taipei or Toronto. Innovation isn't about where you think. It's about how close you are to the problems worth solving. . . #TechInnovation #LocalManufacturing #MakeInIndia #ProductDesign #HardwareInnovation #TechForIndia #ProductDevelopment

Reindustrializing #Europe in the age of AI 🤖”—our latest report outlines what it will take: Amid intensifying global competition in AI and #Robotics, Europe faces a defining moment: reindustrialize or risk falling irreversibly behind. Robotics can help restore industrial sovereignty, address demographic headwinds, and boost productivity. We propose a 5-point strategic roadmap to reposition Europe as a credible competitor alongside the US and China: 1️⃣ A European Robotics Roadmap – Focus on building champions in high-impact, under-robotized sectors: logistics, hospitality, agrifood, healthcare, aerospace, and defense. Prioritize strategic autonomy, not chasing lost ground in humanoids or autonomous vehicles. 2️⃣ Capital Access for Robotics Startups – Address the 7x VC funding gap with the US by scaling Europe’s venture capital market and reinforcing complementary funding streams. 3️⃣ Bridging Innovation and Market – Tackle fragmentation through innovation clusters, regional champions, and greater public-private investment coordination. We recommend increasing the 2028–2034 EU budget by at least 5% with a dedicated robotics allocation. 4️⃣ Upskilling the Workforce – Tackle skill shortages across factory floors and engineering teams. From frontline operators to system integrators, we need a unified "Robot Skills Framework" and modern vocational training. 5️⃣ Smart Regulation – Align AI and robotics regulation to promote innovation. Use regulatory sandboxes, harmonized safety standards, and dynamic, risk-based approaches to support adoption—especially among SMEs. 📘 Download the full report: https://lnkd.in/evxEPDgn #Robotics #AI #IndustrialPolicy #Reindustrialization #Innovation #VentureCapital #FutureOfWork #TechSovereignty #Automation #Manufacturing #Ludonomics #AllianzTrade #Allianz

Tesla has unvelied Optimus Gen 2 and it’s a remarkable leap forward in humanoid robotics. This upgraded model boasts a range of improvements aimed at boosting its functionality and efficiency. It sports Tesla-designed actuators and sensors, a 2-Degree of Freedom (DoF) actuated neck for more human-like head movements, and enhanced balance and mobility. It can now walk 30% faster than its predecessor and features feet that mimic human foot geometry, including articulated toe sections. Even with these enhancements, its weight has been reduced by 10 kilograms without compromising its structural integrity. One standout feature is its advanced hands with 11-DoF and tactile sensing on all fingers. This enables the robot to manipulate delicate objects with impressive precision, as demonstrated in a video where it gently handles an egg. It even showcases its potential in tasks like cooking eggs, indicating its suitability for intricate work. Musk envisions Optimus as a pivotal element in automating physical labor, potentially revolutionizing manufacturing processes. He believes that, in the future, physical work could become optional, with Optimus playing a key role in this transformation. He has gone as far as suggesting that Optimus's development and success could overshadow Tesla's electric car business. While an official release date and price remain unknown, early versions may undergo testing on vehicle production lines, with a commercial version possibly available to customers in three to five years. In the context of Embodied AI versus AGI (Artificial General Intelligence), Optimus Gen 2 signifies a significant stride in Embodied AI. It operates within a physical entity, showcasing AI's potential in performing various physical tasks. AGI, conversely, represents a more advanced form of AI with the ability to understand, learn, and broadly apply intelligence, akin to human cognitive capabilities. Both fields hold immense promise and have distinct implications for technology and society. Progress in Embodied AI, exemplified by Tesla's Optimus Gen 2, underscores the immediate applications of AI in robotics and automation. AGI, though more theoretical at this stage, envisions a future where AI can handle diverse tasks and make decisions with human-like understanding and adaptability. It's a fascinating journey in the realm of AI development. Just imagine the not too distant future when these capabilitesa are fully integrated !! Wow. #ai #gai #robotics #embodiedai

Imagine smarter robots for your business. New research from Google puts advanced Gemini AI directly into robots, which can now understand complex instructions, perform intricate physical tasks with dexterity (like assembly) and adapt to new objects or situations in real time. The paper introduces "Gemini Robotics," a family of AI models based on Google's Gemini 2.0, designed specifically for robotics. They present Vision-Language-Action (VLA) models capable of direct robot control, performing complex, dexterous manipulation tasks smoothly and reactively. The models demonstrate generalization to unseen objects and environments and can follow open-vocabulary instructions. It also introduces "Gemini Robotics-ER" for enhanced embodied reasoning (spatial/temporal understanding, detection, prediction), bridging the gap between large multimodal models and physical robot interaction. Here's why this matters: At scale, this will unlock more flexible, intelligent automation for the future of manufacturing, logistics, warehousing, and more, potentially boosting efficiency and enabling tasks previously too complex for robots as we've imagined in the past. Very, very promising! (Link in the comments.)