Traditional industrial robots are pivotal in performing tasks such as material handling, machine tending, welding, assembly, quality inspection and more. Often these robots operate in isolation away from human workers. However, this approach does not fully maximize productivity as it fails to leverage the potential of human-robot collaboration.

Enter Proxie, a groundbreaking collaborative robot (cobot) developed by Cobot. Unlike traditional robots, Proxie is designed to operate alongside humans in shared workspaces like manufacturing facilities, logistics, warehouses, and hospitals.

Physical AI is transforming how robots like Proxie perceive and perform tasks in the real world. Physical AI models can understand, interact with, and navigate the physical world with generative AI. NVIDIA Omniverse and Isaac empower robots to learn, adapt, and make decisions in complex collaborative environments by combining accelerated computing, multimodal physical AI, and large-scale physically based simulations.

Three computers are needed to train, test, and deploy physical AI for robots - the first computer for training AI models, the second for robotic simulation and validating the entire AI stack, and the third to deploy the AI stack.

Taking a simulation-first approach, Cobot is tapping into NVIDIA Isaac Sim to validate Proxie’s robot software stack and NVIDIA Isaac ROS to efficiently deploy AI models to NVIDIA Jetson onboard computing systems.

A Simulation-First Approach

Proxie is outfitted with an array of sensors including cameras, LiDAR, and inertial sensors. These sensors feed directly into onboard perception AI software, enabling the robot to effectively understand and navigate complex environments.

Before putting Proxie into its first real-world deployment, Cobot started by building a simulation, or digital twin, of their customer’s cross-dock operation. No two factories, warehouses, or spaces are the same, and the dynamic environments make it difficult or impossible to capture all scenarios that Proxie might encounter in real-world operations.

To overcome these challenges, Cobot engineers employed NVIDIA Isaac Sim, a reference application built on NVIDIA Omniverse that enables developers to simulate and test AI-driven robots in physically based virtual environments.

Isaac Sim harnesses the modularity and extensibility of Universal Scene Description (OpenUSD) and the fidelity of NVIDIA RTX ray tracing. This enables developers to craft custom simulators or seamlessly integrate core Isaac Sim technologies into their existing workflows, supporting synthetic data generation, software-in-loop testing, and advanced robot learning algorithms.

To start, Cobot engineers created digital twins of their customer’s site and their test lab. This comprehensive digital twin was populated with digital replicas of real-world objects such as pallets, boxes, carts, dock doors, and shelves. The team leveraged SimReady, or Simulation Ready, assets which serve as digital twin building blocks, preconfigured with semantic labeling, attributes, and physical properties.

These digital twins provide a virtual proving ground for Proxie.

Once the virtual proving ground was built, the engineers imported a 3D version of Proxie, complete with wheels, motors and sensors. In this virtual world, Cobot engineers put Proxie through its paces by running multiple simulation scenarios that captured various operations, which included the placement of obstacles, people, and other robots in Proxie’s path.

With Isaac Sim’s ability to simulate Proxie’s on board sensors and accurately model physics, Cobot engineers were able to test, refine, and optimize Proxie’s planning and control algorithms. By reducing reliance on physical prototypes, Cobot not only enhanced the safety of their testing processes, but also achieved significant improvement in time-to-deployment. Cobot was able to deploy its first few Proxies to start moving carts in a customer site in less than six weeks from when the first Proxie came off the assembly line.

Physical AI Powers Proxie for Autonomous Operation

The final step in Proxie's development involves deploying its advanced AI to ensure seamless real-world operations. Proxie's Scout Sense perception system operates at human eye level, enabling it to make informed decisions with precision and awareness, harmonizing seamlessly with its environment. Scout Sense utilizes NVIDIA Isaac accelerated libraries and AI models running on NVIDIA AGX Jetson Orin modules. Jetson Orin’s high-performance capabilities and low-latency AI processing help ensure that Proxie is responsive to its ever-changing environment.

The team at Cobot is using NVIDIA Isaac ROS, a collection of accelerated computing packages and AI models, to easily deploy ML models onto its robot that is already leveraging the existing ROS 2 ecosystem. By using Isaac ROS, Cobot can take advantage of the GPUs on its NVIDIA AGX Orin compute platform.

The first Orin module handles critical tasks such as Simultaneous Localization and Mapping (SLAM). This technology maps environments while generating cost maps to help the robot navigate safely, while processing extensive camera data for scene perception.

The second Orin module focuses on human-machine interaction. It utilizes large-language models to allow operators to command Proxie using simple voice instructions. Cobot’s proprietary Auditable Collaboration and Planning Framework (ACoP) converts human speech and text into precise commands that Proxie can execute, integrating both perception and cognitive functions.

Alongside the Orin modules, a Jetson Nano supports lower-level operations. This includes translating motion commands into joint movements and managing other essential functions like power management. This comprehensive setup ensures that Proxie operates efficiently and responsively in diverse industrial environments.

Solving Real World Problems

Proxie has been deployed in production for logistics, hospital, and manufacturing operations. Approximately 30 Proxie robots are currently in operations. They have achieved over 5,000 operational hours, moved 16,000 carts, and traveled more than 621 miles (1,000 kilometers) in customer facilities.

Tampa General Hospital has partnered with Cobot to increase operational efficiency and patient experience satisfaction through optimizing cart movement and elevator congestion. This technology is seen as a means to augment healthcare workers' capabilities, freeing them to focus on high quality patient care.

Leading biotech company Moderna is collaborating with Cobot to standardize robotic systems management, reducing costs and enhancing flexibility. This enables Moderna to focus on scientific innovations and to advance drug development and delivery.

Getting Started

Reach out to learn more about Cobot’s pilot program today and see how Proxie seamlessly adapts to customer environments at co.bot/contact

Learn more about NVIDIA Isaac Sim, NVIDIA Isaac ROS and NVIDIA Jetson.