The pandemic has upended the lives of workers around the world. Since February, many businesses deemed “non-essential” have been forced to close their doors for periods of time. People fortunate enough to work remotely are doing so in unprecedented numbers, to the extent that they’ve placed a strain on utilities, e-retailers, and ISPs. Surveys show that even post-pandemic, businesses plan to dramatically increase employees’ share of at-home days.
Roboticists and robotics engineers are among those working from home, but only to the extent that they can. Robotics is a hands-on field — wheels, cogs, arms, motors, gears, and probes need to be tuned in conjunction with computers orchestrating their movements. Most experiments have to be configured and reset in person, and measurements have to be taken with handheld instrumentation. In the event of a critical mechanical or software failure, an expert must be on hand to right the ship.
“During the first few months of quarantine, almost no one had access to the lab spaces, so robot experiments were essentially out of the question during this period,” Brijen Thananjeyan, a Ph.D. student at the University of California, Berkeley’s artificial intelligence lab, told VentureBeat via email.
Devendra Chaplot, a graduate student at Carnegie Mellon’s machine learning department, said he and his team also encountered challenges working with robots during the pandemic. They managed to haul some smaller machines from campus to their homes, but it wasn’t long before logistical and technical limitations reared their ugly heads. Physical tinkering was often impossible because researchers lacked robotics hardware expertise. Chaplot and colleagues had to instead establish connections through home routers, walking each other through calibration and troubleshooting steps.
The home setups also foisted training, testing, and validation restrictions on the researchers, who typically deploy experiments in multiple environments, including Airbnb properties. And the researchers lacked the sensors Carnegie Mellon’s labs are equipped with, limiting their benchmarking capabilities.
Chaplot says the constraints even impacted the types of research he and colleagues pursued. “Robots require a lot of physical space (especially in navigation) and can be loud, [so] experiments have to be limited based on schedules of other people in the household who are also working remotely,” said Chaplot. “This also affects the research problems we decide to tackle. While deciding a project recently, we were consciously thinking of problems that require minimal or no robot experimentation. I have seen some friends also thinking similarly when deciding research projects.”
Roboticists at the Allen Institute for AI encountered similar roadblocks arising from the pandemic. They’d intended to launch the RoboTHOR challenge earlier this year, which would’ve involved deploying navigation algorithms in a robot — the LocoBot — and running it through a physical environment at the nonprofit’s labs. But due to the pandemic, all Allen Institute employees were working from home, preventing them from running experiments on LocoBot for the foreseeable future. They decided to pare down the challenge to only simulated scenes.
Thananjeyan says his work and the projects of other Berkeley students have been less impacted by the health crisis. He and others continue to use the labs, albeit with restricted access to shared spaces and a maximum of two masked people in each lab space. (Researchers are required to complete an online training course before they receive access to the buildings, and they must fill out daily symptom screens and disinfect workspaces before and after each use.) But during the early days of the pandemic, when the labs were shut down entirely, he says teams relied heavily on simulated environments as a testbed for ideas.
“When first testing out a new … algorithm on a robot, we ran many preliminary experiments on workspace-reaching tasks that do not require contact-rich manipulation,” he explained. “Before this, we make sure to thoroughly test methods in simulation to work out any potential issues that may arise. For some experiments, eliminating physical resets is unavoidable, and one of us will need to be present to reset the environment in the event of a failure. We will not need more than a single person present, however, even if multiple people are working with the robot simultaneously.”
Above: The da Vinci Research Kit.
Despite regaining limited lab access, Thananjeyan says he’s increasingly relying on VNC Viewer and other remote apps to manipulate on-campus robots like the da Vinci Research Kit. Via VNC, he’s able to view the workspace using cameras in the setup and control robots programmatically, with various safety mechanisms so they don’t damage themselves or their surroundings. This has the advantage of enabling robots to run overnight or even for days without intervention. Thananjeyan says he’s in the process of setting up a livestream so the robots can be monitored at any time.
“I believe the pandemic pushed researchers to adapt, which took time,” Thananjeyan said. “I think the initial few months of shelter-in-place were tough because lab spaces were completely closed and many of us were unprepared for how long it ended up lasting. However, recent advances in simulation have enabled researchers to still make steady progress by providing virtual environments to test new ideas. While simulation is not a substitute for real-world experiments, it provides a signal about whether methods will be viable on a physical system.”
Researchers at the Max Planck Institute for Intelligent Systems, nonprofit research institute Mila, and New York University have adopted similar approaches to robotics development during the pandemic. Their Dextrous Manipulation Challenge, which kicked off on August 10, provides teams with more than 100 real-robot hours on high-performance systems hosted in Tübingen. Scientists have to solve a series of relatively simple to extremely hard tasks, from pushing a cube to picking up a pen and writing, with the idea that together they could potentially achieve a breakthrough in robotic manipulation.
Back at Carnegie Mellon, assistant professor of mechanical engineering Aaron Johnson’s bio-inspired robot design course had to go fully remote, he told IEEE Spectrum. But none of the teams drastically altered their projects or opted for simulated designs. To get around the logistical challenges, students dropped off and shipped parts to their collaborators, bought simple tools like soldering irons, and had Johnson 3D-print parts and mail them. Each team even managed to put together demo videos from their remote locations.
A team working toward a robotics capstone at the University of California, Santa Cruz took a slightly different tack. With the approval of university administration, they enlisted a graduate assistant in electrical engineering who had permission to access one of the campus’ labs during lockdown. They relayed instructions to her about how to integrate sensors on a Humvee-like robot they’d designed and then tasked her with collecting data from those sensors. From this data, the students created models remotely.
Meanwhile, R&D engineers at Universal Robotics, an Odense, Denmark-based manufacturer of industrial robot arms, have set up labs in their own garages, the company told Robotics Tomorrow. According to a spokesperson, they’re collaborating through proprietary simulation tools and testing robots taken from Universal Robotics’ labs.
Engineers at Boston Dynamics are similarly developing robots from home, at least to the extent possible. In June, the Verge reported that 71 Spot units had been sent home with employees, who braved inclement weather and risk of injury (Spot weighs 71 pounds) to conduct various robustness and validation checks. One employee recreated a slippery surface with a wax sheet on top of a wooden clipboard. Another reconfigured his parents’ basement into something of an obstacle course.
Chris Paxton, a robotics research scientist at Nvidia’s Seattle Robotics lab, says he and his team also experienced less disruption than anticipated. Paxton attributes this to their ability to pivot to simulation, benchmarking, and data processing work.
“There have been a few changes, but I think we’ve done a pretty good job of switching our focus to using some of our internal tools,” he said. “Robotics is very hands-on, but a lot of the stuff that we work on also has to do with software, like learning models and planning algorithms. We can use our simulation tools and work on things like building benchmarks, data sets, and machine learning so we can continue to do good research at full speed.”
Paxton conceded he’s had to refocus projects that entailed lots of real-world experimentation. But he also says the pandemic allowed him and coworkers to work toward reproducible, “quality” robotics research. Reproducibility is a longstanding challenge in robotics, he notes — information found in technical papers is usually not enough to reproduce the results. By focusing on simulation, Paxton believes progress toward simulation-based reproducibility can be accelerated.
“We can use our simulation tools to do better research. I think it’ll be a boon for robotics research as a whole going forward, even if it’s a tough adjustment period,” he said.
