Shelley, Stanford’s autonomous Audi TTS, performs at Thunderhill Raceway Park. (Credit: Kurt Hickman)
Researchers at Stanford University have developed a new way of controlling autonomous cars that integrates prior driving experiences – a system that will help the cars perform more safely in extreme and unknown circumstances. Tested at the limits of friction on a racetrack using Niki, Stanford’s autonomous Volkswagen GTI, and Shelley, Stanford’s autonomous Audi TTS, the system performed about as well as an existing autonomous control system and an experienced racecar driver.
“Our work is motivated by safety, and we want autonomous vehicles to work in many scenarios, from normal driving on high-friction asphalt to fast, low-friction driving in ice and snow,” said Nathan Spielberg, a graduate student in mechanical engineering at Stanford and lead author of the paper about this research, published March 27 in Science Robotics. “We want our algorithms to be as good as the best skilled drivers—and, hopefully, better.”
While current autonomous cars might rely on in-the-moment evaluations of their environment, the control system these researchers designed incorporates data from recent maneuvers and past driving experiences – including trips Niki took around an icy test track near the Arctic Circle. Its ability to learn from the past could prove particularly powerful, given the abundance of autonomous car data researchers are producing in the process of developing these vehicles.
Physics and learning with a neural network
Control systems for autonomous cars need access to information about the available road-tire friction. This information dictates the limits of how hard the car can brake, accelerate and steer in order to stay on the road in critical emergency scenarios. If engineers want to safely push an autonomous car to its limits, such as having it plan an emergency maneuver on ice, they have to provide it with details, like the road-tire friction, in advance. This is difficult in the real world where friction is variable and often is difficult to predict.
To develop a more flexible, responsive control system, the researchers built a neural network that integrates data from past driving experiences at Thunderhill Raceway in Willows, California, and a winter test facility with foundational knowledge provided by 200,000 physics-based trajectories.
This video above shows the neural network controller implemented on an automated autonomous Volkswagen GTI tested at the limits of handling (the ability of a vehicle to maneuver a track or road without skidding out of control) at Thunderhill Raceway.
“With the techniques available today, you often have to choose between data-driven methods and approaches grounded in fundamental physics,” said J. Christian Gerdes, professor of mechanical engineering and senior author of the paper. “We think the path forward is to blend these approaches in order to harness their individual strengths. Physics can provide insight into structuring and validating neural network models that, in turn, can leverage massive amounts of data.”
The group ran comparison tests for their new system at Thunderhill Raceway. First, Shelley sped around controlled by the physics-based autonomous system, pre-loaded with set information about the course and conditions. When compared on the same course during 10 consecutive trials, Shelley and a skilled amateur driver generated comparable lap times. Then, the researchers loaded Niki with their new neural network system. The car performed similarly running both the learned and physics-based systems, even though the neural network lacked explicit information about road friction.
In simulated tests, the neural network system outperformed the physics-based system in both high-friction and low-friction scenarios. It did particularly well in scenarios that mixed those two conditions.
Simple feedforward-feedback control structure used for path tracking on an automated vehicle. (Credit: Stanford University)
An abundance of data
The results were encouraging, but the researchers stress that their neural network system does not perform well in conditions outside the ones it has experienced. They say as autonomous cars generate additional data to train their network, the cars should be able to handle a wider range of conditions.
“With so many self-driving cars on the roads and in development, there is an abundance of data being generated from all kinds of driving scenarios,” Spielberg said. “We wanted to build a neural network because there should be some way to make use of that data. If we can develop vehicles that have seen thousands of times more interactions than we have, we can hopefully make them safer.”
CloudMinds was among the robotics companies receiving funding in March 2019. Source: CloudMinds
Investments in robots, autonomous vehicles, and related systems totaled at least $1.3 billion in March 2019, down from $4.3 billion in February. On the other hand, automation companies reported $7.8 billion in mergers and acquisitions last month. While that may represent a slowdown, note that many businesses did not specify the amounts involved in their transactions, of which there were at least 58 in March.
Self-driving cars and trucks — including machine learning and sensor technologies — continued to receive significant funding. Although Lyft’s initial public offering was not directly related to autonomous vehicles, it illustrates the investments flowing for transportation.
Other use cases represented in March 2019 included surgical robotics, industrial automation, and service robots. See the table below, which lists amounts in millions of dollars where they were available:
Company
Amt. (M$)
Type
Lead investor, partner, acquirer
Date
Technology
Airbiquity
15
investment
Denso Corp., Toyota Motor Corp., Toyota Tsushu Corp.
March 12, 2019
connected vehicles
AROMA BIT Inc.
2.2
Series A
Sony Innovation Fund
March 3, 2019
olofactory sensors
AtomRobot
Series B1
Y&R Capital
March 5, 2019
industrial automation
Automata
7.4
Series A
ABB
March 19, 2019
robot arm
Avidbots
23.6
Series B
True Ventures
March 21, 2019
commercial floor cleaning
Boranet
Series A
Gobi Partners
March 6, 2019
IIoT, machine vision
Broadmann17
11
Series A
OurCrowd
March 6, 2019
deep learning, autonomous vehicles
Cloudminds
300
investment
SoftBank Vision Fund
March 26, 2019
service robots
Corindus
4.8
private placement
March 12, 2019
surgical robot
Determined AI
11
Series A
GV (Google Ventures)
March 13, 2019
AI, deep learning
Emergen Group
29
Series B
Qiming Venture Partners
March 13, 2019
industrial automation
Fabu Technology
pre-Series A
Qingsong Fund
March 1, 2019
autonomous vehicles
Fortna
recapitalization
Thomas H. Lee PArtners LP
March 27, 2019
materlais handling
ForwardX
14.95
Series B
Hupang Licheng Fund
March 21, 2019
autonomous mobile robots
Gaussian Robotics
14.9
Series B
Grand Flight Investment
March 20, 2019
cleaning
Hangzhou Guochen Robot Technology
15
Series A
Hongcheng Capital, Yingshi Fund (YS Investment)
March 13, 2019
robotics R&D
Hangzhou Jimu Technology Co.
Series B
Flyfot Ventures
March 6, 2019
autonomous vehicles
InnerSpace
3.2
seed
BDC Capital's Women in Technology Fund
March 26, 2019
IoT
Innoviz Technologies
132
Series C
China Merchants Capital, Shenzhen Capital Group, New Alliance Capital
March 26, 2019
lidar
Intelligent Marking
investment
Benjamin Capital
March 6, 2019
autonomous robots for marking sports fields
Kaarta Inc.
6.5
Series A
GreenSoil Building Innovation Fund
March 21, 2019
lidar mapping
Kolmostar Inc.
10
Series A
March 5, 2019
positioning technology
Linear Labs
4.5
seed
Science Inc., Kindred Ventures
March 26, 2019
motors
MELCO Factory Automation Philippines Inc.
2.38
new division
Mitsubishi Electric Corp.
March 12, 2019
industrial automation
Monet Technologies
4.51
joint venture
Honda Motor Co., Hino Motors Ltd., SoftBank Corp., Toyota Motor Corp
Bonfire Ventures, Vertex Ventures, London Venture Partners
March 11, 2019
machine vision
Vtrus
2.9
investment
March 8, 2019
drone inspection
Weltmeister Motor
450
Series C
Baidu Inc.
March 11, 2019
self-driving cars
And here are the mergers and acquisitions:
March 2019 robotics acquisitions
Company
Amt. (M$)
Acquirer
Date
Technology
Accelerated Dynamics
Animal Dynamics
3/8/2019
AI, drone swarms
Astori AS
4Subsea
3/19/2019
undersea control systems
Brainlab
Smith & Nephew
3/12/2019
surgical robot
Figure Eight
175
Appen Ltd.
3/10/2019
AI, machine learning
Floating Point FX
CycloMedia
3/7/2019
machine vision, 3D modeling
Florida Turbine Technologies
60
Kratos Defense and Security Solutions
3/1/2019
drones
Infinity Augmented Reality
Alibaba Group Holding Ltd.
3/21/2019
AR, machine vision
Integrated Device Technology Inc.
6700
Renesas
3/30/2019
self-driving vehicle processors
Medineering
Brainlab
3/20/2019
surgical
Modern Robotics Inc.
0.97
Boxlight Corp.
3/14/2019
STEM
OMNI Orthopaedics Inc.
Corin Group
3/6/2019
surgical robotics
OrthoSpace Ltd.
220
Stryker Corp.
3/14/2019
surgical robotics
Osiris Therapeutics
660
Smith & Nephew
3/12/2019
surgical robotics
Restoration Robotics Inc.
21
Venus Concept Ltd.
3/15/2019
surgical robotics
Sofar Ocean Technologies
7
Spoondrift, OpenROV
3/28/2019
underwater drones, sensors
Torc Robotics Inc.
Daimler Trucks and Buses Holding Inc.
3/29/2019
driverless truck software
Surgical robots make the cut
One of the largest transactions reported in March 2019 was Smith & Nephew’s purchase of Osiris Therapeutics for $660 million. However, some Osiris shareholders are suing to block the acquisition because they believe the price that U.K.-based Smith & Nephew is offering is too low. The shareholders’ confidence reflects a hot healthcare robotics space, where capital, consolidation, and chasing new applications are driving factors.
Venus Concept Ltd. merged with hair-implant provider Restoration Robotics for $21 million, and Shanghai Changren Information Technology raised Series A funding of $14.89 million for its Xiaobao healthcare robot.
Aside from Lyft, the biggest reported transportation robotics transaction in March 2019 was Renesas’ completion of its $6.7 billion purchase of Integrated Device Technology Inc. for its self-driving car chips.
The next biggest deal was Weltmeister Motor’s $450 million Series C, in which Baidu Inc. participated.
Lidar also got some support, with Innoviz Technologies raising $132 million in a Series C round, and Ouster raising $60 million. In a prime example of how driverless technology is “paying a peace dividend” to other applications, Google parent Alphabet’s Waymo unit offered its custom lidar sensors to robotics, security, and agricultural companies.
Automakers recognize the need for 3-D modeling, sensors, and software for autonomous vehicles to navigate safely and accurately. A Daimler unit acquired Torc Robotics Inc., which is working on driverless trucks, and CycloMedia acquired machine vision firm Floating Point FX. The amounts were not specified.
Speaking of machine learning, Appen Ltd. acquired dataset annotation company Figure Eight for $175 million, with an possible $125 million more based on 2019 performance. Denso Corp. and Toyota Motor Corp. contributed $15 million to Airbiquity, which is working on connected vehicles.
Service robots clean up
From retail to cleaning and customer service, the combination of improving human-machine interactions, ongoing staffing turnover and shortages, and companies with round-the-clock operations has contributed to investor interest.
The SoftBank Vision Fund participated in a $300 million round for CloudMinds. The Chinese AI and robotics company’s XR-1 is a humanoid service robot, and it also makes security robots and connects robots to the cloud.
According to its filing with the U.S. Securities and Exchange Commission, TakeOff Technologies Inc. raised an unspecified amount for its grocery robots, an area that many observers expect to grow as consumers become more accustomed to getting home deliveries.
On the cleaning side, Avidbots raised $23.6 million in Series B, led by True Ventures. Gaussian Robotics’ Series B was $14.9 million, with participation from Grand Flight Investment.
China’s efforts to develop its domestic robotics industry continued, as Emergen Group’s $29 million Series B round was the largest reported investment in industrial automation last month.
Hangzhou Guochen Robot Technology raised $15 million in Series A funding for robotics research and development and integration.
Data startup Spopondrift and underwater drone maker OpenROV merged to form Sofar Ocean Technologies. The new San Francisco company also announced a Series A round of $7 million. Also, 4Subsea acquired underwater control systems maker Astori AS.
In the aerial drone space, Kratos Defense and Security Solutions acquired Florida Turbine Technologies for $60 million, and Vtrus raised $2.9 million for commercializing drone inspections. Kaarta Inc., which makes a lidar for indoor mapping, raised $6.5 million.
The Robot Reportbroke the news of Aria Insights, formerly known as CyPhy Works, shutting down in March 2019.
Editors Note: What defines robotics investments? The answer to this simple question is central in any attempt to quantify robotics investments with some degree of rigor. To make investment analyses consistent, repeatable, and valuable, it is critical to wring out as much subjectivity as possible during the evaluation process. This begins with a definition of terms and a description of assumptions.
Investors and Investing
Investment should come from venture capital firms, corporate investment groups, angel investors, and other sources. Friends-and-family investments, government/non-governmental agency grants, and crowd-sourced funding are excluded.
Robotics and Intelligent Systems Companies
Robotics companies must generate or expect to generate revenue from the production of robotics products (that sense, think, and act in the physical world), hardware or software subsystems and enabling technologies for robots, or services supporting robotics devices. For this analysis, autonomous vehicles (including technologies that support autonomous driving) and drones are considered robots, while 3D printers, CNC systems, and various types of “hard” automation are not.
Companies that are “robotic” in name only, or use the term “robot” to describe products and services that that do not enable or support devices acting in the physical world, are excluded. For example, this includes “software robots” and robotic process automation. Many firms have multiple locations in different countries. Company locations given in the analysis are based on the publicly listed headquarters in legal documents, press releases, etc.
Verification
Funding information is collected from a number of public and private sources. These include press releases from corporations and investment groups, corporate briefings, and association and industry publications. In addition, information comes from sessions at conferences and seminars, as well as during private interviews with industry representatives, investors, and others. Unverifiable investments are excluded.
Navigating roads less traveled in self-driving cars is a difficult task. One reason is that there aren’t many places where self-driving cars can actually drive. Companies like Google only test their fleets in major cities where they’ve spent countless hours meticulously labeling the exact 3-D positions of lanes, curbs, off-ramps, and stop signs. “The cars…
An Uber self-driving car struck and killed a pedestrian Sunday night in Tempe, Arizona. The preliminary investigation is underway, but Tempe police said the car was in autonomous mode at the time of the crash, was traveling about 40 MPH, and did not show any signs of slowing down. Uber certainly has a recording of…
The trucking industry is heading for an autonomous future. Self-driving trucks have been in the works for several years – Daimler introduced in 2014 the Mercedes-Benz Future Truck 2025 – but developments within the last year show the transformation is moving full steam ahead. The tests being done currently are limited to semi-autonomous systems that…
I recently chaired a UJA Tech Talk on “The Future Of Autonomous Cars” with former General Motors Vice-Chairman Steve Girsky. The auto executive enthusiastically shared his vision for the next 15-25 years of driving – a congestion-free world of automated wheeled capsules zipping commuters to and from work. Girsky stated that connected cars with safety…
Keynotes