Most people understand that an autonomous vehicle drives itself without a human driver. However, beyond that, many people are confused or have misconceptions. Many people are curious and want to learn about how a car can drive itself. Others will accept it and never try to understand how it works. For the curious, here are the answers to some frequently asked questions about how a car can drive itself.
Does an AV use wires embedded in the road?
No, control wires were a very early concept of how to control a vehicle. They are unnecessary and not considered anymore.
Does an AV use two-way satellite communications to control it?
No, this is another myth based on what might be presumed. While GPS satellites provide an AV with its general position, as is done with navigation systems today, there is no need to “connect” any satellite to each car individually to control it.
Does an AV require cellular communications to guide it?
No. AVs drive without a cellular connection to guide them. Later generations of AVs will use mobile, or variations of short distance communications, to communicate with other vehicles and infrastructure such as traffic signals. But, this is not a requirement today.
What steps does an AV follow to drive autonomously?
An AV follows four steps in response to four simple questions, very much like what a human drive does:
Where am I?
What’s around me?
What will happen next?
What should I do?
How does an AV know where it is?
An AV uses GPS navigation to determine generally where it is. Then it accesses a detailed three-dimensional map for that location that specifies objects such as road profiles, curbs, sidewalks, lane markers, crosswalks, traffic lights, trees, shrubs, light posts, telephone poles, etc. These are specified with exact measurements. The AV then uses its sensors to position itself precisely within that detailed map.
How does an AV determine what is around it?
The sensors on the AV are constantly scanning everything around it, including other vehicles, pedestrians, objects, traffic lights, etc. The software on the AV interprets each of these objects to identify what they are and how they are likely to react. This process is called characterization. An AV can see approximately 300 meters or more in all directions, so it able to characterize all surrounding objects.
How does an AV anticipate what will happen next?
Objects characterized as fixed, stay fixed; but dynamic objects like pedestrians, vehicles, bicyclists, etc. are expected to move. The AV’s software predicts the anticipated movement of every dynamic object around it based on the object’s current speed, direction of movement, and the characteristics of the object. For example, a vehicle is expected to move faster than a pedestrian. The software will also anticipate how an object will move based on traffic patterns. As objects move, the AV tracks their movements and updates predicted movements.
How does an AV determine what it should do?
Based on the overall route to where it is going, where it is, and the expected movement of other objects, the AV determines the exact direction, speed, and steering maneuvers it needs to make. It continually monitors all the objects around it, adjusting its movement as necessary.
What sensors are used by an AV?
Sensors are essential to AVs. Generally, there are four types of sensors used in different configurations: video, radar, sonar, and lidar. Lately, there has been some interest in incorporating thermal infrared sensors as well.
How many sensors do AVs have?
The number of sensors varies, but most have an array of sensors. For example, the GM Cruise AV system uses 14 cameras, three articulating radars, five lidars, eight radars, and ten ultra-short radars.
Why are there so many and different sensors?
The sensors need to cover a 360-degree complete view around the vehicle, and they serve different purposes. For example, long-range radar provides cruise control, while short-range radar provides cross-traffic alert and rear collision warning. Cameras identify objects and provide traffic-signal recognition. Lidar creates maps of the environment around the vehicle. Also, different types of sensors work better in different weather conditions.
With so many sensors, doesn’t an AV get confused?
No, all the sensor inputs are “stitched” together using special software that does what is called “sensor fusion.” This provides a single view of where it is and what is around it.
How do video cameras work on an AV?
Video cameras are the primary and least expensive sensors on an AV. Cameras are distributed around the vehicle to record video of all objects surrounding it. However, video needs to be digitized instantly to interpret these objects, and that takes additional computing power. Full HD video has millions of pixels in each video frame.
Are there different types of video cameras?
Yes, there are different types for various functions: narrow-focus cameras for longer distance, wider-focus cameras for shorter range, side-facing cameras, etc.
Are video cameras color or monochrome?
This is a good question. Color cameras are preferable because they use color to interpret objects, such as the color of a traffic light. But, color also requires more computing processing power than monochrome, so it can be more expensive. Tesla uses monochrome video because of the lower cost. Some people believe that this was a reason for a fatal Tesla accident when the vehicle couldn’t distinguish between the side of a white tractor trailer and the clear sky. Although the driver was officially ruled to be at fault.
How does radar work in AVs?
Radar is an object detection system that uses radio waves to determine the range, angle, and velocity of objects. AV radar systems are typically classified by their range: short-range (<30 meters), medium-range (30-80 meters), and long-range (80-200 meters). Radar is excellent for motion measurement, and it requires far less data processing than video.
Are there limitations to radar?
Yes, there are some. For example, it may not correctly identify objects in certain unique conditions, such as a car cutting in front of another, motorcycles staggered in the same lane, identifying the wrong vehicle ahead on a curved road. The AV software can adapt to these situations since they are typically resolved quickly, and other sensors are used in sensor fusion.
How are ultrasonic sensors used?
Ultrasonic proximity detectors measure the distance to nearby objects using sensors mounted in the front and rear bumpers, as well as grills and recesses. These sensors emit acoustic pulses, and the distance to an object is calculated using the time it takes for the pulse to reflect off an object and return.
What the heck is lidar?
Lidar is an acronym for Light Detection And Ranging. It works on the same principle as radar, but it uses light waves instead of radio waves. Lidar was invented in 1960, shortly after the development of the laser. It was first used in the Apollo 15 mission in 1971 to map the surface of the moon.
Why is lidar so important to AVs?
Lidar is the most important of the sensors because it provides a comprehensive and precise high-definition 3D map of everything surrounding the AV. Unlike video, lidar creates digital data, so it doesn’t need to convert video to digital data. Current lidar uses 64-beams of light.
Is that what I see sitting on the top of the roof of an AV?
Yes. Newer versions will be subtler and less visible.
Is lidar expensive?
Yes, it is costly. It's the most expensive of all the sensors.
Is lidar required for an AV?
That is another good question. Many experts believe that it is, but Tesla is developing its AV capabilities without lidar, relying on video instead to maintain a lower vehicle price.
What is a thermal infrared sensor?
It detects and identifies objects based on heat, specifically humans and animals. These sensors can be useful to identify pedestrians hidden behind foliage of other objects. Thermal sensors are not yet used in most AVs, but are being considered for increasing pedestrian identification.
Will AVs require a lot of computing power?
Yes, the amount of computing power required is incredible. For example, Nvidia’s PX Pegasus computing platform for AVs is capable of processing 320 trillion operations per second. While this power is expensive, it is now affordable, which is one of the primary reasons why AVs are now feasible.
How does an AV learn how to drive?
An autonomous vehicle uses artificial intelligence software to drive. It gets its experience from actual miles driven, but it is not only the experience of a single vehicle. It’s the experience of all vehicles from the same developer.
How many miles of driving experience do AVs have?
Millions of miles. By the end of 2018, Waymo AVs had almost ten-million miles of actual driving experience to capture driving data into its AV artificial intelligence systems. Also, it uses simulation systems to create unique and odd driving situations.
How does simulated driving help train AVs?
AV developers use software to simulate an enormous variety of driving conditions that an AV might encounter to supplement the experience it gets from actual encounters. Waymo has trained its vehicles with over five billion miles of simulated driving.
How much experience do AVs need?
They need more experience than humans to achieve the level of safety expected, but the millions of miles of actual driving and billions of miles of simulation shows how important safety is to those developing AVs. In contrast, most states require a person to get only about 50 hours of driving experience to get a driver’s license.
Is the driving experience for an AV limited to that AV?
No, all experience is shared by all AVs from that developer because all its AVs use the same artificial experience software and data. As AVs get more experience, all AVs from that developer are also updated.
Is AV driving experience shared across all developers?
Generally, not. They all have their own artificial intelligence systems and experience databases. Because they have invested billions of dollars developing this, they see it as a competitive advantage. Also, the software systems between AV developers are not compatible. The exceptions are major accidents cases. All developers will make sure they design these cases into their experience database whenever they occur.
Does this mean that some AVs will be safer than others?
This is an awkward question. Yes, there will be different autonomous systems, and they will operate differently. Some will be safer than others in certain situations. It may be something that Consumers Reports, or something similar, does to compare autonomous vehicles in the future.
Will AVs be cautious enough?
They will be programmed, at least initially, to be very cautious. They will generally drive no more than the speed limit, come to a full stop for several seconds at stop signs, stop cautiously for red lights, and defer to pedestrians even if the AV has the right of way. Most likely, they won't be aggressive drivers.
Will AVs be too cautious?
Probably they will be too cautious – at least initially. This will both amuse and upset other drivers. For example, driving only at the speed limit on a highway can sometimes irritate other drivers who are going faster. Over time, AVs will adapt to become a more like human drivers and may go a little faster than the speed limit. There are cases where more cautious drivers can cause accidents because other drivers expect them to be more aggressive in things like pulling out onto a road. Early reports in Waymo's 2018 ARS testing in Arizona indicated that some residents were frustrated with their AVs driving too cautiously. For example, they were criticized for stopping a full three seconds at every stop sign.
Eventually, will AVs have different “personalities”?
They may. Different companies developing AVs may set different driving standards, such as allowing the vehicle to go a few miles-per-hour above the speed limit. Maybe some vehicles will even let the owner, or passengers, select a driving mode that varies its aggressiveness.
Will AVs have difficulty with pedestrians?
Most likely. Human drivers already have problems with them, and AVs will as well. It will be difficult for AVs to determine if someone who is standing on the side of the road, and not at a crosswalk, will try to cross in front. Currently, between 80%-90% of the pedestrian fatalities caused by human drivers occur because of jaywalking.
How will this be addressed?
Some municipalities will take measures to make crosswalks easier to access and monitor jaywalking more. This will protect pedestrians better from human drivers as well.
Are AVs better than human drivers?
In some ways, they will be much better. AVs have better perception than humans. They will be able to see a full 360-degrees. They can precisely compute the speed and slowing of cars ahead of them. They can react instantly because of their rapid computing power. They gain experience from other AVs while humans don’t learn from others. And of course, they don’t drive drunk, distracted, or break traffic laws.
On the other hand, humans have better intuition and instincts to interpret odd situations. For example, they can use body language to determine if the person standing on the side of the road likely to try to cross?