There are many functions or capabilities required in an autonomous vehicle (AV). Some of these technologies may be familiar, while others will be new. Some of these capabilities are available in many cars today, but they also are necessary to provide the foundation for autonomous driving. This chapter will attempt to answer questions about these autonomous driving capabilities in the simplest terms possible.
I’m familiar with GPS navigation systems; how does GPS work in AVs?
Global Positioning Systems (GPS) for car navigation plot routes of travel along a map to a given destination. Some GPS car navigation systems are interconnected with sources of traffic information, enabling them to automatically account for construction and congestion when determining the best route. GPS navigation works the same in AVs.
Does GPS provide sufficient navigation for AVs to drive autonomously?
No, they can’t. Currently, standard GPS is only accurate to within approximately 5 meters or so of an object's actual location. This accuracy is sufficient to position a vehicle on the road in a digital map, but it is not precise enough for determining what lane a vehicle is in or exactly when to start turning. Autonomous vehicles need not only road accuracy, which lets the car know which road it's on, but they also need lane accuracy with precision down to the meter or several feet.
What is blind-spot detection?
Blind-spot detection systems use a variety of sensors and cameras to provide drivers with information about objects that are outside of their range of vision. Cameras provide views from either side of a vehicle that allows drivers to verify that their blind-spot is clear.
What is a surround-view camera?
A surround-view camera system offers a birds-eye view of the car from overhead and shows the vehicle as a moving image on the car's LCD, along with parking lot lane markings, curbs, adjacent cars, garage walls, etc. The camera is not actually above the vehicle. It creates that illusion with camera views that are stitched together into a single video.
What are adaptive headlights?
Each adaptive headlight system works a little differently since they don't all perform the same functions. Some use sensor inputs to determine when the vehicle is turning. The headlights rotate with the turn, which illuminates the road in front of the car, instead of illuminating the side of the road when cornering, or shining off the road entirely.
What is adaptive cruise control?
Adaptive cruise control (ACC), sometimes called autonomous cruise control, was the beginning of semi-autonomous driving. It automatically adjusts the vehicle speed to maintain a safe distance from vehicles ahead. It was initially introduced as cruise control, which enabled drivers to take their foot off the accelerator while the car kept a set speed, but it didn't adjust that speed when approaching too close to a car in front. With adaptive cruise control, the driver sets a speed then the vehicle automatically adjusts its speed based on sensor information using a radar or laser sensor or a dual video camera setup. The vehicle brakes when it detects the car is approaching closer to another vehicle ahead, then it accelerates when the distance separation increases.
What is a lane-departure warning system?
Lane-departure warning (LDW) systems monitor the lane markings on the roadway and notify the driver with an alarm or vibrate the steering wheel whenever the vehicle starts to deviate from its lane. The driver can then take corrective action by steering the car back to the middle of the lane. It only works on roads with sufficient lane markings, and may not work as well if the lane markings are faded. Lane-departure warning systems are designed not to send an alert when the turn signal is on or when the brakes have been applied.
What is a lane-keeping system?
Lane-keeping systems (LKS) also monitor lane markings, but they can take corrective action. A lane-keeping system typically acts to keep the vehicle from drifting. The methods that these systems use to provide corrective actions differ from one system to another. Some of the first lane-keeping systems made use of electronic stability control systems to keep a vehicle in its lane. This was accomplished by applying slight braking pressure to the proper wheels. Modern systems can tap into power steering controls to provide a gentle steering correction.
What is automatic lane-centering and how is it different from lane-keeping?
Automatic lane-centering (ALC) always tries to keep the car centered in the current lane. It provides continuous control across the lane, while lane-keeping systems provide control only near lane boundaries. Controlled steering is primarily implemented through shared braking and steering control services with longitudinal control systems.
Unlike an LKS, which gently steers the vehicle back into its lane by braking the inside wheels while vibrating the steering wheel as a warning, ALC uses adaptive cruise control in conjunction with cameras and steering control electronics to keep the car centered in its lane.
Will ALC/LKS work in the snow?
ALC and LKS use a variety of sensors to read lane markers, but that's a problem when snow is covering them. That makes current ALC/LKS systems unable to function correctly.
Does that mean that AVs will never be able to drive in snow?
No, some innovative solutions are being developed. When driving in snow, human drivers who cannot see the lane markers typically make their best guess to determine road positioning based on other visible markers like curbs, signs, and other cars. Ford is teaching its autonomous cars to do something similar by creating high-fidelity, 3D maps of roads. Those maps include details like the exact position of the curbs and lane lines, trees and signs, along with local speed limits and other relevant rules. The car can use them to figure out, within a centimeter, where it is at any given moment. If the car can't see the lane lines but can see a nearby stop sign, which is on the map, its lidar scanner tells it exactly how far it is from the sign and therefore from the lane lines. There is also a company developing a technology that uses x-rays of the ground underneath the paved road to identify where a vehicle is on the road.
How does an automatic-breaking system work?
Automatic braking systems (ABS) combine sensors and brake controls to help prevent collisions. Some of these systems notify the driver, while others activate the brakes instead of relying on the driver. ABS systems use sensor input such as lasers, radar, or video data. If the sensor detects an object in the path of the vehicle, then the system determines if the speed of the vehicle is higher than the speed of the object in front of it. A significant speed differential may indicate a likely collision, in which case the system automatically activates.
I’ve heard of something called traffic-jam pilot, what is that?
Audi was the first to introduce specific traffic-jam functionality for autonomous driving in its A8 Traffic-Jam Pilot. When the car is in traffic, and the speed falls below 37 mph, the dashboard display shows the icon of a vehicle within white markings, signaling that the system is ready to take over. By pressing the Auto AI button, the car starts to drive itself. Currently, Traffic-Jam Pilot only engages the system if the vehicle is on a limited-access divided highway, has a vehicle directly in front, a line of slow-moving vehicles in adjacent lanes, and the system can identify lane markings and the edge of the roadway (with a barrier or guardrails, for instance).
In some European countries, once the system is engaged, the driver can watch TV, respond to text messages, or have a face-to-face conversation with passengers. The entertainment and productivity features are fully integrated with the vehicle’s interface so that it can adequately warn the driver if needed.
How does an AV recognize speed limits?
Speed-limit detection (SLD) informs the driver of posted speed limits. There are two ways to do this. One uses a camera system that identifies and reads speed-limit signs using image processing. A windshield-mounted camera monitors the area in front of the car looking for road signs. A computer scans the camera image for round-shaped surfaces typical of speed-limit signs. Algorithms filter out all objects that are round-shaped but do not resemble traffic signs. The speed limit is sent to the cockpit display so that the driver is always aware of the current speed limit and can adjust the car's speed accordingly.
Speed-limit identification in other cars, such as those made by General Motors and Mercedes, use a roadway database in the vehicle's navigation system. Using GPS, the instrument cluster displays the speed limit from the database. Drivers need to be careful as this is not always accurate. Speed limits change, and the database is not always updated. Recognition of speed limits is an essential function for AVs.
How does an AV park itself?
There are different types of automatic-parking systems (APS), although they are designed to perform similar tasks. Some automatic parking systems offer hands-free parallel parking, and others only provide some helpful assistance.
Automatic parallel parking is easier for the vehicle’s computer than it is for a human. It follows a simple formula, and computers are better at that:
Pull alongside the front parked car, allowing a 3-foot gap to that car,
Align the back tires with the front parked car’s rear bumper,
Go into reverse and start turning the wheels hard-right,
Then start backing up slowly until reaching a 45-degree angle then stop,
Turn the wheels hard-left,
Then slowly continue to backup parallel with the curb,
If needed, go backward or forward to even out space ahead and behind.
Automatic parallel-parking has been available for less than a decade. Automatic parallel-parking systems use a variety of sensors to determine the approximate size of the space between two parked vehicles, and then the onboard computer calculates the necessary steering angles and velocities to navigate into the parking spot safely.
How does an AV get software updates?
Autonomous vehicles (AVs) are basically computers using a lot of software. Software updates are essential to AVs so that new improvements and software fixes can be rapidly and efficiently transmitted to vehicles. Over-the-air (OTA) updating is already the routine way to update cell phones and iPads, but it is not as easy for cars.
When Tesla introduced the Model S, it had an electronic architecture that enabled every line of code to change remotely. Tesla has some unique advantages in this. The cost of transmitting a lot of software over cellular systems to millions of vehicles can be prohibitive. To overcome this, Tesla uses a process for an over-the-air update of its Autopilot driver-assist system that replaces only the changed code, instead of the entire file, making it more efficient.
But, there is a more significant impediment. By some states’ laws, auto manufacturers can't offer existing customers new features for their vehicles without the car dealerships getting their cut. This restriction contrasts with Tesla, which has done much to highlight the utility of OTA updates. Because Tesla doesn’t have dealers, it isn't restricted by laws that prevent it from rolling out new features to customers without having a dealership as a mandatory middle step.
Are there concerns about software security and hacking with OTA software updates?
Security is a significant issue with OTA updates that could affect the way an AV operates. It will continue to get much attention.
What does a vehicle need to do to be sufficiently autonomous besides the semi-autonomous functions?
To become sufficiently autonomous, a vehicle needs to be able to plan its path, initiate and complete turns, read traffic signs and lights, and avoid obstacles and pedestrians.
What is path planning?
The goal of path planning is to use the information captured in the vehicle's detailed maps to safely direct the vehicle to its destination while avoiding obstacles and following the rules of the road. Although path-planning algorithms will vary in different systems, their navigation objectives, and the sensors used, the following describes a general path-planning algorithm.
The algorithm determines a rough long-range plan for the vehicle to follow while continuously refining the steps along a short-range path (e.g., move into the right lane, drive forward 10m, execute a turn right).
It starts from a set of short-range paths that the vehicle would be capable of completing given its speed, direction, and angular position, and then it removes all those that would either cross an obstacle or come too close to the predicted path of a moving one. For example, a vehicle traveling at 50 mph would not safely be able to complete a right turn 5 meters ahead, therefore that path would be eliminated from the feasible set.
Remaining paths are evaluated based on safety, speed, and any time requirements.
Once the onboard processor identifies the best path it computes, a set of throttle, brake and steering commands and sends them to the actuators.
Altogether, this process takes about 50 milliseconds, although it can be longer or shorter depending on the amount of collected data, available processing power, and complexity of the path-planning algorithm.
How does an AV turn a corner?
Turning corners reliably and accurately is an essential function that AVs must master. Currently, this can't be done with GPS-based navigation, since it's not precise enough. More precise positioning requires detailed HD mapping, which enables a vehicle to know that it is 6 inches or less from a corner and can maneuver the turn.
Are left-hand turns more difficult than right-hand turns?
Instructing an autonomous vehicle to turn left is one of the tougher problems. It’s also a difficult problem for humans, by the way. Left-hand turns cause more than 20% of traffic accidents while right-hand turns cause only about 1%.
The AV needs to estimate the speed, distance, and timing of oncoming cars. They can do that more efficiently than humans can. Humans though will sometimes make judgments on when to risk cutting off an oncoming car based on intangible factors: how long the line is of cars coming toward them, how many cars are waiting behind them, and how long before the light turns red again. (Can you imagine an autonomous car being safe and polite, sitting at an intersection for 20 minutes while angry traffic builds up behind it?)
Then they need to identify how much space is required to turn and compute the proper angle for the turn. Can it complete the turn before the traffic light changes (if there is a traffic signal) and not block the intersection?
Will AVs minimize left-hand turns?
Maybe AVs will adopt the “UPS rule” which routs its 100,000 trucks in a way to avoid left-hand turns, whenever possible.
How does an AV avoid obstacles?
With obstacle avoidance, an AV identifies the current and predicted location of all static and moving obstacles in its vicinity. This is a continuous process of detection and identification.
The different types of obstacles are categorized using a library of pre-determined shape and motion descriptors. The AV matches obstacles it detects to this library. It then predicts the future path of moving objects based on its shape and prior trajectory. For example, if a two-wheeled object is traveling at 40 mph versus ten mph, it is most likely a motorcycle and not a bicycle and will get categorized as such by the vehicle.
Can an AV do evasive steering?
Evasive steering helps an AV steer around an imminent crash. It uses a front-mounted camera and radar detector to monitor the surrounding traffic. If a collision is imminent, the vehicle applies the brakes. Evasive steering turns the car the exact right amount to avoid the obstacle, while also preventing the vehicle from crashing.
How does an AV get onto and off a highway?
Once on the highway, an autonomous vehicle needs to exit the highway safely at the proper time. The Tesla on-ramp and off-ramp function is an example of this. Once on the freeway, the Tesla will determine which lane it needs to be in and when to exit correctly. Besides ensuring that it reaches its intended exit, it will also watch for opportunities to move to a faster lane when the car is behind slower traffic. When it arrives at the exit ramp, it will depart the freeway, slow down and transition control back to the driver. In sufficiently autonomous AVs, the vehicle will continue to control the upcoming turns.
Can AVs read traffic signs?
Traffic-sign recognition (TSR) is a technology that enables a vehicle to recognize traffic signs, e.g. "speed limit" or "children" or "turn ahead." The technology uses image processing techniques to detect traffic signs based on color, shape, etc.
Are traffic signs standardized?
The Vienna Convention on Road Signs and Signals signed in 1968 standardizes traffic signs across different countries. The convention broadly classifies road signs into seven categories designated with letters A-H. This standardization has been essential for the development of traffic-sign recognition systems for global use.
How does an AV detect traffic signals?
Traffic lights pose a unique perception problem for AVs. A previously established map can be used to indicate when and where a vehicle should be able to see a traffic light, but vision is the only way to detect the state of that light, which may include detecting which sub-elements of the light are illuminated. This process includes distinguishing colors (red, yellow, green), as well as identifying and distinguishing colored arrows for turning. Although any vision task may be challenging due to the variety of outdoor conditions, traffic lights have been engineered to be highly visible, emissive light sources that eliminate or substantially reduce illumination-based appearance variations. A camera with fixed gain, exposure, and aperture are calibrated to traffic-light color levels.
Can an AV fail to see a traffic signal?
The most common failure conditions in a traffic light detection (TLD) system are either visual obstructions or false positives such as those induced by the brake lights of other vehicles. By using a map of traffic lights, the vehicle can predict when it should see traffic lights and take actions, such as braking gradually to a stop while alerting the driver, when it can't observe any lights.
How does an AV identify emergency vehicles?
Awareness and identification of approaching emergency vehicles are specific functions that must be managed by autonomous vehicles. The identification usually begins with sound detection of approaching sirens, from which the distance and then the emergency vehicle’s approaching path is determined. This alert makes the car's camera sensors aware of looking for and identifying emergency vehicles, and the software guidance systems control the car appropriately, such as pulling over to the side of the road.
Eventually, all emergency vehicles will have vehicle-to-vehicle communications that will notify other vehicles of their approach and request actions from those other vehicles.
How will an AV communicate with its passengers?
AVs will need a way to communicate with passengers and take direction from them. For example, passengers may want to instruct the vehicle to stop at the next rest area or market. They may want the vehicle to change the destination or stop along the way. Passengers may also want to make inquiries, such as: how long will it take to reach the destination? The car may also need to notify passengers of upcoming actions, such as a stop is coming up for refueling.
Most likely passenger communications will be voice-based, but there will also be a high-quality display in the autonomous vehicle for maps and entertainment selections from onboard integrated entertainment systems.
In autonomous ride services cars, there will also be communications with a customer support center for that service. This communication network would respond to passenger problems or requests, as well as allowing the center to contact passengers.
Will AVs have sophisticated entertainment systems?
Most likely they will have very sophisticated entertainment systems since passengers will want to do something with the extra time. Much of this will come from streaming video, which will put a burden on the cellular system until faster transmission, like 5G, is implemented.