An autonomous car (colloquially referred to as a self-driving car) is an automobile that requires no driver, but uses a collection of sensors and computer logic to drive itself. At this point in time, these cars usually offer some form of human control as well, so that the driver may intervene if needed. However, the goal of this concept is to remove the task of the driver completely, and have the entire cabin section for passengers only.
Getting a car to drive forward between the white and yellow lines of a road is not as difficult as one might think. Sensors and computers have advanced to the point that understanding the surroundings, and reacting to them is entirely feasible. Even fine-tuned maneuvering can be performed by modern technology. The truly complicated part of development is detecting and responding to the unexpected. Poor driving conditions can be an issue as well as road construction, but the erratic driving of others is probably the most serious concern. Those driving under the influence of alcohol can be particularly as issue, since the computer will have a hard time trying to predict what will happen next.
An overlying problem is that experienced human drivers have a certain situational awareness. They know what to expect and prepare for, while the computer driver operates almost entirely from a "snapshot" view. While a human might see a red street light through the trees and begin to slow, the computer will not recognize it until the light is closer. Likewise for many other situations, like a pedestrian crossing the street or brake light coming on up ahead. A human knows what will happen next from experience, but the computer can only try to guess from records and guidelines.
Another consideration is that if autonomous cars become common, what kind of driving skill with future drivers have? At a critical moment when the car tells them to take control, will they be capable of solving the problem? Also, what will the focus of the "psudo-driver" be? If an experienced driver was watching the road as if they were driving, they could probably react to a situation effectively. However, if "nothing ever goes wrong" so they get distracted with a phone, passenger, or even nap, they might be unable to intervene properly when needed.
Despite all of the problems and complications of this development, progress has moved forward with this project. Scientists and engineers have actually been designing autonomous vehicles since the mid-1960s, but crucial innovation happened in 2004 when the Pentagon’s research arm began its Grand Challenge.
In recent years, The United States Government's DARPA Challenge produced some credible self-driving cars, with teams from Stanford and Carnegie-Mellon winning prizes in 2005 and 2007. Google was also joined the race, having announced it has tested autonomous cars (with a safety driver) on public roads. 
Science Fiction Become Reality
Self-driving cars have been a popular concept in fantasy and Science Fiction for many years. This idea goes back at least as far as the robot cabs of Robert Heinlein's Between Planets. Since then they have been in a variety of novels and movies of the Science Fiction genre.