As a young programmer, I was doing software for a self driving electric car project back in 1972-1975. The company built a test and demonstration course with garage/terminal and three stops on a 14 acre site. Getting everything to work as expected was pretty tough back then - there are endless "possibilities" that have to be accounted for. The company came reasonably close to selling the concept to the city of Los Vegas to provide transportation between the airport, the strip, and downtown. However, simulations showed it would be hopelessly congested. The project was canned when one of the vehicles took off on it's own cross country, ran into a creek, turned over and caught fire.
Of course, we were trying to do this with a PDP11 with 24Kbytes of memory
Bottom line, I think this is a tougher problem than many assume. I've also heard that the biggest issue is how to turn control back over to a human when things go wrong and the human is unprepared to handle the situation.