I feel like the NTSB need to draft a min spec for self driving cars and a testing course that involves some of the worst circtimstances to get approved. I feel like all self driving cars should have to have lidar, and other sensors. Computer vision really isn’t working out.
You build a benchmark and tesla will train on that benchmark, says nothing about real world use but gets them signed off.
But yes western society is currently in a hellscape of refusing to do even basic regulation of any new technology so it’ll probably be a good 20 years of murder robots on the streets before anything gets written down.
By “western society” do you mean the US? Because the EU doesn’t seem to have any qualms about regulating new technologies. That seems to be a uniquely American thing.
Which somehow means that Europeans suddenly have headlights that makes sense while we’re over here dying from aftermarket HIDs that should be treated like the VA Highway Patrol treats radar detectors ( rip ‘em out and smash them with a sledgehammer on the side of the road)
To be fair we already have giant metal murder boxes zooming around on the streets. If AI kills even a single person everyone flips out even though over 40,000 people die every year in the US from car accidents. And that is just the deaths, not including injuries. Yet I don’t really see anyone calling for more regulations on driving tests for humans.
People want AI to somehow be perfect when in reality as long as AI is even 1% better than humans that’s saving over 400 lives per year. AI doesn’t get sleepy, distracted, drunk, etc. so it probably already is at least 1% better in most situations. Humans are horrible drivers.
Speak for yourself, chief
But yes western society is currently in a hellscape of refusing to do even basic regulationUS regulations are only written in blood or money. the united states was built on the backs of slaves, and then wage-slaves. literal graveyards filled with workers.
im not disagreeing with you, i just found this comically disparate to history… ie, its always been a regulation hellscape.
But yes western society is currently in a hellscape of refusing to do even basic regulation
Only the Usamerican country.
We Europeans are scratching our heads already for very long: why are they letting these guys do just everything they want?
Not really. The eu does more than most western nations, but it’s generally things that get regulated ten years too late and only a tiny amount compared to what society actually needs. So again, better, massively lax compared to need and comparisons to other periods
We’re the whole West. Didn’t you know?
/s
I don’t think mandating lidar specifically by name is right, seeing as computer vision is definitely a software problem. Instead they should mandate some method to detect objects in any light condition + a performance standard, which in practice during certification could mean lidar. Regulations should be as minimal and specific as possible.
Good point. Mandate the ends rather than the means. If they get better functionality with some new tech in a few years, we don’t want outdated regulations holding the industry back.
computer vision is definitely a software problem.
No, it isn’t.
If it were only software, don’t you think Tesla should be the best of them all, being the pure software shop they are?
But it is a real world problem. Recognizing real objects in real world conditions like weather, natural and artificial lights, temperatures (want some ice on your camera?), winds & storms, all kinds of unforeseen circumstances, other bad drivers, police and firemen…
And that’s why that pure software shop is so bad at it, while all the real carmakers shrug… they are used to it since forever.
You can be the best in the world and still not be good enough.
Driving a car around using a dozen cameras pointing in every direction isn’t something that’s fundamentally impossible. We just can’t do it yet.
And don’t forget vision is what humans use for navigation as well.
And a lot of them are not good at it
As well as human hearing, human touch, human balance / proprioiception, possibly even smell too.
If a person is rear ended, they might not even see the car that hit them, but they know they’re hit based on how the impact moves their body. If a tire blows out on the highway, the first thing a driver might notice is that the steering wheel feels sluggish. I could even imagine a situation where someone driving sees something unusual up ahead and then smells something dangerous, and turns around in time to avoid driving into an active chemical spill. In that situation seeing alone might not be enough to signal the danger.
I would hope that a competent self-driving car design at least incorporates microphones, and some kind of “body” sensors that would notice an impact, notice changes to balance, and so-on.
You can be the best in the world and still not be good enough
So you haven’t recognized that other car brands’ assistance/autonomous systems make less dramatic mistakes?
Literally a different thread about someone dieing from a Cruize self driving car not moving over for an ambulance.
Dying… because he was hit by a human driven car. Maybe the Cruize cars exacerbated the problem, but the original problem was that the victim was hit by a car driven by a human being.
Should a self-driving car face more rigorous tests than actual human drivers? Honest question.
Yep.
Yes, because when there’s an accident with a person driving, you usually know exactly who is legally to blame in an accident. With self-driving, if the car accidentally hits and kills someone, who do you charge for it? There’s no one person you can point to for responsibility for if something goes wrong, like you can for a person responsible for an accident.
Yes. A human brain can handle edge cases it’s never encountered before. Can a self driving car?
-
Ever stop at a red light only to have a police officer wave you through?
-
Ever encounter a car driving the wrong way down a one way street?
-
Ever come across a flooded out stretch of road? (if the road has no lines and the water is still it can be very deceptive looking)
These are a tiny number of things I’ve encountered over the past few years. I’m sure plenty of other drivers can provide other good examples. I’d want to know how a self driving car would handle itself in situations like these.
Those are pretty basic conditions that I hope are already in the training data.
What about a wildfire evacuation? Police might have people driving on the wrong side of the highway to make use of all the lanes. Smoke might be obscuring everything. A human driver would know not to pay attention to any of the road signs in that situation without ever having been trained on it, but would a self-driving car?
Or, how about any situation where a police officer has to have a driver roll down the window to give them instructions for dealing with some unusual situation, like a chemical spill or a landslide.
Or, what about highway signs that have been shot by a shotgun so that it’s hard to read? Or, what about novelty highway signs that a business might put up as a joke?
Self-driving cars definitely need to be tested against a much bigger range of situations than a human driver. Much as we might be baffled by their lack of common sense, the common sense of an average 16-year-old is still off the charts compared to an AI. Having said that, I know how bad many drivers are, and I wouldn’t be surprised if the competent self-driving car organizations (Cruize, Waymo, etc.) are already better than an average driver under 99.9% of common scenarios.
How will the bot car handle itself out in the country? Dirt roads? Deer? Roadblock checkpoints full of bored, mean spirited cops.
How will the bot car handle itself out in the country? Dirt roads?
They don’t go there. They have their limits. Simple as that.
But when the police has ordered them there (for example, the good road must be emptied because of an emergency) then the trouble starts… now imagine not just one or two, but hundreds of them.
-
Human drivers should be facing more rigorous testing regardless. It’s horrifically easy to get a license… and then they never test you again for the rest of your life. That’s just insane when you think about it. My test was in 2002. Feels like I should have to retake it at some point.
And take them away for bad driving. But we don’t because our entire transportation infrastructure, outside of a few cities namely NY, is built around everyone driving a car.
Yes because each person must learn on their own and have limited experience relative to the general public as a whole.
Self driving cars can ‘learn’ from all self driving cars and don’t get tired, forget, or anything like that. While they shouldn’t be held to perfection, they should absolutely be held to a higher standard than a human.
Should a self-driving car face more rigorous tests than actual human drivers? Honest question
First: none of these automated cars would pass a German driver’s license test. By far.
Second: of course you cannot compare tests for humans with tests for machines.
What are the things you think would cause a self-driving car to fail the German test?
Many. But the most obvious currently: they are going too slow.
And maybe the most funny: they are unable to turn their heads (in order to prove to the inspector that they are looking where it is required to look).
Going too slow where, in the city? They’re following traffic and/or speed limits. They also go on highways, but again, there are speed limits. I’m sure they could have a version for Germany where the max speed was adjusted to be appropriate for the autobahn.
As for turning their heads, does Germany make no exceptions for people with disabilities? I’m sure that they could implement a version of “show the instructor you’re paying attention to the right thing”.
Going too slow where, in the city?
Everywhere.
exceptions for people with disabilities?
Well, not that kind of exception where “the requirements are forgotten and we can make everything easier just for you”.
I don’t know this specific thing, but in general, a disabled person must have some kind of aid that fully compensates the disability.
Think of eye glasses: your eyes are bad, so you are required to wear glasses (or contacts). Note that it is on you. You get an extra order written into your license saying that you always have to wear them when driving, and they must fully compensate your vision - otherwise you are not allowed to drive.
Only Tesla self driving cars need to have more rigorous tests. Other brands are fine as it is because they have lidar.
LiDAR isn’t some sort of magic eye. The self driving system is only as good as the software that takes the inputs from cameras, LiDAR, etc., processes them, and ensures safe operation of the car.
Finally someone who actually uses critical thinking instead of being an anti-Elon bandwagoner.
I feel like all them do, have you seen wayze nearly getting black people killed cause it didn’t stop for s cop. And it can’t recognize construction zones.
Five LiDAR sensors hasn’t stopped Cruise from running into a bus, multiple cars, and a fire truck. Maybe self-driving is a myth?
Maybe we should just build buses and trains and pay people good salaries to operate them??
Throw I some pot holes and child pedestrian crossing the street, etc and they’d even come out with a powerful marketing ad.
AND have triplicate back up system that runs in parallel.
Everyone would build to pass the test track. This does get at the problem though: the permutations of scenarios an L5 system has to correctly process is a huge number. Trying to build a system that can do that appears to be beyond anyone’s av system right now. This is why the most advanced deployments are all geofenced. That way at least the traffic signs and signals, lane markings, etc all understood and tested. Even then ‘shit happens’. Untested scenarios still occur. Also the maps are always out of date.
The problem really requires AGI, and nobody has one of those, or if they do it’s a secret.
Pretty much what the UNECE did… there are standards for these things. Tesla doesn’t meet them, which is why FSD ‘beta’ is still ‘seeking regulatory approval’ in the rest of the world.
It’s not really a sensor issue, as much as having software that can interpret the sensor data and act on it. Cameras and lidar effectively provide same thing, distance to objects in 2d/3d. But u need software to process that data and identify where the road is, where little jonny is, and what to do…arguably, the distance measuring problem has been solved for a while with lidar or with cameras, it’s object identification and reaction to that info that’s not solved. You can’t really solve it with traditional if/else programming, while AI gives you only a probability of what something is or what action to do…so the problem is hard.
But ntsb/dmv whatever needs to come up with a way to test and classify autonomous driving software…probably doing real world test and identifying edge cases where it fails.
bUT thAt WOuLD StiFle proDucT iNnoVatIoN!!!
When I found out teslas don’t have LiDAR I nearly shat myself.
Yeah, fun stuff happens when the AI tries to interpret vision of sunlight shining straight into the lens.
Don’t you know those things cost money!?
Sadly cost cutting MBAs seldom concern themselves with silly things like function or necessity.
That wasn’t an educated MBA who cut them it was a stupid CEO who felt it was an unnecessary crutch (his words).
I assure you he was advised what to cut by an MBA he hired.
Have you seen how much these cars cost? For that amount of money it should personally serve me breakfast in bed, let alone having a scanner
Should is carrying a large load there.
They did but then Musk had the genius idea to stop installing them. I still have it in my older model but they changed the software not to use them anymore. Like I said, genius…
Which car do you have that has LiDAR in it? Maybe you’re thinking of RADAR which is different.
Oh, you’re right, it’s radar, sorry.
For an autonomous vehicle without radars or LiDAR they do still drive pretty darn well. AI DRIVR makes really good videos about FSD on YouTube and love it or hate it, it’s quite impressive how well it does despite the the lack of these sensors.
For an autonomous vehicle without radars or LiDAR they do still drive pretty darn well.
That’s a bit like saying “For an old hound dog with only two legs left he is running pretty darn fast”… :-)
Still pushing full self driving even while they are dodging lawsuits by claiming it’s just highway cruise control that customers are abusing
People commonly confuse Autopilot and FSD beta. One is the advanced cruise control and comes on all models while the other is supposed to be autonomous driving and costs $15k extra.
Yeah I’m not getting in a car driven by AI. The tech ain’t where it needs to be for that.
It needs to be regulated to hold manufacturers responsible when their software isn’t good enough. My understanding is that there already probably is enough regulation and government agencies just need to hold Tesla accountable.
Personally, I’m all for cars driven by AI iff it’s better and safer than a human driver. Human drivers make a lot of mistakes and driving is the most dangerous everyday activity many people do. But if the AI isn’t better than a human, that’s a problem. I don’t need AI drivers to be flawless, as that’s an unrealistic bar. I just need them to be undeniably better than humans. Everything I’m hearing about Tesla’s self driving is that they aren’t.
Especially Tesla. I am very into computer vision research but I would never trust a vehicle that relies on only that with 0 LIDAR or other sensing technologies in place.
Great news. You don’t have to. The other people around you moving thousands of pounds of steel will be relying completely on that, and there is nothing you can do to stop it.
Unfortunately you can still be hit by cars that idiots let the AI drive.
Thats why I drive defensively and look both ways before I cross a street.
I’m convinced that the self-driving AI from competent companies is better than human drivers already. The bar isn’t perfection, the bar is the average driver, and the average driver is bad.
Having said that, I’d never get in a car with Tesla’s self-driving solution. Musk polluted the term “full self-driving” to cars that definitely weren’t.
No, the bar is accountability. If the FSD screws up either:
- It’s claimed the driver was responsible, in which case it’s absolutely not FSD
- Someone other than the driver is investigated and cleared/convicted of wrong doing.
It absolutely does not matter that it’s better than an “average” driver. If the best driver in the world screws up, they are still held accountable.
When a service is willing to take responsibility for collisions and driving violations, then we know it works. If the guy asleep at the wheel (which he allegedly can do in an autonomous car) is still the one held responsible, then were not there yet.
That said end-to-end AI totally sounds like equivocal marketing buzz.
When a service is willing to take responsibility for collisions and driving violations
Devil’s advocate: it’s kinda hard to pin the responsibility on Tesla when at the end of the day there was a person driving and the driver’s always responsible.
I’m not disagreeing with you, I’m on team ban-human-drivers
Ideally, we’d get to the point where the driver merely directs the vehicle to where it wants to go, and then the computer system works out all the pathfinding and maneuvering, so that yes, any instance where a vehicle avoidably collides with another thing can be regarded as a malfunction.
I wonder what happens when the car is on a collision course with a golden retriever and the only way not to hit it would be to damage the car. Or same scenario, but the only way not to hit it, is it to hit an 07 Carolla parked on the side of the road. Not saying humans have superior judgement… just wondering if it will be programmed by the theory of actuarial of philosophical science.
That makes me think- will the AI see a kid that’s about to run out from behind a parked car? As a human, if I see a kid run from the house into a row of parked cars, I know he’s still there and will slow down before I get there. But would self driving make that same leap of logic? I’m not sure what the range and capabilities of self driving cars are right now in terms of scanning, but hopefully it would be smart enough to take preventative measures
Right now, car AI has trouble both with kids and non-white persons. That said, when it comes to the things it is good at detecting, the cars respond much more quickly. This came up when an official asked about how it detects brake lights, and the project advisor (from Google, I think) explained that the car doesn’t worry about break lights but instantly detects when a car ahead of it rapidly decelerates, and responds immediately.
I’m pretty sure we can get cars smart enough and sharp enough to drive better than humans. But the recent incident in San Francisco where Cruise driverless taxis blocked an ambulance with a patient in critical condition (resulting in their death), suggests to me we underestimated the layers of logistics necessary to make cars truly autonomous.
Randal Munroe listed a few more incidents we can expect (Obligatory XKCD).
According to other commentors, the need will never arise because the AI cars will be programmed so well it’s impossible to have accidents 🙄… now I see why FSD will never become a reality.
Good question. Neural networks are modelled after how brains learn and process information, so it’s certainly theoretically possible for a neural network (or other machine learning algorithm) to make inferences like that, just like how you’ve learned them with years of experience.
The biggest challenge in any machine learning is finding enough labelled training data. In fact, a friend of mine contributed to a paper in which (no joke) GTA V was used to generate labelled training data for an automous vehicle. Because it’s a game engine, every object in the game is already digitized, and the 3D modelling is accurate enough to be useful, at least. This vehicle used LIDAR so the actual shaders and such didn’t matter as much as the 3D point cloud.
I doubt it. Germany has already implemented (considering implementing) regulations regarding the ethics of autonomous vehicles. As it is, cars are simply trying not to collide with anything and given their reflexes and perception are way faster and more accurate than human beings, they have a better chance of saving both the dog and the other car.
That said, one of the problems we’re seeing with smart devices (that is devices that are software run rather than controlled by simple mechanics) is that companies are keen to abuse the power that gives them, hence the whole John Deere tractors debacle and the development of right-to-repair laws. Also, some BMWs require rental of some of their features (such as seat warmers) which seems to me as less than ethical.
So I hope we’ll get to a point where not only is it anyone’s right to jailbreak their devices (including a self-driving car) but there will be several FOSS options we can choose from. And that means someone who programmed them may actually find a process-layer in which hazard prioritization or victim prioritization is considered.
It is certainly an entertaining idea of speculative fiction that an aggressive driver package is developed, gets popular and then causes a rise in traffic accidents. More likely would be software packages that allow the vehicle to operate despite self-test failures, again leading to a higher traffic collision rate.
Agressive driver package.
That’s already a key plot point in the show Upload (which I really enjoyed)
There are many such hypothetical scenarios based on the trolley problem, but the real answer is that a good self driving system will never end up in that situation in the first place.
So as a dev, you just program to not let that situation arise, then you won’t need to program a solution for that.
Gonna take the concept of AI hallucinations to a whole new level.
That just made me think what if that AI stumbles on video game footage by mistake like NFS or some other screen racing games. Oh boy.
Burnout Revenge’s epic crash challenges.
You mean LLM hallucinations?
Typical ML models are not hallucinating in the same manner.
“There was a ghost! This is ectoplasm!”
You wouldn’t let chatGPT drive a car would you?
DriveGPT. Written by chatGPT proompted by chatGPT. Powered ny Nvidia^tm©®.
Not a fan of Tesla or Musk, but can we differentiate the broad public understanding of the term AI from machine learned control systems? People anthropomorphize the situation into thinking there is an I, Robot style driver enough as it is.
Counterpoint, though, maybe doing so encourages skepticism of Tesla’s capabilities.
It’s not AI it’s VI.
Virtual intelligence is not artificial intelligence.
That’s right, all the mistakes it makes are now powered purely by AI. Isn’t that fantastic?
deleted by creator
What do you think was powering it untill this point?
The current beta is a mix of ai algorithms and human programming. It is also a dead end now as they’ve abandoned that approach.
I guess so?
As a cyclist I really do look forward to the day where good AI is consistently better than the average-to-worst drivers out there; the bar is depressingly low and the stakes are high.
I write (and test) software for a living and my experience with Tesla as a consumer device is that it’s many generations away from being something I would trust.
Also, I’ve seen what happens to product quality when management overrides its engineers in the way elon does- we get pre-alpha quality out there in the wild, being tested on a public that didn’t sign up for that shit
That title and thumbnail are pure poetry.
Felon Must speaks again.
How about that?
