Recent FSD beta videos show how quickly the self driving software can "learn", and why Tesla's "4D" labelling approach is so important.
thedriven.io
SOURCE: TESLA OWNERS SILICON VALLEY
Recent videos released by Tesla Full Self-Driving (FSD) beta testers are showing how quickly the “autonomous” software is learning and why the EV maker’s “4D” approach to its improvement is so important.
Its release followed a “fundamental rewrite” of the FSD software, flagged by Tesla boss Elon Musk at the recent Battery Day, and which now combines imagery gathered from all eight cameras of a Tesla vehicle into a series of linked images, as opposed to image after image.
“The sophistication of the neural net of the car and the overall logic of the car is improved dramatically,”
said Musk at the time.
As previously reported by The Driven, Tesla’s beta self-driving software update has
been in use for some weeks now, and despite a great deal of skepticism does not appear to have yet resulted in any major accidents.
Instead, thanks to many kilometres of data gathered by enthusiastic beta testers and feed back to Tesla, FSD appears to be getting rapidly better.
Testing in the real world is key to Tesla’s strategy to expose the neural net and algorithms that underpin the self-driving software to “edge cases”, that would not otherwise occur in a simulated environment.
So is time. As outlined
in this video by YouTuber
Dr. Know-it-all Knows it all (and first reported on by
Cleantechnica’s Johnna Crider), it is time (the fourth dimension) that will allow Tesla’s FSD software to link object recognition from one instance to another, in order to pass insight gathered in one situation to the next.
As noted by Musk, it is this linking of meaning from image to image, dubbed “quasi-semantics” in the video, that is essential to the correct and safe operation of FSD, particularly for objects that are (from the vehicle’s point of view) constantly moving and at times hidden.
“Good explanation,”
said Musk of the video. “4D is essential for dynamically occluded objects, especially in large intersections with dense vehicle & pedestrian traffic. Frame rate & latency from frame to wheel vector change also important.”
This is where the testers come in.
The more times the software is exposed to a situation, and either successfully navigates it or is disengaged as the tester takes over to correct the situation, the more information is sent back to Tesla’s neural net about how to handle similar manoeuvers, which is then sent out to Tesla vehicles.
(Also take note on his comment about frame rate and latency, which is discussed later in this article and for which Tesla’s “Dojo” neural net training computer will be essential.)
Since going live in late October, a range of videos posted to YouTube and Twitter have shown reported improvements in smooth operation of the software (we note that Tesla owners chosen to test the “fully functional” version of the self-driving software are required to stay attentive and be ready to take over at any time while FSD is active).
A new video series posted by “
Tesla Owners Silicon Valley” on YouTube shows a 14 minute drive with apparently no disengagements of the FSD feature, which uses version 2020.40.8.13 of the software.
“It’s pretty mind blowing that it can literally do all of this by itself,” the driver says, noting that because he is using a GoPro the drive is cut into approximately nine minute segments.
Significantly, a large portion of the 14 minute drive, which includes a range of manouevres including recognising a cyclist, stopping at traffic lights and making both hard and soft turns, and lane changes, is in the rain.
There are reflections on the road which doesn’t seem to obfuscate actual objects being recognised and responded to by the vehicle’s software, either.
“From a side road to a city street to a highway and now it’s starting to rain. I have not had a disengagement in 14 minutes,” he says.
Another example posted on Monday (US time) show the Tesla Model 3 owned by
YouTuber James Locke “learning” to perform a forced 180 degree turn in just two goes.
“This was all done one after the other this afternoon with no new software updates,” he says in the video’s description, adding you can check the time stamps for each attempt.
“It did it on the 2nd try as shown in this video. Each attempt in this video was immediately taken after the last. After I did one attempt I turned around and immediately tried again. These are all this afternoon trying one after the other,” Locke told a viewer.
Each time the manouevre is performed, the vehicle sends data back to Tesla’s neural net, which then deploys the new information back to all Tesla vehicles.
This approach means that Tesla is able to incrementally improve the performance of the self-driving software, so that eventually Tesla will be able to release it on scale to the broader community of Tesla FSD owners.
However, this will still take some time. While Musk has said that
Canada and Norway will be the next markets for FSD beta release, it will be at least 12 months until Tesla’s “Dojo” neural net training computer, which will substantially increase the amount of data that can be processed, is ready to be unleashed.
On Monday (US time), Musk confirmed on Twitter that,” Version 1 is about a year away.”
“Dojo won’t contribute for about a year. It’s mostly a generalized NN training computer, but benchmark we’re tracking is frames/second. Must beat next gen GPU/TPU clusters or it’s pointless.”
en.wikipedia.org
