generally the answer is "long tail reliability". 99% course correctable demos are many many many orders of magnitude easier. extremely high quality high reliability sensors are hard, but were mostly solved by 2018ish. but the planning techniques being used at that point, and for the next 4-5 years, were still hugely classical and heuristics based, always playing catchup to long-tail issues especially in new environments. sufficiently large models with sufficiently powerful onboard processing phase transitioned around 2022 and started significantly driving down long tail reliability issues without the whack-a-mole.

if things were deployed unsupervised at scale earlier they would have crashed, killed people, and blocked traffic, and this was directly known based on simulation and test failure rates multiplied by expected miles at scale. in fact too-early deploys like uber and cruise did exactly that, and were shut down for doing so!

@dwarkesh_sp

Is there a good write up of why self driving cars took a decade+ from working demo rides to deployed at scale?