there are many areas where more effort doesn't matter until u pass a certain capability threshold, and then suddenly effort matters a lot.

if u give a 9 year old a year to write a book, its not going to be a very good book. if they stay at 9 yo capacity then even 1000 years wouldn't be enough for them to write a good book. but almost any reasonably smart and creative adult can write an okay book in a year of dedicated effort, and almost always significantly improve on it (or just come up with a better overall book idea) given 10 years.

what happens when u give someone just as capable as the best of the best human writers 1000 subjective years to write the best book they can? does it really make any sense that it would be only as good as one that was worked on for 1 year?

sure, very likely there are diminishing returns, maybe the first 10 years of focused effort is as valuable as the next 100. still, the further effort will certainly matter.

humans have made massive leaps in all fields over centuries not due to hugely increasing individual genius intelligence (although yes that has happened and has helped), but due to civilization trying a fuckload of things and aggregating knowledge of what works and building on past knowledge. this has aggregated exponentially. even if we only ever get smart-human level AGI, speeding this process up *will cause exponential capability increase at a civilizational level*

@packyM

what is the best argument / evidence that this won't happen to LLM scaling laws?

genuinely curious. pic.twitter.com/xYBNJcm9mW