like my third week at google i ran an eval script six times with slight different parameters bc i kept fucking it up and then the next week my manager told me the script actually kicked off a human eval with paid contractors and had in total cost more than my annual salary

@tekbog

at one of the companies i worked at the data team ran a query that ended up costing over 100k in compute

beautiful stuff really pic.twitter.com/ympiGrSTbG

then he told me that ofc that didn't matter at all and i should run it as many times as i needed to get enough data, but it was good to be aware

to be clear this was not an "incident" or problem of any kind and i continued to run that same script probably almost a hundred times during my time on that team. when ur making even tiny changes to a product seen by hundreds of millions of users per day, its worth it lol

just realized i made a typo in the tweet too god fucking dammit

hey at least im consistent