ChatGPT o3-mini is (probably) out tomorrow, so we should talk about benchmarks

Part of the DeepSeek conversation revolves around "How good is AI anyway?" and I think most of the benchmarks are just acronyms for people—so let's talk about how AI is actually measured!

ChatGPT o3-mini is likely out tomorrow, and it is likely to be very good. The best indications we have are that o3-mini will (better?) than o1 and vastly faster and cheaper. And if you’re thinking “that’s not that great” you’ve probably been hanging around AI too long. It’s more precise to think of o3-mini as the version of the o3 class that was easiest…