What's interesting is that the trend in "Average Weighted Accuracy" seems to increase linearly in the past three years whereas the number of model parameters and required compute is growing exponentially. So accuracy is pretty expensive.
I mean, yes, you have around 100 trillion interconnections in your brain, likely the most complicated biological device on this planet, to create the accuracy you perform. We're just very power efficient when doing that computation.
