Thanks for your input on the matter. The ironic part is, when you want to do something precise and focused, like I don’t know surgery for instance, I would think that a surgeon wielding a scalpel would end up doing a better job at that then one wielding a Swiss Army knife. Don’t get me wrong a Swiss Army knife will help you a bit more if you’re lost in the woods than a scalpel for sure, but executing a specific precise procedure requires precise tools. If you’re focused with those tools you’re gonna have a better outcome 9 times outs out of 10 than the other guy using universal tools.
Also this is not a schilling or FOMO reply, I got into GOOGL C a few years ago when they did the 1:3 split.
Pytorch/tensorflow work with both CUDA and TPUs so this isn't really true... anyone worth their salt in the AI industry already knows pytorch and tensorflow, and maybe JAX.
The main benefit of CUDA is that you can directly access low level GPU kernels via C++/CUDA, whereas for TPUs you must use high-level languages only like pytorch and JAX.
However since TPUs are built only for matrix multiplication, whereas NVIDIA GPUs are built for general purpose compute, you don't actually need the low level access to TPUs like you do for NVIDIA GPUs.
I feel like this might be the equivalent of seeing a guy hit the jackpot at Casino and not being excited b/c he’s still down overall. But what do I know