The Tensor cores will be great for machine learning and the FP32/FP64 fantastic for HPC, but I'd be surprised if there were a lot of applications using both of these features at once. I wonder if there's room for a competitor to come in and sell another huge accelerator but with only one of these two features either at a lower price or with more performance? Perhaps the power density would be too high if everything was in use at once?
Graphcore's IPU is a machine learning variant on that. Power density seems to be ok. The CTO's talks (used to, I'm out of date) talk about dark silicon a lot.
I share your suspicion that fp64 and ML workloads are distinct but can see each running on the same cluster at different times.
> room for a competitor to come in and sell another huge accelerator but with only one of these two features either at a lower price or with more performance?
They'd need fab capacity first. I wouldn't count on it any time soon, and chip gens have short lives.
