Today I learned that the default Kubernetes scheduler is unsuitable for High-Performance-Computing (HPC) applications. HPC schedulers provide a variety of features, that easily become a rather complex constructs, including the ability to reclaim resources, reserve and backfill jobs, etc., compared to the simple scheduling algorithm that the Kubernetes scheduler provides. So given you want to deploy your HPC workload on Kubernetes it might be worth to look for a more advanced scheduler for your workload.

I came across this when reading this article on the CNCF blog talking about an HPC scheduler for Kubernetes. As someone who never bothered with HPC workload, it was quite interesting.