Today I learned how to properly interpret the CFS metrics the linux kernel provides. CFS is the “Completely Fair Scheduler” in the linux kernel, that is taking care of assigning processes to CPU cores. It’s an important element of containers and therefore Kubernetes, by allowing to limit CPU time spend by certain processes, containers and Pods. In Kubernetes it’s controlled by the spec.containers[*].resources.limits.cpu setting. The metric container_cpu_cfs_throttled_periods_total / container_cpu_cfs_periods_total provides the percentage of throttling for each container in a cluster, allowing to optimise workloads. According to the talk linked below, which explains the whole metric in depth, for general purpose compute you want to aim for 0-10% of throttling to strike a reasonable balance between performance and provisioned CPU capacity.

I came across this after trying to answer the question what container_cpu_cfs_throttled_seconds_total is good for and whether one could use it, to calculate the exact CPU limits needed for a process.¹ In order to answer this question, I researched to topic a bit and came across this talk from 2019², explaining one of the bigger bugs in this kernel feature, how it was fixed, but most importantly, how the general feature works in detail.