Control Groups

aka cgroupsv2 aka unified cgroups. Control Groups v2 were unveiled in Linux 4.5 as a means of controlling resource allocation across groups of tasks. Unlike the previous implementation of control groups (introduced in Linux 2.6.24), these cgroups would be under a single hierarchy, usually mounted with a cgroup2-type virtual filesystem at /sys/fs/cgroup. systemd makes fundamental use of cgroups (and indeed, last I checked it couldn't run without them) -- the systemd concept of a slice corresponds to a new cgroup node. I am unaware of anything useful that can be done with cgroupsv1 that can't be accomplished in cgroupsv2.

A great deal of cgroup-related documentation refers to cgroupsv1, especially since it can more or less be run alongside cgroupsv2. Pretty much all information about cpusets (including anything referring to /dev/cpuset or libcpuset) is referring to the obsolete cpuset controller based on cgroupsv1. The "PaxControlGroups" document is cgroupsv1. libcg is cgroupsv1.

According to systemd's documentation, on a machine making use of systemd 205+ as its PID 1, the only legal way to interact with cgroupsv2 is through systemd's APIs (as cgroupsv2 is intended to be a "single-writer" facility). However, systemd exposes only a subset of the full cgroupsv2 functionality. Important cgroupv1 concepts like cpusets and freezer are not currently implemented. One must directly mess with the cgroupsv2 hierarchy to use this functionality, despite systemd's warnings. It tends to work out in the end.

Modern releases of libcgroup seem to be cgroupv2, but don't seem to use the systemd API, and appear to need esoteric compilation flags to work alongside systemd at all? So I'd stay away from it.

CPUsets with cgroupsv2

• Ensure that the cgroups2 filesystem is mounted:

root@schwarzgerat:/sys/fs/cgroup# mount | grep cgroup2
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot)
root@schwarzgerat:/sys/fs/cgroup# 

• Ensure that cpuset is in /sys/fs/cgroup/cgroup.controllers:

root@schwarzgerat:/sys/fs/cgroup# cat cgroup.controllers 
cpuset cpu io memory pids rdma
root@schwarzgerat:/sys/fs/cgroup# 

If it's absent, you're either missing the necessary kernel support, or you have a cgroupv1 implementation mounted.

• Enable the controller in the toplevel:

root@schwarzgerat:/sys/fs/cgroup# cat cgroup.subtree_control 
cpu memory pids
root@schwarzgerat:/sys/fs/cgroup# echo "+cpuset" > cgroup.subtree_control 
root@schwarzgerat:/sys/fs/cgroup# cat cgroup.subtree_control 
cpuset cpu memory pids
root@schwarzgerat:/sys/fs/cgroup# 

• All subdirectories of your cgroup2 filesystem will now have cpuset entries ala:

./user.slice/cpuset.cpus.partition
./user.slice/cpuset.cpus.effective
./user.slice/cpuset.mems
./user.slice/cpuset.mems.effective
./user.slice/cpuset.cpus