By default, a container has no resource constraints and can use as much of agiven resource as the host's kernel scheduler allows. Docker provides waysto control how much memory, or CPU a container can use, setting runtimeconfiguration flags of the docker run command. This section provides detailson when you should set such limits and the possible implications of setting them.
Many of these features require your kernel to support Linux capabilities. Tocheck for support, you can use thedocker info command. If a capabilityis disabled in your kernel, you may see a warning at the end of the output likethe following:
WARNING: No swap limit supportConsult your operating system's documentation for enabling them. See also theDocker Engine troubleshooting guidefor more information.
Understand the risks of running out of memory
It's important not to allow a running container to consume too much of thehost machine's memory. On Linux hosts, if the kernel detects that there isn'tenough memory to perform important system functions, it throws an OOME, orOut Of Memory Exception, and starts killing processes to free upmemory. Any process is subject to killing, including Docker and other importantapplications. This can effectively bring the entire system down if the wrongprocess is killed.
Docker attempts to mitigate these risks by adjusting the OOM priority on theDocker daemon so that it's less likely to be killed than other processeson the system. The OOM priority on containers isn't adjusted. This makes it morelikely for an individual container to be killed than for the Docker daemonor other system processes to be killed. You shouldn't try to circumventthese safeguards by manually setting --oom-score-adj to an extreme negativenumber on the daemon or a container, or by setting --oom-kill-disable on acontainer.
For more information about the Linux kernel's OOM management, seeOut of Memory Management.
You can mitigate the risk of system instability due to OOME by:
- Perform tests to understand the memory requirements of your applicationbefore placing it into production.
- Ensure that your application runs only on hosts with adequate resources.
- Limit the amount of memory your container can use, as described below.
- Be mindful when configuring swap on your Docker hosts. Swap is slower thanmemory but can provide a buffer against running out of system memory.
- Consider converting your container to aservice, and using service-level constraintsand node labels to ensure that the application runs only on hosts with enoughmemory
Limit a container's access to memory
Docker can enforce hard or soft memory limits.
- Hard limits lets the container use no more than a fixed amount of memory.
- Soft limits lets the container use as much memory as it needs unless certainconditions are met, such as when the kernel detects low memory or contention onthe host machine.
Some of these options have different effects when used alone or when more thanone option is set.
Most of these options take a positive integer, followed by a suffix of b, k,m, g, to indicate bytes, kilobytes, megabytes, or gigabytes.
| Option | Description |
|---|---|
-m or --memory= | The maximum amount of memory the container can use. If you set this option, the minimum allowed value is 6m (6 megabytes). That is, you must set the value to at least 6 megabytes. |
--memory-swap* | The amount of memory this container is allowed to swap to disk. See--memory-swap details. |
--memory-swappiness | By default, the host kernel can swap out a percentage of anonymous pages used by a container. You can set --memory-swappiness to a value between 0 and 100, to tune this percentage. See--memory-swappiness details. |
--memory-reservation | Allows you to specify a soft limit smaller than --memory which is activated when Docker detects contention or low memory on the host machine. If you use --memory-reservation, it must be set lower than --memory for it to take precedence. Because it is a soft limit, it doesn't guarantee that the container doesn't exceed the limit. |
--kernel-memory | The maximum amount of kernel memory the container can use. The minimum allowed value is 6m. Because kernel memory can't be swapped out, a container which is starved of kernel memory may block host machine resources, which can have side effects on the host machine and on other containers. See--kernel-memory details. |
--oom-kill-disable | By default, if an out-of-memory (OOM) error occurs, the kernel kills processes in a container. To change this behavior, use the --oom-kill-disable option. Only disable the OOM killer on containers where you have also set the -m/--memory option. If the -m flag isn't set, the host can run out of memory and the kernel may need to kill the host system's processes to free memory. |
For more information about cgroups and memory in general, see the documentationforMemory Resource Controller.
--memory-swap details
--memory-swap is a modifier flag that only has meaning if --memory is alsoset. Using swap allows the container to write excess memory requirements to diskwhen the container has exhausted all the RAM that's available to it. There is aperformance penalty for applications that swap memory to disk often.
Its setting can have complicated effects:
If
--memory-swapis set to a positive integer, then both--memoryand--memory-swapmust be set.--memory-swaprepresents the total amount ofmemory and swap that can be used, and--memorycontrols the amount used bynon-swap memory. So if--memory="300m"and--memory-swap="1g", thecontainer can use 300m of memory and 700m (1g - 300m) swap.If
--memory-swapis set to0, the setting is ignored, and the value istreated as unset.If
--memory-swapis set to the same value as--memory, and--memoryisset to a positive integer, the container doesn't have access to swap.SeePrevent a container from using swap.If
--memory-swapis unset, and--memoryis set, the container can useas much swap as the--memorysetting, if the host container has swapmemory configured. For instance, if--memory="300m"and--memory-swapisnot set, the container can use 600m in total of memory and swap.If
--memory-swapis explicitly set to-1, the container is allowed to useunlimited swap, up to the amount available on the host system.Inside the container, tools like
freereport the host's available swap, not what's available inside the container. Don't rely on the output offreeor similar tools to determine whether swap is present.
Prevent a container from using swap
If --memory and --memory-swap are set to the same value, this preventscontainers from using any swap. This is because --memory-swap is the amount ofcombined memory and swap that can be used, while --memory is only the amountof physical memory that can be used.
--memory-swappiness details
- A value of 0 turns off anonymous page swapping.
- A value of 100 sets all anonymous pages as swappable.
- By default, if you don't set
--memory-swappiness, the value isinherited from the host machine.
--kernel-memory details
Kernel memory limits are expressed in terms of the overall memory allocated toa container. Consider the following scenarios:
- Unlimited memory, unlimited kernel memory: This is the defaultbehavior.
- Unlimited memory, limited kernel memory: This is appropriate when theamount of memory needed by all cgroups is greater than the amount ofmemory that actually exists on the host machine. You can configure thekernel memory to never go over what's available on the host machine,and containers which need more memory need to wait for it.
- Limited memory, unlimited kernel memory: The overall memory islimited, but the kernel memory isn't.
- Limited memory, limited kernel memory: Limiting both user and kernelmemory can be useful for debugging memory-related problems. If a containeris using an unexpected amount of either type of memory, it runs outof memory without affecting other containers or the host machine. Withinthis setting, if the kernel memory limit is lower than the user memorylimit, running out of kernel memory causes the container to experiencean OOM error. If the kernel memory limit is higher than the user memorylimit, the kernel limit doesn't cause the container to experience an OOM.
When you enable kernel memory limits, the host machine tracks "high water mark"statistics on a per-process basis, so you can track which processes (in thiscase, containers) are using excess memory. This can be seen per process byviewing /proc/<PID>/status on the host machine.
By default, each container's access to the host machine's CPU cycles is unlimited.You can set various constraints to limit a given container's access to the hostmachine's CPU cycles. Most users use and configure thedefault CFS scheduler. You can alsoconfigure thereal-time scheduler.
Configure the default CFS scheduler
The CFS is the Linux kernel CPU scheduler for normal Linux processes. Severalruntime flags let you configure the amount of access to CPU resources yourcontainer has. When you use these settings, Docker modifies the settings forthe container's cgroup on the host machine.
| Option | Description |
|---|---|
--cpus=<value> | Specify how much of the available CPU resources a container can use. For instance, if the host machine has two CPUs and you set --cpus="1.5", the container is guaranteed at most one and a half of the CPUs. This is the equivalent of setting --cpu-period="100000" and --cpu-quota="150000". |
--cpu-period=<value> | Specify the CPU CFS scheduler period, which is used alongside --cpu-quota. Defaults to 100000 microseconds (100 milliseconds). Most users don't change this from the default. For most use-cases, --cpus is a more convenient alternative. |
--cpu-quota=<value> | Impose a CPU CFS quota on the container. The number of microseconds per --cpu-period that the container is limited to before throttled. As such acting as the effective ceiling. For most use-cases, --cpus is a more convenient alternative. |
--cpuset-cpus | Limit the specific CPUs or cores a container can use. A comma-separated list or hyphen-separated range of CPUs a container can use, if you have more than one CPU. The first CPU is numbered 0. A valid value might be 0-3 (to use the first, second, third, and fourth CPU) or 1,3 (to use the second and fourth CPU). |
--cpu-shares | Set this flag to a value greater or less than the default of 1024 to increase or reduce the container's weight, and give it access to a greater or lesser proportion of the host machine's CPU cycles. This is only enforced when CPU cycles are constrained. When plenty of CPU cycles are available, all containers use as much CPU as they need. In that way, this is a soft limit. --cpu-shares doesn't prevent containers from being scheduled in Swarm mode. It prioritizes container CPU resources for the available CPU cycles. It doesn't guarantee or reserve any specific CPU access. |
If you have 1 CPU, each of the following commands guarantees the container atmost 50% of the CPU every second.
$ docker run -it --cpus=".5" ubuntu /bin/bashWhich is the equivalent to manually specifying --cpu-period and --cpu-quota;
$ docker run -it --cpu-period=100000 --cpu-quota=50000 ubuntu /bin/bashConfigure the real-time scheduler
You can configure your container to use the real-time scheduler, for tasks whichcan't use the CFS scheduler. You need tomake sure the host machine's kernel is configured correctlybefore you canconfigure the Docker daemon orconfigure individual containers.
Warning
CPU scheduling and prioritization are advanced kernel-level features. Mostusers don't need to change these values from their defaults. Setting thesevalues incorrectly can cause your host system to become unstable or unusable.
Configure the host machine's kernel
Verify that CONFIG_RT_GROUP_SCHED is enabled in the Linux kernel by runningzcat /proc/config.gz | grep CONFIG_RT_GROUP_SCHED or by checking for theexistence of the file /sys/fs/cgroup/cpu.rt_runtime_us. For guidance onconfiguring the kernel real-time scheduler, consult the documentation for youroperating system.
Configure the Docker daemon
To run containers using the real-time scheduler, run the Docker daemon withthe --cpu-rt-runtime flag set to the maximum number of microseconds reservedfor real-time tasks per runtime period. For instance, with the default period of1000000 microseconds (1 second), setting --cpu-rt-runtime=950000 ensures thatcontainers using the real-time scheduler can run for 950000 microseconds for every1000000-microsecond period, leaving at least 50000 microseconds available fornon-real-time tasks. To make this configuration permanent on systems which usesystemd, seeControl and configure Docker with systemd.
Configure individual containers
You can pass several flags to control a container's CPU priority when youstart the container using docker run. Consult your operating system'sdocumentation or the ulimit command for information on appropriate values.
| Option | Description |
|---|---|
--cap-add=sys_nice | Grants the container the CAP_SYS_NICE capability, which allows the container to raise process nice values, set real-time scheduling policies, set CPU affinity, and other operations. |
--cpu-rt-runtime=<value> | The maximum number of microseconds the container can run at real-time priority within the Docker daemon's real-time scheduler period. You also need the --cap-add=sys_nice flag. |
--ulimit rtprio=<value> | The maximum real-time priority allowed for the container. You also need the --cap-add=sys_nice flag. |
The following example command sets each of these three flags on a debian:jessiecontainer.
$ docker run -it \ --cpu-rt-runtime=950000 \ --ulimit rtprio=99 \ --cap-add=sys_nice \ debian:jessieIf the kernel or Docker daemon isn't configured correctly, an error occurs.
GPU
Access an NVIDIA GPU
Prerequisites
Visit the officialNVIDIA drivers pageto download and install the proper drivers. Reboot your system once you havedone so.
Verify that your GPU is running and accessible.
Install nvidia-container-toolkit
Follow the official NVIDIA Container Toolkitinstallation instructions.
Expose GPUs for use
Include the --gpus flag when you start a container to access GPU resources.Specify how many GPUs to use. For example:
$ docker run -it --rm --gpus all ubuntu nvidia-smiExposes all available GPUs and returns a result akin to the following:
+-------------------------------------------------------------------------------+| NVIDIA-SMI 384.130 Driver Version: 384.130 ||-------------------------------+----------------------+------------------------+| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. ||===============================+======================+========================|| 0 GRID K520 Off | 00000000:00:03.0 Off | N/A || N/A 36CP0 39W / 125W | 0MiB / 4036MiB | 0% Default |+-------------------------------+----------------------+------------------------++-------------------------------------------------------------------------------+| Processes: GPU Memory || GPU PID Type Process name Usage ||===============================================================================|| No running processes found |+-------------------------------------------------------------------------------+Use the device option to specify GPUs. For example:
$ docker run -it --rm --gpus device=GPU-3a23c669-1f69-c64e-cf85-44e9b07e7a2a ubuntu nvidia-smiExposes that specific GPU.
$ docker run -it --rm --gpus '"device=0,2"' ubuntu nvidia-smiExposes the first and third GPUs.
Note
NVIDIA GPUs can only be accessed by systems running a single engine.
Set NVIDIA capabilities
You can set capabilities manually. For example, on Ubuntu you can run thefollowing:
$ docker run --gpus 'all,capabilities=utility' --rm ubuntu nvidia-smiThis enables the utility driver capability which adds the nvidia-smi tool tothe container.
Capabilities as well as other configurations can be set in images viaenvironment variables. More information on valid variables can be found in thenvidia-container-toolkitdocumentation. These variables can be set in a Dockerfile.
You can also use CUDA images which sets these variables automatically. See theofficialCUDA imagesNGC catalog page.
