Best practices for building container images

Best practices for building container images

A container that runs well in a data center does not always run well at the edge. An SC//Reliant™ node has limited CPU, memory, and disk, shares them with other containers and virtual machines, and often reaches the internet over a slow or unreliable link. It also runs unattended, so a container that fails needs to recover on its own.

This page describes the requirements every container image must meet to run on the SC//Reliant platform, followed by recommendations for building images and applications that suit these conditions. For a walkthrough of building and deploying a simple image, see Tutorial: Deploying a custom container to an SC//Reliant node.

Requirements

An SC//Reliant node runs Linux on x86_64 processors, so every container image you deploy must meet the following two requirements:

  • It must be built for Linux. Containers share the host's kernel, so an image built for a Windows host cannot run on a node.
  • It must be built for the linux/amd64 platform. An image built for an ARM processor, such as one built on an Apple silicon Mac or a Raspberry Pi, cannot run on a node.

Docker builds for the build machine's own architecture by default. If your build machine has an ARM processor, you must explicitly target linux/amd64, either with --platform=linux/amd64 on the final FROM line of the Dockerfile or with the --platform linux/amd64 option of docker build. To confirm the platform of an image you have built, run the following:

docker image inspect <image_name>:<tag> --format '{{.Os}}/{{.Architecture}}'

The output must be linux/amd64.

Keeping images small

Every image is downloaded to each node that runs it, often over the same link the site uses for its business traffic. A smaller image deploys faster, uses less disk on the node, and leaves less software to patch.

Start from a minimal base image

Choose the smallest base image that runs your application, such as an Alpine, distroless, or slim variant, rather than a full operating system image. Pin the base image to a specific version tag rather than latest, so that rebuilding the image does not silently change what it contains.

Use multi-stage builds

Build your application in one stage and copy only the result into a clean final stage. Compilers, package managers, source code, and test tools stay in the build stage and never reach the node.

The following example builds a Java application with a full JDK and Maven, and then copies only the resulting JAR file into a runtime image that contains the Java Runtime Environment (JRE).

# Build stage: full JDK and build tools. This stage runs on the build
# machine's own architecture, which is fine because the JAR is portable.
FROM maven:3.9-eclipse-temurin-21 AS build
WORKDIR /src
COPY pom.xml .
RUN mvn dependency:go-offline
COPY src ./src
RUN mvn package -DskipTests
# Runtime stage: JRE only, built for the node's platform.
FROM --platform=linux/amd64 eclipse-temurin:21-jre-alpine
WORKDIR /app
COPY --from=build /src/target/app.jar .
USER 1000
ENTRYPOINT ["java", "-XX:MaxRAMPercentage=75", "-jar", "app.jar"]

This example also demonstrates several of the other practices on this page: layers ordered by how often they change, a non-root user, an exec-form ENTRYPOINT so the application receives shutdown signals, and a JVM memory setting that respects the container's memory limit.

Leave development tools out of production images

A production image needs only what the application uses at run time. Do not ship compilers, debuggers, package manager caches, or SDKs. Java images are the most common offender: an application that needs only the JRE is often shipped on a JDK image because of an assumed dependency. Test the application on the runtime-only image rather than assuming it needs the full kit.

Order layers by how often they change

Docker downloads only the layers of an image that a node does not already have. When an image is updated, a node downloads the changed layer and every layer after it, and reuses the rest.

Order the instructions in your Dockerfile from the least to the most frequently changed, and keep separate concerns in separate layers:

  1. The base image and operating system packages.
  2. Third-party libraries and dependencies.
  3. Configuration bootstrapping, such as default configuration files.
  4. Your application code or compiled binary.

With this order, a routine code change produces a small update that downloads quickly, while the large, rarely changed layers stay cached on the node. Copying the dependency manifest and installing dependencies before copying the rest of the source, as in the multi-stage example, is the common way to achieve this.

Designing for the edge

Expect the network to be unavailable

Assume that any service your container depends on, such as a cloud API, a head-office server, or a payment processor, will sometimes be unreachable. Design the application to keep working through an outage rather than failing when a request cannot be sent.

  • Retry with backoff. When a request fails, retry it after an increasing delay rather than immediately and repeatedly.
  • Store and forward. Queue outbound requests or data locally while the network is unavailable, and send them when it returns. A lightweight embedded queue is usually enough.
  • Keep the queue on a volume. Data kept only inside the container is lost when the container is replaced. Write the queue to a data volume so it survives a restart or an image update. On a cluster, use a DRBD-replicated volume so the queue also survives a failover to the other node. See Using data volumes.

Handle errors without exiting

When the main process in a container exits, the container stops, and the service is unavailable until the container is started again. A service that exits on every transient error, such as a dependency that is not ready yet, a port that is briefly unavailable, or a failed connection, is repeatedly unavailable, which disrupts everything that depends on it.

Catch recoverable errors, log them, and retry. Reserve exiting for errors the application genuinely cannot recover from.

Shut down cleanly

A container is stopped whenever it is updated, and on a cluster whenever it moves to the other node. Docker sends the main process a SIGTERM signal and waits a short time before forcibly stopping it. Handle SIGTERM by finishing or saving in-progress work, such as flushing a queue to disk, and then exiting.

Use the exec form of ENTRYPOINT or CMD, for example ENTRYPOINT ["java", "-jar", "app.jar"], so that your application runs as the main process and receives the signal. With the shell form, the signal goes to a shell instead.

Keep configuration out of the image

Build one image and use it at every site. Supply site-specific settings, such as addresses, identifiers, and feature flags, when the container starts, through the Environment Variables or Environment File Content fields of the container definition. See Environment File Content and Environment Variables.

Do not build credentials into an image. Anyone who can pull the image can read them.

Running alongside other workloads

Set resource limits

Your container shares the node's CPU and memory with platform services, other containers, and virtual machines. A container without limits can starve the rest of the node, particularly if it is built on software that expands to use the memory available to it. Java applications are the most common example.

Use the Resource Limitations settings of the container definition to cap the container's memory, swap, and CPU. See the Resource Limitations hash reference. Size the limits from the container's measured usage under load rather than from guesswork. rvc docker stats on the node reports current CPU and memory use for each container.

For Java applications, also tell the JVM to size its heap from the container's limit, for example with -XX:MaxRAMPercentage, as in the multi-stage example.

Control the start order

If a container needs another container to be running first, such as an application that uses a database, set the Depends On field of the container definition to the name of that container. Core services then start before the services that use them.

Depends On controls only the order in which containers start. It does not guarantee that the other service is ready to accept requests. Your application still needs to retry until its dependencies respond, as described in Handle errors without exiting.

Limit log volume

Output that a container writes to standard output and standard error is collected by the node's logging service, identified by the program name docker.<container_name>, and can be forwarded to a log server over the site's network. Verbose logging consumes disk on the node and bandwidth on links and tunnels that the site needs for other traffic.

Set production containers to log warnings and errors, or a concise informational level, rather than debug output. Make the log level configurable through an environment variable so that you can raise it temporarily when you troubleshoot. To keep a noisy container's logs off the network, see Log filters.

Run as a non-root user

Run the container's processes as a non-root user, so that a compromised application has less access to the node. Set the user in the Dockerfile with the USER instruction, or in the User ID and Group ID fields of the container definition. Make sure the user can read and write any data volumes the container uses.

Tagging and delivering images

  • Use a unique version tag for every release, such as 1.4.2, rather than reusing latest or another moving tag. A unique tag makes it clear which version each node runs, and lets you roll back by setting the previous tag. If you must reuse a tag, turn on Force Image Update in the container definition so that nodes pull the new image.
  • Sign your images so that nodes run only images you have approved. See Using container image signing.
  • Stage images ahead of a rollout on sites with slow links, so that the image is already on the node when you deploy. See Using container image staging.
  • Prune unused images on a schedule so that old versions do not fill the node's disk. See Using container image pruning.

© 2026 Scale Computing, Inc. All rights reserved.

Scale Computing, SC//AcuVigil, SC//Connect, and SC//Reliant are trademarks of Scale Computing, Inc.