Author	Nejat Hakan
eMail	nejat.hakan@outlook.de
PayPal Me	https://paypal.me/nejathakan

Containerization with Podman

Welcome to this in-depth exploration of Podman, a powerful and modern engine for developing, managing, and running OCI (Open Container Initiative) containers and pods. In the rapidly evolving landscape of software development and deployment, containers have become a cornerstone technology, offering portability, consistency, and efficiency. Podman distinguishes itself, particularly through its daemonless and rootless architecture, offering significant advantages in security and integration with modern Linux systems.

This material is designed for university students and anyone seeking a thorough understanding of containerization principles specifically applied through Podman. We will start with the fundamentals, gradually building up to more advanced concepts and practical implementations. Each theoretical section is followed by a hands-on workshop, allowing you to immediately apply and solidify your understanding through real-world scenarios. Prepare to dive deep into the world of Podman!

Introduction to Podman and Containerization Concepts

Before diving into Podman's specifics, let's establish a solid foundation by understanding what containers are and why they are so beneficial. We'll then introduce Podman, contrasting it with other containerization tools and highlighting its unique architectural choices.

What is Containerization?

At its core, containerization is a lightweight form of operating system (OS) virtualization. Unlike traditional Virtual Machines (VMs) which virtualize an entire hardware stack (CPU, RAM, storage, network card) and require a full guest OS installation for each instance, containers virtualize the OS itself.

Think of it this way:

• VMs: Like building separate houses, each with its own foundation, walls, plumbing, electricity, and occupants (Guest OS + Apps). They are well-isolated but resource-heavy.
• Containers: Like building apartments within a single large building. All apartments share the main building's foundation and core infrastructure (the host OS kernel), but each apartment has its own secure space, utilities (libraries, binaries), and occupants (the application). They are much lighter and faster to start.

Containers achieve this by leveraging features of the host Linux kernel, primarily:

• Namespaces: These provide isolation for system resources. Each container gets its own view of process IDs (PID namespace), network interfaces (Network namespace), mount points (Mount namespace), inter-process communication (IPC namespace), user IDs (User namespace), and hostname (UTS namespace). To the process inside the container, it looks like it's running on its own dedicated OS.
• Control Groups (cgroups): These limit and account for the resource usage (CPU, memory, disk I/O, network bandwidth) of a collection of processes. This prevents one container from monopolizing host resources and impacting others.
• Layered Filesystems (e.g., OverlayFS): Container images are built in layers. When a container is started, a new writable layer is added on top of the read-only image layers. This makes image distribution efficient (only changed layers need to be transferred) and starting containers fast (no need to copy the entire filesystem).

Benefits of Containerization:

• Consistency: Applications run the same way regardless of where the container is deployed (developer laptop, testing server, production cloud). This eliminates the "it works on my machine" problem.
• Efficiency: Containers require fewer resources than VMs (less RAM, CPU overhead, disk space). More containers can run on the same hardware.
• Speed: Containers can be started, stopped, and created much faster than VMs.
• Portability: OCI-compliant containers can run on any platform supporting the standard (Linux, Windows, macOS, Cloud Platforms).
• Scalability: Easy to scale applications up or down by simply starting or stopping container instances.
• Isolation: Processes within a container are isolated from the host and other containers, improving security and stability.

Introducing Podman

Podman (Pod Manager) is an open-source, OCI-compliant container engine. It provides commands very similar to the Docker command-line interface (CLI), making it familiar to users experienced with Docker. However, Podman has a fundamentally different architecture.

Key Characteristics of Podman:

• Daemonless: Unlike Docker, which relies on a long-running central daemon (dockerd) typically running as root, Podman operates using a fork/exec model. When you run a podman command (like podman run ...), Podman itself directly forks a child process (conmon) which then executes the container runtime (like runc or crun) to create and manage the container. conmon monitors the container's main process, handles logging, and manages TTYs. This direct interaction avoids the single point of failure and potential security risks associated with a privileged daemon.
• Rootless Support: This is perhaps Podman's most significant advantage. Podman was designed from the ground up to run containers entirely as a regular, unprivileged user. It achieves this primarily through user namespaces. When a user runs a rootless container, the UIDs and GIDs inside the container are mapped to a range of unprivileged UIDs/GIDs allocated to that user on the host system. This means that even if a process inside the container thinks it's running as root (UID 0), it's actually running as a non-privileged user ID on the host. This drastically reduces the potential attack surface, as a container breakout wouldn't grant root access to the host system.
• Pod Concept: Podman natively supports the concept of Pods, a term borrowed from Kubernetes. A Pod is a group of one or more containers that share the same network namespace, IPC namespace, and optionally PID namespace. Containers within a pod can communicate with each other via localhost and can coordinate more easily. This is extremely useful for deploying tightly coupled applications (e.g., a web application and its sidecar logging agent or reverse proxy).
• OCI Compliance: Podman adheres to the Open Container Initiative specifications for images and runtimes. This ensures compatibility with other OCI-compliant tools and images (including those built for Docker).
• Systemd Integration: Podman integrates well with systemd, the standard init system on many Linux distributions. It can generate systemd unit files to manage the lifecycle of containers and pods, allowing them to be started automatically on boot, managed like regular services, and have their logs integrated with the systemd journal.

Podman vs. Docker (High-Level):

• Architecture: Podman is daemonless; Docker uses a client-server model with a daemon.
• Security: Podman excels at rootless containers by design; Docker's rootless mode is newer and sometimes considered less mature. Running Docker typically requires root privileges or adding users to the docker group (which grants equivalent privileges).
• Pods: Podman has built-in support for pods; Docker requires Docker Compose for similar multi-container orchestration locally.
• Systemd Integration: Podman offers tighter integration with systemd for service management.
• Build Tool: Both use buildah under the hood for builds, but the user experience via podman build and docker build is very similar. Docker also has BuildKit.
• Command Line: Podman's CLI is largely compatible with Docker's (alias docker=podman often works).

Podman offers a compelling alternative, especially in environments prioritizing security, systemd integration, and running containers without root privileges.

Workshop: First Steps with Podman

Goal: Install Podman (if needed), verify the installation, and run your first simple container to experience the basic workflow.

Prerequisites:

• A Linux system (e.g., Fedora, Ubuntu, Debian, CentOS/RHEL). The installation commands will vary slightly.
• User account with sudo privileges (only needed for installation, not for running rootless containers later).

Steps:

1. Install Podman (Example Commands):

   • On Fedora/CentOS Stream/RHEL:

     sudo dnf install podman
     

   • On Debian/Ubuntu:

     sudo apt-get update
     sudo apt-get install podman
     

   • (Optional) Install slirp4netns and fuse-overlayfs for optimal rootless support if not installed by default:

     # Fedora/RHEL: Usually included or dependency of podman
     # Debian/Ubuntu:
     sudo apt-get install slirp4netns fuse-overlayfs
     

   • Verify Installation: After installation, open your terminal and run:

     podman --version
     

     You should see the installed Podman version number.

2. Explore Podman Information:

   • Get detailed information about your Podman installation, including configured storage, networking, and the OCI runtime being used:

     podman info
     

   • Pay attention to sections like graphRoot (where images and container data are stored), networkBackend (e.g., netavark or cni), and ociRuntime (e.g., crun or runc). If running rootless, you'll see paths within your home directory.

3. Pull a Test Image:

   • Container images are typically downloaded (pulled) from container registries (like Docker Hub or Quay.io). Let's pull a very small image called alpine. Alpine Linux is popular for containers due to its minimal size.

     podman pull alpine
     

   • You will see output showing layers being downloaded and extracted. Podman automatically checks registries like docker.io and quay.io by default (configurable).

4. List Downloaded Images:

   • See the images available locally on your system:

     podman images
     

   • You should see the alpine image listed, along with its tag (likely latest), image ID, creation time, and size.

5. Run Your First Container:

   • Let's run a simple command inside an Alpine container. We'll ask it to print "Hello from Alpine!". The --rm flag automatically removes the container filesystem once it exits.

     podman run --rm alpine echo "Hello from Alpine!"
     

   • Explanation:
     • podman run: The command to create and start a new container.
     • --rm: Cleans up the container after it exits.
     • alpine: The image to base the container on.
     • echo "Hello from Alpine!": The command to execute inside the container.
   • You should see the output: Hello from Alpine!

6. Run an Interactive Container:

   • Sometimes you want to interact with a container's shell. The -it flags enable this. -i keeps STDIN open (interactive), and -t allocates a pseudo-TTY (terminal).

     podman run -it --rm alpine sh
     

   • Explanation:
     • -it: Combined flags for interactive terminal session.
     • sh: The command to run inside the container (the Alpine shell).
   • Your terminal prompt should change, indicating you are now inside the container (e.g., / #).
   • Try running some commands inside the container:

     uname -a  # See kernel info (notice it's the *host's* kernel)
     cat /etc/os-release # See the OS info *inside* the container (Alpine)
     ls /
     exit      # Type exit to leave the container shell
     

   • Since you used --rm, the container is removed upon exiting.

Conclusion: You have successfully installed Podman, pulled an image, and run both simple command containers and interactive containers. You've seen how Podman executes commands within an isolated environment based on a container image. This forms the foundation for all further container operations.

1. Installing and Configuring Podman

Having run your first container, let's take a closer look at the installation process on different Linux distributions and explore the key configuration files that govern Podman's behavior. Understanding configuration is crucial for tailoring Podman to specific needs, especially regarding storage, networking, and registry access.

Installation on Common Distributions

As seen in the first workshop, installation typically involves using the distribution's standard package manager. Here's a slightly more detailed overview:

• Fedora, CentOS Stream, RHEL (and derivatives like AlmaLinux, Rocky Linux): Podman is developed primarily within the Fedora/RHEL ecosystem and is usually available in the default repositories and well-supported.

  sudo dnf update -y
  sudo dnf install podman
  # Optional: podman-compose for docker-compose compatibility
  sudo dnf install podman-compose
  # Optional: Container tools module for more related utilities
  sudo dnf module install container-tools
  

• Debian and Ubuntu (and derivatives): Podman is available in the default repositories of recent Debian/Ubuntu versions. Ensure your system is up-to-date for newer Podman releases.

  sudo apt-get update
  sudo apt-get install podman -y
  # Optional: podman-compose
  # Note: podman-compose might be packaged differently or require pip install
  # sudo apt-get install podman-compose # If available
  # Alternatively (using Python's package manager):
  # sudo apt-get install python3-pip -y
  # pip3 install podman-compose
  

  For rootless mode on Debian/Ubuntu, ensure necessary supporting packages are installed:

  sudo apt-get install -y uidmap slirp4netns fuse-overlayfs
  

  The uidmap package provides newuidmap and newgidmap utilities, essential for setting up user namespaces in rootless mode. slirp4netns provides user-mode networking for rootless containers, and fuse-overlayfs allows unprivileged users to use overlay filesystems.

• Arch Linux: Podman is available in the official repositories.

  sudo pacman -Syu podman
  # Optional: podman-compose
  sudo pacman -Syu podman-compose
  # Ensure dependencies for rootless mode are present
  sudo pacman -Syu slirp4netns fuse-overlayfs
  

• openSUSE: Podman is available via zypper.

  sudo zypper refresh
  sudo zypper install podman
  # Optional: podman-compose
  sudo zypper install python3-podman-compose
  

Verifying Rootless Setup: After installation, especially if you intend to primarily use rootless containers, it's good practice to verify the user namespace setup. Files /etc/subuid and /etc/subgid define the range of subordinate UIDs and GIDs available to users for mapping within user namespaces. They should contain entries like:

your_username:100000:65536

This means your_username can map UIDs/GIDs inside containers to host UIDs/GIDs ranging from 100000 to 165535 (100000 + 65536 - 1). These files are often configured automatically upon user creation or by system tools, but manual configuration might be needed in some scenarios. Podman typically handles this transparently if the shadow-utils (or equivalent) package is correctly installed and configured.

Key Configuration Files

Podman's behavior can be customized through several configuration files. These files use the TOML format. Podman searches for these files in a specific order of precedence:

1. --config=FILE: A file specified via command line flag (highest precedence).
2. ${XDG_CONFIG_HOME}/containers/containers.conf (usually ~/.config/containers/containers.conf) for rootless users.
3. /etc/containers/containers.conf for system-wide configuration (used by root and as fallback for rootless).
4. Default values compiled into Podman (lowest precedence).

Similar lookup paths exist for other specific configuration files like storage.conf and registries.conf, often residing in the same directories.

• containers.conf: This is the main configuration file. It consolidates settings that might previously have been split across storage.conf and registries.conf, although those can still be used for overrides or modularity.

  • Location: /etc/containers/containers.conf (system), ~/.config/containers/containers.conf (user).
  • Key Sections:
    • [engine]: Global options like the default OCI runtime (runtime = "crun"), environment variables to pass into containers, configuration for podman machine (if used), cgroup manager (cgroup_manager = "systemd" or "cgroupfs"), event logging (events_logger = "file" or "journald"), network settings (network_backend = "netavark").
    • [containers]: Default settings applied to containers, such as default time zone (tz = "local"), default sysctl settings, security options (SELinux, AppArmor, seccomp profiles), environment variables.
    • [network]: Network settings like the default CNI/Netavark configuration directory (network_config_dir), default subnet for the bridge network, default bridge name (default_network = "podman").
    • [machine]: Configuration specific to podman machine for managing VMs (e.g., on macOS/Windows).
    • [secrets], [configmaps]: Define default drivers and options for managing secrets and configmaps.

• storage.conf: Controls container storage, including image storage and container root filesystems.

  • Location: /etc/containers/storage.conf (system), ~/.config/containers/storage.conf (user).
  • Key Sections:
    • [storage]: Defines the primary storage driver (driver = "overlay"), the location for container storage (graphroot = "/var/lib/containers/storage" for root, ~/.local/share/containers/storage for rootless), and options specific to the chosen driver.
    • [storage.options]: Driver-specific options. For overlay, this might include mount_program, mountopt.
    • [storage.options.thinpool]: Options if using the (less common now) devicemapper driver with thin provisioning.

• registries.conf (v2 format): Configures access to container image registries.

  • Location: /etc/containers/registries.conf (system), ~/.config/containers/registries.conf (user).
  • Key Features:
    • Search Order: Defines the order in which registries are searched when an image name without a fully qualified domain is used (e.g., podman pull alpine might search docker.io, quay.io, etc.).
    • Blocking/Allowing Registries: Can specify lists of registries that are blocked or allowed.
    • Insecure Registries: Lists registries that can be accessed over HTTP or with invalid TLS certificates (use with extreme caution!).
    • Remapping/Mirrors: Can specify mirror registries to use instead of, or in addition to, the primary registry address. This is useful for local caches or geographically closer mirrors.
    • Credentials: While actual login credentials should be stored securely using podman login (which typically writes to ${XDG_RUNTIME_DIR}/containers/auth.json or ~/.config/containers/auth.json), this file can sometimes point to credential helper locations.

• policy.json: Defines trust policies for container images, specifying requirements for image signatures.

  • Location: /etc/containers/policy.json.
  • Purpose: Allows administrators to enforce rules like "only run images signed by specific keys" or "reject all unsigned images." This is crucial for security in production environments.

Understanding these files allows for fine-grained control over how Podman operates, optimizes storage and networking, and interacts with image registries securely.

Workshop: Exploring Podman Configuration

Goal: Locate and examine Podman's configuration files to understand the default settings on your system. Optionally, make a simple configuration change.

Prerequisites:

• Podman installed from the previous workshop.
• Basic familiarity with terminal commands (ls, cat, mkdir, cp).
• A text editor (like nano, vim, or gedit).

Steps:

1. Identify Configuration File Locations:

   • Use podman info to find the active configuration paths. Run:

     podman info | grep -i -e 'config file' -e graphroot -e OciRuntime -e conmon -e networkBackend
     

   • Note the paths listed for configFile (likely /etc/containers/containers.conf or none if using defaults/user overrides), graphRoot (storage location), ociRuntime, conmon, and networkBackend.

2. Examine System-Wide Configuration (if it exists):

   • Check if the system-wide configuration file exists:

     ls -l /etc/containers/containers.conf
     

   • If it exists, view its contents (use sudo if necessary, but viewing is usually possible without):

     cat /etc/containers/containers.conf
     # Or use a pager like less:
     less /etc/containers/containers.conf
     

   • Look for settings related to cgroup_manager, runtime, network_backend, etc. Notice the TOML syntax ([section], key = "value").

3. Examine Storage Configuration:

   • Check for and view the system-wide storage configuration:

     ls -l /etc/containers/storage.conf
     cat /etc/containers/storage.conf
     

   • Identify the storage driver (likely overlay) and the graphroot path.

4. Examine Registry Configuration:

   • Check for and view the system-wide registry configuration:

     ls -l /etc/containers/registries.conf
     cat /etc/containers/registries.conf
     

   • Look for the unqualified-search-registries list. This shows where Podman looks if you don't specify a full registry name (e.g., docker.io, quay.io).

5. Check for User-Specific Configuration:

   • Look in your home directory for user overrides. Remember, these take precedence for your user when running rootless Podman.

     ls -l ~/.config/containers/containers.conf
     ls -l ~/.config/containers/storage.conf
     ls -l ~/.config/containers/registries.conf
     

   • If these files exist, view their contents. If not, Podman is using the system-wide settings or built-in defaults.

6. (Optional) Create a User Configuration Override:

   • Let's add a default environment variable that will be set in all containers you run as your user.
   • Create the user configuration directory if it doesn't exist:

     mkdir -p ~/.config/containers
     

   • Create or edit the user's containers.conf:

     nano ~/.config/containers/containers.conf
     

   • Add the following content:

     [containers]
     # Add environment variables passed to all containers by default
     env = [
         "MY_DEFAULT_VAR=HelloFromConfig",
         "USER_SPECIFIC_INFO=SetByUserConfig"
     ]
     
     # You could also add other settings here, like default timezone:
     # tz = "Etc/UTC"
     

   • Save and close the file (in nano, press Ctrl+X, then Y, then Enter).

7. Verify the Configuration Change:

   • Run a simple container and check its environment variables:

     podman run --rm alpine env | grep MY_DEFAULT_VAR
     podman run --rm alpine env | grep USER_SPECIFIC_INFO
     

   • You should see the variables MY_DEFAULT_VAR=HelloFromConfig and USER_SPECIFIC_INFO=SetByUserConfig printed, confirming that your user-specific configuration is being applied.

Conclusion: You have now located and inspected the key configuration files for Podman, both system-wide and user-specific. You understand where settings for storage, networking, and registries are defined. By creating a user override file, you've seen how to customize Podman's behavior for your specific user account, demonstrating the configuration hierarchy in action. This knowledge is essential for advanced Podman usage and troubleshooting.

2. Working with Container Images

Container images are the blueprints from which containers are created. They are read-only templates containing the application code, libraries, dependencies, and metadata needed to run an application. Understanding how to find, manage, and build images is fundamental to using Podman effectively.

Understanding Container Images and Layers

As mentioned earlier, container images are not monolithic blobs. They are composed of multiple read-only layers stacked on top of each other. Each layer represents a set of filesystem changes compared to the layer below it. This layered approach has several advantages:

• Efficiency: When pulling an image, only layers not already present locally need to be downloaded. When building images, build steps that haven't changed can reuse cached layers, speeding up the build process.
• Sharing: Multiple images can share common base layers (e.g., an OS layer), saving disk space.
• Versioning: Each layer often corresponds to an instruction in the image build definition (like a RUN or COPY command), making it easier to track changes.

When a container is started from an image, Podman (using the storage driver like OverlayFS) adds a thin writable layer on top of the read-only image layers. Any changes made inside the running container (like writing files, modifying configurations) occur in this writable layer. The underlying image layers remain untouched. When the container is deleted, this writable layer is typically discarded (unless persisted using volumes).

Key Concepts:

• Registry: A server application that stores and distributes container images (e.g., Docker Hub, Quay.io, GitHub Container Registry, private registries).
• Repository: A collection of related images within a registry, usually for different versions or variants of an application (e.g., nginx, python).
• Tag: An alphanumeric label applied to a specific image within a repository, often indicating a version or variant (e.g., nginx:1.21, python:3.9-slim). The tag latest is a convention, usually pointing to the most recent stable version, but its meaning is defined by the image maintainer.
• Digest (Content Addressable Storage): Each image layer, and the image manifest itself, has a unique cryptographic hash (usually SHA256) based on its content. This ensures immutability and allows for precise identification (<repository>@sha256:<hash>). Using digests is more reliable than tags for ensuring you are using the exact same image, as tags can be moved to point to different images over time.
• Manifest: A JSON file describing the image, including its layers, architecture, OS, and other metadata. There can be manifest lists (also called fat manifests) that point to specific image manifests for different architectures (e.g., amd64, arm64).

Finding and Pulling Images

You typically find images using web interfaces of registries (like hub.docker.com) or using the podman search command.

• podman search <term>: Searches configured registries for repositories matching the term.

  podman search nginx
  # By default searches registries listed in registries.conf
  # Example Output:
  # INDEX      NAME                                     DESCRIPTION                                      STARS   OFFICIAL   AUTOMATED
  # docker.io  docker.io/library/nginx                  Official build of Nginx.                         17900   [OK]
  # docker.io  docker.io/bitnami/nginx                  Bitnami Nginx Docker Image                       300
  # quay.io    quay.io/bitnami/nginx                    Bitnami Nginx Container Image                    300
  # ...
  

• podman pull <image_name>[:<tag>|@sha256:<digest>]: Downloads an image from a registry to your local storage.

  • If no registry is specified (e.g., alpine), Podman consults the unqualified-search-registries in registries.conf.
  • If no tag is specified, latest is assumed.
  • You can specify the full path: podman pull docker.io/library/python:3.10-slim
  • You can pull by digest for immutability: podman pull ubuntu@sha256:abc123... (replace with actual digest)

    podman pull redis:7 # Pull version 7 of the official Redis image from Docker Hub
    podman pull quay.io/prometheus/node-exporter:v1.5.0 # Pull specific version from Quay.io
    

Managing Local Images

Once images are pulled or built locally, you can manage them using several commands:

• podman images or podman image ls: Lists images stored locally.

  • Flags: -a (list intermediate layers too), --digests (show digests).

    podman images
    # REPOSITORY                  TAG      IMAGE ID       CREATED        SIZE
    # docker.io/library/redis     7        a8e50a...      3 weeks ago    117 MB
    # docker.io/library/alpine    latest   c1aabb...      4 weeks ago    7.34 MB
    # quay.io/prometheus/node-e.. v1.5.0   b9dd0f...      2 months ago   27.9 MB
    

• podman image inspect <image_name_or_id>: Displays detailed information about an image in JSON format, including its layers, environment variables, entrypoint, command, labels, etc.

  podman image inspect alpine
  # Get specific info using --format (Go template)
  podman image inspect --format '{{.Config.Cmd}}' alpine # Shows the default command
  podman image inspect --format '{{.GraphDriver.Data}}' alpine # Shows storage info
  

• podman tag <source_image>[:<tag>] <target_image>[:<tag>]: Creates an additional tag (an alias) for an existing image. This doesn't duplicate the image data.

  # Tag the downloaded alpine image as 'my-alpine:v1'
  podman tag alpine my-alpine:v1
  podman images # Now shows 'my-alpine' with tag 'v1' pointing to the same IMAGE ID
  

• podman rmi <image_name_or_id> ...: Removes one or more local images.

  • You cannot remove an image that is currently used by a container (stop/remove the container first).
  • You can remove images by name:tag or by image ID.
  • Flag: -f (force removal, even if tagged multiple times or used by stopped containers - use cautiously).

    podman rmi my-alpine:v1 # Remove the tag 'my-alpine:v1'
    podman rmi alpine       # Remove the 'alpine:latest' tag (if only tag, removes image data)
    podman rmi a8e50a       # Remove redis by image ID prefix
    

• podman image prune: Removes unused images (images not associated with any container and without any tags pointing to them, often intermediate build layers or older versions).

  • Flags: -a (remove all unused images, not just dangling ones), -f (force, don't ask for confirmation).

    podman image prune -a -f # Clean up unused images
    

• podman push <image_name>[:<tag>] [destination]: Uploads an image from local storage to a container registry. You need to be logged into the destination registry (podman login <registry_host>). The destination must typically include the registry hostname and your username/organization.

  # Example: Push 'my-alpine:v1' to Docker Hub under username 'myuser'
  podman tag my-alpine:v1 docker.io/myuser/my-custom-alpine:v1
  podman login docker.io # Prompts for username/password
  podman push docker.io/myuser/my-custom-alpine:v1
  

Building Images with Containerfile/Dockerfile

While you can create images by modifying a container and committing its state (podman commit), the standard and reproducible way is to define the image structure in a text file, traditionally called Dockerfile, although Podman often uses the name Containerfile by convention (they are syntactically compatible).

This file contains a series of instructions that podman build executes sequentially to create the image layers.

Common Instructions:

• FROM <base_image>: Specifies the starting image for the build. Every Containerfile must start with FROM.
• LABEL <key>=<value> ...: Adds metadata (labels) to the image (e.g., maintainer, version).
• WORKDIR /path/to/workdir: Sets the working directory for subsequent instructions (RUN, CMD, ENTRYPOINT, COPY, ADD).
• RUN <command>: Executes a command in a new layer. Used for installing packages, compiling code, etc. Each RUN creates a new layer.
• COPY <src> ... <dest>: Copies files or directories from the build context (the directory where podman build is run) into the image filesystem.
• ADD <src> ... <dest>: Similar to COPY, but with added features like extracting local tar archives and downloading files from URLs (use COPY unless you specifically need ADD features).
• ENV <key>=<value>: Sets environment variables within the image.
• ARG <name>[=<default_value>]: Defines build-time variables that can be passed using podman build --build-arg <name>=<value>.
• EXPOSE <port> ...: Informs Podman that the container listens on the specified network ports at runtime (documentation purposes; doesn't actually publish the port).
• USER <user>[:<group>]: Sets the user (and optionally group) to run subsequent commands and the final container process. Important for security (avoid running as root if possible).
• VOLUME ["/path/to/volume"]: Creates a mount point for external volumes. Data written here can be persisted.
• CMD ["executable","param1","param2"] or CMD command param1 param2: Specifies the default command to run when a container starts from this image. Can be overridden when running the container. There can only be one CMD.
• ENTRYPOINT ["executable", "param1", "param2"] or ENTRYPOINT command param1 param2: Configures the container to run as an executable. Arguments passed to podman run are appended to the ENTRYPOINT. If used with CMD, CMD provides default arguments to the ENTRYPOINT. There can only be one ENTRYPOINT.

Building an Image: The command podman build is used to create an image from a Containerfile.

podman build [options] <context_directory_or_url>

• Context Directory: The directory containing the Containerfile and any files needed by COPY or ADD.
• Options:
  • -t <name>[:<tag>]: Tag the resulting image.
  • -f <file>: Specify the path to the Containerfile (defaults to Containerfile or Dockerfile in the context directory).
  • --build-arg <key>=<value>: Pass build-time variables.
  • --no-cache: Do not use cached layers.
  • --squash: Squash all newly built layers into a single new layer (can reduce image size but loses layer history).

Workshop: Building and Managing a Simple Web Server Image

Goal: Create a custom container image based on Alpine Linux that runs a simple Python web server serving a static HTML file. You will write a Containerfile, build the image, run it, and manage it.

Prerequisites:

• Podman installed and working.
• A text editor.
• Basic understanding of shell commands.

Steps:

1. Create Project Directory and Files:

   • Create a directory for your project:

     mkdir podman-web-app
     cd podman-web-app
     

   • Create a simple HTML file named index.html:

     nano index.html
     

     • Add the following content:

       <!DOCTYPE html>
       <html>
       <head>
           <title>My Podman App</title>
       </head>
       <body>
           <h1>Hello from my Containerized App!</h1>
           <p>This page is served by a Python server inside a Podman container.</p>
       </body>
       </html>
       

     • Save and close the file.
   • Create a file named Containerfile (or Dockerfile):

     nano Containerfile
     

     • Add the following content:

       # Use Alpine Linux as the base image (small and efficient)
       FROM alpine:latest
       
       # Add metadata labels
       LABEL maintainer="Your Name <your.email@example.com>"
       LABEL version="1.0"
       LABEL description="Simple Python static web server"
       
       # Set the working directory inside the image
       WORKDIR /app
       
       # Copy the index.html file from the build context to the image's /app directory
       COPY index.html .
       
       # Install Python (needed for the simple HTTP server)
       # Use --no-cache to avoid caching package indexes, reducing image size
       RUN apk update && apk add --no-cache python3
       
       # Expose port 8000 (documentation - does not publish the port)
       EXPOSE 8000
       
       # Set the default command to run when the container starts
       # Runs Python's built-in HTTP server in the /app directory on port 8000
       CMD ["python3", "-m", "http.server", "8000"]
       

     • Save and close the file. Your directory should now contain index.html and Containerfile.

2. Build the Container Image:

   • Run the podman build command from within the podman-web-app directory. The . indicates the current directory is the build context. Tag the image as my-web-app:v1.

     podman build -t my-web-app:v1 .
     

   • Observe the output. You'll see Podman executing each step from your Containerfile: pulling the base image (if not present), running apk add, copying the file, etc. It will likely use cached layers if you re-run the build without changes.

3. Verify the Image:

   • List your local images to see the newly created image:

     podman images
     

     • You should see localhost/my-web-app with the tag v1. (Podman often prefixes images built locally with localhost/ if no registry is specified in the tag).

4. Run a Container from Your Image:

   • Run a container based on your new image. Map port 8080 on your host to port 8000 inside the container. Run it in detached mode (-d) and give it a name (--name).

     podman run -d --name webapp_instance -p 8080:8000 my-web-app:v1
     

   • Verify the container is running:

     podman ps
     # You should see 'webapp_instance' running
     

5. Access Your Web Application:

   • Open a web browser and navigate to http://localhost:8080 (or http://<your-host-ip>:8080 if running on a remote machine/VM).
   • You should see the "Hello from my Containerized App!" page being served.

6. Inspect and Manage:

   • Inspect the running container:

     podman inspect webapp_instance
     

   • View the container's logs (it will show HTTP requests if you refreshed the page):

     podman logs webapp_instance
     

   • Stop the container:

     podman stop webapp_instance
     

   • Remove the container (it was automatically removed if you used --rm, but we didn't here):

     podman rm webapp_instance
     

7. Tag and Prepare for Push (Simulated):

   • Let's tag the image as if preparing to push it to a registry (e.g., Docker Hub, replace yourusername with your actual username if you have one).

     podman tag my-web-app:v1 docker.io/yourusername/my-web-app:v1
     podman images
     # You'll see the new tag pointing to the same image ID
     

   • (Optional) If you have a registry account, you could podman login and podman push docker.io/yourusername/my-web-app:v1.

8. Clean Up:

   • Remove the images you created:

     podman rmi my-web-app:v1
     podman rmi docker.io/yourusername/my-web-app:v1 # Remove the second tag
     # Alternatively, remove by Image ID if both tags are gone
     podman images # Verify they are removed
     

Conclusion: In this workshop, you successfully defined a container image using a Containerfile, including installing dependencies and copying application files. You built the image using podman build, ran a container from it, accessed the application, and managed the container and image lifecycles. This practical experience is key to developing your own containerized applications with Podman.

3. Running and Managing Containers

With images pulled or built, the next step is running containers and managing their lifecycle. This involves understanding the podman run command in more detail, as well as commands for interacting with running or stopped containers.

The podman run Command

The podman run command is the primary way to create and start a container from an image. It has numerous options to control the container's behavior and resources.

podman run [OPTIONS] IMAGE [COMMAND] [ARG...]

Commonly Used Options:

• Detached vs. Foreground:

  • -d, --detach: Run the container in the background (detached mode) and print the new container ID. This is common for servers and services.
  • -it: Run the container interactively, attaching your terminal's standard input, output, and error streams. -i (interactive) keeps STDIN open, -t (tty) allocates a pseudo-terminal. Essential for running shells or interactive applications.

• Container Naming and Cleanup:

  • --name <name>: Assign a specific name to the container. Useful for easy reference instead of using the container ID. Names must be unique.
  • --rm: Automatically remove the container's filesystem when the container exits. Useful for short-lived or temporary containers to avoid clutter. Cannot be used with -d.

• Port Mapping:

  • -p <host_port>:<container_port>[/<protocol>]: Publish (map) a container's port to a port on the Podman host.
    • podman run -p 8080:80 nginx (Maps host port 8080 to container port 80 TCP)
    • podman run -p 127.0.0.1:8080:80 nginx (Maps only on host's loopback interface)
    • podman run -p 8080:80/udp myapp (Maps UDP port)
    • podman run -p 80 (Maps container port 80 to a random available high port on the host)
  • -P, --publish-all: Publish all ports exposed using the EXPOSE instruction in the Containerfile to random host ports.

• Volume Mounting (More details in a later section):

  • -v <host_path_or_volume_name>:<container_path>[:<options>]: Mount volumes or bind mount host directories into the container.
    • Bind mount: podman run -v /path/on/host:/path/in/container ...
    • Named volume: podman run -v my-volume:/data ...
    • Options: ro (read-only), z (shared SELinux label), Z (private SELinux label).

• Environment Variables:

  • -e <key>=<value> or --env <key>=<value>: Set environment variables inside the container.
  • --env-file <file>: Read environment variables from a file (each line in KEY=VALUE format).

• Resource Constraints:

  • --memory <limit> or -m <limit>: Limit memory usage (e.g., 512m, 1g).
  • --cpus <number>: Limit CPU usage (e.g., 1.5 means 1.5 CPU cores).
  • --cpuset-cpus <list>: Bind container to specific CPU cores (e.g., 0, 0-1).

• Security Context:

  • --user <user>[:<group>]: Run the process inside the container as the specified user/group (can be name or UID/GID).
  • --cap-add <capability>, --cap-drop <capability>: Add or drop Linux capabilities (e.g., NET_ADMIN, SYS_ADMIN). Use with caution.
  • --security-opt <option>: Set security options like SELinux labels (label=...) or seccomp profiles (seccomp=...).

• Networking (More details later):

  • --network <network_name>: Connect the container to a specific Podman network (other than the default).
  • --ip <address>: Assign a static IP address within the Podman network (use carefully).
  • --dns <server>: Set custom DNS servers for the container.

• Restart Policies:

  • --restart <policy>: Specify what Podman should do if the container exits.
    • no: (Default) Do not restart.
    • on-failure[:<max_retries>]: Restart only if the container exits with a non-zero status, optionally limiting retries.
    • always: Always restart the container if it stops, regardless of exit status.
    • unless-stopped: Always restart unless the container was explicitly stopped by the user.

• Overriding Image Defaults:

  • [COMMAND] [ARG...]: By providing a command and arguments after the image name, you override the default CMD or provide arguments to the ENTRYPOINT defined in the image.
    • podman run alpine ls -l /etc (Overrides default shell, runs ls -l /etc instead).
  • --entrypoint <command>: Override the ENTRYPOINT defined in the image.

Managing Container Lifecycle

Once containers are created (whether running or stopped), you need commands to manage them:

• podman ps or podman container ls: Lists running containers.

  • Flags:
    • -a, --all: Show all containers (running and stopped).
    • -q, --quiet: Show only container IDs.
    • -s, --size: Show container disk usage (writable layer size).
    • --format <go_template>: Customize output format.
    • --filter <key>=<value>: Filter containers (e.g., --filter status=exited, --filter name=webapp_instance).

• podman stop <container_name_or_id> ...: Stops one or more running containers gracefully (sends SIGTERM, then SIGKILL after a timeout).

  • Flag: -t, --time <seconds>: Set the timeout before sending SIGKILL (default is 10 seconds).

• podman start <container_name_or_id> ...: Starts one or more stopped containers.

• podman restart <container_name_or_id> ...: Restarts one or more running containers (effectively a stop followed by a start).

• podman rm <container_name_or_id> ...: Removes one or more stopped containers. The container's writable filesystem layer is deleted.

  • Flag: -f, --force: Force removal of a running container (sends SIGKILL first). Use with caution, data loss can occur.
  • Flag: -v, --volumes: Remove any anonymous volumes associated with the container. Named volumes are generally not removed by this.

• podman container prune: Removes all stopped containers.

  • Flag: -f, --force: Don't prompt for confirmation.

• podman logs <container_name_or_id>: Fetches the logs (stdout/stderr) of a container.

  • Flags:
    • -f, --follow: Follow log output in real-time.
    • --tail <number>: Show the last N lines.
    • --since <timestamp>, --until <timestamp>: Show logs within a time range.

• podman inspect <container_name_or_id>: Displays detailed low-level information about a container in JSON format (state, configuration, network settings, mounts, etc.).

  • Flag: --format <go_template>: Extract specific pieces of information.

    podman inspect --format '{{.State.Status}}' my_container # Get status
    podman inspect --format '{{.NetworkSettings.IPAddress}}' my_container # Get IP (on default network)
    

• podman exec [OPTIONS] <container_name_or_id> <command> [ARG...]: Executes a new command inside an already running container.

  • Flags:
    • -it: Run interactively with a TTY (e.g., podman exec -it my_container sh).
    • -e <key>=<value>: Set environment variables for the command.
    • -u <user>: Run the command as a specific user inside the container.

      # Get environment variables inside a running nginx container
      podman exec my_nginx_container env
      # Install 'curl' inside a running debian container (assuming apt is available)
      podman exec my_debian_container apt-get update && apt-get install -y curl
      # Open an interactive shell in a running container
      podman exec -it my_running_app /bin/bash
      

• podman cp <src_path> <dest_path> or <container>:<src_path> <host_dest_path> or <host_src_path> <container>:<dest_path>: Copies files/directories between the host and a container.

  # Copy file from host to container
  podman cp ./my-config.conf my_container:/etc/app/config.conf
  # Copy file from container to host
  podman cp my_container:/var/log/app.log ./app.log
  

• podman top <container_name_or_id> [ps_options]: Displays the running processes inside a container, similar to the top or ps command on the host.

Workshop: Deploying and Managing a Multi-Tier Application (Simplified)

Goal: Run two related containers—a simple database (PostgreSQL) and a web application (using the image built previously or a generic one) that could theoretically connect to it (we won't implement the connection logic, just manage the containers). This demonstrates managing multiple dependent services.

Prerequisites:

• Podman installed and working.
• Internet connection to pull images.

Steps:

1. Run the PostgreSQL Database Container:

   • We'll run the official PostgreSQL image. It requires setting a password via an environment variable. We'll run it detached and give it a name.

     podman run -d \
       --name my-postgres-db \
       -e POSTGRES_PASSWORD=mysecretpassword \
       -e POSTGRES_USER=admin \
       -e POSTGRES_DB=mydatabase \
       postgres:14-alpine
     

   • Explanation:
     • -d: Run detached.
     • --name my-postgres-db: Name the container.
     • -e POSTGRES_PASSWORD=...: Sets the required password for the default 'postgres' superuser and the user specified below.
     • -e POSTGRES_USER=admin: Creates a user named 'admin'.
     • -e POSTGRES_DB=mydatabase: Creates a database named 'mydatabase' owned by the specified user.
     • postgres:14-alpine: The image to use (version 14 on Alpine).

2. Verify the Database Container:

   • Check if the container is running:

     podman ps
     

   • Inspect the container to see its IP address (on the default Podman network) and environment variables:

     podman inspect my-postgres-db --format '{{.NetworkSettings.IPAddress}}'
     podman inspect my-postgres-db --format '{{range .Config.Env}}{{println .}}{{end}}'
     

   • View its logs to see initialization messages:

     podman logs my-postgres-db
     # Look for lines indicating the database system is ready to accept connections
     

3. Run a Web Application Container (Simulated Connection):

   • Let's run a simple container that pretends it needs to connect to the database. We'll use the alpine image and just keep it running with a sleep command. We'll set environment variables that a real application might use to find the database.

     podman run -d \
       --name my-simulated-app \
       -e DB_HOST=my-postgres-db \
       -e DB_PORT=5432 \
       -e DB_USER=admin \
       -e DB_NAME=mydatabase \
       -p 8080:80 \
       alpine sleep infinity
     

   • Explanation:
     • -d, --name: As before.
     • -e DB_HOST=...: We use the name of the database container. Podman's built-in DNS service (if using default bridge network) usually allows containers on the same network to resolve each other by name.
     • -p 8080:80: We map port 8080 just as an example (Alpine image doesn't serve anything on port 80 by default).
     • alpine sleep infinity: Use the alpine image and run the command sleep infinity to keep the container running indefinitely without consuming many resources.

4. Manage the Containers:

   • List all running containers:

     podman ps
     # You should see 'my-postgres-db' and 'my-simulated-app'
     

   • Stop the application container:

     podman stop my-simulated-app
     

   • List all containers (including stopped):

     podman ps -a
     # 'my-simulated-app' should show Status 'Exited'
     

   • Start the application container again:

     podman start my-simulated-app
     podman ps # Verify it's running again
     

5. Execute a Command in the Database Container:

   • Let's use podman exec to run the PostgreSQL command-line client psql inside the running database container to verify the user and database were created.

     podman exec -it my-postgres-db psql -U admin -d mydatabase
     

   • Explanation:
     • -it: Interactive terminal.
     • my-postgres-db: The container to execute in.
     • psql -U admin -d mydatabase: The command to run inside (psql utility, connect as user admin to database mydatabase).
   • You should get a psql prompt (e.g., mydatabase=#). You can type \l to list databases or \du to list users. Type \q to exit psql.

6. Clean Up:

   • Stop both containers:

     podman stop my-simulated-app my-postgres-db
     

   • Remove both containers:

     podman rm my-simulated-app my-postgres-db
     

   • Optionally, prune all stopped containers if you have others:

     # podman container prune -f
     

   • Optionally, remove the pulled images if you don't need them:

     # podman rmi postgres:14-alpine alpine
     

Conclusion: This workshop demonstrated how to run multiple containers that might represent different tiers of an application. You practiced using podman run with environment variables, checking container status with podman ps, viewing logs with podman logs, stopping/starting containers, and executing commands inside a running container with podman exec. These are essential skills for managing containerized applications. You also saw a hint of container networking by using the container name (my-postgres-db) as the hostname for the simulated connection.

4. Understanding Podman Networking

By default, containers are isolated from the host network and from each other, except when explicitly configured otherwise. Podman provides robust networking capabilities, allowing containers to communicate with the host, the outside world, and each other in controlled ways. This section explores Podman's networking models and commands.

Podman Network Backends

Podman supports different network backends to implement container networking:

1. Netavark:
   This is the default network backend for new Podman installations (typically Podman v4.0 and later). It's a Rust-based tool developed specifically for Podman and related projects (like CRI-O). It aims to be simpler and more efficient than CNI for Podman's use cases, especially for rootless networking and pods. It works in conjunction with Aardvark-dns, which provides DNS resolution between containers on the same network.

2. CNI (Container Network Interface):
   This was the default backend in earlier Podman versions (v3.x and earlier) and is still widely used by Kubernetes and other container platforms. It uses a plugin-based architecture. Podman typically ships with a set of standard CNI plugins (like bridge, host-local, firewall). When using CNI, Podman invokes these plugins to set up the network for a container.

You can check which backend is active using podman info | grep networkBackend. While the underlying implementation differs, the user-facing podman network commands work similarly with both backends. For most common use cases, the default backend (Netavark or CNI) works transparently.

Default Network (podman)

When Podman is installed, it usually creates a default bridge network, typically named podman.

• Rootful: When running as root, this network often corresponds to a Linux bridge device on the host (e.g., podman0 or cni-podman0) with a specific subnet (e.g., 10.88.0.0/16). Podman manages firewall rules (using iptables or nftables) to allow containers on this bridge to access the outside world via Network Address Translation (NAT) and to handle port forwarding (-p option).
• Rootless: When running as a regular user, the implementation is different.
  • With slirp4netns: This is a common default for rootless. slirp4netns creates a user-mode TCP/IP stack. Containers connect to this stack, which then forwards traffic to the host's network. It doesn't create a bridge device on the host. Performance can be lower than a bridge, and protocols other than TCP/UDP might not work as well. Ping between containers often doesn't work, but TCP/UDP connections usually do. Port forwarding (-p) works by slirp4netns listening on the host port and forwarding to the container.
  • With Pasta (Paste Network Stack): An alternative to slirp4netns aiming for better performance and compatibility, configured via pasta option in network_config_dir for Netavark.
  • With Rootless CNI Bridge (Less Common Default): It's possible, though more complex to set up, to configure a CNI bridge even for rootless mode, but it requires careful configuration of network interfaces and permissions.

Containers started with podman run without a --network flag are typically attached to this default network. Containers on the same default network can usually resolve each other by name (thanks to Aardvark-dns or the CNI dnsname plugin).

Network Modes (--network or --net option)

The --network option in podman run controls how a container connects to the network:

• --network bridge (or omitting the flag): Default mode. Connects the container to the default managed bridge network (podman). The container gets an IP address on the bridge's subnet. This provides isolation from the host network while allowing outbound connections and controlled inbound connections via port mapping.
• --network host: Disables network isolation. The container shares the host's network namespace directly. The container sees the host's network interfaces and IP addresses, and services running in the container bind directly to the host's interfaces. Use with caution, as it bypasses network separation. EXPOSE and -p are irrelevant in this mode.
• --network none: Maximum isolation. The container gets its own network namespace but with only a loopback interface (lo). It has no external network connectivity. Useful for batch jobs or tasks that don't require networking.
• --network container:<id|name>: Joins the network namespace of another existing container. They share the same IP address and network interfaces. Useful for debugging or sidecar patterns where one container needs to monitor the network traffic of another.
• --network <custom_network_name>: Connects the container to a user-defined network you created using podman network create.
• --network slirp4netns:[OPTIONS]: (Primarily Rootless) Explicitly requests the use of slirp4netns even if another default exists, optionally providing specific slirp4netns configuration options.

User-Defined Networks

While the default network is convenient, creating custom networks provides better isolation and organization, especially for multi-container applications. You can create networks where only specific groups of containers can communicate.

• podman network create <network_name>: Creates a new bridge network (by default).

  • Options:
    • --driver <driver> or -d <driver>: Specify network driver (usually bridge).
    • --subnet <subnet>: Specify the IP subnet for the network (e.g., 10.90.1.0/24).
    • --gateway <ip>: Specify the gateway IP address for the subnet.
    • --ip-range <range>: Specify a range within the subnet from which container IPs will be allocated.
    • --opt <key>=<value>: Driver-specific options (less common for basic bridge).
    • --internal: Creates an "internal" network. Containers on this network can communicate with each other, but have no external connectivity (no NAT).

• podman network ls: Lists available Podman networks.

• podman network inspect <network_name>: Shows detailed information about a network (subnet, gateway, connected containers).

• podman network rm <network_name> ...: Removes one or more user-defined networks. A network cannot be removed if containers are still connected to it.

• podman network connect <network_name> <container_name>: Connects an already running container to an additional network. A container can be attached to multiple networks simultaneously.

• podman network disconnect <network_name> <container_name>: Disconnects a running container from a network.

Benefits of Custom Networks:

• Isolation: Containers on different custom networks cannot communicate directly unless explicitly connected to both.
• Service Discovery: Podman's built-in DNS service automatically allows containers on the same user-defined network to resolve each other by their container names. This is crucial for microservices.
• IP Address Management: Allows you to define specific subnets, avoiding potential conflicts with other networks on your host or LAN.

Port Forwarding Explained

When you use -p <host_port>:<container_port>, Podman (via Netavark/CNI and firewall rules, or slirp4netns) sets up a mechanism to forward traffic arriving at <host_port> on the host machine to <container_port> inside the container.

• Rootful: Typically involves iptables or nftables DNAT (Destination Network Address Translation) rules on the host.
• Rootless (slirp4netns): The slirp4netns process listens on the specified <host_port> (limited to ports > 1024 by default for unprivileged users) and forwards the traffic internally to the container.
• Rootless (Pasta): Pasta handles the port forwarding directly.
• Rootless (Bridge): Requires manual setup or specific CNI plugins capable of rootless port forwarding, often more complex.

Workshop: Creating and Using Custom Networks

Goal: Set up two custom networks. Run containers simulating a frontend, backend, and a database. Configure connectivity so the frontend can only talk to the backend, and the backend can only talk to the database.

Prerequisites:

• Podman installed and working.

Steps:

1. Create Custom Networks:

   • Create a network for the frontend-backend connection:

     podman network create frontend-net --subnet 10.10.1.0/24
     

   • Create a network for the backend-database connection:

     podman network create backend-net --subnet 10.10.2.0/24
     

   • List the networks to verify creation:

     podman network ls
     # You should see 'podman', 'frontend-net', and 'backend-net'
     

   • Inspect one of the networks:

     podman network inspect frontend-net
     

2. Run the "Database" Container:

   • Run a simple container (e.g., Alpine with sleep) representing the database. Connect it only to the backend-net.

     podman run -d --name db --network backend-net alpine sleep infinity
     

   • Verify it's connected only to backend-net:

     podman inspect db --format '{{json .NetworkSettings.Networks}}'
     # Output should show details for 'backend-net' only.
     

3. Run the "Backend" Container:

   • Run a container representing the backend service. Crucially, connect it to both frontend-net and backend-net. We'll install ping and curl (or wget) inside it for testing.

     # Start the container connected to backend-net first
     podman run -d --name backend --network backend-net alpine sh -c "apk add --no-cache iputils curl && sleep infinity"
     
     # Now, connect the running backend container to frontend-net
     podman network connect frontend-net backend
     

   • Verify it's connected to both networks:

     podman inspect backend --format '{{json .NetworkSettings.Networks}}'
     # Output should show details for both 'frontend-net' and 'backend-net'
     # Note the different IP addresses assigned on each network.
     

4. Run the "Frontend" Container:

   • Run a container representing the frontend. Connect it only to the frontend-net. Publish a port (e.g., 8080) to simulate user access. Install ping and curl for testing.

     podman run -d --name frontend --network frontend-net -p 8080:80 alpine sh -c "apk add --no-cache iputils curl && sleep infinity"
     

   • Verify it's connected only to frontend-net:

     podman inspect frontend --format '{{json .NetworkSettings.Networks}}'
     

5. Test Connectivity:

   • From Frontend to Backend (Should Work): Use podman exec to ping or curl the backend container by name from the frontend container. This works because they share frontend-net.

     podman exec frontend ping -c 2 backend
     # (Optional) If backend were running a web server on port 80:
     # podman exec frontend curl backend
     

   • From Frontend to Database (Should FAIL): Try to ping the db container from the frontend container. This should fail because they don't share a common network.

     podman exec frontend ping -c 2 db
     # Expect 'bad address' or 'name resolution failure'
     

   • From Backend to Database (Should Work): Ping the db container from the backend container. This works because they share backend-net.

     podman exec backend ping -c 2 db
     

   • From Backend to Frontend (Should Work): Ping the frontend container from the backend container. This works because they share frontend-net.

     podman exec backend ping -c 2 frontend
     

6. Clean Up:

   • Stop and remove the containers:

     podman stop frontend backend db
     podman rm frontend backend db
     

   • Remove the custom networks:

     podman network rm frontend-net backend-net
     

   • Verify networks are gone:

     podman network ls
     

Conclusion: This workshop demonstrated the power of user-defined networks for isolating container communication. You created separate networks and strategically connected containers representing different application tiers. You verified that Podman's built-in DNS allows name resolution only between containers on the same network, enforcing the desired communication flow (Frontend <-> Backend <-> Database). This is a fundamental pattern for building secure and organized multi-container applications with Podman.

5. Persistent Storage with Volumes and Bind Mounts

Containers are ephemeral by design. Their writable layer is discarded when the container is removed. To persist data generated by applications (like databases, user uploads, configuration files) or to provide data to containers, Podman offers two main mechanisms: volumes and bind mounts.

Why Persistent Storage?

Imagine running a database container. If the database stores its data files within the container's writable layer, that data will be lost forever when the container is stopped and removed (e.g., during an upgrade or reconfiguration). Persistent storage solutions decouple the data lifecycle from the container lifecycle.

Bind Mounts

A bind mount maps a file or directory from the host machine's filesystem directly into a container's filesystem.

• Syntax: -v /path/on/host:/path/in/container[:options] or --volume /path/on/host:/path/in/container[:options] or --mount type=bind,source=/path/on/host,target=/path/in/container[,ro,...]
• How it Works: The container directly accesses the specified directory or file on the host. Any changes made inside the container to that path are immediately reflected on the host, and vice-versa.
• Pros:
  • Simple to understand and use for sharing configuration files, source code during development, or accessing host resources.
  • High performance as it's direct filesystem access.
• Cons:
  • Tightly Coupled to Host: Relies on a specific directory structure existing on the host machine, reducing portability. The container configuration might not work if the host path changes or doesn't exist.
  • Permissions Issues: This is a major challenge, especially with rootless containers. The UID/GID of the process inside the container needs appropriate permissions to read/write to the host directory. This can be complex because of user namespace mapping in rootless mode (UID 0 inside might be UID 100000 outside). The :Z or :z options can help Podman manage SELinux labels automatically, but don't solve fundamental UID/GID permission problems.
  • Host Filesystem Clutter: Can lead to application data being scattered across various locations on the host filesystem.

Example: Mount the host's /tmp/app_config directory to /etc/app/conf inside the container.

mkdir /tmp/app_config
echo "setting=value" > /tmp/app_config/app.conf
podman run -v /tmp/app_config:/etc/app/conf:ro --name myapp myimage
# Inside the container, /etc/app/conf/app.conf will exist and be read-only.

Volumes

Volumes are the preferred mechanism for persisting container data. They are managed by Podman and stored in a dedicated area on the host filesystem (within Podman's graphRoot).

• Syntax: -v <volume_name>:/path/in/container[:options] or --volume <volume_name>:/path/in/container[:options] or --mount type=volume,source=<volume_name>,target=/path/in/container[,ro,...]
  • If <volume_name> doesn't exist, Podman creates it automatically.
• How it Works: Podman creates and manages a directory on the host (e.g., under ~/.local/share/containers/storage/volumes/ for rootless, or /var/lib/containers/storage/volumes/ for rootful). This directory is then mounted into the container at the specified path.
• Pros:
  • Managed by Podman: Easier lifecycle management using podman volume commands. Decoupled from the host's specific directory structure.
  • Better Portability: Container configurations using named volumes are more likely to work across different hosts.
  • Permissions Handled Better: Podman often handles ownership and permissions more gracefully with volumes, especially in rootless scenarios, as it controls the volume's storage location. It can often ensure the container's user can write to the volume.
  • Backup/Migration: Easier to back up or migrate volume data as it's centrally managed.
  • Sharing: Volumes can be potentially shared between multiple containers.
  • Volume Drivers: Podman supports volume plugins/drivers, allowing volumes to be stored on external storage systems, cloud storage, etc. (advanced use case).
• Cons:
  • Slightly more abstract than bind mounts; requires podman volume commands for management.
  • Location on host is managed by Podman, less directly accessible than a bind mount path (though you can find it via podman volume inspect).

Types of Volumes:

• Named Volumes: Explicitly created with a name (podman volume create my-data) or implicitly created when first used in podman run -v my-data:/app/data .... These persist until explicitly removed (podman volume rm my-data). This is the recommended type.
• Anonymous Volumes: Created when you only specify the container path in the -v flag: -v /app/data. Podman assigns a random hash as the name. They behave like named volumes but are harder to refer to later. They are typically removed automatically when the container is removed if you use podman rm -v, but not otherwise. Generally, prefer named volumes.

podman volume Commands:

• podman volume create <volume_name>: Creates a new named volume.
• podman volume ls: Lists available volumes.
• podman volume inspect <volume_name>: Shows details about a volume, including its mount point on the host.
• podman volume rm <volume_name> ...: Removes one or more volumes. Volumes currently in use by containers cannot be removed.
• podman volume prune: Removes all unused volumes (volumes not currently attached to any container).

Example: Use a named volume for PostgreSQL data.

# Create a named volume (optional, 'podman run' can create it)
podman volume create pgdata

# Run postgres, mounting the volume to its data directory
podman run -d --name db -v pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=secret postgres:14

# Stop and remove the container
podman stop db
podman rm db

# Run a new postgres container using the *same* volume
podman run -d --name db2 -v pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=secret postgres:14
# The data created by the first container will still be present in the second.

Choosing Between Bind Mounts and Volumes

• Use Volumes for:

  • Persisting application data (databases, user uploads, logs intended for long-term storage).
  • When data needs to be decoupled from the host structure.
  • Sharing data between containers where the host path doesn't matter.
  • Situations where Podman's management features (create, ls, rm, prune) are beneficial.
  • Generally preferred for production and portability.

• Use Bind Mounts for:

  • Sharing configuration files from the host to the container (often read-only).
  • Sharing source code into a container during development for live reloading.
  • Accessing specific host files or devices (e.g., /dev/fuse).
  • Situations where direct access to a specific host path is required and portability is less of a concern.

Workshop: Persisting Data for a Web Application

Goal: Enhance the simple web server from Workshop 3. First, use a bind mount to serve HTML content directly from the host. Second, use a named volume to store application logs generated within the container.

Prerequisites:

• Podman installed and working.
• Project files (index.html, Containerfile) from Workshop 3 (or recreate them). Modify the Containerfile slightly.

Steps:

1. Prepare Project Files:

   • Navigate to your podman-web-app directory (or create it).
   • Ensure index.html exists. Create another page, about.html:

     # In podman-web-app directory
     nano about.html
     

     • Add content:

       <!DOCTYPE html><html><head><title>About Us</title></head>
       <body><h1>About Page</h1><p>Served via bind mount!</p></body></html>
       

     • Save and close.
   • Modify the Containerfile. We no longer need to COPY index.html if we are bind-mounting the content directory. We also don't need the Python web server if we use an official nginx image. Let's switch to nginx. Create a new Containerfile.nginx:

     nano Containerfile.nginx
     

     • Add the following:

       # Use official Nginx image based on Alpine
       FROM nginx:stable-alpine
       
       # Add metadata labels (optional)
       LABEL maintainer="Your Name"
       LABEL description="Nginx serving static files"
       
       # Nginx serves files from /usr/share/nginx/html by default.
       # We will mount our content there using a bind mount later.
       
       # Nginx runs as non-root user 'nginx' by default, which is good.
       
       # Expose port 80 (standard HTTP)
       EXPOSE 80
       
       # Default Nginx command is handled by the base image's ENTRYPOINT/CMD
       # No need to specify CMD here unless overriding behavior.
       

     • Save and close.

2. Build the Nginx Image (Optional but Good Practice):

   • Even though we're using a stock image, sometimes you add custom nginx configs. Let's build our labeled version:

     podman build -t my-nginx-app:v1 -f Containerfile.nginx .
     

3. Run with Bind Mount for Content:

   • Create a directory on your host to hold the web content:

     mkdir ./html_content
     cp index.html about.html ./html_content/
     

   • Run the nginx container, bind-mounting the html_content directory into Nginx's default web root (/usr/share/nginx/html). Use :ro for read-only access from the container if desired, but let's make it writable for now to see changes.

     podman run -d --name webserver_bind \
       -p 8080:80 \
       -v ./html_content:/usr/share/nginx/html:Z \
       my-nginx-app:v1
     

   • Explanation:
     • -v ./html_content:/usr/share/nginx/html: Mounts the local html_content directory to the Nginx web root inside the container. Relative paths like ./html_content are relative to the current working directory where you run the podman command. Using an absolute path ($(pwd)/html_content) is often more reliable.
     • :Z: Important for SELinux: Tells Podman to relabel the host directory (./html_content) so the container's nginx process (running under its own SELinux context) can access it. Use :z if the volume might be shared between multiple containers. If not using SELinux, this option is ignored but doesn't hurt. If you encounter permission errors without it on SELinux systems (like Fedora, RHEL), this is likely the fix.

4. Test Bind Mount:

   • Access http://localhost:8080/index.html and http://localhost:8080/about.html in your browser. You should see the content.
   • Modify Content on Host: Edit the ./html_content/index.html file on your host system. Change the heading. Save the file.
   • Refresh Browser: Refresh http://localhost:8080/index.html. You should immediately see the updated content because the container is directly reading from the host path.
   • Stop and remove this container:

     podman stop webserver_bind
     podman rm webserver_bind
     

5. Run with Named Volume for Logs:

   • Nginx logs to /var/log/nginx by default. Let's redirect these logs to a named volume so they persist even if the container is removed.
   • Create a named volume:

     podman volume create nginx_logs
     

   • Run the container again, this time adding a volume mount for the logs. We still need the bind mount for content.

     podman run -d --name webserver_volume \
       -p 8080:80 \
       -v ./html_content:/usr/share/nginx/html:Z \
       -v nginx_logs:/var/log/nginx \
       my-nginx-app:v1
     

   • Explanation:
     • -v nginx_logs:/var/log/nginx: Mounts the named volume nginx_logs to the /var/log/nginx directory inside the container. Podman will handle permissions.

6. Test Volume Persistence:

   • Access the web server a few times to generate some logs: http://localhost:8080/index.html, http://localhost:8080/about.html, http://localhost:8080/nonexistent_page.
   • Check Logs Inside Container:

     podman exec webserver_volume ls /var/log/nginx
     # Should show access.log, error.log
     podman exec webserver_volume cat /var/log/nginx/access.log
     # Should show your access requests
     

   • Stop and Remove the Container:

     podman stop webserver_volume
     podman rm webserver_volume
     

   • Verify Volume Still Exists and Has Data:

     podman volume ls # nginx_logs should be listed
     podman volume inspect nginx_logs # See its details, including Mountpoint on host
     # Let's run a temporary container just to inspect the volume's contents
     podman run --rm -v nginx_logs:/logs alpine ls /logs
     # Should list access.log, error.log
     podman run --rm -v nginx_logs:/logs alpine cat /logs/access.log
     # Should show the logs generated by the previous container
     

   • Run a New Container Using the Same Volume:

     podman run -d --name webserver_volume_restarted \
       -p 8080:80 \
       -v ./html_content:/usr/share/nginx/html:Z \
       -v nginx_logs:/var/log/nginx \
       my-nginx-app:v1
     

   • Access the server again. Then check the logs inside the new container:

     podman exec webserver_volume_restarted cat /var/log/nginx/access.log
     # You should see the *old* logs plus the *new* access entries. Data persisted!
     

7. Clean Up:

   • Stop and remove the container:

     podman stop webserver_volume_restarted
     podman rm webserver_volume_restarted
     

   • Remove the named volume:

     podman volume rm nginx_logs
     

   • Remove the image (optional):

     podman rmi my-nginx-app:v1
     

   • Remove the host directories (optional):

     rm -rf ./html_content
     

Conclusion: This workshop provided hands-on experience with both bind mounts and named volumes. You used a bind mount to dynamically serve content from the host, observing immediate changes. You then used a named volume to persist Nginx logs across container restarts, demonstrating how volumes effectively decouple data from the container lifecycle. You learned the basic syntax for both and saw the :Z option for handling SELinux with bind mounts. Understanding when and how to use volumes and bind mounts is critical for building stateful, persistent applications with Podman.

6. Orchestrating Containers with Pods

While running individual containers is useful, many applications consist of multiple tightly coupled containers that need to work together closely. Podman borrows the concept of Pods directly from Kubernetes to manage groups of related containers as a single unit.

What is a Pod?

A Pod is a group of one or more containers that are deployed together and share certain Linux namespaces. Specifically, containers within the same pod share:

• Network Namespace: All containers in a pod share the same IP address and port space. They can communicate with each other using localhost. One container can bind to localhost:8000, and another container in the same pod can connect to it directly at that address. This simplifies inter-container communication significantly compared to setting up user-defined networks for every pair.
• IPC Namespace: Containers in a pod share the same Inter-Process Communication resources (like System V IPC semaphores or POSIX shared memory).
• UTS Namespace (Optional): Can optionally share the hostname.
• PID Namespace (Optional): Can optionally share the process ID space, allowing containers within the pod to see and signal each other's processes.

Key Characteristics of Pods:

• Shared Lifecycle (Partially): Pods are created, started, stopped, and removed as a unit using podman pod commands. However, individual containers within the pod still have their own lifecycle managed by podman start/stop/rm <container_id>.
• Resource Sharing: Containers within a pod run on the same node (in Podman's case, the same host machine) and can potentially share storage volumes.
• Atomic Unit: A pod represents the smallest deployable unit.

Why Use Pods? Pods are ideal for scenarios where containers need to cooperate closely:

• Sidecar Containers: A common pattern where a primary application container is augmented by helper containers (sidecars). Examples:
  • A logging agent that collects logs from the main app container and forwards them.
  • A service mesh proxy (like Envoy or Linkerd) handling network traffic for the main app.
  • A monitoring agent exporting metrics from the application.
  • A Git synchronizer pulling configuration updates for the main app. Because they share the network namespace, the sidecar can often interact with the main app via localhost.
• Co-located Applications: Applications designed to work together intimately, perhaps sharing data through local mechanisms (IPC or localhost networking). For instance, a web server and a cache that always need to be deployed together.

The Infra Container: When you create a pod, Podman typically starts a special, very small container called the infra container (often based on an image like k8s.gcr.io/pause or a similar minimal image provided by Podman, e.g., registry.k8s.io/pause). This container's primary role is to "hold" the shared namespaces (network, IPC). It essentially does nothing except sleep. Other containers added to the pod then join the namespaces of this infra container. The infra container is started first when the pod starts and stopped last when the pod stops. You usually don't interact with the infra container directly.

Managing Pods with podman pod

Podman provides a dedicated set of commands for managing pods:

• podman pod create [OPTIONS]: Creates a new pod. A unique ID and name are assigned. Initially, the pod only contains the infra container.

  • Flags:
    • --name <name>: Assign a name to the pod.
    • -p <host_port>:<container_port>, --publish <host_port>:<container_port>: Important: Ports are published at the pod level. Any container within the pod can potentially bind to the specified <container_port>. Traffic arriving at the <host_port> is directed into the pod's shared network namespace.
    • --network <network_name>: Connect the pod (and thus all its containers) to a specific network.
    • --share <namespace>: Specify additional namespaces to share (e.g., pid, uts).

• podman pod ls: Lists existing pods.

  • Flags: Similar filtering (--filter) and formatting (--format) options as podman ps.

• podman pod inspect <pod_name_or_id>: Shows detailed information about a pod, including its state, ID, shared namespaces, and the IDs/names of the containers within it.

• podman pod start <pod_name_or_id> ...: Starts a stopped pod (starts the infra container and any user containers within the pod that aren't already running).

• podman pod stop <pod_name_or_id> ...: Stops a running pod (stops all containers within the pod, including the infra container).

  • Flag: -t, --time <seconds>: Timeout before force-stopping.

• podman pod restart <pod_name_or_id> ...: Restarts a pod.

• podman pod rm <pod_name_or_id> ...: Removes one or more stopped pods. This also removes all containers associated with the pod.

  • Flag: -f, --force: Force removal of a running pod and its containers.

• podman pod top <pod_name_or_id> [ps_options]: Shows processes running across all containers within the pod (if PID namespace is shared, otherwise shows processes for each container separately).

• podman pod prune: Removes all stopped pods.

Running Containers in a Pod

To add a container to an existing pod, you use the regular podman run or podman create command, but specify the target pod using the --pod <pod_name_or_id> option.

# Create a pod, publishing port 8080
podman pod create --name my-web-pod -p 8080:80

# Run an Nginx container *inside* the pod
# Note: No -p flag needed here, as ports are managed at the pod level
podman run -d --pod my-web-pod --name web-server nginx

# Run a logging sidecar container *inside* the same pod
podman run -d --pod my-web-pod --name log-collector my-log-image

Key points when running containers in pods:

• Use --pod <pod_name_or_id> to associate the container with the pod.
• Do not use -p or --publish on podman run for containers inside a pod. Port mapping is defined when the pod is created (podman pod create -p ...).
• Containers within the pod can communicate via localhost:<port>.
• Resource limits (--memory, --cpus) are applied per container, not per pod (though cgroup v2 allows nested limits, which Podman might leverage).

Workshop: Deploying a Web App and Reverse Proxy in a Pod

Goal: Deploy a simple web application (e.g., Python Flask or Node.js, or even just Python's basic server) and an Nginx reverse proxy within the same pod. Nginx will listen on the pod's published port and forward requests to the web application running on a different port accessible via localhost.

Prerequisites:

• Podman installed and working.
• A text editor.

Steps:

1. Create a Simple Web Application:

   • Let's use a minimal Python Flask app. Create a directory:

     mkdir podman-pod-app
     cd podman-pod-app
     

   • Create app.py:

     nano app.py
     

     • Add the following Python code:

       from flask import Flask, jsonify
       import os
       
       app = Flask(__name__)
       
       @app.route('/')
       def home():
           # Read message from environment variable, default if not set
           message = os.environ.get('APP_MESSAGE', 'Hello from Flask!')
           return jsonify({"message": message, "served_by": "Flask App"})
       
       if __name__ == '__main__':
           # Listen on all interfaces within the pod network on port 5000
           app.run(host='0.0.0.0', port=5000)
       

     • Save and close.
   • Create a Containerfile for this Flask app:

     nano Containerfile.flask
     

     • Add the following:

       FROM python:3.9-slim
       
       WORKDIR /app
       
       # Install Flask
       RUN pip install Flask
       
       # Copy the application file
       COPY app.py .
       
       # Set default message (can be overridden)
       ENV APP_MESSAGE="Default Flask Message"
       
       # Expose port 5000 (documentation)
       EXPOSE 5000
       
       # Command to run the application
       CMD ["python", "app.py"]
       

     • Save and close.

2. Build the Flask App Image:

   podman build -t flask-app:v1 -f Containerfile.flask .
   

3. Create an Nginx Configuration for Reverse Proxy:

   • Create an Nginx configuration file nginx.conf:

     nano nginx.conf
     

     • Add the following configuration. This tells Nginx to listen on port 80 and forward requests to the Flask app running on localhost:5000 (because they will be in the same pod/network namespace).

       events {} # Nginx requires events block
       
       http {
           server {
               listen 80;
               server_name localhost;
       
               location / {
                   proxy_pass http://localhost:5000; # Forward to Flask app
                   proxy_set_header Host $host;
                   proxy_set_header X-Real-IP $remote_addr;
                   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                   proxy_set_header X-Forwarded-Proto $scheme;
               }
           }
       }
       

     • Save and close.

4. Create the Pod:

   • Create a pod named webapp-pod. Publish port 8080 on the host, mapping it to port 80 inside the pod's network namespace (which Nginx will listen on).

     podman pod create --name webapp-pod -p 8080:80
     

   • Verify the pod exists (it will contain only the infra container initially):

     podman pod ls
     podman ps -a --pod # Show containers associated with pods
     

5. Run the Flask Application Container in the Pod:

   • Run the flask-app:v1 image inside the pod. Give it a name. We can also override the message using an environment variable.

     podman run -d --pod webapp-pod --name flask_app \
       -e APP_MESSAGE="Running inside a Pod!" \
       flask-app:v1
     

   • Note: No -p flag here. The app listens on port 5000 inside the pod's network namespace.

6. Run the Nginx Reverse Proxy Container in the Pod:

   • Run the official nginx:stable-alpine image inside the same pod. Mount the nginx.conf file you created into the correct location (/etc/nginx/nginx.conf).

     podman run -d --pod webapp-pod --name nginx_proxy \
       -v ./nginx.conf:/etc/nginx/nginx.conf:ro,Z \
       nginx:stable-alpine
     

   • Explanation:
     • --pod webapp-pod: Adds this container to the pod.
     • -v ./nginx.conf:/etc/nginx/nginx.conf:ro,Z: Mounts your custom Nginx config (read-only). Use absolute path $(pwd)/nginx.conf for more robustness. :Z for SELinux compatibility.
     • Nginx will start, read the config, and listen on port 80 within the pod's namespace.

7. Verify Pod and Containers:

   • List running containers, filtering by pod:

     podman ps --pod
     # You should see the infra, flask_app, and nginx_proxy containers listed under webapp-pod
     

   • Inspect the pod:

     podman pod inspect webapp-pod
     

8. Test the Application:

   • Open your web browser and go to http://localhost:8080.
   • You should see the JSON response from the Flask app:

     {
       "message": "Running inside a Pod!",
       "served_by": "Flask App"
     }
     

   • Explanation: Your request hit host port 8080 -> Pod's port 80 -> Nginx container -> proxy_pass to localhost:5000 -> Flask app container -> Response back through Nginx.

9. Check Logs (Optional):

   • View logs for each container:

     podman logs flask_app
     podman logs nginx_proxy
     

10. Clean Up:

    • Stop the entire pod (this stops all containers within it):

      podman pod stop webapp-pod
      

    • Remove the pod (this removes all containers within it):

      podman pod rm webapp-pod
      

    • Verify pod and containers are gone:

      podman pod ls
      podman ps -a --pod
      

    • Remove the Flask app image (optional):

      podman rmi flask-app:v1
      

    • Remove project files (optional):

      cd ..
      rm -rf podman-pod-app
      

Conclusion: In this workshop, you successfully created and managed a Podman pod containing multiple containers (a Flask application and an Nginx reverse proxy). You experienced how containers within a pod share the same network namespace, allowing them to communicate via localhost. You learned how to publish ports at the pod level and how to run containers within a specific pod using podman run --pod. This demonstrates a powerful pattern for deploying co-located, tightly coupled services using Podman, mirroring concepts used in Kubernetes.

7. Running Containers as Systemd Services

While podman run is great for interactive use and development, you often need containers to run reliably as background services, start automatically on boot, and integrate with the system's management tools. Podman provides excellent integration with systemd, the init system used by most modern Linux distributions.

Why Use Systemd for Containers?

• Automatic Startup: Systemd units can be enabled to start automatically when the system boots (or when a user logs in, for user services).
• Lifecycle Management: Use standard systemctl commands (start, stop, restart, status) to manage containerized services just like native system services.
• Dependency Management: Define dependencies between containerized services and native services (e.g., start my app container only after the database service is up).
• Resource Control: Leverage systemd's cgroup management capabilities for fine-grained resource allocation and limits (though Podman also manages cgroups).
• Logging Integration: Container logs (stdout/stderr) are automatically captured by the systemd journal (journald), allowing centralized log viewing and management using journalctl.
• Socket Activation: (Advanced) Start services only when traffic arrives on their designated socket.
• Health Checking & Auto-Restart: Systemd can monitor services and restart them if they fail.

Generating Systemd Unit Files with Podman

Manually writing systemd unit files for containers can be tedious and error-prone. Podman simplifies this dramatically with the podman generate systemd command. This command inspects an existing container or pod and generates a corresponding systemd .service unit file.

Syntax:

podman generate systemd [OPTIONS] <container_or_pod_name_or_id>

Common Options:

• --name, -n: Generate a unit file that starts the container/pod by name (default and recommended). The generated service will be named based on the container/pod name (e.g., container-mywebapp.service or pod-mywebapppod.service).
• --files, -f: Generate the unit file(s) directly into the current directory instead of printing to standard output.
• --new: Create a "new" type unit file. Instead of starting/stopping an existing container definition, this unit file runs podman run (or podman pod start after creating the definition from scratch based on the original) each time the service starts and podman stop/rm when it stops. This ensures a fresh container state on each service start but means container state isn't preserved across restarts unless volumes are used effectively. This is often preferred for stateless services.
• --restart-policy <policy>: Specify the Restart= policy for the systemd unit file (e.g., on-failure, always). Overrides the container's own restart policy for the service definition.
• --container-prefix <prefix>: Set the prefix for the unit name (default: container).
• --pod-prefix <prefix>: Set the prefix for the unit name when generating for a pod (default: pod).
• --separator <separator>: Set the separator character between the prefix and name (default: -).
• --time, -t <seconds>: Set the stop timeout (TimeoutStopSec=) in the generated unit file.
• --no-header: Don't include the informational header comments in the generated file.

Example Workflow:

1. Create and Test: Create your container or pod using podman run or podman pod create / podman run --pod. Test it thoroughly to ensure it works as expected with the desired configuration (ports, volumes, environment variables, etc.).

   # Example: Create a simple nginx container
   podman run -d --name my-webserver -p 8080:80 -v $(pwd)/html:/usr/share/nginx/html:Z nginx
   # Test it...
   

2. Generate Unit File: Once satisfied, generate the systemd unit file. Using --new is often a good choice for services.

   podman generate systemd --new --files --name my-webserver
   # This creates 'container-my-webserver.service' in the current directory
   

3. Inspect the Unit File: Open the generated .service file and examine its contents. You'll see:
   • [Unit] section: Description, dependencies (e.g., Wants=network-online.target).
   • [Service] section: ExecStart (the podman run or podman start command), ExecStop (the podman stop/rm command), Restart policy, user/group information.
   • [Install] section: WantedBy= target (usually default.target or multi-user.target for system services, default.target for user services).

Rootful vs. Rootless Systemd Services

You can manage containers via systemd in two primary contexts:

1. System Services (Rootful):

   • Unit files are placed in /etc/systemd/system/.
   • Managed using standard sudo systemctl [start|stop|enable|disable|status] <service_name>.
   • Containers run as the root user by default (unless --user is specified in the podman run command within the unit file).
   • Suitable for system-wide services. Requires root privileges to manage.

2. User Services (Rootless):

   • Recommended for security and user isolation.
   • Unit files are placed in ~/.config/systemd/user/.
   • Managed using systemctl --user [start|stop|enable|disable|status] <service_name>. No sudo needed.
   • Containers run rootless under the user's context.
   • Services are started automatically when the user logs in and stopped when the last session of the user ends, by default.
   • Linger: To keep user services running even after the user logs out, enable lingering for that user: sudo loginctl enable-linger <username>.
   • Suitable for applications run by specific users, development environments, or any scenario where root privileges are unnecessary.

Quadlet: A Modern Alternative

While podman generate systemd is powerful, managing changes (e.g., updating the image tag, changing a port mapping) requires regenerating the unit file. Quadlet offers a more declarative approach.

• You create a .container, .pod, .volume, or .network file (also using a systemd-like syntax) in /etc/containers/systemd/ (rootful) or ~/.config/containers/systemd/ (rootless).
• These files describe the container, pod, volume, or network you want Podman to manage.
• A systemd generator automatically converts these Quadlet files into transient .service files on the fly during systemd reloads (systemctl daemon-reload or systemctl --user daemon-reload).
• You then manage the service using the Quadlet file name (e.g., systemctl --user start myapp.container).
• Updating the Quadlet file and running daemon-reload is sufficient to apply changes.

Example Quadlet file (~/.config/containers/systemd/my-webserver.container):

[Unit]
Description=My Simple Nginx Webserver Container
After=network-online.target
Wants=network-online.target

[Container]
Image=docker.io/library/nginx:stable-alpine
ContainerName=my-webserver-quadlet
PublishPort=8081:80
Volume=$(pwd)/html:/usr/share/nginx/html:Z
# Add other Podman options here: Environment, Label, etc.

[Install]
WantedBy=default.target

After creating this file, run systemctl --user daemon-reload and then systemctl --user start my-webserver.container. Quadlet provides a cleaner separation between the description of the desired state and the systemd execution mechanism.

Workshop: Running a Container as a Rootless User Service

Goal: Take the Nginx container serving static content (from Workshop 6) and configure it to run as a systemd user service, ensuring it starts automatically when you log in.

Prerequisites:

• Podman installed and working.
• A user account (you should be logged in as this user).
• A simple index.html file.
• Systemd is the init system on your Linux distribution.

Steps:

1. Prepare Content:

   • Create a directory and a simple HTML file:

     mkdir -p ~/systemd-web-test/html
     echo "<h1>Served by Systemd User Service!</h1>" > ~/systemd-web-test/html/index.html
     cd ~/systemd-web-test
     

   • Note the absolute path to the html directory, which we'll need. You can get it with pwd/html, e.g., /home/youruser/systemd-web-test/html.

2. Create the Container Definition (Temporary):

   • Run the container once using podman run to make sure the command is correct. We'll use port 8088 for this example. Use the absolute path for the volume mount.

     # Replace /home/youruser/systemd-web-test/html with the actual absolute path
     podman run -d --name systemd-nginx-test -p 8088:80 \
       -v /home/youruser/systemd-web-test/html:/usr/share/nginx/html:Z \
       nginx:stable-alpine
     

   • Test it by accessing http://localhost:8088. You should see the message.
   • Stop and remove this temporary container; we only needed it to verify the command and create the definition for generate systemd.

     podman stop systemd-nginx-test
     podman rm systemd-nginx-test
     

3. Generate the Systemd Unit File:

   • Use podman generate systemd to create the service file using the --new flag. We'll output it directly to the correct user service directory.
   • Create the user systemd directory if it doesn't exist:

     mkdir -p ~/.config/systemd/user/
     

   • Generate the file:

     podman generate systemd --new --name systemd-nginx-test -f \
       > ~/.config/systemd/user/container-systemd-nginx-test.service
     

     • Note: The command used the name (systemd-nginx-test) of the container we just removed. Podman stored its configuration. The --new flag ensures the generated service uses podman run with that stored configuration. The -f flag is not used here as we redirect the output directly.

4. Examine and Reload Systemd:

   • View the generated file:

     cat ~/.config/systemd/user/container-systemd-nginx-test.service
     

   • Tell systemd to reload its configuration to detect the new user service file:

     systemctl --user daemon-reload
     

5. Enable and Start the User Service:

   • Enable the service to start automatically on login:

     systemctl --user enable container-systemd-nginx-test.service
     

   • Start the service immediately:

     systemctl --user start container-systemd-nginx-test.service
     

6. Check Service Status and Access:

   • Verify the service is running:

     systemctl --user status container-systemd-nginx-test.service
     # Look for 'active (running)'
     

   • Check if the container is running via Podman:

     podman ps
     # You should see the 'systemd-nginx-test' container running
     

   • Access the web server again at http://localhost:8088. It should work.

7. Check Logs with journalctl:

   • View the logs captured by systemd for this service:

     journalctl --user -u container-systemd-nginx-test.service
     

   • Follow logs in real-time (access the web page a few times):

     journalctl --user -f -u container-systemd-nginx-test.service
     # Press Ctrl+C to stop following
     

8. Test Auto-Restart (Simulated):

   • Manually stop the container using Podman (simulating a crash):

     podman stop systemd-nginx-test
     

   • Check the service status immediately. Since the default generated file often has Restart=on-failure, systemd should notice the container stopped unexpectedly (or was stopped) and restart it.

     sleep 5 # Give systemd a moment
     systemctl --user status container-systemd-nginx-test.service
     podman ps # Container should be running again
     

   • Note: If the service used --new, stopping the container via podman stop might not trigger a restart depending on the exact ExecStop= action in the unit file. Killing the container process (podman kill) is a more reliable way to test Restart=on-failure. The generated file might need tweaking (Restart=always) for more robust restarting.

9. Disable and Stop the Service:

   • Stop the service:

     systemctl --user stop container-systemd-nginx-test.service
     

   • Disable it so it doesn't start on the next login:

     systemctl --user disable container-systemd-nginx-test.service
     

   • Verify the container is stopped:

     podman ps -a
     

     (The container might still exist in a stopped state if the --new flag generated an ExecStop=podman stop ... rather than podman stop ... && podman rm .... You may need to podman rm systemd-nginx-test manually or adjust the ExecStop line in the service file and reload).

10. Clean Up:

    • Remove the systemd unit file:

      rm ~/.config/systemd/user/container-systemd-nginx-test.service
      

    • Reload systemd again:

      systemctl --user daemon-reload
      

    • Remove the test directory:

      cd ~
      rm -rf ~/systemd-web-test
      

    • Ensure the container is removed:

      podman rm systemd-nginx-test || true # Ignore error if already gone
      

Conclusion: You have successfully generated a systemd user service unit file from a Podman container definition. You learned how to place this file in the correct location for user services, manage the service lifecycle using systemctl --user, integrate with journalctl for logging, and enable the service for automatic startup. This demonstrates the seamless integration between Podman and systemd for robust service management, especially highlighting the benefits of running services rootlessly. You also briefly touched upon the potential of Quadlet as a more declarative alternative.

8. Podman Security Features - Running Rootless

One of Podman's most acclaimed features is its first-class support for running containers without requiring root privileges on the host. This "rootless" mode significantly enhances security by reducing the potential impact of a container breakout. Let's delve deeper into how rootless containers work and other security aspects of Podman.

The Importance of Rootless Containers

Traditional container runtimes often require a root-privileged daemon. Running containers as root, or giving users access to the Docker socket (which is equivalent to root), poses significant security risks:

• Container Escape: If an attacker compromises an application running inside a container (as root) and finds a vulnerability in the container runtime or the kernel, they could potentially "escape" the container and gain root access on the host system.
• Privilege Escalation: Allowing non-administrator users to run arbitrary containers often implicitly grants them root-level capabilities on the host system via the container daemon.

Rootless containers mitigate these risks. When you run podman as a regular, unprivileged user:

• The Podman processes, the conmon monitor, and the container runtime (runc/crun) all run as your user.
• Crucially, the processes inside the container also run under your user's privileges on the host, even if they appear to be root (UID 0) inside the container.

User Namespaces: The Core Technology

Rootless containers heavily rely on user namespaces, a Linux kernel feature. A user namespace isolates User IDs (UIDs) and Group IDs (GIDs).

• Mapping: When a user namespace is created for a rootless container, a range of UIDs/GIDs inside the namespace is mapped to a different range of unprivileged UIDs/GIDs outside the namespace (on the host system).
• Configuration: This mapping is defined by the /etc/subuid and /etc/subgid files on the host. For example, an entry like myuser:100000:65536 means that the user myuser is allocated 65536 UIDs starting from 100000 on the host.
• How it Works:
  1. You (e.g., myuser, UID 1000) run podman run ....
  2. Podman creates a new user namespace.
  3. Inside this namespace, UID 0 (root) is mapped to your host UID (e.g., 1000).
  4. Other UIDs inside the namespace (e.g., UID 1 to 65535) are mapped to the allocated range on the host (e.g., 100001 to 165535).
  5. The container process starts, believing it runs as UID 0 (root).
  6. However, when this process tries to access host resources (files, devices), the kernel sees it's actually running as your unprivileged host UID (1000) or one of the subordinate UIDs (100001+). It therefore only has the permissions granted to those unprivileged UIDs on the host.

Benefits:
If an attacker breaks out of the rootless container, they gain the privileges of your regular user account on the host, not root privileges. This contains the potential damage significantly.

Limitations and Workarounds for Rootless Mode

While powerful, rootless mode has some inherent limitations due to running without elevated privileges:

• Privileged Ports: Unprivileged users cannot bind to host ports below 1024 (e.g., 80, 443).
  • Workaround 1: Map to a higher host port (-p 8080:80). Use a host-level reverse proxy (like Nginx, HAProxy, or socat) or firewall rules (iptables, nftables) run by root to forward traffic from port 80/443 to the higher port used by the rootless container.
  • Workaround 2: Grant specific capabilities (e.g., CAP_NET_BIND_SERVICE) to the podman executable or specific runtimes, but this elevates privileges and slightly reduces the security benefit.
  • Workaround 3: Modify the system-wide net.ipv4.ip_unprivileged_port_start sysctl value (e.g., sudo sysctl net.ipv4.ip_unprivileged_port_start=80), but this allows any user process to bind low ports, which might have security implications.
• Network Performance: The default slirp4netns network mode for rootless can have performance overhead compared to kernel-level bridge networks used in rootful mode. Pasta aims to improve this. For performance-critical workloads, configuring a rootless CNI bridge or using host networking (--net host, if appropriate) might be considered.
• Mounting Certain Filesystems: Some filesystem operations might require privileges not available in the user namespace.
• Resource Limits (Historically): Older kernels had limitations imposing cgroup resource limits from within user namespaces, but this has improved significantly with cgroup v2. Podman generally handles resource limits correctly in rootless mode on modern systems.
• ICMP Ping: Ping often doesn't work from inside a rootless container using slirp4netns because it lacks privileges to create raw ICMP sockets. Outbound TCP/UDP usually works fine. Ping between containers on a user-defined rootless network using Netavark might work.
• Adding Capabilities: Adding powerful capabilities (--cap-add) might not work as expected, as the user namespace itself restricts the capabilities available even if explicitly added.

Despite these limitations, rootless mode is highly functional for a vast majority of container use cases and is the recommended default for Podman due to its security posture.

Other Podman Security Layers

Beyond rootless execution, Podman leverages several other Linux security mechanisms:

• SELinux (Security-Enhanced Linux): If enabled on the host (common on Fedora, RHEL, CentOS), Podman integrates tightly with SELinux.
  • It assigns specific SELinux labels (e.g., container_t) to container processes.
  • SELinux policies restrict what actions the container_t process can perform on the host system and what files it can access, even if other permissions allow it.
  • This provides Mandatory Access Control (MAC), adding a strong layer of defense against container escapes and privilege escalation.
  • The :Z and :z volume mount options tell Podman to automatically relabel host content to make it accessible to the container's SELinux context.
• AppArmor: On systems using AppArmor (like Debian, Ubuntu), Podman can utilize AppArmor profiles to confine container behavior, similar to SELinux's role.
• Seccomp (Secure Computing Mode): Podman applies a default seccomp filter profile to containers. This profile blocks access to a predefined list of potentially dangerous or unnecessary system calls (syscalls) from within the container. This significantly reduces the kernel's attack surface accessible to container processes. You can customize the profile using --security-opt seccomp=<profile_path.json>. Dropping all syscalls and only allowing specific ones (--security-opt seccomp=unconfined is generally discouraged) provides the tightest security but requires careful profile crafting.
• Capabilities: Linux capabilities break down the monolithic power of root into smaller, distinct privileges (e.g., CAP_NET_ADMIN for network configuration, CAP_SYS_TIME for changing system time). Podman drops most capabilities by default, granting containers only a minimal necessary set. Users can add/drop specific capabilities using --cap-add and --cap-drop, but this should be done cautiously and only when strictly necessary. Running rootless further restricts the effectiveness of many capabilities.
• Read-Only Filesystem: You can run a container with a read-only root filesystem using the --read-only flag. Any required writable paths must then be explicitly provided via volumes (-v or --tmpfs for temporary in-memory storage). This prevents attackers from modifying the container's base image or installing malicious software persistently within the container layer.

By combining rootless execution with namespaces, SELinux/AppArmor, seccomp, and capabilities, Podman provides a multi-layered security approach, making it a strong choice for security-conscious environments.

Workshop: Exploring Rootless Limitations and Security Context

Goal: Experience some of the practical aspects of rootless security, including privileged port limitations and viewing security context information.

Prerequisites:

• Podman installed and configured for rootless execution (this is the default for user installations). You should be running commands as a non-root user.
• Optional: A system with SELinux enabled (like Fedora) to observe SELinux labels.

Steps:

1. Verify Rootless Execution:

   • Run a simple container and check the UID inside vs outside.

     # Run as root inside the container (default for many images)
     podman run --rm alpine id
     # Output inside: uid=0(root) gid=0(root) groups=0(root),...
     
     # Now check the process on the *host* while the container runs briefly
     # Run this in one terminal:
     podman run --name rootless_test alpine sleep 60 &
     # Quickly run this in another terminal (or the same one):
     ps -ef | grep 'sleep 60' | grep -v grep
     

   • Observe the UID in the ps output on the host. It should be your user's UID (e.g., 1000), not root (0), even though id inside the container reported UID 0. This demonstrates the user namespace mapping.
   • Clean up: podman stop rootless_test && podman rm rootless_test

2. Attempt to Bind a Privileged Port (Should Fail):

   • Try to run a container (e.g., nginx) and map host port 80 (a privileged port) to container port 80.

     podman run -d --name low_port_test -p 80:80 nginx:stable-alpine
     

   • This command will likely fail with a "permission denied" or "address already in use" (if another service is using port 80) or similar error related to binding low ports as a non-root user. Note the error message.
   • Clean up any partially created container: podman rm low_port_test || true

3. Bind an Unprivileged Port (Should Succeed):

   • Run the same container but map to a high port (>= 1024), like 8080.

     podman run -d --name high_port_test -p 8080:80 nginx:stable-alpine
     

   • This command should succeed. You can verify by checking podman ps and accessing http://localhost:8080.
   • Clean up: podman stop high_port_test && podman rm high_port_test

4. Inspect Security Options (SELinux Example):

   • If your system uses SELinux (check with getenforce), run a container and inspect its SELinux context.

     podman run -d --name selinux_test alpine sleep 60
     # Check the container process's label on the host
     ps -efZ | grep 'sleep 60' | grep -v grep
     

   • Look for the label associated with the process (e.g., unconfined_u:system_r:container_t:s0:c123,c456). The container_t type is assigned by Podman/SELinux policy.
   • Inspect the container definition to see the applied security options:

     podman inspect selinux_test --format '{{.HostConfig.SecurityOpt}}'
     # You'll likely see entries related to 'label=...' indicating SELinux labeling is active.
     

   • Clean up: podman stop selinux_test && podman rm selinux_test

5. Inspect Seccomp Profile:

   • Inspect a container to see the default seccomp profile path.

     podman run -d --name seccomp_test alpine sleep 10
     podman inspect seccomp_test --format '{{.HostConfig.SecurityOpt}}'
     # Look for an entry like 'seccomp=/usr/share/containers/seccomp.json' (path may vary)
     

   • (Optional) View the contents of that default seccomp profile (it's a JSON file listing allowed/disallowed syscalls).

     # Example path, adjust if needed:
     # less /usr/share/containers/seccomp.json
     

   • Clean up: podman rm seccomp_test || true

6. Inspect Capabilities:

   • Run a container and check its default capabilities using podman inspect.

     podman run -d --name caps_test alpine sleep 10
     podman inspect caps_test --format '{{.HostConfig.CapAdd}} {{.HostConfig.CapDrop}}'
     # By default, CapAdd is likely empty or minimal, CapDrop will list many capabilities.
     

   • Try running a command that requires a specific capability (that is usually dropped). ping often requires CAP_NET_RAW.

     podman run --rm alpine ping -c 1 8.8.8.8
     # This might fail inside a default rootless container depending on network mode.
     # If it fails, try adding the capability:
     podman run --rm --cap-add NET_RAW alpine ping -c 1 8.8.8.8
     # This might still fail in rootless due to user namespace restrictions,
     # but it demonstrates the syntax.
     

   • Clean up: podman rm caps_test || true

Conclusion: This workshop highlighted key practical aspects of Podman's security model, especially in rootless mode. You experienced the privileged port limitation, saw how user namespaces map UIDs, and learned how to inspect the security context applied by Podman, including SELinux labels, seccomp profiles, and Linux capabilities. This understanding reinforces why rootless containers are a major security advantage and demonstrates the multiple layers of defense employed by Podman.

9. Conclusion and Further Steps

Throughout this exploration, we've journeyed from the fundamental concepts of containerization to the specific architecture and features of Podman. We've seen how Podman provides a powerful, secure, and flexible platform for managing containers and pods, distinguishing itself particularly through its daemonless and rootless design.

Key Takeaways:

• Daemonless Architecture: Podman's fork/exec model simplifies the architecture and avoids a single, privileged point of failure compared to daemon-based runtimes.
• Rootless Security: Running containers without root privileges, primarily via user namespaces, drastically reduces the security risks associated with containerization.
• OCI Compliance: Podman works seamlessly with standard OCI images (like Docker images) and runtimes.
• Pod Management: Native support for Kubernetes-style Pods allows for easy management of tightly coupled multi-container applications.
• Systemd Integration: Excellent integration with systemd enables robust management of containers as system or user services, complete with automatic startup, logging, and lifecycle control. Quadlet offers a modern, declarative alternative.
• Rich Feature Set: Podman supports familiar container operations, including building images (Containerfile), managing networks (default and custom), and persistent storage (volumes and bind mounts).
• Layered Security: Podman leverages multiple Linux security features (SELinux, AppArmor, Seccomp, Capabilities) in addition to rootless execution for defense-in-depth.

You've gained hands-on experience through workshops covering installation, image management, container lifecycle, networking, persistent storage, pods, and systemd integration. This practical foundation should empower you to start using Podman effectively for development, testing, and deployment scenarios.

Further Steps and Exploration:

• Podman Compose: Explore podman-compose or alternative tools (like Docker Compose with Podman backend support) for defining and running multi-container applications using familiar docker-compose.yml files.
• Advanced Networking: Investigate more complex networking scenarios, different network drivers (like macvlan), and IPv6 support.
• Building Optimized Images: Learn techniques for creating smaller, more secure container images (multi-stage builds, minimizing layers, using minimal base images like Distroless or optimized Alpine).
• Image Signing and Trust: Explore podman image sign and policy.json for verifying image integrity and enforcing security policies based on image signatures.
• Podman Secrets: Learn how to securely manage sensitive data like passwords and API keys using podman secret create.
• Health Checks: Implement container health checks within Podman (--health-cmd, etc.) or via systemd unit files to ensure service reliability.
• Quadlet In-Depth: Transition from podman generate systemd to using Quadlet's declarative .container and .pod files for managing systemd services.
• Podman Machine: If working on macOS or Windows, explore podman machine for managing a Linux VM where Podman runs your containers.
• Kubernetes Integration (podman kube): Discover how Podman can play/generate Kubernetes YAML definitions (podman kube play, podman kube generate), bridging the gap between local development with Podman and deployment to Kubernetes clusters.
• Advanced Storage: Look into different storage drivers, volume plugins, and snapshot capabilities.

Containerization is a dynamic field, and tools like Podman are constantly evolving. Continue experimenting, read the official Podman documentation and blog, and engage with the community. The skills you've started building here are highly valuable in modern software development and operations. Happy containerizing with Podman!