Author | Nejat Hakan |
nejat.hakan@outlook.de | |
PayPal Me | https://paypal.me/nejathakan |
Advanced Linux Networking - Virtualization and Routing
Introduction
Welcome back! Having mastered the basics of configuration, essential services, and intermediate troubleshooting, we now venture into more advanced territory. The Linux kernel offers an incredibly powerful and flexible networking stack, enabling sophisticated setups far beyond simple client or server roles. Understanding these capabilities is crucial for modern IT infrastructure, including virtualization, containerization, cloud computing, and custom network designs.
This chapter delves into concepts that allow you to create virtual networks within a single Linux host, connect isolated environments, and control how traffic flows between different networks. We'll explore network namespaces for isolation, virtual Ethernet devices (veth pairs) as virtual cables, Linux bridges as virtual switches, and the fundamentals of enabling routing within the kernel. While "advanced," we'll maintain our step-by-step, detailed approach, assuming only the knowledge from the previous chapters.
In this chapter, we will explore:
- Network Namespaces: Creating isolated network environments.
- Virtual Ethernet (veth) Devices: Connecting namespaces and bridges.
- Linux Bridges: Creating virtual Layer 2 switches.
- VLANs (Virtual Local Area Networks): Basic concepts of network segmentation (Conceptual Introduction).
- Kernel IP Forwarding and Basic Routing: Enabling Linux to act as a router.
By the end of this chapter and its workshop, you'll be able to build multi-segment virtual networks on a single Linux machine, a foundational skill for working with containers, VMs, and complex network simulations.
Prerequisites:
- Solid understanding of Chapters 1 & 2 (IP addressing, subnets, gateways, DNS,
ip
command, SSH). - A Linux system (a VM is ideal, as we'll be creating virtual interfaces).
- Root or
sudo
privileges (essential for creating namespaces, links, bridges, and modifying kernel parameters). - The
iproute2
package (provides theip
command, usually installed by default).bridge-utils
might be helpful for older bridge commands butip link
handles most bridge operations now.
1. Network Namespaces: Isolated Network Universes
-
What are Network Namespaces?
- Imagine your Linux system is a large building. Network Namespaces allow you to create separate, isolated apartments within that building, each with its own independent set of network resources.
- Each network namespace gets its own:
- Network interfaces (e.g., its own
lo
, maybe virtual Ethernet devices). - Routing table (its own map of how to reach networks).
- Firewall rules (e.g.,
iptables
ornftables
rules specific to that namespace). - List of network sockets, ports in use, etc.
- Network interfaces (e.g., its own
- Processes running inside a specific namespace can only see and interact with the network resources belonging to that namespace. By default, they are completely unaware of the host system's network configuration or the configurations of other namespaces.
-
Why Use Network Namespaces?
- Isolation: This is the core benefit. Run applications or services in separate namespaces to prevent network conflicts (e.g., two apps wanting to use the same port) or interference. If one application is compromised, the blast radius is limited to its namespace's network view.
- Virtualization/Containers: This is the fundamental technology underpinning container platforms like Docker and LXC/LXD. Each container runs in its own network namespace (among other namespaces like PID, mount, etc.) providing network isolation from the host and other containers.
- Testing & Development: Simulate complex network topologies (multiple clients, servers, routers) on a single physical machine without needing extra hardware. Test network configurations or firewall rules in an isolated sandbox.
- Security: Enhances security by limiting the network visibility and capabilities of processes.
-
Working with Network Namespaces (The
ip netns
command)- The
ip
command (fromiproute2
) is the primary tool. We use theip netns
subcommand. - Listing Namespaces: View all currently defined network namespaces. Initially, this might be empty (only the default "root" namespace exists, which you are in).
- Creating a Namespace: Let's create two namespaces,
ns_web
andns_db
. - Executing Commands within a Namespace: This is how you interact with the namespace's isolated network stack.
Example: Check the network interfaces inside
ns_web
.Example: Bring the loopback interface up insidesudo ip netns exec ns_web ip address show # Output: You'll likely only see the 'lo' (loopback) interface, probably in state DOWN. # This proves the isolation - it doesn't see the host's eth0/wlan0.
ns_web
. - Deleting a Namespace: Remove a namespace when no longer needed.
- Identifying Namespace: Find the namespace associated with a process ID (PID).
- The
Namespaces provide the isolated rooms, but how do we connect them to each other or the outside world? We need virtual wiring.
2. Virtual Ethernet (veth) Devices: The Virtual Network Cables
-
What are veth Devices?
- A
veth
device is not a single interface, but rather a pair of virtual Ethernet interfaces that are directly connected, like two ends of a virtual Ethernet patch cable. - Anything that goes into one end of the pair (
vethA
) immediately comes out the other end (vethB
), and vice versa. They operate at Layer 2 (like a physical cable).
- A
-
Why Use veth Pairs?
- They are the primary mechanism for connecting network namespaces. You can create a veth pair, then move one end into a namespace and keep the other end in the host's namespace (or move it into another namespace, or attach it to a bridge).
- This creates a point-to-point link enabling traffic flow between previously isolated environments.
-
Creating and Using veth Pairs
- We use the
ip link
command. - Creating a Pair: Give unique names to both ends.
- Moving an End into a Namespace: Let's move
veth-guest
into thens_web
namespace (recreate it if you deleted it:sudo ip netns add ns_web
). - Verify Movement:
- Configuring Interfaces: Like physical interfaces, both ends need IP addresses (usually in the same subnet for direct communication) and need to be brought
UP
. - Testing Connectivity: Ping between the ends. If the pings succeed, you've established a network link between the host and the isolated namespace.
- We use the
This point-to-point connection is useful, but what if we want multiple namespaces to talk to each other easily, like computers plugged into the same switch?
3. Linux Bridges: The Virtual Switch
-
What is a Linux Bridge?
- A Linux bridge is a software implementation of a network switch (operating at Layer 2). It's a virtual device that can logically connect multiple network interfaces (physical NICs, veth ends, TAP devices for VMs, etc.) together.
- When interfaces are "enslaved" to a bridge, the bridge learns the MAC addresses of devices reachable through each enslaved interface (its "ports").
- When a frame arrives on one port, the bridge looks at the destination MAC address. If it knows which port leads to that MAC, it forwards the frame only out that specific port. If the destination MAC is unknown, or if it's a broadcast or multicast frame, the bridge floods it out all other ports (just like a basic physical switch).
-
Why Use Linux Bridges?
- Connecting Virtual Machines (VMs): A classic use case. Create a bridge, attach the host's physical NIC to it, and attach the virtual NICs of your VMs. The VMs then appear to be directly on the physical network.
- Connecting Containers/Namespaces: Instead of numerous point-to-point veth pairs, create a bridge. Create veth pairs, put one end in each namespace, and attach the other end of all pairs to the bridge. All namespaces connected this way can communicate directly within the same subnet.
- Building Complex Virtual Networks: Bridges are essential components for creating multi-tier application environments, network simulations, etc.
-
Creating and Using Bridges
- We primarily use
ip link
. Thebridge
command (frombridge-utils
) provides more advanced options, especially for things like STP (Spanning Tree Protocol) or VLAN filtering on the bridge ports. - Creating a Bridge:
- Activating the Bridge: A bridge needs to be
UP
to function. - Attaching Interfaces (Enslaving): Let's create two namespaces (
ns_web
,ns_db
) and connect them viabr0
.# 1. Create namespaces sudo ip netns add ns_web sudo ip netns add ns_db # 2. Create veth pair for ns_web sudo ip link add veth-web type veth peer name veth-web-br # 3. Move one end into ns_web sudo ip link set veth-web netns ns_web # 4. Attach the other end to the bridge br0 sudo ip link set veth-web-br master br0 # 5. Repeat for ns_db sudo ip link add veth-db type veth peer name veth-db-br sudo ip link set veth-db netns ns_db sudo ip link set veth-db-br master br0 # 6. Bring UP the bridge ports (the ends attached to the bridge) sudo ip link set veth-web-br up sudo ip link set veth-db-br up # 7. (Optional but useful) View bridge connections bridge link show # Shows which interfaces are enslaved to which bridges
- Configuring Interfaces within Namespaces: Since both namespaces are connected to the same bridge (Layer 2 domain), assign them IP addresses in the same subnet.
# Configure ns_web sudo ip netns exec ns_web ip addr add 192.168.100.10/24 dev veth-web sudo ip netns exec ns_web ip link set dev lo up sudo ip netns exec ns_web ip link set dev veth-web up # Configure ns_db sudo ip netns exec ns_db ip addr add 192.168.100.20/24 dev veth-db sudo ip netns exec ns_db ip link set dev lo up sudo ip netns exec ns_db ip link set dev veth-db up
-
Testing Connectivity: Ping between the namespaces.
The bridgesudo ip netns exec ns_web ping -c 3 192.168.100.20 sudo ip netns exec ns_db ping -c 3 192.168.100.10
br0
facilitates this communication. -
(Optional) Assigning IP to the Bridge Interface: You can assign an IP address to the bridge device (
br0
) itself. This allows the host system to communicate directly with the namespaces connected to the bridge on that subnet. It can also allow the bridge to act as a gateway if routing is enabled.
- We primarily use
4. VLANs (Virtual Local Area Networks): Basic Concepts
- What are VLANs? Imagine your physical switch (or our Linux bridge) connects devices from different departments (Sales, Engineering). By default, they are all in one large broadcast domain. A broadcast from one device goes to all others. VLANs allow you to logically segment this single switch/bridge into multiple, isolated broadcast domains based on tagging Ethernet frames (using the IEEE 802.1q standard). Each VLAN is assigned a unique ID number (1-4094).
- Why Use VLANs?
- Security: Isolate traffic between groups, even on the same switch.
- Organization: Group devices logically regardless of physical location.
- Performance: Reduce broadcast traffic scope.
- How Linux Handles VLANs (Basic):
- You can create virtual sub-interfaces tagged with a specific VLAN ID on top of a parent interface (physical NIC, bond, bridge).
- Example: Create VLAN 10 and VLAN 20 interfaces on
eth0
.# Create VLAN 10 interface, linked to eth0 sudo ip link add link eth0 name eth0.10 type vlan id 10 # Create VLAN 20 interface, linked to eth0 sudo ip link add link eth0 name eth0.20 type vlan id 20 # Bring them up sudo ip link set eth0.10 up sudo ip link set eth0.20 up # Assign IPs (usually from different subnets) sudo ip address add 10.0.10.1/24 dev eth0.10 sudo ip address add 10.0.20.1/24 dev eth0.20
- How it works: Traffic sent out
eth0.10
will automatically have an 802.1q tag with ID 10 added. Only traffic coming ineth0
tagged with ID 10 will be received by theeth0.10
interface. The underlying physical switch connected toeth0
must also be configured to handle these VLAN tags (e.g., using trunk ports). - Bridge VLAN Filtering: Configuring a Linux bridge to properly switch tagged traffic between specific ports and VLANs is more complex, involving
bridge vlan
commands to set port VLAN IDs (PVIDs) and allowed VLANs. This is beyond this introductory scope but is crucial for advanced bridge usage. The key takeaway here is that Linux can create interfaces that handle VLAN tags.
5. Kernel IP Forwarding and Basic Routing
- Why is Routing Needed? Bridges and switches operate at Layer 2, connecting devices within the same IP subnet. When a device needs to send traffic to an IP address in a different subnet, it needs a router (gateway). The router sits between networks, examines the destination IP address (Layer 3), consults its routing table, and forwards the packet towards the correct network.
-
The Linux Kernel as a Router: Any Linux machine can function as a router if:
- It has interfaces connected to two or more different IP networks/subnets.
- IP forwarding is enabled in the kernel.
- It has appropriate entries in its routing table (either learned automatically or configured manually).
-
Enabling IP Forwarding: By default, Linux does not forward packets between interfaces for security reasons. To turn a Linux box into a router, you must enable it:
(Note:# Check current setting (0 = disabled, 1 = enabled) cat /proc/sys/net/ipv4/ip_forward # Or: sysctl net.ipv4.ip_forward # Enable temporarily (lost on reboot) sudo sysctl -w net.ipv4.ip_forward=1 # Enable permanently: Edit /etc/sysctl.conf or a file in /etc/sysctl.d/ # Find or add the line: net.ipv4.ip_forward=1 # Then apply persistent settings: sudo sysctl -p
net.ipv6.conf.all.forwarding=1
does the same for IPv6). -
Basic Routing Example: Let's reconsider the
ns_web
andns_db
example, but put them in different subnets and use the host as a router between them.- Setup:
- Create
ns_web
,ns_db
. - Create veth pair
veth-web
<=>veth-web-host
. Moveveth-web
tons_web
. - Create veth pair
veth-db
<=>veth-db-host
. Moveveth-db
tons_db
.
- Create
- Configure IPs:
At this point, the host can ping both
# ns_web network: 10.0.1.0/24 sudo ip netns exec ns_web ip addr add 10.0.1.10/24 dev veth-web sudo ip netns exec ns_web ip link set dev veth-web up sudo ip address add 10.0.1.1/24 dev veth-web-host # Host's IP in web net sudo ip link set dev veth-web-host up # ns_db network: 10.0.2.0/24 sudo ip netns exec ns_db ip addr add 10.0.2.20/24 dev veth-db sudo ip netns exec ns_db ip link set dev veth-db up sudo ip address add 10.0.2.1/24 dev veth-db-host # Host's IP in db net sudo ip link set dev veth-db-host up
10.0.1.10
and10.0.2.20
, butns_web
cannot pingns_db
because they are in different subnets and lack routes. - Enable Forwarding on Host:
- Add Routes in Namespaces: Each namespace needs a route telling it how to reach the other network. The "gateway" for this route will be the host's IP address in their own network segment.
# In ns_web: To reach 10.0.2.0/24, go via host's IP 10.0.1.1 sudo ip netns exec ns_web ip route add 10.0.2.0/24 via 10.0.1.1 # In ns_db: To reach 10.0.1.0/24, go via host's IP 10.0.2.1 sudo ip netns exec ns_db ip route add 10.0.1.0/24 via 10.0.2.1 # Alternatively, add default routes if the host is the only way out: # sudo ip netns exec ns_web ip route add default via 10.0.1.1 # sudo ip netns exec ns_db ip route add default via 10.0.2.1
- Test Connectivity:
The host kernel, with forwarding enabled, now receives the packet from
sudo ip netns exec ns_web ping -c 3 10.0.2.20 # Should now work! sudo ip netns exec ns_db ping -c 3 10.0.1.10 # Should also work!
ns_web
onveth-web-host
, sees the destination is10.0.2.20
, knows that network is reachable viaveth-db-host
, and forwards the packet out that interface tons_db
.
- Setup:
Conclusion
This chapter introduced powerful constructs for creating virtual networks within Linux. Network namespaces provide isolation, veth pairs act as connections, and bridges serve as virtual switches. You've also seen the basic concept of VLAN tagging and, crucially, learned how to enable IP forwarding and configure static routes to make Linux act as a router between different networks. These capabilities are the bedrock of container networking (like Docker networks), virtual machine networking (like KVM/QEMU bridges), and complex network simulations or security architectures.
Let's solidify this understanding by building a routed virtual network in the workshop.
Workshop: Building a Routed Virtual Network
Goal:
Create two isolated network namespaces representing two different departments/subnets (sales
and dev
). Create a third namespace (router
) to act solely as a router connecting these two subnets. Configure IP addresses, enable IP forwarding in the router namespace, and set up static routes to allow communication between the sales
and dev
namespaces, routed via the router
namespace.
Scenario:
sales
namespace: Network172.16.10.0/24
dev
namespace: Network172.16.20.0/24
router
namespace: Has interfaces in both networks (172.16.10.1
and172.16.20.1
) and forwards traffic between them.
Diagram:
[ sales ns ] <---- veth-s ----> [ router ns ] <---- veth-r ----> [ dev ns ]
(172.16.10.10/24) (172.16.10.1/24) (172.16.20.1/24) (172.16.20.10/24)
^ ^
| GW: 172.16.10.1 | GW: 172.16.20.1
+-------------------- Route via router -----------------+
Prerequisites:
- Linux system (VM recommended).
- Root or
sudo
privileges. iproute2
package installed.
Steps:
1. Cleanup Previous Environment (Optional but Recommended):
If you have leftover namespaces, bridges, or veth pairs from previous examples, it's best to remove them to avoid conflicts.
# List existing namespaces and delete if necessary
# sudo ip netns list
# sudo ip netns delete <name> ...
# List links and delete previous veth/bridges if necessary
# ip link show type veth
# ip link show type bridge
# sudo ip link delete <device_name> ...
2. Create Network Namespaces:
We need three namespaces.
echo "Creating namespaces: sales, dev, router..."
sudo ip netns add sales
sudo ip netns add dev
sudo ip netns add router
sudo ip netns list # Verify
- Explanation: We create the three distinct network environments using
ip netns add
.
3. Create veth Pairs:
We need two pairs: one connecting sales
to router
, another connecting router
to dev
.
echo "Creating veth pairs..."
# Pair 1: sales <-> router
sudo ip link add veth-s type veth peer name veth-s-rtr
# Pair 2: dev <-> router
sudo ip link add veth-d type veth peer name veth-d-rtr
echo "Verifying veth pairs created in host:"
ip link show type veth
- Explanation: We create two virtual cable pairs.
veth-s
will go into thesales
ns,veth-s-rtr
into therouter
ns. Similarly,veth-d
goes intodev
, andveth-d-rtr
intorouter
.
4. Move veth Ends into Namespaces:
Assign the correct end of each pair to its designated namespace.
echo "Moving veth ends into respective namespaces..."
# Pair 1 ends
sudo ip link set veth-s netns sales
sudo ip link set veth-s-rtr netns router
# Pair 2 ends
sudo ip link set veth-d netns dev
sudo ip link set veth-d-rtr netns router
echo "Verifying veth locations (example: check router ns):"
sudo ip netns exec router ip link show | grep veth
- Explanation: We use
ip link set netns
to place the interfaces.veth-s-rtr
andveth-d-rtr
should now reside within therouter
namespace, whileveth-s
is insales
andveth-d
is indev
.
5. Configure sales
Namespace:
Assign IP, bring interfaces up, and set the default route towards the router.
echo "Configuring 'sales' namespace (172.16.10.0/24)..."
# IP address for veth-s inside sales
sudo ip netns exec sales ip addr add 172.16.10.10/24 dev veth-s
# Bring up loopback and veth-s
sudo ip netns exec sales ip link set dev lo up
sudo ip netns exec sales ip link set dev veth-s up
# Add default route via router's IP on this segment (172.16.10.1)
sudo ip netns exec sales ip route add default via 172.16.10.1
echo "Sales configuration verification:"
sudo ip netns exec sales ip addr show dev veth-s
sudo ip netns exec sales ip route show
- Explanation: We configure the
sales
end of the network: set its IP, activate the interface, and crucially, tell it that the default gateway (for reaching any network it doesn't know directly) is172.16.10.1
(which will be the router's IP on this link).
6. Configure dev
Namespace:
Similar configuration for the dev
namespace on its separate subnet.
echo "Configuring 'dev' namespace (172.16.20.0/24)..."
# IP address for veth-d inside dev
sudo ip netns exec dev ip addr add 172.16.20.10/24 dev veth-d
# Bring up loopback and veth-d
sudo ip netns exec dev ip link set dev lo up
sudo ip netns exec dev ip link set dev veth-d up
# Add default route via router's IP on this segment (172.16.20.1)
sudo ip netns exec dev ip route add default via 172.16.20.1
echo "Dev configuration verification:"
sudo ip netns exec dev ip addr show dev veth-d
sudo ip netns exec dev ip route show
- Explanation: We configure the
dev
end: set its IP in the172.16.20.0/24
network, activate the interface, and set its default gateway to172.16.20.1
(the router's IP on its link).
7. Configure router
Namespace:
The router needs IPs on both its interfaces and needs IP forwarding enabled.
echo "Configuring 'router' namespace..."
# IP address for veth-s-rtr (connected to sales)
sudo ip netns exec router ip addr add 172.16.10.1/24 dev veth-s-rtr
# IP address for veth-d-rtr (connected to dev)
sudo ip netns exec router ip addr add 172.16.20.1/24 dev veth-d-rtr
# Bring up loopback and both veth interfaces
sudo ip netns exec router ip link set dev lo up
sudo ip netns exec router ip link set dev veth-s-rtr up
sudo ip netns exec router ip link set dev veth-d-rtr up
# CRITICAL: Enable IP forwarding within the router namespace
echo "Enabling IP forwarding in router namespace..."
sudo ip netns exec router sysctl -w net.ipv4.ip_forward=1
echo "Router configuration verification:"
sudo ip netns exec router ip addr show
sudo ip netns exec router sysctl net.ipv4.ip_forward # Should show '= 1'
sudo ip netns exec router ip route show # Should show routes for directly connected nets
- Explanation: The router namespace gets configured with IPs on both its interfaces, acting as the gateway address for each respective network. The most critical step here is enabling
net.ipv4.ip_forward=1
within the router namespace. This allows its kernel to forward packets received on one interface (e.g.,veth-s-rtr
) out another (e.g.,veth-d-rtr
) if the destination IP matches the network on the other side.
8. Test Connectivity:
Can sales
ping dev
through the router?
echo "Testing connectivity from sales (172.16.10.10) to dev (172.16.20.10)..."
sudo ip netns exec sales ping -c 4 172.16.20.10
- Expected Output: Successful ping replies.
- Troubleshooting: If this fails:
- Double-check all IP addresses and subnet masks (
/24
). - Verify all relevant interfaces are
UP
(ip link show
inside each namespace). - Verify the default routes in
sales
anddev
point to the correct router IPs (ip route show
). - Crucially verify IP forwarding is enabled in the
router
namespace (sudo ip netns exec router sysctl net.ipv4.ip_forward
). This is the most common mistake.
- Double-check all IP addresses and subnet masks (
Test the other direction:
echo "Testing connectivity from dev (172.16.20.10) to sales (172.16.10.10)..."
sudo ip netns exec dev ping -c 4 172.16.10.10
- Expected Output: Successful ping replies.
9. (Optional) Trace the Route:
See the path the packets take. (You might need sudo apt install traceroute
or sudo dnf install traceroute
first on your host, then execute it via ip netns exec
).
- Expected Output: Should show hop 1 as the router's IP on the sales side (
172.16.10.1
) and hop 2 as the final destination (172.16.20.10
). This explicitly confirms the router namespace is forwarding the traffic.
10. Cleanup:
Remove the namespaces, which also removes the veth pairs associated with them.
echo "Cleaning up namespaces..."
sudo ip netns delete sales
sudo ip netns delete dev
sudo ip netns delete router
echo "Verification:"
sudo ip netns list
ip link show type veth # Should be empty if no other veths exist
Workshop Conclusion:
Excellent! You have successfully constructed a multi-segment network using isolated namespaces and configured a dedicated namespace to act as a router between them. You practiced creating veth pairs, assigning IPs, enabling kernel IP forwarding, and setting static routes. This practical exercise demonstrates how Linux can be used to build complex, isolated network environments entirely in software, mimicking physical network setups and providing the foundation for understanding technologies like container networking.