Author	Nejat Hakan
eMail	nejat.hakan@outlook.de
PayPal Me	https://paypal.me/nejathakan

Backup Server - UrBackup

Introduction to UrBackup

Welcome to this comprehensive guide on UrBackup, a powerful open-source client-server backup system designed for individuals and businesses looking to self-host their backup solution. In an era where data is paramount, having a reliable and controllable backup strategy is not just a good practice; it's a necessity. UrBackup offers a robust feature set that allows you to protect data from various sources, including workstations and servers, across different operating systems.

What is UrBackup?

UrBackup is an easy-to-set-up, open-source client-server network backup system. It operates by having a central UrBackup server that manages backups from multiple clients installed on the machines you wish to protect. Its primary goal is to provide a solution that is both highly efficient and simple to manage, without sacrificing features or reliability.

Key characteristics of UrBackup include:

Client-Server Architecture: A central server orchestrates and stores backups from client software installed on target machines.
Open-Source: UrBackup is free to use, modify, and distribute under the GNU Affero General Public License (AGPL) v3+. This transparency allows for community vetting and contributions.
Cross-Platform: Clients are available for Windows, Linux, and macOS, while the server component is primarily designed for Linux and Windows (with Linux often being the preferred platform for stability and performance in self-hosted scenarios).
Multiple Backup Types: It supports both file backups (backing up specific folders and files) and image backups (creating a full snapshot of a disk or partition).
Efficiency: UrBackup excels at incremental backups. After an initial full backup, subsequent backups only transfer and store changed data blocks, significantly reducing backup time and storage space. It uses data deduplication to further optimize storage.
Web Interface: A comprehensive web-based user interface allows for easy management of the server, clients, backups, and restores from anywhere on your network (or even the internet, if configured).

Why choose UrBackup for self-hosting?

Self-hosting your backup solution with UrBackup offers several compelling advantages:

Complete Data Control and Privacy: Your data remains on your hardware, under your control. You are not reliant on third-party cloud providers and their terms of service, pricing changes, or potential privacy concerns. This is crucial for sensitive data.
Cost-Effectiveness: Apart from the initial hardware investment for your backup server and storage, UrBackup itself is free. This can lead to significant cost savings compared to commercial backup software licenses or ongoing cloud storage fees, especially as your data grows.
Flexibility and Customization: You can tailor the backup server hardware, storage capacity, network configuration, backup schedules, and retention policies precisely to your needs.
Learning Opportunity: Setting up and managing your own backup server is an excellent way to deepen your understanding of networking, server administration, storage technologies, and data protection principles.
No Vendor Lock-in: Being open-source, you are not tied to a specific vendor. You have the freedom to migrate or modify the system as your requirements evolve.

Key Features and Benefits

UrBackup boasts a rich feature set. Let's explore some of the most significant ones in detail:

File Backups:
- Functionality: Allows you to select specific directories and files for backup. This is ideal for protecting user documents, application data, and configuration files.
- Incremental Forever: After the first full backup, UrBackup performs incremental backups indefinitely. It intelligently reconstructs full backups from the server-side when needed for restoration, saving space and time on the client.
- Deduplication: Identical files, even across different clients, are stored only once on the server, significantly reducing storage space. This deduplication also works for parts of files.
- Pros: Faster for small, frequent changes; granular restoration of individual files/folders; less storage for text-heavy data.
- Cons: System recovery is more complex than with image backups as the OS and applications need to be reinstalled first.
Image Backups:
- Functionality: Creates a complete block-level copy of a disk or partition. This is essential for bare-metal recovery (restoring an entire system, including the OS, applications, settings, and data, to a new or repaired hard drive).
- Supported Formats: UrBackup can create images in VHD (Virtual Hard Disk) and VHDZ (compressed VHD) formats for Windows clients using Volume Shadow Copy Service (VSS) for consistency, and raw image formats for Linux and Windows. VHD/VHDZ files can often be mounted directly for file access.
- Incremental Images: Similar to file backups, image backups are also incremental, saving significant space and time after the initial full image.
- Pros: Enables quick full system recovery (Bare Metal Restore); captures everything on the selected volume.
- Cons: Initial full image backup can be large and time-consuming; restoring individual files can be slightly less direct than from a file backup (though UrBackup facilitates this).
Efficient Incremental Backups: UrBackup's core strength lies in its highly optimized incremental backup strategy. It uses client-side hashing to detect changed file blocks. Only these changed blocks are sent to the server. This minimizes network traffic and backup duration, making frequent backups feasible even over slower links.
Global Deduplication: As mentioned, UrBackup deduplicates data at the file and sub-file block level across all connected clients. If multiple clients have the same operating system files or common documents, these are stored only once on the backup server.
User-Friendly Web Interface: The web UI provides a centralized dashboard for:
- Monitoring backup status of all clients.
- Viewing logs.
- Configuring server settings (storage paths, backup windows, users, etc.).
- Managing client settings (backup paths, frequencies, quotas).
- Initiating manual backups and restores.
- Downloading client installers.
Cross-Platform Client Support:
- Windows: Full support for file and image backups, including VSS for consistent snapshots of live systems.
- Linux: Robust file backup capabilities. Image backups are typically done for entire disks or partitions in raw format. Scripting can be used for more complex snapshotting (e.g., with LVM).
- macOS: Client support is available, typically focusing on file backups.
Security:
- Encrypted Transfer: Backup data can be encrypted during transmission between the client and server.
- Secure Client Authentication: Clients authenticate with the server to prevent unauthorized access.
- HTTPS for Web UI: The web interface can be secured using SSL/TLS certificates.
Flexible Restore Options:
- Web-based Restore: Users (if permitted) or administrators can browse backups and select files/folders for restoration to the original client or download as a ZIP archive.
- Restore CD/USB: UrBackup provides a bootable restore environment for performing bare-metal restores from image backups.
- File-level Restore from Images: Even from image backups, UrBackup often allows browsing and restoring individual files via the web interface.
Internet Backups: UrBackup can be configured to back up clients over the internet. This is useful for remote workers, laptops, or even off-site backups of small servers, provided you have sufficient bandwidth and have secured the server appropriately.
Low Resource Usage: Both the server and client components are designed to be lightweight and efficient, minimizing their impact on system performance.

UrBackup Components

Understanding the core components is key to grasping how UrBackup works:

UrBackup Server (urbackup_srv): This is the central application that runs on your designated backup server machine (Linux or Windows). Its responsibilities include:
- Listening for client connections.
- Managing client configurations and backup schedules.
- Receiving, processing, and storing backup data.
- Performing deduplication.
- Maintaining the backup database and metadata.
- Serving the web interface.
- Handling restore requests.
UrBackup Client (UrBackupClientBackend and optional tray icon/CLI tools): This software is installed on each machine (Windows, Linux, macOS) that you want to back up. Its functions are:
- Connecting to the UrBackup server.
- Identifying files and disk blocks that need to be backed up.
- Hashing data to detect changes.
- Sending new or changed data to the server.
- Receiving commands from the server (e.g., to start a backup).
- Facilitating restores to the local machine.
Web Interface: Accessed via a web browser, this is the primary administrative console for managing the entire UrBackup system. It communicates with the UrBackup Server component.

Comparison with other backup solutions (briefly)

While many backup solutions exist (e.g., Bacula, Bareos, Duplicati, Restic, commercial cloud backups), UrBackup carves out a niche for self-hosters due to its:

Ease of Setup: Compared to more complex enterprise solutions like Bacula/Bareos, UrBackup is significantly easier to get up and running.
Combined File and Image Backups: Offering both in an integrated, efficient manner is a strong point.
Efficient Incremental and Deduplication: Particularly good for network efficiency and storage savings.
No Ongoing Cost (for the software): Ideal for budget-conscious self-hosters. Solutions like Duplicati or Restic are excellent for file-based, encrypted backups to various cloud storage backends but lack the centralized server management and native image backup capabilities of UrBackup in the same way. Commercial cloud backups offer convenience but reduce control and can become expensive. UrBackup offers a balance of control, features, and cost for those willing to manage their own server.

This introduction should provide a solid foundation for understanding what UrBackup is and why it's a compelling choice for your self-hosted backup needs. The following sections will guide you through setting it up, configuring it, and mastering its advanced features.

Workshop Introduction to UrBackup

Project: Planning Your UrBackup Deployment

Before diving into installation and configuration, a little planning goes a long way in ensuring your UrBackup deployment meets your needs effectively. This workshop will guide you through the essential planning steps.

Objective: To create a basic deployment plan document for your UrBackup setup, considering your specific backup requirements, resources, and network environment.

Steps:

Identify Your Backup Needs:
- What to back up?
  - List the machines (servers, workstations, laptops) you intend to back up.
  - For each machine, determine what data is critical. Is it specific user directories (e.g., /home/user/Documents, C:\Users\User\Documents), entire application data folders, databases, or entire system disks (for bare-metal recovery)?
  - Example:
    - Desktop-PC (Windows 10): C:\Users\MyUser\Documents, C:\Users\MyUser\Pictures, C:\Users\MyUser\Desktop, Full system image.
    - Laptop (Ubuntu Linux): /home/mylaptopuser/work_projects, /home/mylaptopuser/.ssh, /etc/.
    - HomeServer (Debian): /var/www/html, /srv/git, /opt/appdata, Full system image for OS disk.
- Which type of backup?
  - For each data set, decide if a file backup or an image backup is more appropriate. Often, a combination is best (e.g., daily file backups for user data, weekly image backups for system recovery).
- How often to back up?
  - Determine the backup frequency for each client/data type. Critical data might need hourly or daily backups, while less critical systems might be fine with weekly backups. Consider your Recovery Point Objective (RPO) – how much data can you afford to lose?
  - Example: Desktop-PC Documents: Daily file backups. Desktop-PC System Image: Weekly image backups.
- How long to keep backups?
  - Define your retention policy. How many old versions of files do you need? How far back should you be able to restore? This is your Recovery Time Objective (RTO) consideration in part, but more about data versioning.
  - UrBackup uses settings like min/max full file backups, min/max incremental file backups, and similar for images. Think about this in terms of days, weeks, or months.
  - Example: Keep 7 daily incremental file backups, 4 weekly full file backups. Keep 2 monthly image backups.
Estimate Storage Requirements:
- For each client, estimate the total size of the data to be backed up (for both initial full backups and anticipated changes).
- Consider UrBackup's deduplication and compression, which can significantly reduce actual storage used, but it's wise to be conservative in initial estimates. A rough rule of thumb might be 1.5x to 3x the total source data size to accommodate multiple versions and images, depending on data type and change rate.
- Example:
  - Desktop-PC: 100GB (Documents) + 200GB (Image). Initial full = 300GB.
  - Laptop: 50GB (Files). Initial full = 50GB.
  - HomeServer: 30GB (Files) + 100GB (Image). Initial full = 130GB.
  - Total initial full data: ~480GB.
  - Estimated storage with retention and growth: Perhaps 1TB - 2TB to start.
Choose Suitable Server Hardware/VM:
- CPU: A modern multi-core CPU is beneficial, especially if you have many clients or use compression. Even a Raspberry Pi 4 can run UrBackup for a few clients, but a dedicated machine with an x86 CPU (e.g., Intel Core i3/i5/Ryzen 3/5 or better) is recommended for more serious use.
- RAM: Minimum 2GB RAM. 4GB+ is recommended, especially with many clients or large backups, as UrBackup uses RAM for caching and database operations.
- Storage:
  - OS Drive: A small, fast SSD for the UrBackup server's operating system and UrBackup database is ideal for performance.
  - Backup Storage Drive(s): Large capacity HDDs are common. Consider RAID for redundancy (e.g., RAID 1, RAID 5, RAID 6, or ZFS equivalents). The filesystem choice on this storage is also important (e.g., Btrfs, ZFS for advanced features, or XFS/ext4 for simplicity).
- Virtual Machine (VM): Running UrBackup server in a VM is a viable option. Ensure the VM has dedicated access to sufficient CPU, RAM, and fast storage for backups (e.g., pass-through a dedicated HDD/SSD array or use a robust virtual disk on fast storage).
Network Considerations:
- Bandwidth:
  - For local clients (on the same LAN), a Gigabit Ethernet connection (1 Gbps) is standard. If backing up many clients simultaneously or very large amounts of data, ensure your network switches and server NIC can handle the load.
  - For internet clients, your internet connection's upload speed at the server location and download speed at the client location will be the bottleneck. Calculate if this is feasible for your data volumes and backup windows.
- Server IP Address: The UrBackup server should ideally have a static IP address on your local network so clients can reliably find it.
- Port Forwarding (for Internet Clients): If you plan to back up clients over the internet, you'll need to configure port forwarding on your router to direct UrBackup traffic to the server. Default ports are TCP/UDP 55413, 55414, and 55415.
- Dynamic DNS (for Internet Clients): If your server's public IP address is dynamic, set up a Dynamic DNS (DDNS) service so clients can find the server using a consistent hostname.

Create a Simple Deployment Plan Document:

Based on the points above, create a simple text file or spreadsheet.

Example Document Snippet:

UrBackup Deployment Plan

1. Backup Scope:
    - Client1 (Win10-Desktop):
        - Data: C:\Users\UserX\Docs, C:\Users\UserX\Photos (File Backup)
        - System: C:\ (Image Backup)
        - Frequency: Files daily, Image weekly
        - Retention: Files - 14 days incremental, 4 weekly fulls. Image - 2 monthly.
    - Client2 (Linux-Laptop):
        - Data: /home/usery/projects (File Backup)
        - Frequency: Files daily
        - Retention: Files - 30 days incremental, 2 monthly fulls.

2. Estimated Storage:
    - Client1: ~150GB initial, ~400GB target with retention.
    - Client2: ~20GB initial, ~60GB target with retention.
    - Total Target: ~460GB. Planned Server Storage: 2TB (HDD, RAID1).

3. Server Specification:
    - Type: Dedicated Physical Machine (Old PC)
    - CPU: Intel Core i5-4570
    - RAM: 8GB DDR3
    - OS Drive: 120GB SSD (for Linux OS + UrBackup DB)
    - Backup Drive: 2x 2TB HDD (RAID 1, Btrfs)
    - OS: Ubuntu Server 22.04 LTS

4. Network:
    - Server IP: 192.168.1.10 (Static)
    - Clients: All local LAN initially.
    - Internet Access: Not planned initially, but will configure firewall rules if needed later.
      (Ports: 55413, 55414, 55415 TCP/UDP)

5. Security Considerations:
    - Strong admin password for UrBackup WebUI.
    - Server OS regularly updated.
    - Firewall enabled on server.
    - Future: HTTPS for WebUI.

By completing this planning workshop, you'll have a much clearer picture of your UrBackup setup, making the subsequent installation and configuration phases smoother and more aligned with your actual backup objectives. This document will serve as a valuable reference as you proceed.

1. Setting Up the UrBackup Server

With your planning complete, it's time to install the heart of your backup system: the UrBackup server. This section will guide you through the prerequisites, installation process (with a focus on Linux), and initial configuration.

Prerequisites

Before you begin the installation, ensure your chosen server machine or virtual machine meets the following requirements:

Hardware Recommendations:
- CPU: While UrBackup can run on low-power devices like a Raspberry Pi for a small number of clients, a more robust CPU (e.g., dual-core x86_64 or better) is recommended for optimal performance, especially with multiple clients, compression, or deduplication. For university student projects, a VM with 2+ vCPUs is a good starting point.
- RAM: A minimum of 2GB of RAM is suggested. 4GB or more is preferable for smoother operation, especially if you plan to back up several clients or large datasets. RAM is used for database operations and caching.
- Storage:
  - Operating System Drive: A separate drive (preferably an SSD) for the server's operating system and the UrBackup database can significantly improve web interface responsiveness and backup processing speeds. 30-60GB should be sufficient for the OS and UrBackup application.
  - Backup Storage Drive(s): This is where the actual backup data will reside. The size depends entirely on your estimated storage needs from the planning phase. Large capacity HDDs are common.
    - Important: The chosen path for backup storage should have ample free space and be dedicated to UrBackup if possible.
    - Consider using a filesystem that supports snapshots and data integrity, like Btrfs or ZFS, if you're comfortable with them. Otherwise, ext4 or XFS are solid choices on Linux.
Operating System (OS) Choice:
- Linux (Recommended): UrBackup is well-supported and widely used on Linux distributions. Debian or Ubuntu LTS (Long Term Support) versions are excellent choices due to their stability and large communities. This guide will primarily focus on Debian/Ubuntu. Other distributions like CentOS/RHEL/Fedora are also usable.
- Windows: UrBackup server can also run on Windows. However, for a dedicated, self-hosted server, Linux is often preferred for its stability, performance, and resource efficiency in server roles.
- Other (FreeBSD/NAS): Some NAS devices offer UrBackup server packages or allow installation via Docker, which can be a convenient option if you have a capable NAS.
Network Configuration:
- Static IP Address: It is highly recommended to configure a static IP address for your UrBackup server on your local network. This ensures that clients can consistently connect to the server without issues caused by IP address changes (which can happen with DHCP).
- Internet Connectivity: Required for downloading the UrBackup server package and its dependencies.
- Firewall: Be prepared to configure your server's firewall (e.g., ufw on Ubuntu, firewalld on CentOS, or Windows Firewall) to allow incoming connections on UrBackup's default ports:
  - TCP 55413: For client communication (Internet mode).
  - TCP 55414: For the web interface.
  - UDP 55415: For client discovery on the local network.
  - TCP 55415: For client communication (Local network mode). (Note: UDP and TCP on 55415)

Installation on Linux (Debian/Ubuntu Focus)

The recommended way to install UrBackup server on Debian-based systems like Ubuntu is by using the official UrBackup repository, which provides up-to-date versions.

Update Your System: Always start by ensuring your system's package list and installed packages are up to date.
```
sudo apt update
sudo apt upgrade -y
```
Add the UrBackup Repository: UrBackup provides a Personal Package Archive (PPA) for Ubuntu and similar systems. For Debian, the process involves adding the OpenSUSE Build Service repository.
- For Ubuntu:
```
sudo add-apt-repository ppa:uroni/urbackup
sudo apt update
```
- For Debian: You'll need to add the repository key and the repository source. Replace Debian_11 (Bullseye) or Debian_10 (Buster) with your Debian version. You can find the correct repository URLs on the UrBackup download page if these change.
```
# Example for Debian 11 (Bullseye)
# Import GPG key
wget -qO - https://download.opensuse.org/repositories/home:/uroni/Debian_11/Release.key | sudo apt-key add -

# Add repository
echo "deb https://download.opensuse.org/repositories/home:/uroni/Debian_11/ ./" | sudo tee /etc/apt/sources.list.d/urbackup.list
sudo apt update
```
  Note: Always verify GPG keys and repository URLs from the official UrBackup website for security.
Install the urbackup-server Package: Once the repository is added and the package list updated, install the UrBackup server package:
```
sudo apt install urbackup-server
```
During the installation, you might be prompted for the path where backups should be stored. You can enter it now or configure it later via the web interface. It's generally a good idea to have a dedicated partition or disk mounted for this purpose (e.g., /mnt/backups or /var/urbackup). Example: If you have a separate drive mounted at /mnt/urbackup_storage, you would enter this path.
Initial Configuration Steps (Post-Installation):
- Backup Storage Path: If you didn't set this during installation, or if you want to change it, this is the most crucial first step. The UrBackup server service needs a directory with sufficient space and write permissions for the urbackup user (which is created during installation).
  - If you choose a new path, ensure the urbackup user has ownership or write access:
```
# Example: if your storage path is /mnt/urbackup_data
sudo mkdir -p /mnt/urbackup_data
sudo chown urbackup:urbackup /mnt/urbackup_data
sudo chmod 750 /mnt/urbackup_data # Or more restrictive if appropriate
```
    You will then set this path in the web interface settings.
- Service Status: The UrBackup server should start automatically after installation. You can check its status:
```
sudo systemctl status urbackup-server.service
# Or for older systems without systemd:
# sudo service urbackup-server status
```
  If it's not running, you can try starting it:
```
sudo systemctl start urbackup-server.service
```
  And enable it to start on boot:
```
sudo systemctl enable urbackup-server.service
```

Installation on Windows (Brief Overview)

While Linux is often preferred for stability in self-hosting, you can also run the UrBackup server on Windows:

Download the Installer: Go to the official UrBackup website (urbackup.org) and download the Windows server installer (.exe file).
Run the Installer: Execute the downloaded installer. The setup wizard will guide you through the process.
Backup Storage Path: You will be prompted to specify a folder where backups will be stored. Choose a location with ample disk space, preferably on a separate drive/partition.
Firewall: The installer usually attempts to configure the Windows Firewall automatically. If you use a third-party firewall, you may need to add exceptions for UrBackup manually (ports 55413, 55414, 55415).
Service: The UrBackup server runs as a Windows service. You can manage it from the Services administrative tool (services.msc).

Accessing the Web Interface

Once the UrBackup server is installed and running, you can access its web management interface:

URL: Open a web browser on a machine that can reach the UrBackup server on your network. Navigate to: http://YOUR_SERVER_IP:55414 Replace YOUR_SERVER_IP with the static IP address you configured for the UrBackup server. For example, http://192.168.1.10:55414.
First Login and Admin User:
- On the first visit, UrBackup might present some initial setup steps or take you directly to the main interface.
- By default, there is an admin user. The default password for the admin user is often admin or may be blank initially, or set during installation if prompted. It is CRITICAL to change this default password immediately for security.
- Go to the "Settings" page in the web interface.
- Navigate to the "Users" tab or section.
- Select the admin user and set a strong, unique password.

Understanding the Dashboard (Status Page)

The main page you'll see after logging in is usually the "Status" or "Dashboard" page. This page provides an at-a-glance overview of your backup system:

Server Status: Information about server uptime, version, and storage usage.
Client List: A list of connected clients, their online status, last backup times, and backup progress.
Recent Activities: Logs of recent backup jobs, restores, and server events.
Backup Statistics: Summaries of data backed up, storage efficiency, etc.

Familiarize yourself with this page, as it will be your primary point for monitoring daily operations.

Basic Server Settings

After securing the admin user, explore the "Settings" page to configure essential server parameters:

General -> Server:
- Backup storage path: This is the most important setting. Verify or set the correct path to your dedicated backup storage location. If you change this, the server may need to re-index files or might require a restart.
General -> Server -> Backup Window:
- Specify the time window during which backups are allowed to run (e.g., 00:00-23:59/1-7 for all day, every day, or 22:00-06:00/1-5 for weekdays between 10 PM and 6 AM). This helps manage network and server load.
General -> Server -> Max active backups:
- Max simultaneous backups: Controls how many clients can back up at the same time. Setting this too high can overload your server's CPU, RAM, disk I/O, or network bandwidth. Start with a conservative number (e.g., 2-4) and adjust based on your server's performance.
Permissions and Users: While more advanced, ensure you understand how to manage users if you plan to give others access or restrict client management.
Email Notifications: Configure SMTP server settings to receive email alerts for backup successes, failures, or other important events. This is highly recommended.

Take your time to explore the various settings. The UrBackup Server Administration Manual (available on the website) provides detailed explanations for each option. However, for now, ensuring the storage path is correct and the admin password is changed are the top priorities.

Workshop Setting Up the UrBackup Server

Project: Install UrBackup Server on a Linux Virtual Machine

This workshop will guide you through installing the UrBackup server on a fresh Ubuntu Server virtual machine. This provides a safe and isolated environment for learning.

Objective: To successfully install and perform the initial configuration of an UrBackup server on an Ubuntu Server VM, accessible via its web interface.

Prerequisites for this Workshop:

A host machine with virtualization software installed (e.g., VirtualBox, VMware Workstation Player/Pro, Hyper-V).
An Ubuntu Server ISO image (e.g., Ubuntu Server 22.04 LTS). Download from the official Ubuntu website.
Basic familiarity with creating and managing virtual machines.
At least 2 vCPUs, 2GB RAM (4GB recommended), and two virtual disks for the VM: one ~30GB for the OS, and another ~50-100GB (or dynamically sized) for backup storage.

Steps:

Set Up a New Ubuntu Server VM:
- In your virtualization software, create a new virtual machine.
- Allocate resources: 2 vCPUs, 2-4GB RAM.
- Create a primary virtual hard disk (e.g., 30GB) for the Ubuntu Server OS installation.
- Crucially, add a second virtual hard disk (e.g., 50GB or larger, can be thin-provisioned/dynamically allocated). This will be dedicated to UrBackup storage.
- Configure the VM's network adapter to "Bridged Adapter" mode. This will allow the VM to get an IP address from your router, making it accessible from other machines on your LAN, including your host machine. If Bridged mode causes issues, NAT with port forwarding can also work but is more complex for initial access.
- Attach the Ubuntu Server ISO to the VM's virtual CD/DVD drive and boot the VM.
Install Ubuntu Server:
- Follow the on-screen prompts to install Ubuntu Server.
- Language, Keyboard: Select your preferences.
- Network: It should attempt to get an IP via DHCP. Note down the IP address if shown, or you can find it later using ip a. For this workshop, a DHCP-assigned IP is fine, but remember a static IP is best for a real server.
- Storage Configuration:
  - Choose "Custom storage layout".
  - You should see your two virtual disks (e.g., sda and sdb).
  - Set up sda (your 30GB disk) as the boot disk with a root partition (/) using ext4.
  - Leave sdb (your 50GB+ disk) unformatted for now. We will partition and format it manually for backups.
- Profile Setup: Create a user account (e.g., uradmin with a strong password).
- SSH Setup: Install the OpenSSH server when prompted. This allows you to connect to the VM via SSH from your host machine, which is convenient for copying commands.
- Server Snaps: Skip installing any additional server snaps for now.
- Let the installation complete and reboot the VM. Remove the ISO from the virtual CD/DVD drive.
Post-Installation VM Configuration & Update:
- Log in to the VM using the credentials you created.
- Find the VM's IP address: ip a (look for the address under your main network interface, like ens33 or enp0s3).
- From your host machine, try to SSH into the VM: ssh uradmin@VM_IP_ADDRESS. This confirms network connectivity.
- Inside the VM (either via console or SSH session), update the system:
```
sudo apt update
sudo apt upgrade -y
```
Prepare the Backup Storage Disk:
- List available block devices to identify your second virtual disk (e.g., /dev/sdb):
```
lsblk
```
- Use fdisk or parted to create a single primary partition on this disk. Example using fdisk for /dev/sdb:
```
sudo fdisk /dev/sdb
# Inside fdisk:
# n (new partition)
# p (primary)
# 1 (partition number)
# Enter (default first sector)
# Enter (default last sector)
# w (write changes and exit)
```
- Format the new partition (e.g., /dev/sdb1) with a filesystem like ext4:
```
sudo mkfs.ext4 /dev/sdb1
```
- Create a mount point for this partition:
```
sudo mkdir /mnt/urbackup_storage
```
- Mount the partition:
```
sudo mount /dev/sdb1 /mnt/urbackup_storage
```
- To make the mount persistent across reboots, add it to /etc/fstab. First, get the UUID of the partition:
```
sudo blkid /dev/sdb1
```
  Copy the UUID value. Then edit /etc/fstab:
```
sudo nano /etc/fstab
```
  Add a line like this (replace YOUR_UUID_HERE with the actual UUID):
```
UUID=YOUR_UUID_HERE /mnt/urbackup_storage ext4 defaults 0 2
```
  Save and exit. Test the fstab entry by unmounting and remounting:
```
sudo umount /mnt/urbackup_storage
sudo mount -a
df -h # Check if /mnt/urbackup_storage is mounted
```
Install urbackup-server:
- Add the UrBackup PPA for Ubuntu:
```
sudo add-apt-repository ppa:uroni/urbackup
sudo apt update
```
- Install the server package:
```
sudo apt install urbackup-server
```
  During installation, when prompted for "Path to backup storage area:", enter /mnt/urbackup_storage.
- Verify the service is running:
```
sudo systemctl status urbackup-server.service
```

Configure Firewall (ufw): Ubuntu Server often comes with ufw (Uncomplicated Firewall) installed but inactive. Let's enable it and allow UrBackup ports.

sudo ufw allow ssh       # If not already enabled, for SSH access
sudo ufw allow 55413/tcp # UrBackup client (Internet mode)
sudo ufw allow 55414/tcp # UrBackup Web UI
sudo ufw allow 55415/udp # UrBackup client discovery
sudo ufw allow 55415/tcp # UrBackup client (LAN mode)
sudo ufw enable          # This will enable the firewall, answer 'y'
sudo ufw status verbose  # Check the status

Access the Web Interface:
- Open a web browser on your host machine (or any machine on the same network as the VM).
- Navigate to http://VM_IP_ADDRESS:55414 (e.g., http://192.168.1.123:55414).
- You should see the UrBackup web interface.
Initial Web UI Configuration:
- Change Admin Password:
  - Log in. The default admin password might be admin or something simple.
  - Go to the "Settings" tab.
  - Click on the "Users" sub-tab.
  - Select the admin user.
  - Enter a new, strong password in the "Password" and "Retype password" fields.
  - Click "Save". You might be logged out and asked to log in again with the new password.
- Verify Backup Storage Path:
  - Still in "Settings", go to "General" -> "Server".
  - Confirm that the "Backup storage path" is correctly set to /mnt/urbackup_storage. If not, correct it and save. The urbackup user (created by the package installation) needs write permissions to this directory. The installer usually handles this if the path is given during setup. If you set it manually later, you might need:
```
sudo chown -R urbackup:urbackup /mnt/urbackup_storage
sudo chmod -R 750 /mnt/urbackup_storage
```
Explore the Dashboard:
- Go back to the "Status" page. It will be mostly empty since no clients are connected yet.
- Familiarize yourself with the layout: server information, client list area, activity log.

Congratulations! You have successfully installed and performed the basic configuration of an UrBackup server on a Linux VM. It's now ready to accept client connections and start backing up data. Keep this VM running for the next workshops.

2. Installing and Configuring UrBackup Clients

With your UrBackup server up and running, the next step is to install the UrBackup client software on the machines you wish to protect. This section covers downloading client installers, installing them on Windows and Linux, and basic client management from the server's web interface.

Supported Operating Systems

UrBackup provides client software for a range of popular operating systems:

Windows: Extensive support for versions from Windows XP/Server 2003 up to the latest Windows 10/11 and Windows Server editions. It integrates with Volume Shadow Copy Service (VSS) for consistent backups of live systems.
Linux: Clients are available for various distributions. Installation can be via pre-compiled binaries, package managers (on some distributions), or by compiling from source. Supports file backups and raw image backups.
macOS: Client software is also available for macOS, typically offering similar functionality to the Linux client for file backups.

Downloading Client Installers

There are two primary ways to obtain the client installers:

From Your UrBackup Server Web Interface (Recommended):
- Navigate to your UrBackup server's web UI (http://YOUR_SERVER_IP:55414).
- On the "Status" page or sometimes via a dedicated "Add Client" or "Downloads" link (the UI may vary slightly with versions), you will find links to download the client installers for different operating systems.
- Downloading from your server is convenient because these clients are often pre-configured or easily discover your server on the local network.
From the Official UrBackup Website (urbackup.org):
- You can also download the latest client versions directly from the official website.
- This is useful if you want the absolute latest version or if you're preparing installers for a machine not yet on the same network as the server.
- If you download from the website, you might need to manually point the client to your server's IP address during or after installation.

Installing the Windows Client

The Windows client installation is straightforward using a graphical installer.

Download the Installer:
- Obtain the .exe installer for Windows from your UrBackup server or the official website.
Run the Installer:
- Execute the installer on the Windows machine you want to back up. You'll typically need administrative privileges.
- The setup wizard will appear.
Installer Options:
- Components: You'll usually have options to install:
  - UrBackup Client Backend: The core service that performs backups. This is essential.
  - UrBackup Client Tray Icon: A system tray utility that shows backup status, allows manual initiation of backups (if permitted by server settings), and access to client logs and settings. This is recommended for user workstations.
  - Potentially options related to CDFS/UDF file system drivers if you plan to use the restore CD functionality extensively with certain types of media.
- Silent Install: For deployment on multiple machines, the installer supports command-line switches for silent/unattended installation (e.g., /S). Refer to the UrBackup documentation for specific parameters.
Client Discovery / Server Configuration:
- Automatic Discovery (LAN): If the UrBackup server is on the same local network, the client will attempt to discover it automatically using UDP broadcasts on port 55415. If successful, it will connect, and the client will appear in the server's web UI shortly.
- Manual Server IP Entry: If automatic discovery fails (e.g., different subnets, firewall blocking broadcasts, or client downloaded from the website), the client might prompt you for the server's IP address or hostname during installation.
  - If not prompted during installation, you can configure it post-install via the client's tray icon settings (if installed) or by modifying configuration files (e.g., C:\Program Files\UrBackup\server_ident.txt - this file is created by the client to store the server's identity it should connect to).
  - The server_ident.txt file usually contains the server's hostname or IP. You may also need to configure C:\Program Files\UrBackup\urbackup\data\settings.cfg or use the urbackupclientctl command-line tool for advanced settings.
Initial Client Configuration (Server-Side):
- Once the client connects to the server, it will appear in the "Status" page of the server's web UI. Initially, it might have default settings.
- The server administrator then configures what to back up for this client:
  - Go to the "Settings" tab on the server UI.
  - Select the newly added client from the dropdown list (or manage it via group settings).
  - Under the "File Backups" tab for that client, specify the "Directories to backup". For example: C:\Users or more specific paths like C:\Users\JohnDoe\Documents;C:\Users\JohnDoe\Pictures. Use semicolons to separate multiple paths on Windows.
  - Under the "Image Backups" tab, specify the volumes to back up as images (e.g., C).
  - Adjust backup frequency, retention, and other settings as needed.
- Settings Inheritance: Clients can inherit settings from "Default" group settings or other custom groups you create on the server. You can also override these with client-specific settings.

Installing the Linux Client

Installing the UrBackup client on Linux can be done in a few ways. Using pre-compiled binaries or scripts provided by UrBackup is often the easiest for common distributions.

Download the Client:
- From your UrBackup server's web UI or the official website, download the appropriate Linux client installer script or binary package (e.g., UrBackup Client Linux X.Y.Z.sh).
- These are often shell scripts that detect your distribution and architecture and download the correct binaries.
Make the Installer Executable and Run It:
```
# Example for a downloaded .sh installer
chmod +x UrBackupClientLinux_*.sh
sudo ./UrBackupClientLinux_*.sh
```
The script will typically:
- Install necessary dependencies.
- Download and install the client binaries (e.g., to /usr/local/sbin/urbackupclientbackend and other locations).
- Set up an init script or systemd service file to start the client backend automatically.
- Attempt to discover the server.
Client Discovery and Server Configuration:
- Similar to the Windows client, the Linux client will try to auto-discover the server on the LAN.
- If discovery fails or if you need to specify the server manually, this is often done by creating or editing configuration files or using the urbackupclientctl command-line utility.
- The main configuration file for what to back up for file backups is often /etc/urbackup/backup_dirs.txt (or a similar path like /usr/local/etc/urbackup/backup_dirs.txt). Each path to back up should be on a new line. Example:
```
/home
/etc
/var/www
```
- Alternatively, and often preferably for centralized management, you can configure backup paths directly from the UrBackup server's web interface once the client connects, just like with Windows clients. The server settings will override local client configurations like backup_dirs.txt if "Separate settings for this client" is enabled and paths are defined on the server.
Compiling from Source (Advanced):
- For unsupported distributions or specific needs, you can compile the client from source code available on GitHub or the UrBackup website. This requires development tools (compiler, make, etc.) and libraries. The process typically involves ./configure, make, and sudo make install.
Headless Clients (Servers):
- For Linux servers that don't have a graphical environment, the client runs as a background daemon (urbackupclientbackend).
- Management and configuration are done either via server-side settings or by editing configuration files on the client and using urbackupclientctl.

Installing the macOS Client

The macOS client installation is generally similar to Windows, using a .dmg or .pkg installer.

Download: Get the macOS client installer from your UrBackup server or the official site.
Install: Open the downloaded file and follow the on-screen instructions. It will typically install the client application and a background service.
Configuration: Like other clients, it will attempt to discover the server. If manual configuration is needed, it might involve settings within the client application or modifying property list (.plist) files. Server-side configuration of backup paths and schedules is also the primary method.

Client Status and Management in the Web UI

Once clients are installed and have successfully connected to the UrBackup server, they will appear on the "Status" page of the web UI. For each client, you can typically see:

Name: The hostname of the client machine.
Online Status: Indicates if the client is currently connected to the server.
IP Address: The client's current IP address.
Last Seen: When the client last communicated with the server.
Last File Backup / Last Image Backup: Timestamp and status of the most recent backups.
Actions: You can often initiate manual backups (file or image) for a client from here.
Logs: Access detailed logs for each client's backup activities.

Clicking on a client's name usually takes you to a more detailed view for that client, showing backup history, current activity, and allowing access to restore options.

The "Settings" tab in the server UI is where you define what gets backed up for each client (or groups of clients), backup intervals, retention policies, and other client-specific configurations. Centralized management via the server UI is one of UrBackup's key strengths.

Workshop Installing and Configuring UrBackup Clients

Project: Install UrBackup Client on a Windows VM and a Linux VM, connect them to your UrBackup server, and configure basic backup paths.

Objective: To have two client machines (one Windows, one Linux) successfully connected to your UrBackup server and listed in its web interface, with initial backup paths configured.

Prerequisites:

Your UrBackup server VM (from Workshop 1) should be running and accessible on the network. Note its IP address.
Virtualization software (VirtualBox, VMware, etc.).
A Windows ISO (e.g., Windows 10 evaluation, or any licensed version you have).
A Linux Desktop ISO (e.g., Ubuntu Desktop, Linux Mint). This makes it easy to use a browser on the Linux client for downloading, though a server version would also work.

Steps:

Part 1: Setting up the Client VMs

Create a Windows Client VM:
- In your virtualization software, create a new VM.
- Allocate resources (e.g., 2 vCPUs, 2-4GB RAM, 40-60GB disk).
- Install Windows using your ISO. Complete the out-of-box experience (OOBE).
- Ensure it has network connectivity (Bridged Adapter recommended, same network as UrBackup server).
- Install guest additions/VMware Tools for better performance and usability.
Create a Linux Client VM:
- Create another new VM.
- Allocate resources (e.g., 1-2 vCPUs, 1-2GB RAM, 20-30GB disk).
- Install your chosen Linux Desktop distribution.
- Ensure it has network connectivity (Bridged Adapter).
- Install guest additions/VMware Tools.
- It's helpful if this Linux VM has a graphical environment and a web browser for this workshop.

Part 2: Installing the UrBackup Client on Windows VM

Download UrBackup Client for Windows:
- On your Windows Client VM, open a web browser.
- Navigate to your UrBackup server's web interface: http://YOUR_URBACKUP_SERVER_IP:55414.
- On the "Status" page (or look for a download link), find and download the "UrBackup Client for Windows" installer (.exe file).
Install the Windows Client:
- Run the downloaded installer on the Windows Client VM.
- Accept the license agreement.
- Choose components to install (default is usually fine: Backend and Tray Icon).
- The installer should automatically detect your UrBackup server if it's on the same network. If it asks for the server name/IP, ensure it's correct or enter it if blank.
- Complete the installation. The UrBackup tray icon should appear in the system tray.
Verify Connection on Server:
- Go back to your UrBackup server's web interface.
- Refresh the "Status" page. Your Windows Client VM should appear in the list of clients, likely with an "Online" status. It might say "No recent backup" or similar.
Configure Backup Paths for Windows Client (Server-Side):
- In the UrBackup server UI, go to "Settings".
- From the "Client" dropdown menu, select your newly added Windows client.
- Go to the "File Backups" tab for this client.
- In the "Directories to backup" field, enter a path. For testing, you can use something simple like C:\Users\YourUserName\Documents (replace YourUserName with the actual user folder on the VM). Or, create a test folder C:\TestData_Win and put a few files in it, then use C:\TestData_Win as the path.
- Click "Save" at the bottom of the settings page.

Part 3: Installing the UrBackup Client on Linux VM

Download UrBackup Client for Linux:
- On your Linux Client VM, open a web browser.
- Navigate to your UrBackup server's web interface: http://YOUR_URBACKUP_SERVER_IP:55414.
- Find and download the "UrBackup Client for Linux" installer script (usually a .sh file). Save it to your Downloads folder or home directory.
Install the Linux Client:
- Open a terminal on the Linux Client VM.
- Navigate to the directory where you downloaded the script (e.g., cd ~/Downloads).
- Make the script executable: chmod +x UrBackupClientLinux_*.sh (replace * with the actual version).
- Run the script with sudo: sudo ./UrBackupClientLinux_*.sh.
- The script will ask some questions or perform actions. Follow its prompts. It should install the client and start the urbackupclientbackend service.
- It will attempt to discover the server.
Verify Connection on Server:
- Go back to your UrBackup server's web interface.
- Refresh the "Status" page. Your Linux Client VM should now also appear in the list of clients.
Configure Backup Paths for Linux Client (Server-Side):
- In the UrBackup server UI, go to "Settings".
- From the "Client" dropdown menu, select your newly added Linux client.
- Go to the "File Backups" tab for this client.
- In the "Directories to backup" field, enter a path. For testing, use your user's home directory documents folder, e.g., /home/yourlinuxuser/Documents. Or, create a test folder /home/yourlinuxuser/testdata_linux and put a few files in it, then use this path.
- Click "Save".

Part 4: Check Client Status

Review Client Status on Server:
- On the UrBackup server's "Status" page, both your Windows and Linux clients should be listed.
- They might show "Online, idle" or similar.
- The "Last file backup" column will likely indicate that no backup has been performed yet.

You have now successfully installed UrBackup clients on both a Windows and a Linux virtual machine, connected them to your server, and configured basic backup paths from the server's web interface. In the next workshop, you'll perform your first backups and restores.

3. Performing Your First Backups and Restores

Now that your UrBackup server is operational and clients are connected, it's time to perform the core functions: backing up data and restoring it. This section explains the types of backups UrBackup offers, how to initiate and monitor them, understand logs, and perform basic file and image restores.

Types of Backups

UrBackup supports two primary types of backups, each serving different purposes:

File Backups:
- What they are: File backups involve selecting specific directories and files on the client machine to be copied to the UrBackup server. UrBackup intelligently backs up only new or changed files (or parts of files) after the initial full backup.
- When to use them: Ideal for protecting user data (documents, photos, project files), application configurations, and any data where granular, file-level access and versioning are important.
- Pros:
  - Efficient for data that changes frequently but where the overall system structure remains stable.
  - Allows for easy browsing and restoration of individual files or folders from different points in time.
  - Typically faster for daily operations after the initial full backup due to incremental transfers.
  - Excellent storage efficiency due to deduplication, especially for text-based data or multiple users with similar files.
- Cons:
  - Does not back up the operating system, installed applications, or system state directly.
  - Restoring a full system from only file backups requires reinstalling the OS and applications first, then restoring the data files, which can be time-consuming.
Image Backups (Volume Backups):
- What they are: Image backups (also known as volume or disk backups) create a complete, block-by-block snapshot of an entire hard disk partition or volume (e.g., the C: drive on Windows, or /dev/sda1 on Linux).
- When to use them: Primarily used for disaster recovery, specifically for Bare Metal Restore (BMR). If a hard drive fails or a system becomes unbootable, an image backup can restore the entire system to a working state on a new or repaired drive, including the OS, applications, settings, and data.
- Formats:
  - Windows: UrBackup typically uses Microsoft's VHD (Virtual Hard Disk) or VHDZ (compressed VHD) format, leveraging the Volume Shadow Copy Service (VSS) to ensure data consistency even while the system is running. VHD/VHDZ files can often be mounted directly in Windows or other OSes to browse their contents.
  - Linux: Image backups are usually created in a raw format (a direct byte-for-byte copy of the partition).
- Pros:
  - Enables fast and complete system recovery.
  - Captures everything on the backed-up volume, reducing the chance of missing critical system files.
  - UrBackup also performs incremental image backups, saving space and time after the initial full image.
- Cons:
  - Initial full image backups can be very large and take a considerable amount of time and network bandwidth.
  - Restoring individual files from an image backup can be slightly less direct than from a file backup, though UrBackup's web interface often allows browsing image contents for file-level restores.
  - Image backups consume more storage space on the server compared to file backups of only user data, even with incrementals.

A common strategy is to use both: frequent file backups for user data and less frequent image backups for system recovery.

Initiating Backups

UrBackup offers several ways to start a backup process:

Automatic (Scheduled) Backups:
- This is the primary mode of operation. In the UrBackup server settings (either global, group, or client-specific), you define:
  - Backup intervals: How often incremental and full backups should occur (e.g., incremental file backups every 2 hours, full file backups every 7 days; incremental image backups every 24 hours, full image backups every 30 days).
  - Backup windows: The times of day/days of the week when backups are permitted to run.
- Clients automatically start backups according to these schedules when they are online and within the allowed backup window.
Manual Backups from the Server Web UI:
- Administrators can trigger backups on demand:
  - Go to the "Status" page on the server web UI.
  - For the desired client, there are usually buttons or actions like "Start file backup" or "Start image backup" (these might be under a dropdown menu like "Actions" or "...")
  - You can choose between an incremental or a full backup.
Manual Backups from the Client UI (if enabled and tray icon installed):
- If the UrBackup client tray icon is installed (common on Windows workstations) and server settings permit, users can right-click the tray icon and select options to start a file or image backup for their own machine.
- This permission is controlled on the server (Settings -> Client -> Permissions -> Allow client to start backups).

Monitoring Backup Progress

You can monitor the progress of ongoing backups in several places:

Server Web UI - "Status" Page:
- Clients with active backups will show their progress (e.g., percentage complete, current file being transferred, estimated time remaining, transfer speed).
Server Web UI - "Activities" Page:
- This page provides a real-time log of server and client activities, including backup indexing, file transfers, and completion status.
Client Tray Icon (if installed):
- The tray icon usually changes appearance or shows notifications during a backup. Hovering over it or opening its status window often provides progress details.
Client Logs:
- The client software maintains its own logs, which can be viewed for detailed progress (e.g., on Windows, via the tray icon; on Linux, often in /var/log/urbackupclient.log or similar).

Understanding Backup Logs

Logs are crucial for verifying successful backups and troubleshooting any issues.

Server Logs:
- Accessible via the "Logs" tab in the server web UI.
- Shows activities for all clients, server events, errors, and warnings.
- The main server log file is typically /var/log/urbackup.log on Linux.
- Look for messages indicating successful completion (e.g., "Backup Succeeded") or any error messages.
Client Logs:
- Provide more detailed information about the backup process on a specific client.
- Windows: Accessible via the tray icon ("Log File") or in C:\Program Files\UrBackup\urbackup.log.
- Linux: Typically located in /var/log/urbackupclient.log or /usr/local/var/log/urbackupclient.log.
- Client logs show details like files being scanned, VSS operations (Windows), files transferred, and any errors encountered on the client side (e.g., permission denied for a file).

Common things to look for in logs:

Successful start and end of backups.
Number of files/amount of data transferred.
Any errors related to accessing files (permissions), VSS failures (Windows), network connectivity issues, or disk space problems on the server.

Restoring Files

Restoring files is a common task. UrBackup provides a user-friendly way to do this via the web interface.

Using the Web Interface (Most Common):
- Log in to the UrBackup server web UI.
- Go to the "Backups" tab.
- Select the client from which you want to restore files.
- You'll see a list of available backups (both file and image backups). Choose a file backup from the desired date/time.
- You can now browse the directory structure of that backup, just like a file explorer.
- Navigate to the file(s) or folder(s) you want to restore.
- Restore Options:
  - Select items and click "Restore selected": This will attempt to restore the selected items back to their original location on the client machine. The client must be online.
  - Select items and click "Download selected as ZIP": This will package the selected files/folders into a ZIP archive and download it to the computer you are using to access the web interface. This is useful for quick access or if the client is offline.
  - Some versions may offer "Restore to client" with options to specify an alternative path on the client.
Using the Client (If available and configured):
- For some clients (especially Windows with the tray icon), there might be a restore option directly accessible from the client interface. This functionality might be limited compared to the server's web UI and depends on server permissions.

Restoring Images (Brief Overview - More in Advanced Section)

Restoring an entire system from an image backup is a more involved process, typically used for bare-metal recovery.

Bootable Restore Media:
- UrBackup provides a Restore CD/USB image (downloadable from the UrBackup website or sometimes from your server UI).
- You need to burn this ISO to a CD/DVD or write it to a USB flash drive to create bootable recovery media.
Booting from Restore Media:
- Boot the target machine (the one you want to restore to, which could be the original machine with a new hard drive, or a different compatible machine) from the UrBackup Restore CD/USB.
Restore Process:
- The restore environment will load. It will guide you through:
  - Connecting to your UrBackup server (network configuration might be needed).
  - Selecting the client whose image you want to restore.
  - Choosing the specific image backup (by date).
  - Selecting the target hard drive on the machine where the image will be restored.
  - Initiating the restore. This process can take a long time, depending on the image size and network/disk speed.
- Detailed steps for Bare Metal Restore will be covered in the "Advanced Restore Techniques" section.

For now, the focus is on understanding that image restores are possible and require special boot media. You can also often mount VHD/VHDZ image backups on a running system to extract individual files if needed, providing another way to access data from images without a full system restore.

Workshop Performing Your First Backups and Restores

Project: Backup and Restore Files for Both Your Windows and Linux Client VMs.

Objective: To perform a successful file backup for each client VM, verify its completion, and then restore a deleted file to each client, confirming data integrity.

Prerequisites:

Your UrBackup server VM is running.
Your Windows Client VM and Linux Client VM (from Workshop 2) are running, connected to the UrBackup server, and have basic backup paths configured (e.g., C:\TestData_Win and /home/yourlinuxuser/testdata_linux).

Steps:

Part 1: Prepare Test Data and Initiate Backups

Create Test Data on Windows Client VM:
- On your Windows Client VM, navigate to the folder you configured for backup (e.g., C:\TestData_Win). If it doesn't exist, create it.
- Inside this folder, create a few sample files:
  - A text file: my_document.txt (add some unique text inside).
  - Another text file: important_notes.log (add some different text).
  - (Optional) Copy a small image file into this folder.
Create Test Data on Linux Client VM:
- On your Linux Client VM, navigate to the folder you configured for backup (e.g., /home/yourlinuxuser/testdata_linux). If it doesn't exist, create it using mkdir ~/testdata_linux.
- Inside this folder, create a few sample files using the terminal or a text editor:
  - echo "This is a test file for Linux backup." > ~/testdata_linux/linux_test1.txt
  - echo "Another important piece of information." > ~/testdata_linux/linux_notes.md
  - (Optional) Copy a small script or configuration file into this folder.
Initiate File Backup for Windows Client (from Server UI):
- Open the UrBackup server web UI in your browser.
- Go to the "Status" page.
- Find your Windows Client VM in the list.
- Click the "Actions" button (or "..." menu) associated with the Windows client and select "Start file backup". Choose "Incremental file backup" (since it's the first, it will effectively be a full backup of the configured paths).
- Observe: The status for the Windows client should change to show the backup in progress (e.g., "Indexing...", "Transferring...").
Monitor Windows Client Backup:
- Keep an eye on the "Status" page and the "Activities" page for progress.
- You can also check the UrBackup tray icon on the Windows Client VM for its status.
- Wait for the backup to complete. It should eventually say "ok" or show a recent successful backup time. Check the "Logs" for any errors.
Initiate File Backup for Linux Client (from Server UI):
- Once the Windows client backup is complete (or if your server can handle concurrent backups), find your Linux Client VM on the "Status" page.
- Click its "Actions" button and select "Start file backup" (Incremental).
- Observe: The status for the Linux client should change.
Monitor Linux Client Backup:
- Monitor its progress on the "Status" and "Activities" pages.
- Wait for the backup to complete successfully. Check "Logs".

Part 2: Perform File Restores

Simulate Data Loss on Windows Client VM:
- On your Windows Client VM, go to C:\TestData_Win.
- Delete one of the files you created, for example, my_document.txt. (Move it to Recycle Bin and empty it, or Shift+Delete).
Restore Deleted File to Windows Client (from Server UI):
- In the UrBackup server web UI, go to the "Backups" tab.
- Select your Windows Client VM from the "Client" dropdown.
- You should see at least one file backup listed. Click on it (usually the most recent one).
- Browse the backup: Click through the C drive, then TestData_Win (or your configured path).
- You should see the files that were backed up, including my_document.txt.
- Select the checkbox next to my_document.txt.
- Click the "Restore selected" button.
- A confirmation might appear. Proceed with the restore.
- Observe: The "Activities" page on the server should show the restore operation.
Verify Restoration on Windows Client VM:
- On your Windows Client VM, check the C:\TestData_Win folder.
- The my_document.txt file should reappear.
- Open it and verify its content is the same as what you originally created.
Simulate Data Loss on Linux Client VM:
- On your Linux Client VM, open a terminal.
- Navigate to your test data directory: cd ~/testdata_linux.
- Delete one of the files: rm linux_test1.txt.
Restore Deleted File to Linux Client (from Server UI):
- In the UrBackup server web UI, go to the "Backups" tab.
- Select your Linux Client VM from the "Client" dropdown.
- Click on its most recent file backup.
- Browse the backup: Click through home, then yourlinuxuser, then testdata_linux.
- You should see the backed-up files, including linux_test1.txt.
- Select the checkbox next to linux_test1.txt.
- Click "Restore selected".
Verify Restoration on Linux Client VM:
- On your Linux Client VM, in the terminal, list the files in ~/testdata_linux: ls -l ~/testdata_linux.
- The linux_test1.txt file should be present again.
- View its content: cat ~/testdata_linux/linux_test1.txt. Verify it's correct.

Part 3: (Optional) Explore Image Backup (Initiation only)

Initiate Image Backup for Windows Client (from Server UI):
- First, ensure an image backup path is configured: On the server UI, go to "Settings", select your Windows client, go to the "Image Backups" tab. In "Volumes to backup", enter C. Save settings.
- On the "Status" page, for your Windows client, click "Actions" -> "Start image backup" (Incremental).
- Observe: This will take much longer than the file backup. Monitor its progress. You don't need to wait for it to complete fully for this workshop if time is limited, but observe how it starts and progresses. Note the difference in data size being processed compared to the file backup.
- Image backup restoration (Bare Metal Restore) is a more advanced topic covered later.

Congratulations! You have successfully performed your first file backups and restores using UrBackup. You've learned how to initiate backups, monitor their progress, and recover data – the fundamental operations of any backup system. This practical experience forms the basis for understanding more advanced features.

4. Advanced Server Configuration

Once you're comfortable with basic UrBackup server setup and operations, you can delve into more advanced configurations to tailor the system to your specific needs, enhance security, and improve management. This section covers user management, backup storage strategies, network settings for internet clients, email notifications, and server maintenance tasks.

User Management and Permissions

By default, UrBackup has a single admin user with full control. For environments where multiple people might need access, or if you want to grant limited access (e.g., for users to restore their own files), UrBackup provides a flexible user and permission system.

Creating Users and Groups:
- Access the server web UI and navigate to "Settings" -> "Users".
- You can create new individual users or groups. Groups are useful for applying a common set of permissions to multiple users.
- When creating a user, you'll set a username and password.
Assigning Permissions:
- Permissions are granular and can be assigned to users or groups.
- Permissions are typically managed per client or for "all clients." This means you can grant a user rights over specific machines they are responsible for.
- Common permissions include:
  - status: View the status and logs of clients.
  - settings: Change settings for clients (e.g., backup paths, schedules).
  - restore: Perform restores for clients.
  - backup_inc: Start incremental backups.
  - backup_full: Start full backups.
  - logs: View detailed logs.
  - all_nobackup: All permissions except starting new backups (useful for a monitoring role).
  - all: All permissions (administrator rights for the selected client(s) or all clients).
- You can assign these rights specifically for "File backups" or "Image backups."
Use Cases for Different User Roles:
- System Administrator (admin): Has all rights for all clients and server settings.
- Departmental IT / Team Lead: Might have all rights for clients belonging to their department/team but no access to server-wide settings or other departments' clients.
- End-Users (Self-Service Restore): Can be given status and restore rights only for their own machine. This allows them to recover their own files without administrator intervention. This requires careful consideration of data privacy if users share machines.
- Auditor/Monitoring Role: Might have status and logs rights for all clients but no ability to change settings or perform restores.

Backup Storage Management

Effectively managing your backup storage is crucial for long-term reliability and performance.

Choosing the Right Filesystem for Backups:
- The filesystem on which your Backup storage path resides can impact performance, reliability, and features.
- Btrfs: A modern Linux filesystem with built-in support for checksums (data integrity), compression, and efficient CoW (Copy-on-Write) snapshots. UrBackup can leverage Btrfs snapshots for its internal workings, potentially making operations like consolidating incremental backups faster and more space-efficient. If using Btrfs, UrBackup can store files directly without creating VHDs for file backups if certain options are enabled, leading to space savings.
- ZFS: Another advanced filesystem (common on FreeBSD, also available for Linux via ZFSonLinux) offering excellent data integrity, snapshots, compression, and RAID capabilities (ZRAID). It's very robust but can be more complex to set up and manage and is generally more RAM-hungry.
- XFS: A high-performance journaling filesystem, well-suited for large files and large storage volumes. It's stable and mature.
- ext4: The default filesystem for many Linux distributions. It's reliable, well-tested, and generally performs well. It lacks some of the advanced features of Btrfs or ZFS but is simpler to manage.
- NTFS (Windows): If running the UrBackup server on Windows, your backup storage will likely be on an NTFS volume. NTFS supports features like file compression and shadow copies, but the advanced filesystem benefits are more pronounced with Linux-based filesystems like Btrfs/ZFS when combined with UrBackup's architecture.
Configuring Multiple Backup Storage Paths:
- UrBackup allows you to add multiple storage paths (Settings -> General -> Server -> Additional backup storage paths).
- This can be useful for:
  - Tiered Storage: Using faster, more expensive storage (SSDs) for recent backups or metadata, and slower, larger capacity storage (HDDs) for older archives. (UrBackup doesn't explicitly manage tiers this way out-of-the-box, but you could manually assign specific clients or backup types to different paths).
  - Expanding Storage: Adding a new disk/array without having to move existing data.
  - Client Grouping: Directing backups from different groups of clients to different storage locations.
- UrBackup will distribute new client data across the available paths. However, an existing client's data will generally remain on the path it was initially assigned to unless manually moved.
Understanding Retention Settings:
- Located under Settings -> Client (or Default) -> File Backups / Image Backups.
- Minimum number of full file/image backups: UrBackup will keep at least this many full backups.
- Maximum number of full file/image backups: UrBackup will delete older full backups once this limit is exceeded (and minimums are met).
- Minimum number of incremental file/image backups: Keeps at least this many incrementals associated with their respective fulls.
- Maximum number of incremental file/image backups: Deletes older incrementals.
- These settings interact to define your overall retention window (how far back you can restore). Plan these carefully based on your Recovery Point Objectives (RPO) and storage capacity.
Archive Settings and Data Retention Policies:
- UrBackup has an "Archiving" feature (Settings -> General -> Server -> Archiving).
- If enabled, UrBackup can periodically archive backups that meet certain criteria (e.g., older than X days).
- Archived backups are typically marked as read-only and might be subject to different cleanup rules or could be moved to slower/cheaper storage manually. This feature helps in managing long-term retention.

Network Settings

Fine-tuning network settings is important for performance and for enabling backups over the internet.

Configuring Internet/WAN Backups:
- UrBackup clients can back up to a server over the internet. This is extremely useful for remote laptops or off-site backups.
- Server Configuration:
  - The UrBackup server must be reachable from the internet on its designated ports (TCP 55413, TCP/UDP 55415).
  - NAT & Port Forwarding: If your server is behind a NAT router (most home/office setups), you need to configure port forwarding on your router to direct incoming traffic on these ports to the internal IP address of your UrBackup server.
  - Firewall: Ensure your server's firewall and any network firewalls allow these ports.
  - Dynamic DNS (DDNS): If your server's public IP address is dynamic (changes periodically), set up a DDNS service (e.g., No-IP, Dynu, DuckDNS). This gives your server a consistent hostname (e.g., myurbackupserver.ddns.net) that clients can use.
- Client Configuration (Internet Mode):
  - When installing the client or in its settings, you specify the server's public hostname (from DDNS) or public IP address.
  - The client needs to be configured to operate in "Internet mode." This usually involves setting the internet_mode_enabled=true and internet_server=your.ddns.hostname parameters in the client's settings or using the urbackupclientctl set-settings command.
  - In the UrBackup server settings (Settings -> General -> Server -> Internet settings), you can configure specific parameters for internet clients, like separate bandwidth throttles or different backup windows.
- Security: Backing up over the internet requires robust security. Ensure your server OS is hardened, use strong passwords, and consider enabling encrypted transfer for backups. HTTPS for the web UI is also essential if it's exposed.
Bandwidth Throttling:
- To prevent UrBackup from consuming all available network bandwidth (especially important for internet clients or on busy LANs):
- Server-Side: Settings -> General -> Server -> "Max backup speed for local network" and "Max backup speed for internet clients". You can set speeds in Kbit/s or Mbit/s.
- Client-Side: Settings -> Client (or Default) -> General -> "Max backup speed". This allows per-client or per-group throttling.
- The most restrictive setting (client or server) usually takes precedence.
Client-Specific Network Settings:
- You can configure clients to connect via a specific IP/hostname, which is useful if DNS resolution is an issue or if you have multiple network interfaces on the server.

Email Notifications and Reporting

Getting notified about backup status is critical for proactive management.

Configuring SMTP Settings:
- Navigate to Settings -> Mail on the server UI.
- You need to provide details for an SMTP server that UrBackup can use to send emails:
  - Server name: SMTP server address (e.g., smtp.gmail.com, smtp.office365.com, or your local mail relay).
  - Server port: SMTP port (e.g., 587 for TLS, 465 for SSL, 25 for unencrypted).
  - User name & Password: Credentials for the SMTP server if authentication is required.
  - Sender email address: The "From" address for notification emails.
  - Encryption: Choose None, SSL, or TLS based on your SMTP provider's requirements.
- Send a test email to verify the configuration.
Setting up Email Alerts:
- Once SMTP is configured, go to Settings -> Mail -> "When to send e-mails".
- You can configure UrBackup to send emails for various events:
  - When a backup succeeds.
  - When a backup fails or completes with errors (most important!).
  - When a client computer has not connected for a certain period.
  - Server issues (e.g., low disk space).
  - Specify recipient email addresses. Multiple addresses can often be comma-separated.
Daily/Weekly Backup Reports:
- UrBackup can send summary reports of backup activities.
- Configure this under Settings -> Mail -> "Report settings". You can choose the frequency (daily, weekly) and content of these reports.

Server Maintenance

Regular maintenance helps keep your UrBackup server running smoothly.

urbackup_srv remove-unknown command:
- This command-line tool (run on the server) is used to remove data for clients that are no longer configured on the server or data that is otherwise "orphaned."
- sudo urbackup_srv remove-unknown
- It's good practice to run this periodically, especially after deleting clients, to reclaim storage space. Always ensure you have backups of critical data before running cleanup commands.
urbackup_srv cleanup command:
- This command helps clean up old backups based on your retention settings and can also remove temporary files.
- sudo urbackup_srv cleanup -a <percentage_of_free_space_to_keep> (e.g., -a 10%)
- The server usually runs cleanup tasks automatically based on its internal schedule, but manual runs can be useful.
Database Maintenance (Vacuuming):
- UrBackup uses SQLite databases to store metadata. Over time, these databases can become fragmented.
- sudo urbackup_srv vacuum
- This command optimizes the database, potentially improving performance and reclaiming some disk space used by the database itself. It's good to run this occasionally (e.g., monthly or quarterly), but be aware it can take time and make the server unresponsive during the process.
Updating the UrBackup Server Software:
- Regularly check for and install updates to the UrBackup server package. Updates often include bug fixes, security patches, and new features.
- If installed via a package manager (like apt on Debian/Ubuntu):
```
sudo apt update
sudo apt install urbackup-server
```
- Always read release notes before updating, and consider backing up your UrBackup server configuration and database (though UrBackup often internally backs up its DB before upgrades). The backup storage itself is your primary data.

Workshop Advanced Server Configuration

Project: Configure Email Notifications for Backup Failures, Create a Limited User Role, and Prepare for Internet Client Access (Port Forwarding).

Objective: To enhance server management by setting up email alerts, implementing role-based access control, and configuring the network for potential internet-based clients.

Prerequisites:

Your UrBackup server VM is running.
Access to an SMTP server for sending emails. You can use:
- Your personal email provider's SMTP (e.g., Gmail, Outlook.com - you might need to enable "less secure app access" or generate an app-specific password for Gmail).
- A transactional email service like SendGrid (offers a free tier), Mailgun, etc.
- A local SMTP relay if you have one set up.
Access to your router's configuration page (if you want to do the actual port forwarding part).

Steps:

Part 1: Configure Email Notifications

Gather SMTP Server Details:
- You'll need: SMTP server address, port, username, password, and encryption type (SSL/TLS).
- Example for Gmail (requires "App Password" if 2FA is enabled, or "Less Secure App Access" enabled if not):
  - Server: smtp.gmail.com
  - Port: 587 (for TLS) or 465 (for SSL)
  - Username: Your Gmail address (e.g., your.email@gmail.com)
  - Password: Your Gmail password or App Password.
  - Encryption: TLS (STARTTLS) or SSL.
Configure SMTP in UrBackup Server UI:
- Log in to your UrBackup server web UI as admin.
- Go to "Settings" -> "Mail".
- Fill in the SMTP server details:
  - Server name: (e.g., smtp.gmail.com)
  - Server port: (e.g., 587)
  - User name for server: (e.g., your.email@gmail.com)
  - Password for server: Your SMTP password.
  - Sender email address: (e.g., urbackup-server@yourdomain.com or your actual email)
  - Recipient email address for test e-mail: Your email address to receive the test.
  - Encryption type for SMTP connection: Choose appropriate (e.g., "TLS").
- Click "Save".
Send a Test Email:
- After saving, click the "Send test e-mail" button.
- Check your recipient email inbox (and spam folder) for the test email from UrBackup.
- If it fails, double-check your SMTP settings, credentials, and any firewall rules that might block outgoing SMTP traffic from your UrBackup server VM. Check the UrBackup server logs for errors.
Configure Alerts for Backup Failures:
- Still in "Settings" -> "Mail".
- Under "When to send e-mails for backups", ensure options like "A backup failed or completed with errors" or similar are checked.
- Enter one or more recipient email addresses in "Recipients of backup reports".
- You can also configure "Report settings" to send daily/weekly summaries.
- Click "Save".
- (To test this, you'd need to make a backup fail, e.g., by trying to back up an inaccessible path on a client. You can try this later if desired.)

Part 2: Create a Limited User Role

Create a New User:
- Go to "Settings" -> "Users".
- Click "Add user".
- Username: limited_user
- Password: Set a strong password (e.g., P@sswordL1m1t3d!).
- Retype password: Confirm the password.
- Click "Save".
Assign Permissions to the New User:
- The new user limited_user will appear in the list.
- You will now see a matrix of clients (including "All clients" and your specific client VMs) and permissions.
- Let's assume you want this user to only be able to view status and restore files for your "Windows Client VM" (use the actual name of your client).
- Find the row for your Windows Client VM.
- In that row, check the boxes for the following permissions for limited_user:
  - status
  - restore (under "File backups")
  - restore (under "Image backups", if you also want them to be able to initiate image restores for that client)
  - logs
- Ensure all other boxes for this user, especially settings and all, are unchecked for all clients (including "All clients").
- Click "Save" at the bottom of the Users page.
Test the Limited User Account:
- Log out of the admin account in the UrBackup web UI.
- Log in as limited_user with the password you set.
- Verify:
  - Can this user see the "Status" page?
  - If they go to "Backups", can they select the Windows Client VM and see its backups?
  - Can they browse a backup and see the "Restore selected" option? (You don't have to perform a restore).
  - Can they access "Settings"? (They shouldn't be able to, or should see very restricted settings if any).
  - Can they see/manage the Linux Client VM? (They shouldn't, based on the permissions we set).
- Log out and log back in as admin.

Part 3: Prepare for Internet Client Access (Network Configuration)

This part involves understanding what's needed. Actually making your server publicly accessible has security implications and might be beyond a quick workshop if you're on a complex network.

Identify UrBackup Ports:
- As a reminder, UrBackup uses:
  - TCP 55413: Client communication (Internet mode)
  - TCP 55414: Web interface
  - UDP 55415: Client discovery (LAN)
  - TCP 55415: Client communication (LAN mode)
- For internet clients, you primarily need to expose TCP 55413 and TCP 55414 (if you want to access the web UI from the internet, which requires caution and HTTPS). Port 55415 (TCP) is also used for internet communication if the client is set to "internet mode" but can connect using this more direct method.
Find Your UrBackup Server's Internal IP Address:
- On your UrBackup Server VM, use ip a to get its current IP address (e.g., 192.168.1.123). This is the IP you'll forward ports to.
(Conceptual/Actual) Configure Port Forwarding on Your Router:
- Access Your Router: Open a web browser and go to your router's admin page (commonly 192.168.1.1 or 192.168.0.1). Log in with your router's admin credentials.
- Locate Port Forwarding Settings: This section is usually called "Port Forwarding," "Virtual Servers," "NAT Forwarding," or similar.
- Create Port Forwarding Rules: You'll need to create rules for:
  - Rule 1 (Client Communication):
    - Service Name/Description: UrBackup_Client
    - External Port(s): 55413 (and potentially 55415 if you intend to use it for internet clients)
    - Internal Port(s): 55413 (and 55415)
    - Protocol: TCP
    - Internal IP Address: Your UrBackup Server VM's IP (e.g., 192.168.1.123)
    - Enable the rule.
  - Rule 2 (Web Interface - Use with Caution, HTTPS Recommended):
    - Service Name/Description: UrBackup_WebUI
    - External Port(s): 55414
    - Internal Port(s): 55414
    - Protocol: TCP
    - Internal IP Address: Your UrBackup Server VM's IP
    - Enable the rule.
    - Security Note: Exposing the HTTP web interface directly to the internet is risky. It's strongly recommended to set up HTTPS (covered in a later security workshop) if you need external web UI access.
- Save/apply the changes on your router.
(Conceptual) Set Up Dynamic DNS (DDNS):
- If your public IP address (assigned by your ISP) changes, clients won't be able to find your server reliably.
- Sign up for a DDNS service (e.g., No-IP.com, DuckDNS.org - many offer free plans).
- Create a hostname (e.g., myurbackup.ddns.net).
- Install a DDNS update client on your UrBackup server VM or on another machine on your network, or configure DDNS directly on your router if it supports your chosen DDNS provider. This client will keep your DDNS hostname updated with your current public IP address.
Configure UrBackup Server for Internet Mode (Optional settings):
- In the UrBackup server UI, go to "Settings" -> "General" -> "Server" -> "Internet settings".
- Here you can set:
  - Hostname for internet server: Enter your DDNS hostname here (e.g., myurbackup.ddns.net). This helps the server generate correct client installers for internet mode.
  - Specific bandwidth limits for internet clients.
  - Enable/disable "Allow backups via internet".
- Save changes.

Testing Internet Client (Beyond this Workshop's Scope for full setup): To truly test this, you would need a client machine on a different network (e.g., a laptop tethered to a mobile hotspot or at a friend's house). You would install the UrBackup client on it and configure it to connect to your DDNS hostname. This setup can be complex and requires careful security considerations (especially HTTPS for the web UI).

For this workshop, the main goals were to understand the port forwarding requirements and to successfully set up email notifications and user roles. The internet client setup is a conceptual preparation.

You have now configured several advanced aspects of your UrBackup server, making it more manageable and robust. Email notifications will keep you informed, user roles allow for delegated access, and you understand the steps needed for potential internet client support.

5. Fine-Tuning Client Backups

Once clients are connected and basic backups are running, you can significantly enhance the efficiency and specificity of your backup strategy by fine-tuning client backup settings. This involves using client-specific configurations, applying include/exclude filters, understanding advanced options for both file and image backups, and managing client identities.

Client-Specific Settings vs. Group Settings

UrBackup offers a hierarchical approach to client configuration:

Global Default Settings: Found under "Settings" -> "Default" (or sometimes just under the main "Settings" categories if no client is selected). These apply to all clients unless overridden.
Group Settings: You can create client groups (Settings -> Users -> "Add group" or manage existing ones). Clients can be assigned to a group. The group can have its own set of backup settings that override the global defaults for members of that group. This is useful for applying consistent policies to, for example, all "Linux Servers" or all "Marketing Department Workstations."
Client-Specific Settings: For any individual client, you can enable "Separate settings for this client." This allows you to define unique backup parameters (paths, schedules, retention, etc.) that override both group and global default settings for that particular client.

When to use which:

Global Defaults: For settings that are common across your entire backup environment (e.g., default backup window if most clients should adhere to it, default number of incremental backups).
Group Settings: When you have distinct categories of clients that require similar, but different, configurations from the global default or other groups (e.g., servers might need more frequent backups and longer retention than workstations; Windows clients might have different default image backup settings than Linux clients).
Client-Specific Settings: For individual machines with unique requirements that don't fit neatly into a group or deviate from the group's standard (e.g., a specific developer's machine needing a very particular set of folders backed up, or a critical server needing a more aggressive backup schedule than other servers in its group).

Start with sensible global defaults, use groups for logical client categorizations, and resort to client-specific settings only when necessary to keep management streamlined.

Advanced File Backup Settings

These settings are typically found under "Settings" -> (Select Client or Default/Group) -> "File Backups".

Include/Exclude Filters: Powerful for precisely controlling what gets backed up.
- Directories to backup (Inclusion): The primary field, where you list paths to be backed up (e.g., /home; C:\Users).
- Excluded files (global filter): Files/patterns listed here will be excluded from all backups on this client/group. Example: *.tmp;*.temp;*cache*/;*.bak
- Included files (global filter): Files/patterns listed here act as an inclusion list if the "Directories to backup" is very broad (like / or C:\). This is less commonly used than exclusions. It's usually better to specify include directories directly.
- Syntax:
  - Paths are separated by semicolons (;) on Windows and newlines or semicolons on Linux (newlines in the text area are often preferred for readability on Linux).
  - Wildcards: * (matches any sequence of characters), ? (matches any single character).
  - Paths can be absolute (e.g., /var/log/, C:\Windows\Temp) or relative if within an already included directory (less common for top-level excludes).
  - To exclude a directory and its contents, end the path with a slash: temp_files/ or C:\TempFolder\.
- Order of Operations: Exclusions generally take precedence. If a file matches an exclude pattern, it won't be backed up even if it's in an included directory.
- Testing: After setting filters, it's wise to run a test backup and carefully check the logs or browse the backup to ensure the filters are behaving as expected.
Backup of Symbolic Links, Hard Links:
- Follow symbolic links: If checked, UrBackup will back up the target file/directory that the symlink points to. If unchecked (default), it backs up the symlink itself. Be cautious with this, as following symlinks can lead to backing up data outside the intended scope or create backup loops if symlinks point to parent directories or already included locations.
- Backup hard links as links: If checked (default), hard links are backed up efficiently, storing the data once and linking subsequent occurrences. If unchecked, each hard link instance would be backed up as a separate copy of the data.
Specific File Types/Attributes (Windows):
- Backup NTFS alternate data streams: Important for some applications that store metadata in ADSs. Usually good to keep enabled.
- Backup EFI partition: If performing file backups, this allows backing up the contents of the EFI system partition (ESP), which contains bootloader files. This is more relevant if you are not doing full image backups that would capture the ESP anyway.
Default Directories for New Clients:
- Under "Settings" -> "Default" (or a group) -> "File Backups". You can specify default directories that will be automatically configured for any new client that joins the server and falls under these default/group settings. This speeds up onboarding new clients.
snapshot.sh and pre_backup.sh / post_backup.sh Scripts (Client-Side):
- These are powerful hooks for advanced users, especially on Linux clients.
- You can place executable scripts in C:\Program Files\UrBackup (Windows) or /usr/local/etc/urbackup/scripts (or similar path on Linux, check client documentation) on the client machine.
- pre_filebackup.sh / post_filebackup.sh (and similar for image backups):
  - pre_...: Runs before the backup process starts. Useful for quiescing databases (e.g., mysqldump), creating temporary LVM snapshots, or preparing application data. If the script exits with a non-zero status, the backup may be aborted.
  - post_...: Runs after the backup attempt (regardless of success or failure). Useful for cleaning up (e.g., removing LVM snapshots, restarting services) or triggering notifications.
- snapshot.sh (for image backups on Linux): If this script exists and is executable, UrBackup will run it to create a snapshot of the volumes to be backed up. The script should output the path to the snapshot device(s). UrBackup will then back up from these snapshots and run the script again with a "remove" argument afterwards. This is essential for consistent image backups of live Linux systems, especially those using LVM or filesystems that support snapshotting.

Advanced Image Backup Settings

These are typically found under "Settings" -> (Select Client or Default/Group) -> "Image Backups".

Choosing Image Format (VHD, VHDZ, raw):
- VHD (Virtual Hard Disk): Standard Microsoft format. Can be mounted on Windows and other OSes. Good compatibility.
- VHDZ (Compressed VHD): UrBackup's default for Windows clients. It's a VHD file compressed using zlib, with an index for faster seeking. Offers good compression and decent performance. Files are typically named .vhd.gz but are in VHDZ format.
- Raw: A bit-for-bit copy of the partition.
  - Pros: Can be slightly faster to create (no container overhead/conversion), universally mountable with tools like losetup and mount on Linux. Sometimes preferred for Linux image backups.
  - Cons: No inherent compression within the format itself (though UrBackup can compress the transfer and the raw file can be stored on a compressed filesystem on the server). Can be less convenient to mount on Windows without third-party tools.
- UrBackup often handles the choice automatically based on the client OS (VHDZ for Windows, raw for Linux). You can sometimes influence this via settings like "Raw copy image backup" or by how volumes are specified.
Volume Shadow Copy Service (VSS) on Windows:
- Essential for consistent backups: VSS is a Windows framework that allows applications (like SQL Server, Exchange) and the system to create consistent point-in-time copies (shadow copies or snapshots) of volumes, even while files are in use.
- UrBackup client for Windows utilizes VSS by default to ensure that image backups (and file backups, if configured) are taken from a stable snapshot, preventing data corruption or inconsistencies from open files.
- Troubleshooting VSS issues (e.g., VSS writer errors in Windows Event Log) is a common task when Windows backups fail.
Using Raw Disk Images and Specific Partition Backups:
- When specifying volumes for image backup (e.g., C, D, or all, all_nonusb), UrBackup determines how to back them up.
- For Linux, you might specify partitions like /dev/sda1, /dev/vg0/lv_root.
- The "Backup bootable flag" or similar options ensure that metadata required to make the restored image bootable is included.
Image Backup Compression and Encryption Settings:
- Compression:
  - VHDZ itself is compressed.
  - Even for raw images, UrBackup compresses data during transfer.
  - The actual storage on the server can further benefit from filesystem-level compression (e.g., if using Btrfs or ZFS for the backup storage path).
  - Settings like "Transfer mode for image backups" might offer options like "Raw" or "Hashed" which influence how data is processed and compressed.
- Encryption:
  - UrBackup offers "Encrypted transfer" which encrypts data in transit between client and server.
  - For at-rest encryption of backups on the server, this is typically handled by:
    - Encrypting the entire backup storage filesystem (e.g., using LUKS on Linux).
    - UrBackup itself doesn't currently offer built-in per-client, end-to-end encryption of the backup data files stored on the server in a way that the server admin cannot decrypt them (unlike some other tools like Duplicati or Restic). The focus is on securing the server and the transfer.

Client-Side Hashing and Transfer Modes

UrBackup is very efficient with data transfer due to client-side hashing.

Before transferring any file or block of an image, the client calculates its hash.
It sends this hash to the server. The server checks if it already has data with this hash (from a previous backup of this client or even another client due to global deduplication).
If the server has the data, it tells the client not to send it. Only new/changed data blocks are actually transferred.
"Local network" vs. "Internet" mode:
- The server can be configured with different settings (e.g., bandwidth limits, backup windows) for clients connecting via the local network versus the internet.
- Clients automatically switch or can be configured for one mode. This affects which settings apply and potentially how aggressively they try to minimize data transfer (e.g., internet mode might be more conservative with bandwidth).

Managing Client Identities and Authentication

server_ident.txt (Client-Side) and Server Identity:
- When a client first connects to a server, it receives a server identity key, which it stores (e.g., in C:\Program Files\UrBackup\server_ident.txt on Windows). This ensures the client only talks to the correct UrBackup server it initially paired with, preventing accidental connection to a rogue server.
Client Authentication:
- Clients also have their own identity that the server recognizes.
- In Settings -> General -> Server -> "Client authentication method," you can choose how clients authenticate. "Simple file based token authentication" is common.
Dealing with Reinstalled Clients or Changed Hostnames:
- If you reinstall a client's OS but want it to resume backing up as the same client (to continue its backup history and deduplication benefits), you need to restore its client identity. This is usually stored in files like client_ident.key and client_ident.privkey (paths vary by OS, e.g., C:\Program Files\UrBackup\urbackup or /usr/local/var/urbackup/). Back these up before reinstalling and restore them afterwards before the new client contacts the server.
- If a client's hostname changes, the server might see it as a new client. You may need to manually delete the "old" client entry on the server (after ensuring data is migrated or no longer needed) or use the "Assume this client is old client X" feature in the client settings on the server (if available) to merge identities. This area can be tricky, so careful identity management is key.

Workshop Fine-Tuning Client Backups

Project: Implement Include/Exclude Rules for a Windows Client and Configure Client-Specific Backup Windows for Linux vs. Windows Clients.

Objective: To gain practical experience with filtering backup content using include/exclude rules and to configure different backup schedules for different types of clients.

Prerequisites:

Your UrBackup server VM, Windows Client VM, and Linux Client VM are all running and connected.
Basic file backups are working for both clients.

Steps:

Part 1: Implement Include/Exclude Rules (Windows Client)

Prepare Folder Structure and Files on Windows Client:
- On your Windows Client VM, create the following folder structure inside C:\ProjectAlpha (if C:\ProjectAlpha doesn't exist, create it):
```
C:\ProjectAlpha\
    ├── important_docs\
    │   ├── report_final.docx
    │   └── budget_q3.xlsx
    ├── temp_files\
    │   ├── session_data.tmp
    │   └── app_debug.log
    └── media\
        ├── vacation_pic.jpg
        └── old_video.mp4
```
- Create empty files with these names (e.g., right-click -> New -> Text Document, then rename and change extension). Add a little unique text to report_final.docx and budget_q3.xlsx just for later verification if you wish.
Configure Backup Path for C:\ProjectAlpha:
- In the UrBackup server UI, go to "Settings".
- Select your Windows Client VM from the "Client" dropdown.
- Go to the "File Backups" tab.
- In "Directories to backup", ensure C:\ProjectAlpha is listed. If you had C:\TestData_Win before, you can replace it or add C:\ProjectAlpha as a new path separated by a semicolon (e.g., C:\TestData_Win;C:\ProjectAlpha). For this workshop, let's focus just on C:\ProjectAlpha to keep it simple. So, set it to: C:\ProjectAlpha
- Click "Save".
Add Exclude Rules:
- Still in the "File Backups" settings for the Windows client:
- Find the field "Excluded files (global filter)".
- We want to exclude all .tmp and .log files, and the entire media directory. Add the following, ensuring each rule is on a new line or separated by semicolons:
```
*.tmp
*.log
C:\ProjectAlpha\media/
```
  (Note: Using C:\ProjectAlpha\media/ specifically targets that folder. A more general media/ would exclude any folder named media within the backup scope.)
- Click "Save".
Run a File Backup for the Windows Client:
- Go to the "Status" page.
- For your Windows Client VM, click "Actions" -> "Start file backup" (Incremental).
- Wait for the backup to complete.
Verify Exclusions:
- Once the backup is complete, go to the "Backups" tab.
- Select your Windows client and then click on the latest file backup.
- Browse the backup:
  - Navigate into C -> ProjectAlpha.
  - Check important_docs: report_final.docx and budget_q3.xlsx SHOULD be present.
  - Check temp_files: session_data.tmp and app_debug.log SHOULD NOT be present.
  - Check media folder: The media folder itself (and its contents vacation_pic.jpg, old_video.mp4) SHOULD NOT be present.
- This confirms your exclude rules are working.

Part 2: Configure Client-Specific Backup Windows

Currently, your clients are likely using the default server backup window (probably 24/7). Let's change this.

Define Backup Window for Windows Client (Workstation - Daytime):
- In the UrBackup server UI, go to "Settings".
- Select your Windows Client VM. Ensure "Separate settings for this client" is effectively enabled (it usually is if you're editing it directly).
- Go to the "File Backups" tab (or "General" tab, location of backup window can vary slightly or be inherited). Look for "Backup window for file backups".
- Set it to something representing daytime hours on weekdays, e.g.: 09:00-17:00/1-5 (This means 9 AM to 5 PM, Monday to Friday. 1-5 represents Mon-Fri).
- Click "Save".
Define Backup Window for Linux Client (Server - Nighttime):
- In the UrBackup server UI, go to "Settings".
- Select your Linux Client VM.
- Go to the "File Backups" tab (or "General" for backup window settings).
- Set its "Backup window for file backups" to nighttime hours, e.g.: 22:00-05:00/1-7 (This means 10 PM to 5 AM, every day of the week).
- Click "Save".
Verify Adherence to Backup Windows (Conceptual Test):
- Observe Client Behavior: If you try to manually start a backup outside of a client's configured backup window (from the server UI), UrBackup will usually queue it and wait until the window opens.
- Check Server Logs/Status: The server logs or client status might indicate "Backup window missed" or "Waiting for backup window."
- To test this practically:
  - Note the current time.
  - Adjust one of the client's backup windows so that the current time is outside of it. For example, if it's 3 PM, set the Windows client's window to 10:00-14:00/1-5.
  - Try to initiate a manual file backup for that client from the "Status" page.
  - Observe its status. It should show as queued or waiting, not immediately starting. The "Next backup time" might reflect the start of the next window.
  - Remember to set the backup windows back to your desired settings (09:00-17:00/1-5 for Windows, 22:00-05:00/1-7 for Linux) after testing.

You have now successfully applied include/exclude filters to control what data gets backed up and configured different backup schedules for different clients. These are powerful techniques for optimizing your backup strategy, saving storage space, reducing network traffic, and ensuring backups run at appropriate times.

6. Backup Verification and Data Integrity

Performing backups is only half the battle; ensuring those backups are valid, complete, and restorable is equally, if not more, important. This section emphasizes the critical need for backup verification and discusses UrBackup's built-in mechanisms, manual verification techniques, and broader considerations for data integrity.

Importance of Backup Verification

Why Backups Can Fail Silently:
- Software bugs (in the OS, backup software, or drivers).
- Hardware issues (failing disks on client or server, RAM errors, network problems).
- Filesystem corruption on the client or server.
- Unforeseen interactions with other software (e.g., antivirus).
- Human error in configuration (e.g., incorrect include/exclude rules, wrong paths).
- Malware interference. A backup process might report "success" at a superficial level, but the actual data backed up could be incomplete, corrupted, or unrestorable. Without verification, you might only discover this when you critically need the data, which is too late.
The Cost of Unrecoverable Data:
- Loss of critical business information, customer data, financial records.
- Loss of irreplaceable personal data (photos, documents).
- Significant downtime and recovery effort.
- Reputational damage.
- Legal or compliance penalties in some contexts. The potential cost of data loss far outweighs the effort required to implement a robust verification strategy.

UrBackup's Built-in Verification Mechanisms

UrBackup incorporates several features to help ensure data integrity during the backup process:

Hashing During Transfer and Storage:
- Client-Side Hashing: Before sending data, the client hashes files or blocks of files.
- Transfer Integrity: The server can re-verify hashes of transferred data to detect corruption during transit (though this can be disabled for performance, it's generally good to have some level of check).
- Server-Side Storage: UrBackup stores data and associates it with these hashes. Deduplication relies on these hashes; if a block with a specific hash already exists, it's not stored again. This implicitly means the server "trusts" the integrity of previously stored, hashed blocks.
urbackup_srv verify-hashes Command (Server-Side):
- This is a crucial command-line tool provided with the UrBackup server.
- sudo urbackup_srv verify-hashes [--client <clientname>] [--path <backup_storage_path>] [--full]
- Functionality: It reads the backup data from the storage path and recalculates the hashes of the files/blocks. It then compares these recalculated hashes against the hashes stored in the UrBackup database for those files/blocks.
- Usage:
  - Run it for all clients: sudo urbackup_srv verify-hashes
  - Run for a specific client: sudo urbackup_srv verify-hashes --client myclient1
  - The --full option might perform more exhaustive checks but can take longer.
- Output: It will report any mismatches or errors found, indicating potential corruption in the stored backup data.
- Frequency: It's advisable to schedule this command to run periodically (e.g., weekly or monthly) as part of your maintenance routine. It can be resource-intensive, so run it during off-peak hours.
- Limitations: This verifies that the data on the backup server matches what the server thinks it should be. It doesn't inherently verify that the original source data was correctly captured if there was an issue during the initial hashing on the client or if client-side data was already corrupt.

Performing Manual Verification

Built-in tools are helpful, but manual verification provides the highest level of confidence.

Test Restores of Random Files and Folders:
- This is the most common and highly recommended verification method.
- Process:
  1. Periodically (e.g., weekly or monthly), select a few random clients.
  2. For each selected client, choose a few random files or small folders from different parts of their backup set (e.g., a document, a spreadsheet, a configuration file).
  3. Restore these files to an alternate location on the original client machine or to a different test machine. Avoid restoring directly over live production data unless you are absolutely certain and it's a planned test.
  4. Compare the restored files with the originals (if still available) or with known good copies. Use file comparison tools (e.g., diff on Linux/macOS, WinMerge on Windows) or check file hashes (MD5, SHA256) to ensure they are identical.
- Benefits: Directly tests the end-to-end restorability of your data. Catches issues that purely server-side hash verification might miss.
Test Restores of Entire Images (Mounting VHD/VHDZ, or Full System Restore to a Test VM):
- For image backups, verification is more involved but equally critical.
- Mounting VHD/VHDZ Files:
  1. Locate the VHD/VHDZ image file on your UrBackup server's storage path.
  2. Copy it to a test machine (or access it directly if the server OS allows mounting).
  3. On Windows: Use Disk Management (Action -> Attach VHD) to mount the VHD/VHDZ.
  4. On Linux: Use tools like guestmount (from libguestfs-tools) or qemu-nbd to mount VHD/VHDZ or raw images.
  5. Once mounted, browse the filesystem within the image. Check for file integrity, directory structure, and try opening a few files.
- Full System Restore to a Test VM (Best Test for BMR):
  1. Perform a full Bare Metal Restore (BMR) using the UrBackup Restore CD/USB.
  2. Instead of restoring to the original physical machine, restore the image to a new, empty virtual machine.
  3. Attempt to boot the VM. Verify that the OS loads, applications run (if any specific ones are critical), and key data is accessible.
  4. This is the most comprehensive test of your image backup's viability for disaster recovery. It's time-consuming, so it might be done less frequently (e.g., quarterly or semi-annually) for critical systems.

Automated Verification Scripts (Conceptual)

For more advanced users, scripting can help automate parts of the verification process:

Using post_backup.sh or post_imagebackup.sh Scripts (Client-Side or Server-Side):
- These scripts run after a backup completes.
- A server-side script (if UrBackup offers such hooks, or triggered by monitoring backup completion) could:
  - Automatically mount the latest image backup (if VHD/raw and tools are available on the server).
  - Run fsck (filesystem check) on the mounted image.
  - Check for the existence and integrity (e.g., hash) of a predefined set of "canary" files within the backup.
  - Send a notification based on the script's findings.
- This requires significant scripting expertise and careful design to avoid false positives/negatives.

Monitoring for "Bit Rot" and Storage Health

Data integrity isn't just about the backup process; it's also about the long-term health of your storage media. "Bit rot" refers to the silent degradation of data stored on media over time.

Filesystem-Level Checks:
- Btrfs: Use btrfs scrub regularly. Btrfs stores checksums for data and metadata. Scrubbing reads all data and verifies it against the checksums. If redundant copies exist (e.g., RAID1 on Btrfs), it can automatically repair corruption.
- ZFS: Use zfs scrub. ZFS also uses checksums extensively and can self-heal if configured with redundancy.
- These tools are excellent for detecting and correcting silent corruption on the backup storage itself.
SMART Monitoring for Hard Drives:
- Use S.M.A.R.T. (Self-Monitoring, Analysis, and Reporting Technology) tools (e.g., smartmontools on Linux) to monitor the health of your hard drives on the UrBackup server.
- Regularly check SMART attributes for indicators of impending drive failure (e.g., reallocated sectors, pending sectors, temperature).
- Proactively replace drives that show signs of failing.
RAID Health:
- If using hardware or software RAID, regularly monitor the RAID array's status to ensure all disks are online and healthy, and that there are no rebuilds or degradation issues.

A comprehensive backup verification strategy combines UrBackup's built-in features, regular manual test restores, and proactive monitoring of your storage hardware and filesystems. It's an ongoing process, not a one-time setup.

Workshop Backup Verification and Data Integrity

Project: Perform Manual Verification of File Backups and Mount an Image Backup for Inspection.

Objective: To practice essential backup verification techniques: restoring and comparing files, and mounting an image backup to check its contents.

Prerequisites:

Your UrBackup server, Windows Client VM, and Linux Client VM are running.
You have successful recent file backups for both clients.
You have a successful recent image backup for your Windows Client VM (VHD or VHDZ format). If not, configure one (e.g., backup drive C:) and run an image backup for the Windows client before starting this workshop.

Steps:

Part 1: Manual File Backup Verification (Windows Client)

Identify Files for Test Restore:
- On your UrBackup server web UI, go to "Backups".
- Select your Windows Client VM and click on its latest file backup.
- Browse the backup. Identify 3-5 diverse files from different locations within the backup (e.g., a .txt file from C:\TestData_Win, a file from C:\ProjectAlpha\important_docs if you did the previous workshop). Note their full paths and names.
Restore Files to an Alternate Location:
- On the Windows Client VM, create a new folder for testing, e.g., C:\RestoreTest_Win.
- Back in the UrBackup server web UI (browsing the Windows client's backup):
  - Select the files you identified.
  - Instead of clicking "Restore selected" (which goes to original location), look for an option to "Download selected as ZIP". Download the ZIP file to your host machine or directly to the Windows Client VM.
  - Extract the contents of the ZIP file into the C:\RestoreTest_Win folder on the Windows Client VM.
  - (Alternatively, if your UrBackup version supports restoring to an alternate path directly on the client, use that feature to restore to C:\RestoreTest_Win.)
Compare Restored Files with Originals:
- On the Windows Client VM:
  - For each restored file in C:\RestoreTest_Win, locate its original counterpart (e.g., in C:\TestData_Win or C:\ProjectAlpha).
  - Method 1: Manual Comparison (for text files): Open both the original and the restored version of a text file in Notepad or another editor. Visually compare their content.
  - Method 2: Using a File Comparison Tool (Recommended):
    - Install a file comparison tool like WinMerge (free, open-source) on your Windows Client VM.
    - Use WinMerge to compare the original file with its restored version. WinMerge will highlight any differences. They should be identical.
  - Method 3: Hash Comparison (for any file type):
    - Open PowerShell on the Windows Client VM.
    - Calculate the hash of an original file: Get-FileHash C:\path\to\original\file.ext
    - Calculate the hash of the corresponding restored file: Get-FileHash C:\RestoreTest_Win\path\within\zip\file.ext
    - The Hash values (SHA256 by default) should be identical.
- Repeat for all files you restored. If all are identical, your file backup is verified for these files.

Part 2: Manual File Backup Verification (Linux Client - if time permits or for practice)

Repeat steps 1-3 for your Linux Client VM:
- Identify 3-5 files from its latest file backup.
- Download them as a ZIP or restore to an alternate path (e.g., /home/yourlinuxuser/restoretest_linux).
- On the Linux Client VM, use the diff command to compare text files: diff /path/to/original/file.txt /home/yourlinuxuser/restoretest_linux/path/to/restored/file.txt (No output means files are identical).
- For binary files or any file, use sha256sum or md5sum to compare hashes: sha256sum /path/to/original/file.ext sha256sum /home/yourlinuxuser/restoretest_linux/path/to/restored/file.ext (The hash strings should match).

Part 3: Mount and Inspect a Windows Image Backup (VHD/VHDZ)

Locate the Image Backup File on the Server:
- On your UrBackup Server VM (the Linux VM running urbackup-server):
  - Navigate to your backup storage path (e.g., /mnt/urbackup_storage).
  - Inside this path, there will be a directory named after your Windows client. Navigate into it.
  - Look for a subdirectory corresponding to the date/time of your image backup.
  - Inside, you should find the VHD or VHDZ file (e.g., Image_C_190911-2310.vhd.gz or similar - the .gz extension is used for VHDZ by UrBackup). Note its full path.
Mount the VHD/VHDZ Image on the Windows Client VM (or another Windows machine):
- Option A: Copy VHD(Z) to Windows Client VM (if server is Linux and client is Windows):
  - This is the simplest for this workshop. Copy the VHD/VHDZ file from the UrBackup server to your Windows Client VM (e.g., using SCP, or if your VMs share a folder, or via a network share). This might take time if the image is large.
  - Once copied to the Windows Client VM (e.g., to C:\TempImage\my_image.vhd.gz):
    - If it's a .vhd.gz (VHDZ), you may need to rename it to .vhd for Windows Disk Management to recognize it directly, or use 7-Zip to extract the .vhd from the .gz archive if it's just standard gzip compression over a VHD. UrBackup's VHDZ is a special indexed GZ format, so direct mounting might be tricky.
    - Let's assume for simplicity your UrBackup server created a plain .vhd file or you extracted it.
- Option B: Direct Mount from Server if Server is Windows (Not our workshop setup): If your UrBackup server was on Windows, you could mount it there directly.
- Option C (Advanced Linux Server): On the Linux UrBackup server, you could use qemu-nbd and guestmount to mount the VHD/VHDZ, but this is more complex than needed for this workshop's verification goal. For this workshop, we'll focus on mounting on a Windows system.
- Mounting Procedure (assuming you have the VHD/VHDZ file on a Windows system, like your Windows Client VM):
  1. On the Windows Client VM, open Disk Management (press Win+X and select "Disk Management").
  2. In Disk Management, click Action -> Attach VHD.
  3. Browse to and select your VHD/VHDZ file. Check the "Read-only" box to prevent accidental writes to the backup image. Click "OK".
  4. The image should mount, and a new disk/volume will appear in Disk Management. It might automatically be assigned a drive letter. If not, right-click the mounted volume and select "Change Drive Letter and Paths..." to assign one (e.g., E:).
Browse and Verify Contents of Mounted Image:
- Open File Explorer on the Windows Client VM.
- Navigate to the newly assigned drive letter (e.g., E:).
- You are now browsing the filesystem of your Windows image backup as it was at the time of the backup.
- Verify:
  - The overall directory structure (e.g., Windows, Users, Program Files folders are present).
  - Check for a few key files or folders you know should exist.
  - Try opening a text file or viewing an image file from the mounted image to ensure readability.
  - You are not testing a full boot, just data accessibility and basic integrity.
Unmount the Image:
- In Disk Management, find the disk corresponding to your mounted VHD.
- Right-click the disk (e.g., "Disk 2" - usually indicated as "VHD") and select "Detach VHD".
- Confirm if prompted.

Part 4: (Optional) Run verify-hashes on the Server

Run urbackup_srv verify-hashes:
- On your UrBackup Server VM (Linux), open a terminal.
- Run the command for your Windows client (replace YourWindowsClientName with its actual name as shown in UrBackup UI):
```
sudo urbackup_srv verify-hashes --client YourWindowsClientName
```
  Or for all clients:
```
sudo urbackup_srv verify-hashes
```
- Observe: This command can take a while, especially for large backups. It will output progress and report any errors if hash mismatches are found. Successful completion without errors indicates that the data stored on the server is consistent with the server's database records for that data.

This workshop has given you hands-on experience with crucial backup verification steps. Regularly performing test restores and image inspections, combined with using UrBackup's verify-hashes tool and monitoring storage health, will greatly increase your confidence in your backup system's ability to protect your data.

7. Advanced Restore Techniques

While basic file restores are straightforward, UrBackup offers more powerful restoration capabilities, especially when dealing with entire system recovery from image backups. This section covers Bare Metal Restore (BMR), considerations for restoring to dissimilar hardware, mounting image backups for file access, and file-level restores from images.

Bare Metal Restore (BMR)

Bare Metal Restore is the process of restoring an entire operating system, along with applications, settings, and data, to a computer that may have no OS installed (i.e., "bare metal") or whose existing system is corrupt. This is typically done using an image backup.

Creating and Using the UrBackup Restore CD/USB:
1. Download the Restore Media ISO:
  - Go to the official UrBackup website (urbackup.org).
  - Navigate to the "Download" section and find the link for the "UrBackup Restore CD/USB" (it's usually an ISO file, e.g., urbackup-restore.iso). Download the latest version.
  - This ISO is a bootable Linux environment containing the necessary tools to connect to your UrBackup server and perform a restore.
2. Create Bootable Media:
  - USB Drive (Recommended): Use a tool like Rufus (Windows), balenaEtcher (Windows, macOS, Linux), or dd (Linux) to write the downloaded ISO image to a USB flash drive. This will make the USB drive bootable.
    - Example using dd on Linux (be extremely careful with of=/dev/sdx to select the correct USB drive, as this will overwrite it): sudo dd if=/path/to/urbackup-restore.iso of=/dev/sdx bs=4M status=progress oflag=sync
  - CD/DVD: Alternatively, you can burn the ISO image to a CD or DVD using any disc burning software.
3. BIOS/UEFI Settings:
  - On the target machine (where you want to restore), you'll need to enter the BIOS/UEFI setup during boot (usually by pressing Del, F2, F10, F12, or Esc).
  - Configure the boot order to prioritize booting from the USB drive (or CD/DVD drive) before the internal hard drive.
Requirements for BMR:
- A successful image backup of the source system stored on your UrBackup server.
- The UrBackup Restore CD/USB.
- The target machine must be able to boot from the restore media.
- Network connectivity from the restore environment to your UrBackup server. The restore environment usually supports DHCP for automatic network configuration or allows manual IP setup.
- Hardware Compatibility:
  - Similar Hardware: BMR works best when restoring to hardware identical or very similar to the original machine.
  - Dissimilar Hardware: Restoring to different hardware can be challenging due to driver issues (see below).
- Sufficient disk space on the target hard drive to accommodate the restored image.
Step-by-Step BMR Process (General Outline):
1. Boot from Restore Media: Insert the UrBackup Restore USB/CD into the target machine and boot from it.
2. UrBackup Restore Environment: The restore environment will load. It typically presents a menu-driven or graphical interface.
3. Network Configuration: Ensure the network is configured (DHCP usually works). You might need to provide network details if it's a complex network.
4. Connect to UrBackup Server: Enter the IP address or hostname of your UrBackup server and credentials if required by your server setup (though often client identity from the backup is used).
5. Select Client and Backup: Choose the client whose image you want to restore and then select the specific image backup (by date/time) from the list provided by the server.
6. Select Target Disk/Partitions:
  - The restore interface will show the hard drives detected on the target machine.
  - You need to select the target disk where the image will be written.
  - Partitioning:
    - You might have options to restore the original partition layout or manually partition the target disk.
    - If restoring to a disk of a different size, adjustments might be needed. The target partition for the OS usually needs to be at least as large as the original.
    - The restore tool will attempt to recreate partitions as they were in the image.
7. Start Restore: Confirm your selections and start the restore process. This will overwrite the selected target disk.
8. Monitor Progress: The restore tool will show progress. This can take a significant amount of time depending on image size, network speed, and disk I/O speed.
9. Post-Restore Steps:
  - Once the restore completes, remove the restore media.
  - Reboot the machine.
  - The system should boot into the restored operating system.
  - Windows: May require reactivation. Driver issues might appear if hardware is different.
  - Linux: Bootloader (GRUB) issues can occur, especially if disk identifiers (/dev/sdX) have changed. You might need to chroot into the restored system from a live Linux CD and run update-grub or grub-install. Also, check /etc/fstab for correct UUIDs or device names.

Restoring to Dissimilar Hardware (Challenges and Strategies)

Restoring an OS image to hardware significantly different from the original is a common challenge. The primary issue is hardware drivers (storage controller, network card, chipset, video).

Windows:
- Driver Injection:
  - Sysprep (System Preparation Tool): Before taking an image of a Windows machine intended for deployment to dissimilar hardware, running sysprep /generalize /oobe /shutdown removes system-specific information (like SIDs and hardware drivers) and prepares the image for deployment. The restored image will then go through a mini-setup (OOBE - Out-Of-Box Experience) on first boot, redetecting hardware. This is the most robust method but must be done on the source machine before the backup used for BMR.
  - DISM (Deployment Image Servicing and Management): If you have the necessary drivers for the new hardware, you can sometimes inject them into the restored (but offline) Windows image using DISM from a Windows PE environment or another Windows installation. This is more advanced.
  - Universal Restore Tools: Some commercial backup products have "universal restore" features that attempt to inject drivers automatically or assist in the process. UrBackup's restore media has some capabilities to load drivers, but it's not as automated as some specialized tools.
- Stop Errors/BSODs: Often, booting a Windows image on very different hardware (especially different storage controllers - IDE vs. AHCI vs. RAID) will result in a Blue Screen of Death (BSOD), typically 0x0000007B INACCESSIBLE_BOOT_DEVICE.
Linux:
- Kernel Modules: The Linux kernel is generally more adaptable to hardware changes, as many drivers are included as modules. However, issues can still arise:
  - Storage Controller Drivers: Ensure the kernel in the restored image has the necessary modules for the new storage controller. This is usually handled if using a generic kernel.
  - Network Drivers: Similar to storage controllers.
  - initramfs/initrd: The initial RAM disk might need to be rebuilt on the restored system to include necessary drivers for booting, especially if critical hardware like the disk controller has changed significantly. This can be done by chrooting into the restored system and running update-initramfs -u -k all.
  - Bootloader (GRUB): As mentioned, GRUB may need to be reinstalled or reconfigured (grub-install /dev/sdX, update-grub).
  - /etc/fstab: Ensure mount points use UUIDs rather than device names like /dev/sda1 to avoid issues if disk enumeration changes.
Virtual-to-Physical (V2P) and Physical-to-Virtual (P2V):
- P2V: Backing up a physical machine and restoring its image to a virtual machine is generally easier. VMs use standardized virtual hardware, so driver issues are less common (though guest additions/tools should be installed in the VM post-restore). UrBackup is often used for P2V.
- V2P: Restoring a VM's image to physical hardware is much more prone to dissimilar hardware issues.

Mounting and Accessing Image Backups

You don't always need a full BMR to get data from an image backup. UrBackup's image files (VHD, VHDZ, raw) can often be mounted to access their contents.

Mounting VHD/VHDZ Files:

Windows:
- Use Disk Management (Action -> Attach VHD). Browse to the .vhd or .vhd.gz (VHDZ) file. Windows 10/11 can often mount VHDZ directly.
- Once mounted, it appears as another disk/volume with a drive letter.

Linux:

For VHD:

sudo apt install libguestfs-tools # Installs guestmount and other tools
sudo mkdir /mnt/vhd_image
sudo guestmount -a /path/to/image.vhd -m /dev/sda1 --ro /mnt/vhd_image
# Replace /dev/sda1 with the actual partition inside the VHD you want to mount
# (use fdisk -l image.vhd to see partitions within the VHD)

For VHDZ: VHDZ is a gzipped VHD with an index. guestmount might handle some VHDZ files directly. If not, you might need to decompress it first or use tools specifically designed for VHDZ. The UrBackup server itself can often browse VHDZ contents for file-level restores.

Using qemu-nbd (for VHD, VHDX, QCOW2, raw):

sudo apt install qemu-utils
sudo modprobe nbd max_part=8
sudo qemu-nbd --connect=/dev/nbd0 /path/to/image.vhd_or_raw
sudo mount /dev/nbd0p1 /mnt/my_image # Assuming the first partition
# After use:
sudo umount /mnt/my_image
sudo qemu-nbd --disconnect /dev/nbd0

Accessing Files Directly: Once mounted, you can browse the filesystem within the image backup like any other disk and copy files/folders as needed. This is much faster than a full BMR if you only need a few items.
Using libguestfs on Linux: This suite of tools provides powerful capabilities for accessing and modifying virtual machine disk images (including VHD, raw, QCOW2, etc.) without needing to boot the VM or fully mount the image in some cases. Tools like guestfish allow scripted access.

File-Level Restore from Image Backups

UrBackup's web interface often allows you to perform file-level restores directly from image backups without manually mounting them.

How UrBackup Facilitates This:
- When you browse an image backup in the "Backups" tab of the web UI, the UrBackup server has logic to parse the filesystem structure within the image file (VHD, VHDZ, or raw image of a known filesystem like NTFS, ext4).
- It presents this structure to you, allowing you to navigate folders and select individual files or directories for restoration.
When This is Preferable:
- You need only a few files or a specific folder from an image backup.
- The original system is still running, but some files were corrupted or deleted, and the latest file backup doesn't have the version you need (but an older image backup might).
- It's faster and less disruptive than a full BMR or manual mounting if you just need a quick file recovery.
- You want to compare a file from an image backup with the current live file.

The specific capabilities for browsing and restoring files from images can depend on the image format, the filesystem within the image, and the UrBackup server version.

Workshop Advanced Restore Techniques

Project: Perform a Bare Metal Restore of your Windows Client VM to a New, Empty Virtual Machine.

Objective: To simulate a disaster recovery scenario by restoring an image backup of your Windows client to a new VM, effectively performing a P2V-like operation (or V2V if source was already a VM).

Prerequisites:

Your UrBackup server VM is running and accessible.
You have a recent, successful image backup of your Windows Client VM (backing up the C: drive). Ensure this image backup is complete.
Download the UrBackup Restore CD ISO from the official UrBackup website (urbackup.org -> Download -> Restore CD/USB). Save the ISO file to your host machine.
Virtualization software (VirtualBox, VMware, etc.).

Steps:

Verify Image Backup:
- In the UrBackup server UI, go to "Backups", select your Windows Client VM, and confirm a recent, complete image backup exists for the C: drive (or system volume).
Create a New, Empty "Target" Virtual Machine:
- In your virtualization software, create a brand new virtual machine. Let's call it "Windows_Restored_VM".
- Configuration:
  - OS Type: Select the same OS type as your original Windows Client VM (e.g., Windows 10 64-bit).
  - RAM: Assign at least the same amount of RAM as the original Windows Client VM, or slightly more if possible (e.g., 2-4GB).
  - Hard Disk:
    - Create a new virtual hard disk.
    - Make its size equal to or slightly larger than the original Windows Client VM's C: drive. For example, if the original C: drive was 60GB, make this new disk 60GB or 65GB. Do not make it smaller.
    - Do not install an OS on this disk. It should be blank/unformatted.
  - Network Adapter: Configure it to "Bridged Adapter" (or NAT if Bridged gives issues, but Bridged is easier for the restore environment to reach the UrBackup server). Ensure it's on the same network as your UrBackup server.
  - CD/DVD Drive: Configure the VM's virtual CD/DVD drive to use the urbackup-restore.iso file you downloaded.
Boot the "Windows_Restored_VM" from the UrBackup Restore ISO:
- Start the "Windows_Restored_VM".
- It should boot from the ISO image. If it tries to boot from the empty hard disk (e.g., shows "No bootable medium found"), restart the VM and ensure the BIOS/UEFI boot order is set to boot from CD/DVD first, or use the boot menu (often F12) to select the CD/DVD drive.
- The UrBackup Restore environment should load. You might see a boot menu (e.g., graphical or text-based). Choose the default option.
Navigate the Restore Environment:
- The restore environment will typically present a series of dialogs or a menu.
- Language/Keyboard: Select if prompted.
- Network Configuration: It will usually attempt to configure the network via DHCP. If successful, great. If not, you may need to configure it manually (IP address, subnet, gateway, DNS) so it can reach your UrBackup server. Ensure it can ping your UrBackup server's IP.
- UrBackup Server Connection:
  - You'll be asked for the UrBackup server's IP address or hostname. Enter the IP address of your UrBackup Server VM.
  - It might ask for a username/password if your server is configured for specific client authentication for restores; often, it discovers clients based on what the server reports.
Select Client and Backup for Restore:
- The restore client should connect to your UrBackup server and list available clients that have image backups.
- Select your original Windows Client VM from the list.
- It will then list available image backups for that client. Select the most recent complete image backup of the C: drive.
Select Target Disk and Partitioning:
- The restore tool will display the hard disks available in the "Windows_Restored_VM". You should see the blank virtual hard disk you created (e.g., /dev/sda).
- Select this disk as the target for the restore.
- Partitioning Options:
  - You'll likely be asked how to partition the target disk. Common options:
    - "Restore original partition layout" (or similar): This is usually what you want. It will try to recreate the partitions as they were in the backup image.
    - "Use entire disk" or manual partitioning options.
  - Select the option to restore the original layout or use the entire disk for the image.
- Confirm Overwrite: You will be warned that the target disk will be overwritten. Double-check you've selected the correct (empty) disk in your new VM and confirm.
Initiate and Monitor the Restore:
- Start the restore process.
- The tool will show progress (percentage, speed, ETA). This can take a while (minutes to hours depending on image size and VM performance). Be patient.
- You can also monitor activity on your UrBackup server's "Activities" page.
Post-Restore and First Boot:
- Once the restore completes successfully, the restore tool will indicate this.
- Remove Restore Media: Shut down the "Windows_Restored_VM". Crucially, before restarting, detach the urbackup-restore.iso from its virtual CD/DVD drive. Otherwise, it will just boot back into the restore environment.
- Boot from Hard Drive: Start the "Windows_Restored_VM" again. It should now attempt to boot from its virtual hard drive, which contains the restored Windows system.
Verify the Restored System:
- Windows Boot: Observe if Windows boots successfully. It might take a bit longer on the first boot as it detects the "new" virtual hardware.
- Log In: You should be able to log in with the same username and password as your original Windows Client VM.
- Check Data: Verify that your desktop, documents, and any test files you had are present.
- Functionality: Check if basic applications work.
- Drivers: Windows might install new drivers for the virtual hardware. You may need to install/update guest additions/VMware Tools in this restored VM for optimal performance.
- Activation: Windows might prompt for reactivation since the "hardware" has changed (from the perspective of the OS). This is normal.

Troubleshooting (If it doesn't boot):

INACCESSIBLE_BOOT_DEVICE (Windows BSOD): This is the most common issue if there's a significant storage controller difference. In a V2V scenario with similar hypervisors, it's less likely but possible. The UrBackup restore environment attempts to handle some common scenarios.
Bootloader Issues (Linux): For Linux restores, you might need to use the restore media's tools or a separate Linux Live CD to chroot into the restored system and run grub-install and update-grub.
Ensure you selected the correct image and target disk.

This workshop provides a vital experience in performing a full system recovery, which is a cornerstone of any disaster recovery plan. Successfully restoring to a new VM demonstrates the power and utility of UrBackup's image backup capabilities.

8. UrBackup Security Hardening

While UrBackup is a powerful tool, like any server application (especially one managing critical backup data and potentially accessible over a network or the internet), it needs to be secured. Hardening your UrBackup deployment involves securing the server OS, the UrBackup application itself, client communications, and network access.

Securing the UrBackup Server OS

The security of the operating system hosting the UrBackup server is fundamental. If the OS is compromised, the backup server and its data are also at risk. (Assuming a Linux server, e.g., Ubuntu/Debian).

Regular Updates and Patching:
- Keep the server OS and all installed packages up to date to protect against known vulnerabilities.
- sudo apt update && sudo apt dist-upgrade -y (for Debian/Ubuntu)
- Consider configuring automatic security updates for critical patches.
Firewall Configuration (iptables, ufw):
- Implement a host-based firewall and only allow necessary incoming connections.
- For UrBackup, these are typically:
  - TCP 55413: Client communication (Internet mode)
  - TCP 55414: Web interface
  - UDP 55415: Client discovery (LAN)
  - TCP 55415: Client communication (LAN mode)
- Also allow SSH (default TCP 22) but consider changing its port and/or restricting access to specific IPs.
- Example with ufw (Uncomplicated Firewall) on Ubuntu:
```
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow ssh # Or your custom SSH port
sudo ufw allow 55413/tcp comment 'UrBackup Client Internet'
sudo ufw allow 55414/tcp comment 'UrBackup WebUI'
sudo ufw allow 55415/udp comment 'UrBackup Discovery LAN'
sudo ufw allow 55415/tcp comment 'UrBackup Client LAN'
sudo ufw enable
```
Disabling Unused Services:
- Reduce the attack surface by disabling or uninstalling any services and applications that are not essential for the server's function.
- Use sudo systemctl list-units --type=service --state=running to see active services.
Using Strong Passwords and SSH Key-Based Authentication:
- Ensure all user accounts on the server (especially root and administrative users) have strong, unique passwords.
- For SSH access, prioritize key-based authentication over password authentication.
  - Disable password authentication in /etc/ssh/sshd_config: PasswordAuthentication no
  - Disable root login via SSH: PermitRootLogin no (and use sudo from a regular user account).
Filesystem Permissions for Backup Storage:
- The UrBackup server process runs as a specific user (e.g., urbackup on Linux).
- Ensure the backup storage path (e.g., /mnt/urbackup_storage) has appropriate permissions:
  - The urbackup user must have read and write access.
  - Other non-privileged users should not have write access, and ideally, no read access unless explicitly needed.
  - Example: sudo chown -R urbackup:urbackup /mnt/urbackup_storage and sudo chmod -R 750 /mnt/urbackup_storage (owner rwx, group rx, others no permissions). Adjust as needed.

Securing the UrBackup Web Interface

The web interface is a primary control point and needs robust protection.

Using HTTPS (SSL/TLS Certificates):
- By default, the UrBackup web interface is served over HTTP, meaning login credentials and data are sent unencrypted. This is a major security risk, especially if accessed over untrusted networks or the internet.
- Setup a Reverse Proxy with HTTPS: The recommended way to enable HTTPS is to use a web server like Nginx or Apache as a reverse proxy in front of the UrBackup web interface. The reverse proxy handles SSL/TLS termination.
  - Nginx Example: Nginx listens on port 443 (HTTPS), decrypts the traffic, and forwards it to UrBackup on http://127.0.0.1:55414.
  - SSL Certificates:
    - Let's Encrypt: Free, automated, and trusted certificates. Ideal for publicly accessible servers with a domain name. Use certbot to obtain and renew certificates.
    - Self-Signed Certificates: Can be used for internal networks or testing, but browsers will show trust warnings. Not recommended for internet-facing services.
- Configure UrBackup to only listen on 127.0.0.1 for its web interface if using a local reverse proxy, preventing direct HTTP access from the network. (Settings -> General -> Server -> "IP address to bind web server to": 127.0.0.1).
Strong Admin Passwords:
- Enforce a strong, unique password for the UrBackup admin user and any other administrative accounts created within UrBackup.
- Change default passwords immediately.
Restricting Access by IP Address (if applicable):
- If the web UI should only be accessible from specific IP addresses or subnets, configure this in your reverse proxy (Nginx/Apache) or firewall.
- Nginx example: allow 192.168.1.0/24; deny all; within the server block.
Web Application Firewall (WAF) Considerations:
- For internet-facing services, consider using a WAF (e.g., ModSecurity with Apache/Nginx, or a cloud-based WAF). A WAF can help protect against common web attacks like SQL injection, XSS, etc. This is an advanced measure.
HTTP Security Headers:
- Configure your reverse proxy to send security-enhancing HTTP headers like Strict-Transport-Security (HSTS), X-Content-Type-Options, X-Frame-Options, Content-Security-Policy.

Client Security

Protecting clients and their communication with the server is also vital.

Client Authentication Methods:
- UrBackup uses token-based authentication by default for clients. When a client first connects, it's "discovered" and added.
- You can configure "Client authentication method" in server settings (Settings -> General -> Server).
- Ensure new clients require approval if you are in a less trusted environment.
- The server_ident.pwd on the client helps verify the server's identity to prevent man-in-the-middle (MITM) attacks on first connection if pre-seeded.
Encrypting Backups:
- Encrypted Transfer (In-Transit Encryption):
  - UrBackup encrypts backup data during transmission between the client and server if "Encrypted transfer" is enabled (Settings -> Client -> Advanced -> "Transfer mode"). This protects against eavesdropping on the network. It uses TLS for this.
- At-Rest Encryption (Data on Server):
  - UrBackup itself does not currently provide granular, end-to-end, client-controlled encryption of the backup data stored on the server (where the server admin cannot access the plaintext).
  - To protect data at rest on the server:
    - Full Disk Encryption (FDE): Encrypt the entire operating system drive and/or the dedicated backup storage drives using technologies like LUKS (Linux) or BitLocker (Windows Server). This protects data if the physical server or drives are stolen. The encryption key is typically entered at boot time or managed by a TPM.
    - This means the server OS can decrypt the data once booted, and thus the UrBackup server process can access it.
- Consider the threat model: who are you protecting the data from?
Protecting Client Backup Settings from Unauthorized Changes:
- On the client machine, local administrative users can potentially alter UrBackup client settings or stop the service.
- Server-side settings generally override client-side configurations for paths, schedules, etc. This is a good control.
- Limit administrative access on client machines where possible.

Network Security for Backups

Isolating Backup Traffic (VLANs):
- In larger or more secure environments, consider placing the UrBackup server and clients (or at least the server's backup network interface) on a dedicated VLAN.
- This isolates backup traffic from general network traffic, improving security and potentially performance.
- Use firewall rules between VLANs to strictly control communication.
VPNs for Internet Clients:
- For clients backing up over the internet, using a VPN (Virtual Private Network) connection to the UrBackup server's network provides an additional layer of security.
- The client connects to the VPN, and then UrBackup traffic routes through the encrypted VPN tunnel, as if the client were on the local network.
- This can simplify firewall rules (as you only need to expose the VPN port to the internet) and adds strong encryption and authentication. Common VPN solutions include OpenVPN or WireGuard.

Auditing and Logging

Regularly Review UrBackup Server Logs:
- Check the "Logs" and "Activities" tabs in the UrBackup web UI for any suspicious activity, repeated failed logins, or unusual errors.
System Logs (syslog, journalctl) on the Server:
- Monitor the server OS system logs for security-related events (e.g., SSH login attempts, sudo usage, firewall blocks).
- Tools like logwatch can summarize logs and email reports.
Setting up Centralized Logging:
- If managing multiple servers (including the UrBackup server), consider sending logs to a centralized logging server (e.g., using rsyslog to forward to an ELK stack - Elasticsearch, Logstash, Kibana - or Graylog). This allows for easier correlation of events and more sophisticated analysis and alerting.

Security is not a one-time task but an ongoing process of vigilance, updates, and adaptation to new threats. A layered security approach is always best.

Workshop UrBackup Security Hardening

Project: Secure the UrBackup Web Interface with HTTPS using Nginx as a Reverse Proxy and Generate a Self-Signed SSL Certificate for local testing.

Objective: To enhance the security of your UrBackup server by encrypting web interface traffic using HTTPS. This workshop focuses on a self-signed certificate for local environments; for public-facing servers, Let's Encrypt would be preferred.

Prerequisites:

Your UrBackup Server VM (Linux-based, e.g., Ubuntu) is running.
UrBackup server is installed and accessible via HTTP on port 55414.
You have sudo access on the UrBackup Server VM.
openssl command-line tool (usually pre-installed on Linux).
Nginx web server. If not installed, you'll install it.

Steps:

Install Nginx (if not already installed):
- On your UrBackup Server VM, open a terminal.
- Update package lists:
```
sudo apt update
```
- Install Nginx:
```
sudo apt install nginx -y
```
- Start and enable Nginx service:
```
sudo systemctl start nginx
sudo systemctl enable nginx
```
- Verify Nginx is running by opening a web browser on your host machine and navigating to http://YOUR_URBACKUP_SERVER_VM_IP. You should see the Nginx welcome page.
Generate a Self-Signed SSL Certificate:
- We'll create a directory to store our SSL certificate and key.
```
sudo mkdir -p /etc/nginx/ssl
```
- Generate the self-signed certificate and private key using OpenSSL:
```
sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout /etc/nginx/ssl/urbackup-selfsigned.key \
    -out /etc/nginx/ssl/urbackup-selfsigned.crt
```
  - -x509: Output a self-signed certificate instead of a certificate request.
  - -nodes: Don't encrypt the private key (for a server, you usually don't want to enter a passphrase on startup).
  - -days 365: Certificate validity period.
  - -newkey rsa:2048: Generate a new RSA private key of 2048 bits.
  - -keyout: Path to save the private key.
  - -out: Path to save the certificate.
- You will be prompted for information for the certificate (Country Name, State, Locality, Organization, Common Name, etc.).
  - For "Common Name (e.g. server FQDN or YOUR name)": Enter the IP address of your UrBackup Server VM or a hostname you use to access it locally (e.g., urbackup.local). If you use an IP, browsers will still complain more, but it will work for testing. Using a hostname defined in your host machine's hosts file that points to the VM's IP is better.
  - The other fields can be filled as you see fit or left blank if allowed.
- Secure the private key:
```
sudo chmod 600 /etc/nginx/ssl/urbackup-selfsigned.key
```

Configure Nginx as a Reverse Proxy for UrBackup:

Create a new Nginx server block configuration file for UrBackup:
```
sudo nano /etc/nginx/sites-available/urbackup-proxy
```

Paste the following configuration into the file. Replace your_urbackup_server_vm_ip_or_hostname with the actual IP address or hostname you entered as "Common Name" for your SSL certificate.

server {
    listen 80;
    server_name your_urbackup_server_vm_ip_or_hostname; # Or just use the IP

    # Redirect all HTTP traffic to HTTPS
    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl http2;
    server_name your_urbackup_server_vm_ip_or_hostname; # Or just use the IP

    ssl_certificate /etc/nginx/ssl/urbackup-selfsigned.crt;
    ssl_certificate_key /etc/nginx/ssl/urbackup-selfsigned.key;

    # Recommended SSL/TLS settings (Optional, but good practice)
    # ssl_protocols TLSv1.2 TLSv1.3;
    # ssl_prefer_server_ciphers on;
    # ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384;
    # ssl_session_cache shared:SSL:10m;
    # ssl_session_timeout 1d;
    # ssl_session_tickets off;

    # Add HSTS header (Optional, but good for security)
    # add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;

    location / {
        proxy_pass http://127.0.0.1:55414; # UrBackup web UI runs here
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # For WebSocket support (UrBackup uses WebSockets for live updates)
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

Save and close the file (Ctrl+X, then Y, then Enter in nano).

Enable the Nginx Site and Test Configuration:
- Create a symbolic link from sites-available to sites-enabled:
```
sudo ln -s /etc/nginx/sites-available/urbackup-proxy /etc/nginx/sites-enabled/
```
- Remove the default Nginx site if it's enabled and might conflict (listening on port 80):
```
# Check if default is enabled: ls -l /etc/nginx/sites-enabled/
# If 'default' is present:
sudo rm /etc/nginx/sites-enabled/default
```
- Test the Nginx configuration for syntax errors:
```
sudo nginx -t
```
  If it shows "syntax is ok" and "test is successful", proceed. If not, review your urbackup-proxy file for typos.
Restart Nginx:
```
sudo systemctl restart nginx
```

Adjust Firewall (ufw):

Allow HTTPS traffic and remove direct access to port 55414 if you plan to only use Nginx.

sudo ufw allow https   # This is an alias for port 443/tcp
sudo ufw allow http    # Port 80 for HTTP to HTTPS redirect
# Optional: If you want to force all access through Nginx,
# you might consider deleting the direct rule for 55414 later,
# but for now, let's keep it for testing.
# sudo ufw delete allow 55414/tcp
sudo ufw reload

Test HTTPS Access:
- Open a web browser on your host machine.
- Navigate to https://your_urbackup_server_vm_ip_or_hostname (use the same IP/hostname you used for server_name and the SSL certificate's Common Name).
- Expect a Browser Warning: Because you are using a self-signed certificate, your browser will display a security warning ("Your connection is not private," "Warning: Potential Security Risk Ahead," etc.). This is normal.
  - You'll need to click "Advanced" and then "Proceed to [IP/hostname] (unsafe)" or "Accept the Risk and Continue."
- You should now see the UrBackup web interface, but the connection is secured with HTTPS (padlock icon, though with a warning due to the self-signed cert).
- Test HTTP redirection: Try navigating to http://your_urbackup_server_vm_ip_or_hostname. It should automatically redirect you to HTTPS.
(Optional but Recommended) Configure UrBackup to Bind Web Server to localhost:
- Log in to the UrBackup web interface (via HTTPS now).
- Go to "Settings" -> "General" -> "Server".
- Find the setting "IP address to bind web server to".
- Change it from 0.0.0.0 (or blank) to 127.0.0.1.
- Click "Save".
- The UrBackup server will restart its web service. Now, the UrBackup web UI on port 55414 will only be directly accessible from the server VM itself, not from the network. Nginx, running on the same VM, can still proxy to it. This prevents bypassing your HTTPS proxy.
- If you had the direct ufw allow 55414/tcp rule, you could now safely remove it.

You have successfully configured Nginx as a reverse proxy to serve your UrBackup web interface over HTTPS using a self-signed certificate. While a self-signed certificate is fine for a local lab/homelab, remember that for any internet-facing service, a certificate from a trusted Certificate Authority like Let's Encrypt is essential. This setup significantly improves the security of your UrBackup management.

9. Scaling and Performance Optimization

As your backup needs grow—more clients, larger data volumes, or more frequent backups—the performance of your UrBackup server can become a bottleneck. This section explores hardware considerations, server and client-side optimizations, filesystem choices, and monitoring techniques to help you scale your UrBackup deployment effectively.

Hardware Considerations for Larger Deployments

The capacity of your UrBackup server hardware is the primary factor in its ability to handle a large load.

CPU Cores and Speed:
- Hashing and Compression: UrBackup performs hashing on the client and server for deduplication and change detection. Compression (especially for VHDZ images or if enabled for transfer) is CPU-intensive. More CPU cores and higher clock speeds directly benefit these operations.
- Concurrent Operations: Each active backup, restore, or maintenance task (like cleanup or verification) consumes CPU resources. A server with more cores (e.g., 4, 6, 8, or more) can handle more simultaneous operations smoothly.
- Recommendation: For a growing environment, a modern multi-core CPU (e.g., Intel Core i5/i7/i9, AMD Ryzen 5/7/9, or server-grade Xeon/EPYC) is advisable.
RAM (Memory):
- Database Caching: UrBackup uses SQLite databases for metadata. The operating system will cache frequently accessed parts of these databases and other UrBackup data in RAM, significantly speeding up lookups and operations. More RAM means more effective caching.
- Concurrent Backups: Each active backup process consumes some RAM.
- Filesystem Caching: The OS also uses RAM to cache data being written to or read from the backup storage.
- Recommendation: 4GB is a bare minimum for a small server. For larger deployments (10+ clients or terabytes of data), 8GB, 16GB, or even 32GB+ of RAM can make a substantial difference. Use ECC RAM for server builds for enhanced reliability.
Storage I/O (Input/Output):
- This is often the most critical bottleneck in backup systems.
- Backup Storage Drive(s):
  - HDDs (Hard Disk Drives): Suitable for bulk storage due to cost per TB. For performance with many clients or large backups, use multiple HDDs in a RAID array (RAID 5, 6, 10, or ZFS equivalents like RAIDZ1/Z2/Z3). 7200 RPM drives are preferred over 5400 RPM. NAS-grade or enterprise-grade HDDs are more reliable for 24/7 operation.
  - SSDs (Solid State Drives): Offer significantly higher IOPS (Input/Output Operations Per Second) and lower latency than HDDs.
    - For Backup Data: Using SSDs for the primary backup storage path is expensive for large volumes but provides the best performance.
    - For UrBackup Database/Metadata: It is highly recommended to place the UrBackup server's internal databases (usually in /var/urbackup or C:\ProgramData\UrBackupServer\urbackup by default if not a separate partition) on an SSD. This dramatically improves web interface responsiveness, indexing speed, and overall server agility.
    - SSD Caching: Some storage solutions (like ZFS with L2ARC/SLOG, or Bcache on Linux) allow using SSDs as a cache for slower HDD arrays, which can be a good compromise.
- Storage Controller: A good quality HBA (Host Bus Adapter) or RAID controller is important for performance and reliability if using multiple internal drives.
Network Bandwidth:
- Server NIC: A Gigabit Ethernet (1 Gbps) network interface is standard. For very large environments or many concurrent high-speed backups, consider a 2.5 Gbps, 5 Gbps, or 10 Gbps NIC on the server and a compatible network switch.
- Network Infrastructure: Ensure your switches and routers can handle the backup traffic load, especially if multiple clients are backing up simultaneously.
- Client NICs: Clients also need adequate network bandwidth.

Optimizing UrBackup Server Settings

The UrBackup server UI offers several settings that can be tuned for performance:

Settings -> General -> Server:
- Max simultaneous backups: Controls how many clients can back up at the same time.
  - Too low: Underutilizes server resources, backups take longer to complete overall for all clients.
  - Too high: Can overwhelm CPU, RAM, disk I/O, or network, leading to very slow backups for everyone or server instability.
  - Tuning: Start with a conservative number (e.g., 2-5 based on your CPU cores and disk speed) and monitor server load (CPU, disk I/O). Gradually increase if the server has spare capacity.
- Max simultaneously running non-backup server maintenance tasks: Controls concurrency for tasks like cleanup, verification. Default is usually 1.
- Max backup speed for local network / Max backup speed for internet clients: Global bandwidth throttling. Useful for preventing UrBackup from saturating your network, but setting it too low will slow down backups.
Settings -> Client (or Default/Group) -> Advanced:
- Transfer mode for file backups / Transfer mode for image backups:
  - Options like "Hashed" (default), "Raw", "Compressed-Hashed".
  - "Hashed" involves calculating hashes on the client and server to transfer only changed blocks, good for deduplication and incremental efficiency.
  - "Raw" might be slightly faster for initial full backups if network is very fast and CPU is a bottleneck, but less efficient for incrementals.
  - Experimentation might be needed for specific scenarios, but defaults are usually well-chosen.
- Client side file hashing during file backups: Enabling this (default) allows the client to do more work, potentially reducing server load for deduplication lookups.
Database Choices:
- UrBackup uses SQLite as its default database backend. SQLite is embedded and generally very efficient for UrBackup's workload.
- For extremely large deployments (thousands of clients, vast numbers of files), some users have experimented with other backends if they perceive SQLite as a bottleneck, but this is highly advanced and not officially supported for easy switching. Sticking with SQLite and optimizing the hardware (especially SSD for the database path) is usually the best approach.
Adjusting Backup Windows:
- Stagger backup windows for different groups of clients to distribute the load on the server over time, rather than having all clients try to back up simultaneously.
- For example, Group A backs up from 10 PM - 12 AM, Group B from 12 AM - 2 AM, etc.

Filesystem Choices and Performance (for Backup Storage Path)

The filesystem on your backup storage can significantly impact performance, especially for UrBackup's I/O patterns (many small files, random reads/writes during deduplication and consolidation).

Btrfs:
- Copy-on-Write (CoW): Can be beneficial for how UrBackup stores and links data, especially with its snapshotting capabilities.
- Built-in Compression (e.g., zstd, lzo): Can save space and, in some cases (CPU faster than disk I/O), improve throughput by reducing the amount of data written to disk.
- Potential Fragmentation: CoW filesystems can become fragmented over time with heavy random writes, which might impact performance. Regular balancing (btrfs balance) might be needed.
- UrBackup has specific optimizations when using Btrfs (e.g., using reflinks for file copies).
ZFS:
- Robustness and Data Integrity: Excellent features for preventing data corruption.
- Adaptive Replacement Cache (ARC): Advanced RAM caching that can greatly improve read performance. ZFS benefits significantly from ample RAM.
- Compression: Supports various compression algorithms.
- Snapshots: Powerful snapshot capabilities.
- Complexity and Resource Usage: Can be more complex to set up and manage than ext4/XFS and typically requires more RAM.
XFS:
- Known for good performance with large files and high concurrency.
- Scales well to large storage volumes.
- A solid, mature choice, often recommended for general-purpose file servers and can work well for UrBackup.
ext4:
- Default for many Linux distributions. Reliable and well-understood.
- Performance is generally good, but may not have the advanced features or same level of optimization for specific backup workloads as Btrfs or ZFS.
- Simpler to manage.
RAID Configurations:
- RAID 0 (Striping): Best performance, no redundancy. Not recommended for backup storage unless data is replicated elsewhere.
- RAID 1 (Mirroring): Good read performance, write performance similar to a single disk, full redundancy.
- RAID 5 (Striping with Parity): Balances performance and capacity with single-disk failure tolerance. Suffers from write penalty and slow rebuilds, especially with large HDDs.
- RAID 6 (Striping with Dual Parity): Better than RAID 5, tolerates two disk failures. Higher write penalty.
- RAID 10 (Stripe of Mirrors): Excellent performance (reads and writes) and good redundancy. More expensive in terms of usable capacity. Often the best choice for performance-critical HDD arrays.
- ZFS RAIDZ1/Z2/Z3: ZFS's integrated RAID solutions, generally preferred over mdadm or hardware RAID when using ZFS due to better data integrity and integration with the filesystem.

Client-Side Optimizations

Optimizations on the client can also reduce server load and backup times.

Throttling Client Background Processes:
- UrBackup client has settings (often in urbackupclient.ini or via urbackupclientctl) to limit its own CPU usage or I/O priority to minimize impact on the user's work.
- Server-side, you can also set "Background priority for file backups on clients" (Settings -> Client -> Advanced).
Staggering Client Backup Schedules:
- As mentioned for server load, avoid having all clients start backups simultaneously by using different intervals or backup windows.
Excluding Unnecessary Temporary or Cache Files:
- Use precise include/exclude rules (see Section 5) to avoid backing up volatile temporary files, browser caches, sleep images (hiberfil.sys), page files (pagefile.sys), large log files that aren't needed, etc. This reduces backup size, time, and server processing.

Monitoring Performance

To optimize, you first need to identify bottlenecks.

Using Server OS Tools (Linux):
- top / htop: Monitor CPU usage (per core, overall, urbackupsrv process), RAM usage, swap usage. Look for high CPU load, low idle time, or excessive swapping.
- iostat -xz 1 (from sysstat package): Shows disk I/O statistics per device (reads/sec, writes/sec, await times, %util). High %util (near 100%) or high await times indicate a disk bottleneck.
- vmstat 1: Reports on processes, memory, paging, block IO, traps, and cpu activity.
- iftop or nload: Monitor network bandwidth usage per interface or connection. Check if your NIC is saturated.
- dstat -fcailmnpsy: A versatile tool that combines information from vmstat, iostat, ifstat, netstat and more.
UrBackup Server Logs and Statistics:
- Web UI "Status" Page: Shows active backups, speeds, and basic server info.
- Web UI "Activities" Page: Provides a log of ongoing and recent tasks.
- Web UI "Statistics" Page: Can show historical data on backup sizes, durations, etc.
- Server Log File (/var/log/urbackup.log): Contains detailed information, including performance-related messages or warnings.
Identifying Bottlenecks:
- CPU Bound: If CPU usage is consistently high (e.g., urbackupsrv process at 100% on one or more cores) and disk/network are not saturated, you might be CPU bound. Consider reducing Max simultaneous backups, using less CPU-intensive compression, or upgrading the CPU.
- Disk I/O Bound: If disk %util is 100% and CPU/network have spare capacity, your storage is the bottleneck. Consider faster disks (SSDs), a better RAID configuration, optimizing filesystem, or moving the UrBackup database to an SSD.
- RAM Bound: If the system is swapping heavily (si/so columns in vmstat are high), you need more RAM. Insufficient RAM also impacts caching effectiveness.
- Network Bound: If your server's NIC is saturated (e.g., 1Gbps link is constantly near 125 MB/s) and CPU/disk have capacity, the network is the bottleneck. Consider a faster NIC/switch or reducing concurrent backups/throttling clients.

Performance tuning is an iterative process: monitor, identify bottleneck, make one change, monitor again to see the impact.

Workshop Scaling and Performance Optimization

Project: Simulate a High-Load Scenario, Monitor Server Performance, and Experiment with Max simultaneous backups.

Objective: To observe how your UrBackup server performs under increased load, identify potential bottlenecks using OS monitoring tools, and see the effect of adjusting the Max simultaneous backups setting.

Prerequisites:

Your UrBackup server VM (Linux) is running.
At least 2-3 client VMs (Windows or Linux) are running and configured to back up to your server. If you only have one or two, try to get another one set up for this workshop. The more clients, the better the simulation.
Basic file backup paths configured on each client. Ensure these paths contain a moderate amount of data (e.g., 1-5 GB per client). You can use fsutil file createnew dummyfile.bin <size_in_bytes> on Windows or fallocate -l 1G dummyfile.bin on Linux to quickly create large dummy files in their backup paths.
SSH access to your UrBackup server VM to run command-line monitoring tools.

Packages for monitoring tools installed on the UrBackup server VM:

sudo apt update
sudo apt install htop iotop iftop sysstat # iotop is an alternative to iostat for live I/O monitoring

Steps:

Prepare Clients for Concurrent Backups:
- On each client VM, ensure there's a decent amount of data in the configured backup paths (e.g., a few GB). If you added dummy files, that's good.
- In the UrBackup server UI ("Settings" -> "Client" or "Default"), ensure the backup intervals are such that they won't all start automatically right away, or that you can easily trigger them manually. For this test, manual triggering is best.
Set Initial Max simultaneous backups:
- In the UrBackup server UI, go to "Settings" -> "General" -> "Server".
- Set Max simultaneous backups to a value equal to the number of client VMs you have for this test (e.g., if 3 clients, set it to 3). If you have many clients (5+), maybe start with a slightly lower number like 3 or 4 to avoid completely overwhelming a small lab VM.
- Click "Save".
Prepare Monitoring on the UrBackup Server VM:
- Open multiple SSH terminal windows to your UrBackup server VM. This allows you to view different monitoring tools simultaneously.
- Terminal 1: htop
```
htop
```
  (Observe CPU usage per core, memory usage, load average, urbackupsrv process)
- Terminal 2: iotop or iostat
  - For iotop (live I/O per process):
```
sudo iotop -oP # -o only show processes doing I/O, -P processes not threads
```
  - For iostat (summary disk stats, updates every 2 seconds):
```
iostat -xzcms 2 # Shows extended stats, kB/s, merged stats, color, for all block devices
```
    (Focus on %util for your backup storage drive, await times, and read/write speeds rkB/s, wkB/s)
- Terminal 3: iftop
```
sudo iftop -NnPp # -N resolve port numbers, -n no hostname lookup, -P show ports, -p promiscuous off
```
  (Observe network traffic on your server's main network interface)
Initiate Backups Simultaneously:
- Go to the UrBackup server web UI "Status" page.
- For each of your test client VMs, quickly click "Actions" -> "Start file backup" (Incremental). Try to start them all within a few seconds of each other.
Observe and Record Performance Metrics:
- As the backups run concurrently, closely watch your monitoring tools for about 5-10 minutes (or until backups complete).
- htop:
  - What is the overall CPU utilization? Are any cores maxed out at 100%?
  - How much CPU is the urbackupsrv process (or its threads) using?
  - How is memory (RAM) usage? Is swap being used?
  - What is the load average? (A load average higher than the number of CPU cores for extended periods indicates the system is overloaded).
- iotop/iostat:
  - What is the %util for your backup storage disk? Is it consistently near 100%?
  - What are the read/write speeds (MB/s or kB/s)?
  - Are await times high (e.g., >20-50ms for HDDs, >1-5ms for SSDs consistently can indicate a bottleneck)?
  - Which processes are doing the most disk I/O (from iotop)?
- iftop:
  - What is the total network throughput (send/receive)?
  - Is it approaching the limit of your VM's network interface or your physical network segment (e.g., near 125MB/s for a 1Gbit link)?
- UrBackup Web UI:
  - How fast are the individual backups progressing (as reported in the UI)?
  - Are there any warnings or errors?
- Note down your observations. Try to identify the primary bottleneck: Is it CPU, Disk I/O, RAM, or Network? For a typical lab VM with HDD-backed virtual disks, Disk I/O is often the first bottleneck.
Reduce Max simultaneous backups and Repeat:
- Wait for all current backups to finish.
- In the UrBackup server UI ("Settings" -> "General" -> "Server"), reduce Max simultaneous backups to a lower value (e.g., 1 or 2). Click "Save".
- Again, simultaneously initiate file backups for all your client VMs.
- Observe the performance metrics again.
  - How does CPU usage compare to the previous run?
  - How does disk I/O utilization (%util) compare? Is it still 100%, or does it have more "breathing room"?
  - Are individual backup speeds faster or slower? (Individual might be faster if the server isn't thrashing, but overall time for all clients might be longer).
  - Is the server generally more responsive?
Analyze and Discuss:
- Compare your observations from the two runs (high concurrency vs. low concurrency).
- What was the apparent bottleneck in the high-concurrency scenario? Did reducing concurrency alleviate that specific bottleneck?
- How did reducing concurrency affect:
  - Server resource usage (CPU, disk)?
  - Perceived server responsiveness?
  - Individual client backup speed?
  - Overall time to back up all clients (if you let them finish)?
- Based on your server VM's resources, what do you think an optimal Max simultaneous backups value might be? (It's a balance between utilizing resources and not overwhelming them).
- Discuss how you would apply these monitoring techniques and tuning steps in a real-world scenario with more powerful hardware. For example, if disk I/O was the bottleneck on your HDD-based VM, what hardware change would most directly address this (e.g., SSD for metadata, faster RAID array for data)? If CPU was maxed out, what would be the solutions?

This workshop provides a practical feel for performance monitoring and the impact of a key tuning parameter. In a real environment, this process would be ongoing, especially as the number of clients or data volume grows. Remember to set Max simultaneous backups back to a reasonable default for your lab (e.g., 2) after the workshop.

10. Troubleshooting Common UrBackup Issues

Even with a well-configured system, issues can arise. Being able to effectively troubleshoot is a key skill for any self-hosted service. This section covers common problems encountered with UrBackup servers and clients, log analysis techniques, useful command-line tools, and where to find help.

Common Server Issues

Server Not Starting:
- Symptoms: Web interface is inaccessible; sudo systemctl status urbackup-server.service shows failed or inactive state (Linux); UrBackup Server service in services.msc is stopped or fails to start (Windows).
- Possible Causes & Solutions:
  1. Check Server Logs: This is the absolute first step.
    - Linux: /var/log/urbackup.log (primary log) and system logs (journalctl -u urbackup-server.service or /var/log/syslog).
    - Windows: C:\Program Files\UrBackupServer\urbackup.log (if it exists, might be named differently or path might be in ProgramData) and Windows Event Viewer (Application and System logs).
    - Look for error messages like "FATAL:", "ERROR:", or stack traces around the time of the startup attempt.
  2. Storage Path Issues: A very common cause.
    - Path Does Not Exist: The configured "Backup storage path" in UrBackup settings (e.g., /mnt/backups, D:\UrBackupStorage) is not currently accessible.
      - Linux: Is the drive mounted? Use df -h or lsblk to check. If it's an external drive or network mount, ensure it's mounted correctly. Check /etc/fstab for persistent mount errors.
      - Windows: Is the drive letter correct and the drive online in Disk Management?
    - Incorrect Permissions: The user account under which the UrBackup server process runs (typically urbackup on Linux, or the service account like Local System or a dedicated user on Windows) does not have read/write permissions to the specified backup storage path.
      - Linux: sudo chown -R urbackup:urbackup /path/to/storage and sudo chmod -R 750 /path/to/storage (or appropriate permissions).
      - Windows: Check the Security tab in the folder's Properties.
    - Disk Full or Filesystem Read-Only: The storage volume is full, or the filesystem has encountered errors and been remounted in read-only mode (common on Linux after critical errors).
      - Linux: df -h to check space. dmesg | grep "EXT4-fs error" (or similar for XFS/Btrfs) for filesystem errors. Run fsck on the unmounted filesystem if errors are suspected.
      - Windows: Check available space in File Explorer. Run chkdsk /f D: (replace D: with your drive) from an admin command prompt.
    - Filesystem Corruption: Severe corruption on the backup storage filesystem can prevent UrBackup from accessing its data.
  3. Database Issues: UrBackup uses SQLite databases (e.g., urbackup/backup_server.db, urbackup/backup_server_files.db, urbackup/backup_server_links.db, urbackup/backup_server_settings.db).
    - Corruption: These database files can become corrupted due to improper shutdowns, disk errors, or software bugs. The server log will usually indicate database errors (e.g., "database disk image is malformed", "SQLite error").
    - Solution: UrBackup often creates backups of its databases (e.g., .db-wal.bak, _TEST files, or in a db_backup subfolder). You might be able to restore from a recent backup by stopping the server, renaming the corrupted .db file, and renaming a valid backup to the original .db name. Refer to UrBackup documentation or forums for specific database recovery steps. The urbackup_srv repair-database command can sometimes help.
  4. Port Conflicts: Another application might be using one of UrBackup's required ports (default: TCP 55413, TCP 55414, UDP/TCP 55415).
    - Linux: sudo ss -tulnp | grep -E '55413|55414|55415' or sudo netstat -tulnp | grep -E '55413|55414|55415'.
    - Windows: Use netstat -ano | findstr "55413" (and for other ports) or Resource Monitor (resmon.exe) -> Network -> Listening Ports.
    - If a conflict exists, either stop the other application or reconfigure UrBackup (or the other app) to use different ports. UrBackup port settings are in the advanced server settings.
  5. Configuration File Errors: The main server configuration file (/var/urbackup/settings.cfg on Linux, or in the UrBackup Server program data folder on Windows) might be corrupted or contain invalid syntax. This is rare.
  6. Insufficient RAM/Resources: If the server is extremely constrained for RAM, it might fail to start critical processes. Check dmesg or system logs for Out-Of-Memory (OOM) killer messages.
Web Interface Inaccessible or Unresponsive:
- Symptoms: Browser times out, shows connection refused, or the UI is extremely slow.
- Possible Causes & Solutions:
  1. Server Not Running: See "Server Not Starting" above.
  2. Firewall Blocking Port 55414: Ensure the server's host firewall (e.g., ufw, firewalld, Windows Firewall) and any network firewalls between your browser and the server allow incoming TCP connections on port 55414.
  3. UrBackup Server Listening on Wrong IP: If UrBackup is configured to bind its web server to a specific IP (e.g., 127.0.0.1 for use with a reverse proxy) and you're trying to access it directly via another IP. Check "IP address to bind web server to" in UrBackup server settings.
  4. Reverse Proxy Issues: If using Nginx/Apache as a reverse proxy, the proxy itself might be misconfigured or not running. Check proxy logs.
  5. Server Overload: If the server is extremely busy (e.g., many concurrent backups, cleanup tasks, verification), the web UI might become slow or unresponsive. Check server load (CPU, disk I/O, RAM).
  6. Database Load: Heavy database activity can slow down UI responsiveness. This is more likely if the database is on a slow HDD or if there are very many clients/files.
Backups Not Starting or Stuck:
- Symptoms: Clients report backups are pending but never start, or progress stays at 0% or a low percentage for a very long time.
- Possible Causes & Solutions:
  1. Backup Window Restrictions: The current time is outside the configured backup window for the client or globally. Check "Backup window" settings (global and client-specific).
  2. Max simultaneous backups Reached: The server is already handling its maximum configured number of concurrent backups. Other backups will be queued.
  3. Client Offline or Unreachable: The client machine is powered off, not connected to the network, or firewalled from the server.
  4. Server Storage Full: The backup storage path on the server has run out of space. UrBackup will stop accepting new backups. Check with df -h.
  5. Client-Side Issues:
    - VSS problems (Windows): Volume Shadow Copy Service errors on the Windows client can prevent image backups or even file backups (if "Enable backup of open files" is used via VSS). Check Windows Event Viewer (Application and System logs) on the client for VSS errors (Event IDs 8193, 12293, etc.). Run vssadmin list writers and vssadmin list providers on the client to check status.
    - Client service stopped/crashed: The UrBackupClientBackend service on the client might not be running.
    - Client misconfiguration: Incorrect server address/port on the client, or client identity issues.
  6. Network Issues: Unreliable network connection between client and server, high packet loss, or insufficient bandwidth.
  7. Indexing Phase Taking Long: For very large file systems or many small files, the initial indexing phase (where the client scans for changes) can take a long time. Check client logs for progress.
  8. Server Overload: As above, an overloaded server may struggle to start new tasks.
High Server Resource Usage (CPU, Disk, RAM):
- Symptoms: Server is slow, unresponsive, or fans are running constantly. Monitoring tools show high utilization.
- Possible Causes & Solutions:
  1. Too Many Concurrent Backups: Reduce Max simultaneous backups.
  2. Cleanup/Maintenance Tasks: Server cleanup (removing old backups), verification (verify-hashes), or database vacuuming can be resource-intensive. Schedule these for off-peak hours.
  3. Inefficient Backup Configuration: Backing up very large numbers of tiny files, or very large files with high change rates, can be demanding.
  4. Filesystem Choice: Some filesystems are more CPU or I/O intensive for UrBackup's workload. (e.g. Btrfs CoW can have overhead).
  5. Hardware Bottlenecks: Insufficient CPU power, slow HDDs (especially for the database), or not enough RAM. Refer to performance optimization section.
  6. Compression: High compression levels are CPU intensive.

Common Client Issues

Client Not Connecting to Server / "Offline":
- Symptoms: Client shows as "Offline" in server UI; client tray icon reports no server connection.
- Possible Causes & Solutions:
  1. Client Service Not Running: Ensure UrBackupClientBackend service is running on the client machine (check services.msc on Windows, systemctl status urbackupclientbackend or ps aux | grep urbackupclientbackend on Linux).
  2. Incorrect Server Address/Name on Client:
    - The client might be configured with the wrong server IP address, hostname, or port (if not using defaults).
    - On Windows, this can sometimes be in C:\Program Files\UrBackup\urbackup\data\settings.cfg or discovered settings. Use urbackupclientctl to set server details.
    - On Linux, settings might be in /etc/urbackup/client.conf or similar, or via urbackupclientctl.
  3. Firewall Blocking Client Outbound or Server Inbound:
    - Client's firewall might be blocking outbound connections from UrBackupClientBackend.exe or urbackupclientbackend.
    - Server's firewall might be blocking inbound connections from the client's IP on ports 55413 or 55415.
  4. Network Connectivity Issues: General network problems (DNS resolution failure if using hostname, routing issues, no physical link). Ping the server from the client and vice-versa.
  5. Server Not Running or Unreachable: The UrBackup server itself might be down.
  6. Client Identity Mismatch (server_ident.txt / server_ident.key):
    - If the server was reinstalled or its identity changed, clients will refuse to connect to the "new" server unless their server_ident.txt (Windows) or server_ident.key/server_ident.pwd (Linux) is updated or removed to allow re-pairing.
    - If the client was reinstalled and you want it to connect as an "old" client, its own identity (client_ident.key, client_ident.privkey) needs to be restored.
Backups Failing on Client:
- Symptoms: Backups start but fail with errors reported in server UI logs or client logs.
- Possible Causes & Solutions:
  1. Check Client Logs: Crucial for details.
    - Windows: C:\Program Files\UrBackup\urbackup.log or via Tray Icon -> Log file.
    - Linux: /var/log/urbackupclient.log or /usr/local/var/urbackupclient.log.
  2. VSS Errors (Windows Image/File Backups):
    - As mentioned above, check Windows Event Viewer for VSS errors.
    - Common causes: Low disk space on volumes involved in VSS, conflicting backup software, problematic VSS writers (run vssadmin list writers to see their state; restart problematic writer services or the "Volume Shadow Copy" service).
    - Try rebooting the Windows client.
  3. Permission Denied (File Backups): The UrBackup client (running as SYSTEM on Windows or often root/urbackup user on Linux) cannot access certain files or directories specified for backup due to restrictive permissions.
  4. Files in Use / Locked Files (File Backups without VSS): If not using VSS for file backups on Windows, or on Linux for files locked exclusively, UrBackup might not be able to back them up. VSS (Windows) or custom pre/post scripts with LVM snapshots (Linux) can help.
  5. Insufficient Disk Space on Client (for VSS snapshots): VSS needs some free space on the volume being snapshotted to store the shadow copy.
  6. Network Interruption During Backup: If the connection drops mid-backup. UrBackup usually attempts to resume.
  7. Unsupported Filesystem Features or Corruption (Client-Side): Trying to back up from a heavily corrupted filesystem on the client can cause errors.
  8. Special Files or Paths: Problems backing up pseudo-filesystems (/proc, /sys on Linux if accidentally included), or very long paths/filenames on Windows (though UrBackup generally handles these well).
Client Using Excessive Resources:
- Symptoms: Client machine is slow during backups.
- Possible Causes & Solutions:
  1. CPU Usage: Hashing can be CPU intensive.
    - On the server, adjust "Background priority for file backups on clients" (Settings -> Client -> Advanced).
    - On the client, urbackupclientctl set-settings --background-priority <value> or edit client settings file if applicable.
  2. Disk I/O: Client disk can be busy during scanning and reading.
    - Exclude volatile temporary directories that change frequently.
    - Ensure client hard drive is healthy and not overly fragmented.
  3. Throttling: Use client-side bandwidth or CPU throttling options if available through urbackupclientctl or client config files (less common to configure directly on client vs server).

Log Analysis Techniques

Effective log analysis is key to diagnosing most problems.

Start with the UrBackup Server Log: /var/log/urbackup.log (Linux). This gives an overview and often points to client-side issues.
Examine Client Logs for Specific Failures: /var/log/urbackupclient.log (Linux), C:\Program Files\UrBackup\urbackup.log (Windows).
Correlate Timestamps: Match timestamps between server and client logs to understand the sequence of events.
Look for Keywords: ERROR, FATAL, WARNING, failed, permission denied, VSS, timeout, database.
Increase Log Verbosity (If Needed):
- Server: Settings -> Advanced -> "Log level" (e.g., change from "info" to "debug"). Be cautious, debug logs can be very verbose and fill up disk space quickly. Change back after troubleshooting.
- Client: Often controlled by urbackupclientctl set-settings --log-level debug or in a client config file.
Check System Logs:
- Linux: journalctl -xe, dmesg, /var/log/syslog.
- Windows: Event Viewer (Application, System, and sometimes specific logs for VSS).

Useful Command-Line Tools

urbackup_srv (Server-side, run as root/admin):
- urbackup_srv remove-unknown: Cleans up orphaned data.
- urbackup_srv cleanup -a <percentage>%: Manages disk space based on retention.
- urbackup_srv verify-hashes: Verifies integrity of stored backups.
- urbackup_srv vacuum: Optimizes server databases.
- urbackup_srv repair-database: Attempts to repair corrupted server databases.
- urbackup_srv test-server-speed: Basic performance test for storage.
- urbackup_srv dump-db-infos: Dumps some database statistics.
urbackupclientctl (Client-side, run as root/admin):
- urbackupclientctl status: Shows current client status.
- urbackupclientctl start -f/-i: Manually start file/image backup (incremental). Add -F/-I for full.
- urbackupclientctl list-backupdirs: Shows directories configured for file backup.
- urbackupclientctl add-backupdir -d /path/to/backup: Adds a directory.
- urbackupclientctl set-settings --server-url <URL> --internet-mode <true/false> ...: Configure various client settings. Use urbackupclientctl set-settings --help for options.
- urbackupclientctl browse: Browse backups (limited functionality).
- urbackupclientctl restore-start ...: Initiate restores.
Standard OS Tools:
- Networking: ping, traceroute/tracert, netstat/ss, ip a/ipconfig, nslookup/dig.
- Disk: df, du, lsblk, iostat, iotop, fsck, chkdsk.
- Process/Memory: top, htop, ps, vmstat, Task Manager, Resource Monitor.

Seeking Further Assistance

If you're stuck:

Official UrBackup Forums: (forums.urbackup.org) This is the primary place for community support. Search for existing topics before posting a new one. Provide detailed information:
- UrBackup server and client versions.
- Operating systems involved.
- Relevant log snippets (use code tags).
- Clear description of the problem and steps taken.
UrBackup Server Administration Manual: Available on the UrBackup website. Contains detailed explanations of settings and features.
GitHub Issues (for bugs): If you suspect a software bug, you can check or report it on the UrBackup GitHub repository, but forums are generally for support questions.

Troubleshooting is a methodical process: observe symptoms, form a hypothesis, test it (often by checking logs or making a small change), and repeat until the root cause is found and resolved.

Workshop Troubleshooting Common UrBackup Issues

Project: Diagnose and Resolve a Simulated Client Backup Failure (VSS Issue on Windows).

Objective: To practice using logs and diagnostic tools to identify why a client backup is failing and then implement a common solution.

Prerequisites:

Your UrBackup server VM is running.
A Windows Client VM is running and configured for UrBackup.
Basic file backups from the Windows Client VM were previously working. Image backups should also be configured for the C: drive.
Administrator access to both the UrBackup server and the Windows Client VM.

Scenario Simulation (Manual Intervention): We will simulate a common VSS (Volume Shadow Copy Service) issue on the Windows client that prevents image backups.

Steps:

Simulate a VSS Writer Problem on Windows Client VM:
- On your Windows Client VM:
  - Open Services (services.msc).
  - Find a VSS-related service that is often problematic or can be safely stopped for a short test. A good candidate is "SQL Server VSS Writer" if SQL Server Express or another version is installed (often installed by other apps). If not, find another non-critical service that might register as a VSS writer, or even temporarily stop the "Volume Shadow Copy" service itself briefly to simulate a problem (but restart it afterwards).
  - For this workshop, let's try to disrupt a writer: If you have Microsoft SQL Server installed (even Express, sometimes installed by Visual Studio or other dev tools), find "SQL Server VSS Writer". Right-click and Stop it.
  - If you don't have SQL Server, as an alternative, you can try stopping the "Cryptographic Services" and then try to run vssadmin list writers. It might show some writers in a failed state. Remember to restart it after this step.
  - Verify a VSS problem: Open an administrative Command Prompt or PowerShell on the Windows Client VM and run:
```
vssadmin list writers
```
    Look through the output. If you successfully stopped a writer service, you might see a writer listed with a "State" other than [1] Stable (e.g., [7] Failed, [8] Failed, [9] Failed, [10] Failed, or [11] Failed) or "Last error" indicating a problem (e.g., "Retryable error", "Non-retryable error", "Writer error"). If all writers are stable, your attempt to break one might not have worked, or the problematic service wasn't critical. The goal is to create a scenario where an image backup might fail due to VSS.
  - Note: If "SQL Server VSS Writer" was stopped, leave it stopped for the next step.
Attempt an Image Backup:
- Go to your UrBackup server web UI.
- On the "Status" page, find your Windows Client VM.
- Click "Actions" -> "Start image backup" (Incremental or Full for drive C:).
Observe the Failure:
- Monitor the backup progress in the UrBackup UI ("Status" and "Activities").
- The image backup will likely fail, or get stuck, or complete with errors.
- Note any error messages shown in the UrBackup UI for this client's activity.
Troubleshooting - Step 1: Check UrBackup Server Logs:
- In the UrBackup server UI, go to "Logs".
- Filter for logs related to your Windows Client VM and the failed backup attempt.
- Look for messages like "Error creating shadow copy", "VSS error", or messages indicating the client aborted or failed. The server log might be general, pointing towards a client-side issue.
Troubleshooting - Step 2: Check UrBackup Client Logs:
- On the Windows Client VM:
  - Open the UrBackup client log file: C:\Program Files\UrBackup\urbackup.log (or access via Tray Icon -> Log file).
  - Scroll to the entries corresponding to the failed backup attempt.
  - Look for detailed error messages. You should see more specific VSS-related errors, references to shadow copy creation failures, or error codes.
Troubleshooting - Step 3: Check Windows Event Viewer on Client:
- On the Windows Client VM, open Event Viewer (eventvwr.msc).
- Navigate to "Windows Logs" -> "Application".
- Look for Error or Warning events around the time of the backup attempt.
  - Source: VSS (Volume Shadow Copy Service) - These are highly relevant.
  - Source: VolSnap (Volume Snapshot Driver)
  - Source: UrBackupClientService (if it logs specific errors here).
- Note down any relevant Event IDs and error messages. Common VSS error Event IDs include 8193, 12293, 12289. The error messages will often point to a specific VSS writer or a general VSS problem.
Troubleshooting - Step 4: Re-check VSS Writers Status:
- On the Windows Client VM, in an administrative Command Prompt/PowerShell, run:
```
vssadmin list writers
```
  Confirm if any writers are in a failed state or reporting errors. If "SQL Server VSS Writer" was stopped, it should show as failed or not present.
Implement a Solution:
- Identify the problematic writer: Based on vssadmin list writers and Event Viewer, if a specific writer is failing (like our simulated "SQL Server VSS Writer" being stopped), the solution is often to restart the service associated with that writer.
- Go back to Services (services.msc) on the Windows Client VM.
- Find the service for the problematic writer (e.g., "SQL Server VSS Writer").
- Right-click and Start it. (If it was "Cryptographic Services", restart that).
- If the "Volume Shadow Copy" service itself was stopped, restart it. Ensure its Startup Type is "Manual".
- Run vssadmin list writers again. All writers should ideally now report [1] Stable and "No error".
Retry the Image Backup:
- Go back to the UrBackup server UI.
- Attempt the image backup for the Windows Client VM again.
Verify Success:
- Monitor the backup. It should now (hopefully) complete successfully.
- Check UrBackup server and client logs to confirm no VSS errors.

Discussion Points:

How did the different log sources (UrBackup server, UrBackup client, Windows Event Viewer) provide different pieces of the puzzle?
Why is vssadmin list writers a critical command for Windows backup troubleshooting?
What are other common causes of VSS failures (e.g., low disk space for shadow storage, conflicts with other backup software, needing a reboot)?
If restarting a writer service didn't fix it, what would be your next steps (e.g., checking specific application logs for that writer, rebooting the client, researching the specific VSS error code online)?

This workshop demonstrates a common troubleshooting workflow for client-side backup failures, emphasizing the importance of systematic log checking and using OS-specific diagnostic tools.

11. Backup and Restore of the UrBackup Server Itself

While UrBackup is designed to back up your clients, it's crucial to remember that the UrBackup server itself is a critical piece of infrastructure containing valuable configuration, historical data, and potentially unique server-side encryption keys or identities. Protecting the UrBackup server ensures you can quickly recover your entire backup environment in case of server hardware failure, OS corruption, or other disasters.

Why Backup the UrBackup Server?

Configuration Data: The server stores all client configurations (backup paths, schedules, retention policies), server settings (storage paths, users, groups, network settings), and client identities. Recreating this manually for many clients would be extremely time-consuming and error-prone.
Backup Database and Metadata: UrBackup maintains databases that track all backed-up files, versions, deduplication information, and backup history. Losing this metadata would mean your existing client backups on the storage might become difficult or impossible for a new UrBackup server instance to recognize and use efficiently. You might lose the ability to browse past versions or perform deduplicated restores.
Client Trust Relationships: Clients store the identity of the server they trust. If you rebuild a server from scratch, clients might not trust it, requiring re-authentication or reconfiguration on each client.
Server-Side Scripts or Customizations: Any custom scripts, SSL certificates for the web UI, or specific OS-level configurations for the UrBackup server would be lost.
Downtime Reduction: Having a backup of the UrBackup server allows for much faster recovery of your backup services compared to rebuilding from scratch.

What to Backup on the UrBackup Server

The key components to back up on your UrBackup server (assuming a Linux installation) are:

UrBackup Database Files:
- These are the most critical. They are typically located in /var/urbackup/ (or a similar path depending on installation, e.g., /usr/local/var/urbackup/).
- Key files include:
  - backup_server.db
  - backup_server_files.db
  - backup_server_links.db
  - backup_server_settings.db
  - And potentially others like backup_server_hashes.db, backup_server_clients.db.
- Also include any journal files like *.db-shm, *.db-wal.
- UrBackup often creates its own backups of these databases in a subfolder like /var/urbackup/db_backup/ or by appending _TEST or .bak during certain operations. Including these is also a good idea.
UrBackup Configuration Files:
- settings.cfg: Usually in /var/urbackup/ or the main UrBackup data directory. Contains server settings.
- Client-specific configuration files if you've made manual edits not managed by the database (less common).
UrBackup Identity Files:
- server_ident.key
- server_ident.privkey
- server_ident.pub
- These are critical for clients to recognize and trust your server. Usually located in /var/urbackup/ or /usr/local/var/urbackup/.
Web Interface SSL Certificates and Keys (if using HTTPS):
- If you've set up a reverse proxy (like Nginx or Apache) with SSL certificates for the web UI, back up these certificate files (e.g., from /etc/nginx/ssl/, /etc/letsencrypt/live/).
Reverse Proxy Configuration Files:
- The Nginx or Apache configuration files for your UrBackup reverse proxy site (e.g., /etc/nginx/sites-available/urbackup-proxy).
(Optional but Recommended) Operating System Configuration:
- While not strictly UrBackup files, relevant OS configs like:
  - /etc/fstab (if you have specific mount points for backup storage).
  - Network configuration files.
  - Firewall rules (ufw configurations, iptables save files).
  - Any custom scripts related to UrBackup server management.

What NOT to typically back up as part of the "UrBackup server backup":

The Actual Client Backup Data Storage Path: (e.g., /mnt/backups, D:\UrBackupStorage). This data is the primary output of UrBackup, not its internal configuration. This data should be protected by its own means (RAID, filesystem snapshots, replication to another server if desired for DR, etc.). Backing this up again with another tool is usually redundant and resource-intensive, unless it's part of a separate disaster recovery strategy for the backup data itself (e.g., replicating to an offsite location). The focus here is on backing up the UrBackup server application state and configuration.

Backup Methods for the UrBackup Server

Stopping UrBackup Server and Copying Files (Safest for Database Consistency):
- This is the most straightforward and reliable method to ensure database consistency.
- Steps:
  1. Stop the UrBackup server service: sudo systemctl stop urbackup-server.service
  2. Use rsync, tar, or cp -a to copy the critical directories and files (listed above) to a separate backup location (e.g., another disk, a network share, a USB drive).
    - Example using tar:
      sudo tar -czvf /path/to/backup_location/urbackup_server_backup_$(date +%Y%m%d).tar.gz \ /var/urbackup \ /etc/nginx/ssl \ /etc/nginx/sites-available/urbackup-proxy \ # Add other relevant OS config files
    - Example using rsync:
      sudo rsync -avz --delete /var/urbackup/ /path/to/backup_location/urbackup_config_backup/var_urbackup/ # Repeat rsync for other paths
  3. Restart the UrBackup server service: sudo systemctl start urbackup-server.service
- Pros: Ensures database files are not being written to during the copy, leading to consistent backups.
- Cons: Requires downtime for the UrBackup server during the backup process. The duration depends on the size of /var/urbackup (mainly databases).
Using Filesystem Snapshots (LVM, Btrfs, ZFS):
- If your UrBackup server's OS or the partition hosting /var/urbackup is on a filesystem that supports atomic snapshots (like LVM, Btrfs, or ZFS), this is an excellent method.
- Steps:
  1. Create a temporary snapshot of the filesystem/volume containing /var/urbackup.
  2. Mount the snapshot (if necessary).
  3. Back up the relevant files from the snapshot (UrBackup server can continue running).
  4. Unmount and delete the snapshot.
- Pros: Minimal or no downtime for the UrBackup server. Provides a point-in-time consistent copy.
- Cons: Requires the underlying OS/filesystem to support snapshots. Adds complexity if not already using such a setup.
Using UrBackup's Internal Database Backup Feature (Partial Solution):
- UrBackup regularly creates backups of its own databases (e.g., in /var/urbackup/db_backup/).
- You can back up this db_backup directory.
- Pros: No UrBackup server downtime needed to back up these already-created database copies.
- Cons: Only covers the databases. You still need to separately back up configuration files (settings.cfg), identity files, SSL certs, etc. The frequency and reliability of these internal backups should be understood. This is more of a supplementary method.
Third-Party Backup Tools (e.g., another UrBackup client, Duplicati, Restic):
- You could theoretically install another backup client (even another instance of UrBackup client, carefully configured not to interfere) on your UrBackup server itself to back up its configuration directories to a different storage location or server.
- This needs careful planning to avoid backup loops or excessive resource usage.
- Using a different tool like Duplicati or Restic to back up /var/urbackup and other config files to a cloud storage or separate server can also work. Ensure the tool can handle open SQLite files gracefully or schedule the backup when UrBackup is stopped or less active.

Frequency and Retention:

Back up your UrBackup server configuration frequently enough to capture significant changes (e.g., daily or weekly, or after any major configuration change).
Keep several versions of these backups.

Restoring the UrBackup Server

The restore process depends on the scenario (e.g., restoring to the same hardware after OS reinstall, or migrating to new hardware).

Prepare the New/Restored Server Environment:
- Install the same operating system as the original UrBackup server.
- Install the same version of UrBackup server software that was running when the backup was taken (or a newer compatible version, but same version is safest for initial restore). Do not start or configure it yet beyond basic installation.
Stop the UrBackup Server Service (if it started automatically):
```
sudo systemctl stop urbackup-server.service
```
Restore the Backed-Up Files:
- Copy or extract your backed-up UrBackup data (/var/urbackup/ contents, identity files, settings.cfg) to their original locations on the new server. Ensure permissions and ownership are correct (e.g., sudo chown -R urbackup:urbackup /var/urbackup).
- Restore SSL certificates and reverse proxy configurations if applicable.
- Restore any other OS-level configurations you backed up.
Verify Backup Storage Path:
- Ensure the actual client backup data storage (e.g., /mnt/backups) is accessible and correctly configured in the restored /var/urbackup/settings.cfg or will be recognized. If migrating to new hardware, you might need to attach the old storage drives or restore this data from its own separate backup/replication.

Start the UrBackup Server Service:

sudo systemctl start urbackup-server.service

Test Thoroughly:
- Access the web interface. Check if settings, users, and clients are present.
- Verify clients can connect and that their backup history appears correct.
- Perform a test backup and restore for a couple of clients.
- Check server logs for any errors.

Important Considerations:

Server Identity: Restoring server_ident.key, server_ident.privkey, and server_ident.pub is crucial for clients to trust the restored server without needing reconfiguration.
Database Integrity: If restoring databases, ensure they were backed up consistently.
Backup Storage Location: The path to your main backup storage (where client data resides) is defined in settings.cfg and within the databases. If this path changes on the new server (e.g., different mount point or drive letter), you'll need to update it carefully, potentially by editing settings.cfg before starting UrBackup and then letting UrBackup re-index or adjust if needed. This can be complex. It's best to try and maintain the same backup storage path if possible.

Workshop Backing Up and Restoring the UrBackup Server Configuration

Project: Perform a manual backup of your UrBackup server's configuration and databases, simulate a "server loss" by renaming the configuration directory, and then restore it.

Objective: To practice the steps involved in backing up and restoring the critical application state of an UrBackup server.

Prerequisites:

Your UrBackup server VM (Linux) is running.
You have sudo access.
A separate location on your server VM to store the backup (e.g., /tmp/urbackup_server_config_backup or a temporary directory in your user's home). In a real scenario, this backup would be on a different machine or disk.

Steps:

Part 1: Backing Up the UrBackup Server Configuration

Identify Key Directories and Files:
- The primary directory is /var/urbackup/ (this contains databases, settings.cfg, identity files).
- If you set up Nginx with SSL in a previous workshop:
  - Nginx config: e.g., /etc/nginx/sites-available/urbackup-proxy
  - SSL certs: e.g., /etc/nginx/ssl/

Create a Backup Destination:

mkdir -p ~/urbackup_server_manual_backup
# This creates a directory in your current user's home directory.

Stop the UrBackup Server Service:

This ensures database consistency for the copy.

sudo systemctl stop urbackup-server.service
sudo systemctl status urbackup-server.service # Verify it's inactive (dead)

Copy the UrBackup Data and Configuration:

Using rsync (preserves permissions and is efficient):

sudo rsync -avz /var/urbackup/ ~/urbackup_server_manual_backup/var_urbackup_copy/

If you have Nginx configs for UrBackup, back them up too:

# Example, adjust paths if yours are different
sudo rsync -avz /etc/nginx/sites-available/urbackup-proxy ~/urbackup_server_manual_backup/nginx_config/
sudo rsync -avz /etc/nginx/ssl/ ~/urbackup_server_manual_backup/nginx_ssl/

Verify the files are copied to ~/urbackup_server_manual_backup.

Restart the UrBackup Server Service (for now):
- (We'll stop it again before the "restore" part of the simulation)
```
sudo systemctl start urbackup-server.service
sudo systemctl status urbackup-server.service # Verify it's active (running)
```
- Quickly check if your web UI is still accessible to confirm UrBackup is working.

Part 2: Simulate Server Configuration Loss

Stop the UrBackup Server Service:

sudo systemctl stop urbackup-server.service

"Lose" the UrBackup Configuration Data:
- We'll rename the /var/urbackup directory to simulate it being deleted or corrupted. Be careful with mv and rm commands!
```
sudo mv /var/urbackup /var/urbackup_OLD
ls /var/ # Verify /var/urbackup is gone and /var/urbackup_OLD exists
```
- If you also backed up Nginx configs, you could simulate their loss by renaming them too, but /var/urbackup is the most critical for UrBackup itself.
Attempt to Start UrBackup Server (It Should Fail or Start with Defaults):
```
sudo systemctl start urbackup-server.service
sudo systemctl status urbackup-server.service
```
- It will likely fail to start properly, or if it does start, it will create a new, empty /var/urbackup directory with default settings.
- If you try to access the web UI, it would be like a fresh install (no clients, default admin password if it even loads).
- Stop it again if it started: sudo systemctl stop urbackup-server.service

Part 3: Restoring the UrBackup Server Configuration

Ensure UrBackup Server is Stopped:

sudo systemctl status urbackup-server.service # Should be inactive (dead)

Restore the Backed-Up Data:
- Copy the backed-up /var/urbackup data back to its original location:
```
sudo rsync -avz ~/urbackup_server_manual_backup/var_urbackup_copy/ /var/urbackup/
```
  (Note the trailing slashes: source_dir/ copies contents, source_dir copies the directory itself. Here we want to restore the contents into a newly created or existing /var/urbackup)
  - If /var/urbackup was recreated empty by a failed start attempt, this rsync will populate it. If it doesn't exist, rsync will create it.
- Ensure correct ownership and permissions (rsync with -a should preserve them, but good to double-check or re-apply):
```
sudo chown -R urbackup:urbackup /var/urbackup
sudo chmod -R 750 /var/urbackup # Or your original restrictive permissions
```
- If you simulated loss of Nginx configs, restore them too:
```
# Example
sudo rsync -avz ~/urbackup_server_manual_backup/nginx_config/urbackup-proxy /etc/nginx/sites-available/
sudo rsync -avz ~/urbackup_server_manual_backup/nginx_ssl/ /etc/nginx/ssl/
```

Start the UrBackup Server Service:

sudo systemctl start urbackup-server.service
sudo systemctl status urbackup-server.service # Verify it's active (running)

Verify the Restoration:
- Web Interface: Open the UrBackup web UI.
  - Are your previous settings restored?
  - Are your users and groups present?
  - Are your clients listed with their history (they might show as offline if client VMs aren't running, but their configuration should be there)?
- Logs: Check /var/log/urbackup.log for any errors during startup. It should start normally and recognize its existing databases.
- Test a Client: If your client VMs are running, see if they connect. Try a small backup/restore operation if possible.
- If you restored Nginx configs, ensure HTTPS access is working as before.
Cleanup (Optional):
- Once you're sure the restore was successful, you can remove the simulated "lost" data and the temporary backup:
```
sudo rm -rf /var/urbackup_OLD
rm -rf ~/urbackup_server_manual_backup
```

This workshop demonstrates the fundamental process of backing up and restoring your UrBackup server's application state. In a real-world scenario, you would automate this backup process and store the backups securely on a separate system or offsite.

Conclusion

Throughout this comprehensive guide, we have journeyed from the foundational concepts of UrBackup to its advanced configurations and operational best practices. You have learned how to install and configure the UrBackup server, manage clients across different operating systems, perform various types of backups and restores, fine-tune performance, secure your deployment, troubleshoot common issues, and even protect the UrBackup server itself.

UrBackup stands out as a powerful, flexible, and cost-effective open-source solution for self-hosting your backup infrastructure. By taking control of your backups, you gain unparalleled data privacy, customization, and the invaluable opportunity to deepen your understanding of server administration and data protection principles.

The workshops included were designed to provide practical, hands-on experience, transforming theoretical knowledge into tangible skills. From planning your deployment and setting up the server, to executing your first backups and restores, navigating advanced configurations, and simulating recovery scenarios, these exercises aimed to build your confidence and competence in managing an UrBackup system.

Key Takeaways:

Planning is Paramount: A successful backup strategy begins with careful planning of what to back up, how often, retention policies, and storage requirements.
File vs. Image Backups: Understand the distinct advantages and use cases for both file-level and full image backups to create a comprehensive data protection plan.
Server and Client Synergy: The client-server architecture of UrBackup allows for centralized management and efficient, deduplicated backups.
Verification is Non-Negotiable: Backups are only useful if they are restorable. Regular verification through test restores and integrity checks is critical.
Security is an Ongoing Process: Protecting your backup server and data requires a multi-layered security approach, including OS hardening, HTTPS for the web UI, secure client communication, and robust network security.
Performance is Scalable: With appropriate hardware and tuning, UrBackup can scale to accommodate growing backup needs. Monitoring is key to identifying and addressing bottlenecks.
Troubleshooting Skills are Essential: Knowing how to read logs, use diagnostic tools, and approach problems systematically will save you time and ensure the reliability of your backup system.
Protect Your Protector: The UrBackup server itself needs to be backed up to ensure a swift recovery of your entire backup environment.

Self-hosting a backup server like UrBackup is a rewarding endeavor. It places you in full command of your data's destiny, offering peace of mind in an increasingly data-driven world. The knowledge and skills you've acquired here form a solid foundation for not only managing UrBackup effectively but also for tackling other self-hosting projects and understanding broader IT infrastructure concepts.

Remember that the world of technology is ever-evolving. Continue to explore UrBackup's documentation, engage with its community forums, and stay updated on best practices in data protection and server administration. Your commitment to learning and diligent management will ensure your UrBackup deployment remains a reliable guardian of your valuable digital assets.