Cloud-Native Immutable Backups: Restic, S3 Object Lock (WORM), Rabin Fingerprinting & rclone Backends

Executive Summary: While BorgBackup excels on SSH-accessible POSIX servers, modern multi-cloud and object-storage architectures require a single static backup binary capable of speaking natively to AWS S3 / MinIO / Cloudflare R2 / Backblaze B2, SFTP, rclone, and rest-server without requiring a server-side daemon. This architectural guide details the production deployment of Restic, utilizing Rabin Karp Polynomial Fingerprinting, AES-256-CTR + Poly1305-AES authenticated encryption, S3 Object Lock (COMPLIANCE / GOVERNANCE WORM) immutability, and declarative scheduling via resticprofile.

• Zero-Daemon Object Storage

  ---

  Push deduplicated encrypted packfiles directly to S3-compatible buckets (MinIO, Wasabi, B2, R2), SFTP, or 40+ cloud providers via rclone:.

• S3 Object Lock (WORM) Immutability

  ---

  Combine Restic with S3 Object Lock or rest-server --append-only to enforce hardware/cloud-level Write-Once-Read-Many ransomware protection.

• Rabin Polynomial Chunking

  ---

  Split files deterministically along content boundaries using sliding Rabin fingerprints (512 KiB–8 MiB) stored in consolidated pack blobs.

• Declarative resticprofile Orchestration

  ---

  Manage multi-repository profiles, systemd timers, ionice/nice CPU/IO throttling, and Prometheus node_exporter textfile metrics cleanly.

---

1. Architecture: Rabin Fingerprinting, Packfiles & Multi-Backend Storage

Unlike traditional archivers (tar), Restic implements a content-addressable storage (CAS) repository format. Files are split using a random irreducible degree-53 polynomial in $\text{GF}(2)[x]$ (Rabin Fingerprinting). Individual encrypted data and tree blobs are bundled into ~16 MiB Packfiles inside data/00/..data/ff/ to avoid S3 small-object API request penalties:

%%{init: {'flowchart': {'wrappingWidth': 600, 'subGraphTitleMargin': {'top': 15, 'bottom': 15}}}}%%
flowchart TB
    subgraph ClientNode["Production Linux Node (Restic Go Binary)"]
        FS["Source Filesystem<br/>+ Stdin DB Streams"]
        Rabin["Rabin Polynomial Chunker<br/>SHA-256 Content Addressing"]
        Packer["zstd Compression +<br/>AES-256-CTR / Poly1305 Pack"]
    end

    subgraph StorageBackends["Zero-Knowledge Remote Targets"]
        S3WORM[("S3 / MinIO / B2 Bucket<br/>Object Lock WORM Enabled")]
        RestSrv[("Self-Hosted rest-server<br/>--append-only + Prometheus")]
    end

    FS --> Rabin
    Rabin -->|"Deduplicated Blobs"| Packer
    Packer -->|"HTTPS PUT ~16MiB Packs"| S3WORM
    Packer -->|"HTTP/2 TLS mTLS"| RestSrv

Restic vs. BorgBackup Architectural Decision Matrix

Engineering Criterion	Restic (Go Static Binary)	BorgBackup (Python + C Extensions)
Server-Side Requirement	None (Dumb storage: S3, B2, SFTP, HTTP, rclone)	Requires borg binary installed on remote SSH server for deduplication speed
Multi-Host Shared Repo	Native (Multiple hosts can concurrently back up into the same S3 bucket to share deduplication across nodes)	Single-host locking per repository recommended
Chunking Algorithm	Rabin Karp Polynomial over $\text{GF}(2^{53})$	Buzhash cyclic polynomial rolling hash
Encryption Primitive	AES-256-CTR + Poly1305-AES MAC (scrypt KDF)	AES-256-CTR + HMAC-SHA256 or AES-256-OCB / BLAKE2b

---

2. Repository Initialization & Streaming Database Backups

Install the latest Restic binary (sudo apt install restic && sudo restic self-update) and store your repository credentials in a restricted /etc/restic/env.conf (chmod 0600) file:

/etc/restic/env.conf

export RESTIC_REPOSITORY="s3:https://s3.eu-central-003.backblazeb2.com/nejat-sovereign-backups/prod-cluster"
export RESTIC_PASSWORD_FILE="/etc/restic/repo.pw"
export AWS_ACCESS_KEY_ID="003a1b2c3d4e5f0000000001"
export AWS_SECRET_ACCESS_KEY="K003..."
export RESTIC_COMPRESSION="max"

Initialize the repository and verify streaming stdin backups for PostgreSQL without writing unencrypted SQL dumps to disk:

Initializing Restic & Streaming PostgreSQL Directly into an Encrypted Snapshot

source /etc/restic/env.conf

# 1. Initialize the content-addressable repository
restic init

# 2. Stream pg_dumpall directly into Restic via stdin with a virtual filename
sudo -u postgres pg_dumpall --clean \
  | restic backup --stdin --stdin-filename postgresql_cluster.sql --tag database,postgres

# 3. Back up filesystem paths excluding virtual/cache filesystems
restic backup /etc /opt /var/vmail \
  --one-file-system \
  --exclude-caches \
  --exclude="/opt/*/logs" \
  --tag filesystem

---

3. Automated Scheduling & Pruning via resticprofile

Rather than writing fragile custom bash wrappers around restic backup and restic forget --prune, deploy resticprofile—a declarative configuration manager that generates native systemd timers, enforces ionice/nice resource limits, and exports Prometheus .prom textfile metrics:

/etc/resticprofile/profiles.yaml

version: "1"

global:
  ionice: true
  ionice-class: 2
  ionice-level: 6
  nice: 15
  priority: low
  prometheus-save-to-file: "/var/lib/node_exporter/textfile_collector/restic.prom"

default:
  repository: "s3:https://s3.eu-central-003.backblazeb2.com/nejat-sovereign-backups/prod-cluster"
  password-file: "/etc/restic/repo.pw"
  env-file: "/etc/restic/s3.env"
  compression: "auto"
  lock: "/run/resticprofile-default.lock"

  backup:
    source:
      - "/etc"
      - "/opt"
      - "/var/vmail"
    exclude-caches: true
    one-file-system: true
    schedule: "*-*-* 02:30:00"
    schedule-permission: system
    check-before: false
    run-before:
      - "docker exec postgres-db pg_dumpall -U postgres > /opt/db_backups/pg_cluster.sql"
    run-after:
      - "shred -u /opt/db_backups/pg_cluster.sql"

  retention:
    before-backup: false
    after-backup: true
    keep-daily: 14
    keep-weekly: 8
    keep-monthly: 12
    keep-yearly: 3
    prune: true

  check:
    schedule: "Sun *-*-* 04:00:00"
    read-data-subset: "10%"

Install the generated systemd timers with a single command:

Activating Systemd Timers via resticprofile

sudo resticprofile schedule --all
sudo resticprofile status

---

4. Self-Hosted Immutable Target: rest-server in Append-Only Mode

If you also maintain a local NAS or off-site Linux server, run Restic's official high-performance HTTP/2 backend (rest-server) with --append-only and --private-repos enabled:

/opt/rest-server/compose.yaml

services:
  rest-server:
    image: restic/rest-server:latest
    container_name: restic-rest-server
    restart: unless-stopped
    ports:
      - "127.0.0.1:8000:8000/tcp"
    environment:
      OPTIONS: "--append-only --private-repos --prometheus --no-auth"
    volumes:
      - /mnt/zfs_vault/restic:/data:Z

Pruning an --append-only Repository Safely

While --append-only is active on rest-server (or when using an S3 IAM key restricted to s3:PutObject and s3:GetObject without s3:DeleteObject), the production server can create new snapshots (restic backup) but cannot delete old snapshots. Schedule restic forget --prune from an isolated management workstation or a dedicated local cron job on the storage server itself using separate administrative credentials.

---

5. Verification, Bit-Rot Scrubbing & Diff Inspection

Verify cryptographic pack integrity (including reading a random 10% subset of actual data blobs from object storage to detect silent bit-rot) and compare changes between snapshots:

Auditing Snapshot Diffs & Performing Partial Data Verification

source /etc/restic/env.conf

# 1. List all snapshots grouped by host and tag
restic snapshots

# 2. Compare added/modified/removed files and byte delta between the last two snapshots
restic diff $(restic snapshots --json | jq -r '.[-2].short_id') $(restic snapshots --json | jq -r '.[-1].short_id')

# 3. Verify repository structural index AND download/verify 10% of encrypted data packs
restic check --read-data-subset=10%