OpenSimulator · EN

FSAssets: hot backups and online pre-copy

Production backups, online rsync pre-copy, SQL consistency and restore validation.

Online copying is not the same as a consistent recovery point. All paths are examples only; do not execute commands without confirming your actual directories, services and SQL databases.

Define hot backups and their limits

Copying files while services are writing is a useful pre-copy, but does not automatically form a consistent restore point. FSAssets links SQL metadata to disk payloads. Preserve SQL and files as a recoverable pair; include any active temporary spool in the plan.

Inventory storage paths and backup destination

Before copying, identify configured BaseDirectory, SpoolDirectory, the actual SQL table/database, and every writer. Examples assume data and spool are both under /srv/opensim/fsassets/. If not, back up all real paths separately. Ensure enough free bytes and inodes on a distinct volume.

Generic example to adapt and test, not a ready-to-run production path.

findmnt -T /srv/opensim/fsassets
df -h /srv/opensim/fsassets /mnt/backup
df -i /srv/opensim/fsassets /mnt/backup
# Les chemins sont fictifs : vérifier BaseDirectory et SpoolDirectory.

Low-impact pre-copy while online

Start a first rsync pass without --delete, with deliberately reduced CPU and disk priority. It can be restarted but may include files captured at different times. nice and ionice affect scheduling depending on the storage and kernel; they cannot guarantee zero production impact.

Generic example to adapt and test, not a ready-to-run production path.

nice -n 19 ionice -c2 -n7 rsync -a --info=progress2 \
  /srv/opensim/fsassets/ \
  /mnt/backup/fsassets/

Keep a long copy running across SSH disconnects

For long operations, run rsync inside screen or tmux and detach. Reconnect using screen -r and record the final exit code. If the session disappears, inspect the actual process; a missing screen session does not mean rsync is still working.

Generic example to adapt and test, not a ready-to-run production path.

screen -S fsassets_backup
# Dans screen : lancer la commande rsync précédente.
# Détacher : Ctrl+A, puis D.
screen -ls
screen -r fsassets_backup
# Après sortie de screen, contrôler les journaux et le code de retour.

Preview differences with --dry-run

After the first pass, run rsync -a --dry-run to see what rsync would transfer without writing. Trailing slashes here mean copying a directory’s contents. With active writers, results may change immediately. An empty difference list does not prove SQL consistency, byte-for-byte equality or successful recovery.

Generic example to adapt and test, not a ready-to-run production path.

rsync -a --dry-run --itemize-changes \
  /srv/opensim/fsassets/ \
  /mnt/backup/fsassets/
# Ne change rien. À répéter à l'état figé.

Create a consistent restore point

The easiest consistency boundary is a short no-write window: gracefully stop or block every process that writes to FSAssets, including the ROBUST asset service and any other writers. While writes are stopped, finish rsync, take a database-consistent SQL backup appropriate to the tables, and retain files and dump as one recovery set. Measure actual duration instead of promising a sub-minute outage.

Generic example to adapt and test, not a ready-to-run production path.

# PSEUDO-PROCÉDURE : adapter à l'architecture réelle.
# 1. Bloquer/arrêter TOUTES les écritures FSAssets et vérifier les logs.
rsync -a --info=progress2 \
  /srv/opensim/fsassets/ \
  /mnt/backup/fsassets/
# 2. Tant que les écritures restent arrêtées :
mysqldump --single-transaction --routines --triggers --events \
  --databases opensim > /mnt/backup/opensim.sql
# 3. Vérifier les codes de sortie et conserver ce lot sous un identifiant commun.

SQL backups without shutting down the grid: conditions

An InnoDB --single-transaction dump provides a transactional SQL view under suitable MySQL conditions but does not synchronize the FSAssets payload bytes. A strictly no-pause backup requires a coordinated, tested snapshot or recovery-point mechanism encompassing files, spool and SQL. Do not label a 02:00 SQL dump and a 03:00 file copy as one atomic backup.

Protect backups and handle retention

Copy to a dedicated destination, distinguishing backups from storage migration. Overwriting a single destination each time offers no history. Use dated backups, snapshots or managed retention; avoid --delete on the only recovery copy. Keep a protected copy on another host or off-site.

Verify copy and payload integrity

Check rsync’s exit status, the SQL dump and permissions, then run a final --dry-run while writes remain stopped. -a preserves symlinks, timestamps and permissions, but not necessarily ACLs, xattrs or hardlink relationships: evaluate -HAX --numeric-ids and required privileges where applicable. Content hashing is more expensive and should be scheduled.

Generic example to adapt and test, not a ready-to-run production path.

rsync -a --dry-run --itemize-changes \
  /srv/opensim/fsassets/ \
  /mnt/backup/fsassets/
df -h /mnt/backup
df -i /mnt/backup
ls -lh /mnt/backup/opensim.sql

Prove a restore before trusting the backup

Restore to an isolated environment: SQL and FSAssets tree from the same recovery set, matching path format and service version. Read known assets, validate inventory and textures, and inspect errors. Never run a restore experiment on production data.

Disk migration: one additional final step

For disk relocation (different from a backup), pre-copying can reduce the final stoppage. After halting writes and verifying synchronization, prepare the mount, change the active path or create a symlink, and test before retiring the original store. See the migration chapter for mount and rollback precautions.

Related FSAssets chapters

Backup and integrity · Disk migration · Inodes and capacity