It's been griping me for YEARS that #syncoid can only give a progress bar for the CURRENT replication task, not for the entire job (when it involves multiple datasets).

The problem is that it relies on an external program for the progress meter, which means you can only meter the one task at a time.

That's gonna change real soon. We gonna have simultaneous progress bars for the current task, and for the ENTIRE set of tasks:

https://github.com/jimsalterjrs/perlpv/

I realised something important today. Encrypted #ZFS datasets will not automatically be sent encrypted with `zfs send`. To ensure the destination receives the dataset in encrypted form, you need to include the send option `w` or `--raw` for a raw stream. The same applies when using #Syncoid, which I'm now happily relying on for snapshots and scheduled #backups.

#admin #gnu #linux #cli

Upgraded the OS on my #HomeLab #KVM #hypervisor and its #Sanoid + #Syncoid #ZFS backup target #server today.

It never ceases to amaze me that upgrading from one #Ubuntu LTS version to the next is way faster than installing the monthly updates on our #Windows servers at work.

Here's my #ZFS diskio, as provided by LibreNMS, as I use #syncoid to copy my #FreeBSD jails from two hosts into one.

The next major step is copying over all the jail configuration. I'm moving from a monolithic /etc/jail.conf to one-file-per jail in /etc/jail.conf.d/*.conf

There will be about 40 jails on this new host.

Using syncoid to backup the zfs snapshots of my Home Assistant installation.

https://dan.langille.org/2022/12/27/using-syncoid-to-backup-zfs-snapshots-home-assistant/

#FreeBSD #zfs #syncoid

I finally have a compelling reason to start copying a #ZFS file system from one host to another. I am using #sanoid on #FreeBSD so, naturally, it seems I should use #syncoid (since they both come from the same package). The primary purpose of this replication is backup; this is a #bhyve instance I want to keep running.

Are you using synoid already for replication between hosts? Please share your recipe.

@sebsauvage #ZFS mirrors et/ou des datasets avec copies=2 et/ou avec #sanoid+#syncoid pour les sauvegardes distantes.

Tests de résilience à la corruption : https://www.openoid.net/testing-the-resiliency-of-zfs-set-copiesn/
> The TL;DR here is that copies=n is better than nothing... but not by a long shot, and you do give up a lot of performance for it. Conclusion: play with it if you like, but limit it only to extremely important data, and don't make the mistake of thinking it's any substitute for device redundancy, much less backups.

@somenxavier If you have the freedom to choose the file-system, and the time investment needed to learn it, #ZFS match all your criterias: you can either use directly the zfs subcommands like snapshot/delete/send/receive or use a higher level tool like #sanoid and #syncoid.

Tip: go to the hidden .zfs/ directory where a ZFS dataset is mounted to browse its snapshots.

If and only if you're using it on a single disk setup, check also #Btrfs for the same features.

@carlchenet
- #ZFS pour les fichiers ; avec un peu de #Perl pour aider : #sanoid (création et expiration des snapshots) et #syncoid (envoi incrémental des snapshots chiffrés via #ssh)
- #rsync vers #ZFS pour les fichiers hors #ZFS
- #NixOS+#Git pour les confs de / et de ~
- #ZFS pour chaque BDD #PostreSQL (aucun dump requis)

À noter que #ZFS ne requiert pas la clé de déchiffrement pour la vérification régulière de l'arborescence des sommes de contrôle des données (zpool scrub)

@ntnsndr clearly more demanding than other solutions (eg. #rclone, #rsync, #BorgBackup) but I'm using #OpenZFS on #Linux for backups, with the help of #sanoid (for taking and pruning snapshots) and #syncoid (for sending them incrementally and permanently encrypted over #ssh)
https://en.wikipedia.org/wiki/ZFS
https://github.com/jimsalterjrs/sanoid
FYI, their author co-hosts a weekly podcast about #ZFS sysadmin https://2.5admins.com/
Also I'm aware of at least one commercial cloud provider https://zfs.rent

@Ninjatrappeur to limit that stress, even with #ZFS which resilvers only the data used and not the whole disk, I still stick with mirroring (it's like RAID1) and avoid the more sophisticated raidz/dRAID. And since using #ZFS, I also keep many external backups up to date, because it's easy even without decrypting datasets (FYI, I'm using #sanoid & #syncoid, either with the help of their #NixOS modules or with standalone #bash scripts)

@luc il y a aussi le couple #sanoid (pour prendre et supprimer automatiquement les snapshots) et #syncoid (pour transférer les snapshots par #ssh) en #Perl. Je l'utilise désormais pour mes #sauvegardes #ZFS.
https://github.com/jimsalterjrs/sanoid

#syncoid supporte bien les fonctionnalités permises par #ZFS : incrémentale, résumable, via #mbuffer, avec ou sans déchiffrement / décompression.

L'intégration à #NixOS nécessite encore un correctif que j'essaye de pousser ici : https://github.com/NixOS/nixpkgs/pull/83904