TL;DR @ bottom

Over the last year I’ve dove head first into selfhosting various services and apps on an UNRAID server. It’s my first real experience with Linux short of fiddling with a dual boot of Pop!_OS. So that alone has already been quite the ride. Thankfully, UNRAID supports docker compose and has it’s own Community App store where there are very few extra steps to get an app to work short of installing dependencies (MariaDB, Postgres, Redis, etc).

I am trying to be militant in my backups of mostly important files. Critical documents and photos, that sort of thing. Right now I have two copies on the same Win10 PC (two different drives) that backup nightly to iDrive. I recently started testing Backblaze Personal as well from the same computer. What I want to do is get to a point where the other Win10 PCs, MacOS laptops, and my yet to be built Linux desktop are all backed up to the server locally, then shuttled off to something like B2 on a nightly basis. Seemingly there is absolutely no consensus on the best cross platform app to backup the various clients. I currently have Time Machine backing up the laptops to the server, but that’s it. Nothing else in the house is touching the server outside of Immich pulling photos from phones when we’re back on network.

I keep trying to wrap my head around the CLI only options like Restic, Borg, and Kopia but I can’t get to a point that I’m confident enough to ensure I’m backed up and safe as far as data integrity. So that led me to apps with GUIs. I’ve tried UrBackup, BackupPC, KopiaUI, and Duplicacy. That last one is the only one I can get to work reliably and that is backing up server data to B2 right now. I’m still in my trial period for it so nothing set in stone in that regard.

I did see Vorta for Borgbase, but it doesn’t have a Win10. UrBackup refuses to work with MacOS for whatever reason. KopiaUI can’t see directories on the server which I’m guessing is permissions issues and the client side won’t accept an http://ip:port address, instead requiring https and I’m not sure if that’s possible to get around. BackupPC I can’t get to connect to the clients and that project seems somewhat stale with no updates in the last few years.

I’m comfortable hunting for solutions but seemingly the various support forums have a lot of assumptions in regards to prior knowledge of these products and I’m just not connecting the dots.

TL;DR: I apologize that this is so lengthy, but I’m honestly just not sure what to do at this point. I need (in my mind) the following as a solution:

  • Compatibility with Win10, MacOS, and Linux clients
  • Can back up to a local repository on UNRAID
  • Has a GUI (because I apparently am too stupid for CLI-only)
  • Would be nice to also backup to remote buckets like B2 or others but I can leverage something like Rclone GUI for that

Anyone have any advice on either solutions or resources I can dig into to accomplish this?

  • somedaysoon@lemmy.worldEnglish
    4·
    1 year ago

    CLI is the best way… it’s not difficult, borg is really easy to use, what are you hung up on? I can probably help if you have questions on it.

    I don’t use Windows, but you could syncthing important files to your server, then use borg to make legit backups of all the important files on your server. I store all my important files on one of my servers. Basically, all my PCs act more like thin clients… as far as files go anyway. I also backup my entire phone in a similar way as I’m suggesting you for Windows using syncthing for all my photos/videos/documents/etc.

    If you want, you can read this which describes my entire backup solution.

    I like borg because once you figure out what directories you want to backup, then you just put them in a script and schedule to run however often you like it. I also set healthchecks to notify me if it does not run… and for even further peace of mind, I have the output from the script sent to my Telegram just in case there were any errors.

    Here is an example of my config with some changes made to post publicly. This is backing up all the important files from my main server to two other servers. You could also do the same to a local location and/or a remote location like borgbase or backblaze. This server has all the important files for everything, so this one script backs up all my docker services on this server, all my phone backups, all my desktop/laptop backups, and all my music/tv/movies. Literally everything except the docker services running on the other servers.

    #!/bin/bash
#get date
NOW=$(date +"%y.%m")

#notify healthchecks script ran
curl http://thinkcentre.lan:7055/ping/W0WfGWIWDUDP5blEV00pOw/thinkbox-backup-script

#setup env
export BORG_RSH="ssh -i ~/somedir/thekey"
export BORG_PASSPHRASE="password"
export BORG_RELOCATED_REPO_ACCESS_IS_OK=yes


#stop all docker containers
docker stop $(docker ps -a -q)


#run nas backup to 5tb drive (connected to rockpro64 running dietpi)
borg create --stats user@192.168.1.200:/mnt/storage/borg::tv.$NOW /srv/dev-disk-by-uuid-70d630c3-5019-446a-ab57-ef796cfdab76/media/tv/completed.shows /srv/dev-disk-by-uuid-70d630c3-5019-446a-ab57-ef796cfdab76/media/tv/misc
borg create --stats user@192.168.1.200:/mnt/storage/borg::common.$NOW /srv/dev-disk-by-uuid-3a8447fe-a660-4ac7-ae46-2de19b6d59c1/
borg create --stats user@192.168.1.200:/mnt/storage/borg::music.$NOW /srv/dev-disk-by-uuid-70d630c3-5019-446a-ab57-ef796cfdab76/media/music
borg create --stats user@192.168.1.200:/mnt/storage/borg::public.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/public --exclude '/srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/public/downloads/*' --exclude '/srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/public/temp'
borg create --stats user@192.168.1.200:/mnt/storage/borg::backups.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/backups
borg create --stats user@192.168.1.200:/mnt/storage/borg::thinkb.home.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/home
borg create --stats user@192.168.1.200:/mnt/storage/borg::thinkb.docker.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/docker


#run nas backup to 2tb drive (connected to thinkc)
#this one excludes backups/.archive/borg for space
borg create --stats user@192.168.1.125:/shared/storage/borg::tv.$NOW /srv/dev-disk-by-uuid-70d630c3-5019-446a-ab57-ef796cfdab76/media/tv/completed.shows /srv/dev-disk-by-uuid-70d630c3-5019-446a-ab57-ef796cfdab76/media/tv/misc
borg create --stats user@192.168.1.125:/shared/storage/borg::common.$NOW /srv/dev-disk-by-uuid-3a8447fe-a660-4ac7-ae46-2de19b6d59c1/
borg create --stats user@192.168.1.125:/shared/storage/borg::music.$NOW /srv/dev-disk-by-uuid-70d630c3-5019-446a-ab57-ef796cfdab76/media/music
borg create --stats user@192.168.1.125:/shared/storage/borg::public.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/public --exclude '/srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/public/downloads/*' --exclude '/srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/public/temp'
borg create --stats user@192.168.1.125:/shared/storage/borg::backups.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/backups --exclude '/srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/backups/user/.archive/borg/*' 
borg create --stats user@192.168.1.125:/shared/storage/borg::thinkb.home.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/home
borg create --stats user@192.168.1.125:/shared/storage/borg::thinkb.docker.$NOW /srv/dev-disk-by-uuid-b5cb9459-0144-494b-af8c-89db1f742a83/docker


#these services need to start before other services that depend on them
docker start redis
docker start nginx
#start all the other services
docker start $(docker ps -a -q)
    • Father_Redbeard@lemmy.mlOP
      4·
      1 year ago

      That’s the rub. I keep seeing that it’s easy and the best way and folks fawn over Restic and Borg (for seemingly good reasons) so I start to read the documentation on setting it up and my eyes glaze over and I’m lost because there are assumptions made regarding the readers knowledge beforehand. Which is fair in this context. Even you example, which thank you by the way, assumes I know scripting. Yeah…not so much. I will look into what you’ve written and linked though. Maybe it’ll click for me. I also looked at some of the wrappers for both Borg (borgmatic) and Restic (autorestic) and it’s still over my head. I get the general concept that instead of writing it all down, you’re creating configs ahead of time, similar to your script example. But I’m not at a spot where I can dive into that sort of thing yet. I need a ELI5, for CLI stuff I guess…

      • somedaysoon@lemmy.worldEnglish
        2·
        1 year ago

        You can get by with a GUI wrapper for borg, that is another option. I know OpenMediaVault even has it built in to the WebGUI, but I know you are using UnRaid so you’d have to see if they have anything. Otherwise you could maybe find a docker container for Vorta or something to spin up.

        But on a server platform it will greatly benefit you to learn to use a terminal because pretty much all the best and most efficient tooling is going to use it. You may possibly see yourself running into this problem over and over where you want a GUI to do something when GUI tooling on the server platform just doesn’t quite exist or isn’t as efficient or reliable for many different reasons.

        I would say just take small steps, don’t try to take it all in at once. When you have free time, sit down with it, and just be patient and try to understand what you can, and if you get frustrated just put it down and walk away. It’s not a race and you can always go back later. Also, trial and error in small tests with data that won’t matter. That goes for any terminal based tooling. And every new tool you learn, it will get easier and easier.

        And like I said before, if you do have questions feel free to hit me up. I started a selfhosting series on YouTube and have a few videos up, but I will actually be redoing it and trying to make it even more beginner friendly. Talking to someone that doesn’t have much experience with a terminal may be beneficial to knowing what needs to be addressed in them.

        • Father_Redbeard@lemmy.mlOP
          3·
          1 year ago

          All valid points and I appreciate the input. My frustration comes from a lot of the tutorials online for these types of tools already start ahead of my knowledge. I have some terminal knowledge and have been messing around on. VPS I rent just so that if things go really bad, I’m not hosing my production server. And that’s been ok. I’ve even stood up an instance of Miniflux and Wallabag for my Read It Later stack and it’s working really well. I absolutely do need to learn more about using terminal though. I’ve always had good success learning by doing, with a project in mind rather than reading courses only and using their course materials for the actual practice portions. And example would be using digital footage of my own kids to learn video editing instead of the sample clips that came with the book/online class.

          I’ll take you up on the offer for help. I will try again and when I get stuck I’ll shout. I appreciate your input!