Talk:Backing Up via Rsync

Rsync Daemon

Why turn on the rsync daemon at all? This could easily be accomplished simply by running

rsync -HrlptgoDPvvS --force --delete --del --stats -e ssh root@IPADDRESS:/ /media/pre-backup/

Also, since -P includes --progress, there's no reason to call it a second time.

-JackieRipper

We don't have a root password set up on the Pre, it's using sudo therefore that's why ssh won't work. Rsyncd.conf gives the rsync daemon root privileges so it can mirror the device. I'm be open to suggestions, but I'm trying to make this fit in with the next steps guide and limit the number of steps.

Thanks for pointing out the redundancy on --progress.

-hopspitfire

Ahhh yes. I keep forgetting that not everyone set up keys for root access.

-JackieRipper

You dont need to be root to use the rsync over ssh to backup, the pervious command will work if you are using the username and password set up in the optware setup.

rsync -HrlptgoDPvvS --force --delete --del --stats -e username@IPADDRESS:/ /media/pre-backup/

I also dont think you need the "ssh" in the command the new versions of rsync use it automatically. I have writen up another tutorial of how to back up the pre without setting up the rsync daemon. check it out here. I have not done a restore yet so I still have to look into that but I see no reason why it cant be done over ssh as long as the partition is r/w and the user has sudo configured.

-elpollodiablo

The problem with running 'rsync -avz' (in your article) is that the files in your backup directory won't get removed when they're not present on the Pre, only overwritten. Have you tested your command above with a normal user account, and did you run into any permission issues? I tested the restore process and it works so far with the methods in the article. -hopspitfire 16:07, 13 October 2009 (UTC)

Yes you are correct mine is not set up to delete folder/files that are no longer on the Pre. I didnt want to do that, I wanted it to back up all the files and keep a backup in case I needed to go back to files I had which I deleted from my Pre. I am also only overwriting the files that have changed. For a general backup utility your command would be best. I have not tested the command above but I have used the one in my article to back up my pre. There was a few problems one with the /dev/gadget folder it backedup one file then it seems to froze on me. so I exclude that folder and ran it again. It restarted on me in /sys/devices/ I excluded that folder and it all seemed to work, but there were alot of errors in the /sys folder. so I am probably going to exclude that one as well. why are you excluding the /proc folder? here is my final code: rsync -avz --size-only --exclude gadget/ --exclude sys/ --progress / username@sshserverIP:/Directory/To/Backup/To

--Elpollodiablo1 01:09, 14 October 2009 (UTC)

Additions

I've added these to my .profile on the Pre:

exclude1="--exclude=/proc --exclude=/sys --exclude=/tmp/webdiskcache"
exclude2="--exclude=/media/internal/AUDIO --exclude=/media/internal/Place_iTunes_Uses"
syncopt1="-az --partial --progress --stats --delete-after --bwlimit=300 --numeric-ids"
syncpath=" / me@$IPADDRESS:~/PreRSync/ "
alias Sync='d=`date +%Y%m%d.%H%M%S` ; sudo rsync --rsync-path=sudo\ rsync $syncopt1 -b --backup-dir=~/PreRSync.bak/$d $syncpath $exclude1 $exclude2'

On the server in the /etc/sudoers I've added something like this (not sure this is the most secure thing to do):

me     ALL=(ALL) ALL, NOPASSWD: /usr/bin/rsync

Now I can simply login to the Pre and run Sync, it prompts for a password at the server but hostkeys could be used to avoid that. The addition of the dated backup options means that I get a copy of any removed or overwritten files at the server. I use something called cwrsync which I believe is the cygwin rsync command without installing all of cygwin, might be handy and simple enough for windows users. --Dolio 07:02, 1 October 2009 (UTC)

Nice additions and thanks for sharing, feel free to add them to the article. I'll have to try these myself. -hopspitfire 07:05, 1 October 2009 (UTC)

Pre Rebooting Issue

If you don't exclude some of the directories then the Pre reboots when the backup process touches them. -hmagoo

Can you specify which files/directories prompt the reboot? -hopspitfire 02:57, 20 August 2009 (UTC)

I could narrow it down to proc and/or sys, that's narrow enough for me to exclude both, any progress on restore testing? I'm not running an emulator. -hmagoo

I tested the restore process and it works. Can you run the backup process and send the output to a file (rsync ... > /media/internal/rsync.log) and post it? -hopspitfire 00:49, 24 August 2009 (UTC)

rsync-outputs.tar.gz this was the console and log output from running a backup. I had already a backup in place in the destination but excluded /dev, /sys/ and /proc initially, ran it this time without those exclusions to test it out (again, as I saw this reboot the very first time I tried this method). rebooted right after this line in the console, similar in the log.

rsync: read errors mapping "/sys/devices/platform/lcd-controller/ctrl_reg_dump" (in root): No data available (61)

-hmagoo

I ran the backup again excluding /sys and everything completed, only errors I got in the console were:

rsync: send_files failed to open "/proc/sys/kernel/sched_nr_migrate" (in root): Permission denied (13)

rsync: send_files failed to open "/proc/sys/net/ipv4/route/flush" (in root): Permission denied (13)

rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1524) [generator=3.0.5]

-hmagoo

Thanks for attaching your log. /sys and /proc don't need to be included in the backup, but /dev does (to initialize devices on boot). After testing the backup and restore, the easiest way is just mkdir the /sys and /proc directories and exlude them in the backup. Does the pre still restart when you excluded those directories?? -hopspitfire 21:23, 26 August 2009 (UTC)

Shouldn't /dev exist fine after a restore and before rsync backup restore? I guess if you made some strange volume changes, but still. --NetWhiz 21:59, 26 August 2009 (UTC)

I was using /dev as an example of a directory that _shouldn't_ be excluded in the backup process, so /dev will exist after a backup prior to a restore (otherwise the Pre won't boot because it can't initialize the mapper devices for storage). -hopspitfire 22:37, 26 August 2009 (UTC)

You mean it SHOULD be excluded b/c you will not need to restore it. --NetWhiz 22:54, 26 August 2009 (UTC)

(failed=reboot).. failed with no excludes. successful with /dev/,/proc,/sys excluded. successful with /sys excluded. but with those three error messages.--Hmagoo 23:52, 26 August 2009 (UTC)

@NetWhiz: /dev needs to be populated on the actual filesystem (before devfs/udev get loaded) for a *nix system to boot.

@Hmagoo: Thanks, I went ahead and fixed the lines in the article.

-hopspitfire 00:12, 27 August 2009 (UTC)

I understand, BUT when you do a restore with WebOS Doctor it will already be there. Are we not talking about the same thing or are we just cross talking? --NetWhiz 01:11, 27 August 2009 (UTC)

OH, I'm talking about a restore using rsync ;) (in this guide, after the intial webOS Doctor). I still don't know if we should be overwriting the entire system (with the rsync backup, including /dev). Any ideas on this? My reasoning for doing a full system overwrite is version compatibility. -hopspitfire 01:16, 27 August 2009 (UTC)

LOL! I would say NOT to overwrite /dev b/c it should already be setup correctly and there is nothing anyone should be doing in there with any mod anyway that I have seen. Trying to overwrite some of the virtual devices can be a BAD thing as some have seen (crashes, freezes, etc.) Just my thoughts. --NetWhiz 01:58, 27 August 2009 (UTC)

Error in /etc/rsyncd.conf on host?

Shouldn't been in the /etc/rsyncd.conf on the host (Backing Up via Rsync#Setting up the host 2) instead of "[root]" "[pre-backup]"? On restoring process came "unknown module" until I changed it. --Pcworld 18:24, 23 August 2010 (UTC) (sorry for my bad English please!)

Restore From Backup

It worked, and very well. I had a Pre+ with a broken display. Preware was not installed, everything had to be done from the Novaterm command line. I was able to backup the broken Pre to my PC, and then restore the backup onto a new Pre.

--Kiransingh 17:32, 13 June 2011 (UTC)

Restore From Backup / Palm Profile Issues

It did not work in my case (Pre-, WebOS 1.4.5): The restore actually worked, but after a few hours a message about not being signed on my palm profile popped up and the device rebooted to first use.

--IDG 20:00, 18 July 2011