Author Topic: File synchronization between 2 desktops on different locations  (Read 2393 times)

0 Members and 1 Guest are viewing this topic.

Offline BillBoh1971

  • Administrator
  • *****
  • Posts: 605
  • Gender: Male
    • View Profile
Ok, here's the problem. Soon i'm gonna move to a new home and a few days a week i will still come here. I'll have a computer and internet access on both places. The old desktop will have Puppy Linux as OS, the other one will probably dual boot a Ubuntu flavor and Puppy.

I want to keep my files synchronized. I have not the slightest idea what to do. I know there's something like Ubuntu One, but i don't think it works for Puppy. I'd like to have something like pushing on a button before going to the other place, and when arriving there pushing again on a button to download the updated files.

Any computer expert in here who can give me some advice? Gany? BG Jester?

Offline Ganymedes

  • Administrator
  • *****
  • Posts: 3270
  • Gender: Male
    • View Profile
Re: File synchronization between 2 desktops on different locations
« Reply #1 on: May 02, 2010, 09:13:58 AM »
Well, there is synchronization software around. Personally, I never use them and do not believe in them in my use. There are many reasons, but I work with different files on different computers and there are files, like large video files or large pictures, that I definitely do not want to automatically move around (because that kind of synchronization would take forever).

So, I guess in that kind of work the main problem is to decide, which is the Master place and which are the copies ... that is generally speaking not always very easy to solve and is a subject to continuous hassle, which at the minimum takes time and effort. You could have as a master an external USB disk (small 500 GB costs 90 euros). You could also use Internet services for that - however, then you would be totally dependent on them to work and using that might be slow - but depending on your actual case, they could work.

Obviously, you could use a smaller and cheaper media, if your data sets are small and if what you synchronize is organized in such a manner that the synchronized dataset is very small (even if you have a lot of other data that you move very seldom).

Then there is the question of the backup - you cannot have the backup on the same media as the master. I create backups manually, too, because then I have a total control on how different things are spread on different backup disks and what is the interval of the backup.

As for the technology for synchronization (and backup is the same), here are some commands to give you some hints (subject for detailed instructions later). They work only one way - these do not synchronize per se - just make the target the same as what the destination is:

In Linux:

rsync -urv /home/bill/source_directory/ /mnt/media/target_directory/

and

rsync -urv --delete-before /home/bill/source_directory/ /mnt/media/target_directory/

This copies and updates files on the target to be the same as they are on the source. Directory names are imaginative (could be anything but these deal with directories). The latter really deletes extra files on the target.


In Windows:

robocopy /e /v /Z  c:\data\source_directory\   d:\target_directory\

robocopy /e /v /Z /mir  c:\data\source_directory\   d:\target_directory\

The latter with "/mir" gets rid of all the extra files on the target.

Be VERY CAREFUL when trying out with --delete or mir (the first choices are safe), because if your target is wrong, it will first, very quickly I have to remind you, delete everything from the target that the source does not have ... and if the target was wrong, basically everything will go.

It is much safer if you do a script, when using "mirror", because then typos are not possible when running. If not obvious, for backup purposes, these commands are best put into a script and then all you need to do, is to click an icon on the desktop and it will run.

Both run incrementally - meaning that after you have run once for everything, the second time you run, it will only deal with changes. As things are not changing always much, these commands typically run very quickly. Also, if you make any big changes to directory structures, you do not need to worry, because when using "mirroring", everything will be exactly the same on the target (without any old and new directories confusing the matter) ... of course it will take much longer in this case.

Well, in short, if you organize your data well, which is recommended in all cases, with these commands you can do whatever you want. Robocopy.exe is a small Windows executable that you get from Windows Server resource kit (has been around since Windows NT 3.5.1) or from elsewhere. Basically robocopy is the only safe way to copy files in Windows (well, there are GUIs built on top of robocopy, but the basis is the same). Rsync is built in with most Linux systems.

I hope this explain. If not obvious, there are MANY other options and MANY usages for both rsync and robocopy. They definitely worth checking out since they form the basis for single-directional synchronization, safe backup and fast incremental backup. I use them basically all the time with large datasets and with datasets with lots of files and through unreliable or slow network connections. In many cases there are no real substitutes for them.
« Last Edit: May 02, 2010, 09:52:44 PM by Ganymedes »
Long ago, set into rhyme - Ours the right - Our reason to be here.

Offline BillBoh1971

  • Administrator
  • *****
  • Posts: 605
  • Gender: Male
    • View Profile
Re: File synchronization between 2 desktops on different locations
« Reply #2 on: May 03, 2010, 01:43:23 AM »
Well, after trying a few things, i settled for Dropbox. It's a small app which runs for Windows, Mac and the Gnome desktop. Although Puppy doesn't use Gnome, i managed to make it work (just a few "cosmetic" things don't work). It also works cross-platform (at least if the same version of Dropbox is installed on each computer). It's also incredibly fast for a remote service. All i can say is: it seems like a fantastic app, both for sync as for backup. Files are being updated automatically on each registered computer AND on the Dropbox host. Hope i stay as happy about it as i am now.
« Last Edit: May 03, 2010, 01:45:30 AM by BillBoh1971 »

Offline Ganymedes

  • Administrator
  • *****
  • Posts: 3270
  • Gender: Male
    • View Profile
Re: File synchronization between 2 desktops on different locations
« Reply #3 on: May 03, 2010, 10:37:01 AM »
Very good that you found a good WEB-based application that works with your amount of data. That is of course kind of an ideal solution to this.

Long ago, set into rhyme - Ours the right - Our reason to be here.