The DAM Forum
July 28, 2021, 06:50:58 AM *
Welcome, Guest. Please login or register.

Login with username, password and session length
News:
 
   Home   Help Search Login Register  
Pages: [1]
  Print  
Author Topic: Syncback operation  (Read 12006 times)
Kevin Forsmo
Newbie
*
Posts: 31



View Profile
« on: June 16, 2009, 08:01:23 PM »

So I'm testing out Syncback since I'm on Windows and am looking for a good sync option.  I'm currently using LR2 as my all-in-one DAM progam, and will eventually probably use EM as well (with all the headaches that will entail).

I've hit my first major snag, which may only occur while I'm screwing with folder structure, so I'd appreciate anyone's input that uses the program.

I'm simulating a sync between desktop and notebook of a working directory, which contains the primary media files that have yet to be archived.  Let's say I have both locations as exact copies of each other (fresh start for syncing).  If, within LR, I move a set of files into a new, child directory (select files, right-click on folder on the left, choose "move to new subfolder" or similar, don't have it running right now), then for cataloging purposes LR keeps track of everything nicely.  As opposed to moving using the file system (FS), that is.  If Syncback is set to synchronize, though, then the files are not recognized as moved on the updated drive but rather it wants to copy the files from the stale drive to the updated drive in the old location and copy the files from the updated drive to the stale drive in the new location.  Am I missing something basic, like I should be using a mirror operation and choosing source/dest based on what I know is updated vs. stale?  I worry what will happen in the event that I start making changes to both data sets, which may be a goal in the future.

I thought initially that the hashing method of comparison would help but no luck.

Thanks in advance,

Kevin
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #1 on: June 18, 2009, 06:37:56 PM »

Kevin,
Are you doing work to both sets of files?  If possible, you should only be working on the primary, and then backing up to the second copy.  It sounds like you are saying you sometimes work on each set of files.
Peter
Logged
BoglePhoto
Full Member
***
Posts: 112

BoglePhoto BoglePhoto
View Profile WWW
« Reply #2 on: June 18, 2009, 09:44:36 PM »

Peter,

Don't you recommend that you mirror the primary drive (the C: or other location on your main workstation with the photos) to the swap drive and not "sync" it so that the files you change on the primary drive are copied to the swap, but if you delete or change them from the primary drive (either no longer used, deleted, or archived away to the archive server) they are changed or deleted from the swap? I thought that was the procedure you recommended. Syncback does this very efficiently, as I recently learned when I had a numbering issue, and cleaned up the numbering of files by renaming on my primary drive, and it cleaned it up in the swap. deleting the old named files and copying the new named files correctly. This keeps me from junking up the drive with old named strays or duplicates.

If you are doing two back ups with the swap (I have two 750 gb hard drives in a small USB box from Sans Digital which recognizes separate drives, labeled N and O), do you do two separate mirrors of the C drive photos, once to N and then to O over the night, or do you recommend mirroring N to O? My thought on the O is to take that off site for a week, and bring it back to mirror it weekly or more frequently to make sure I have an off site backup of my C photos that is not too old.

Last point. On the laptop, if you ingest on the road, and use two external usb drives to hold the backups on ingestion, when do you start pruning the laptop and the two usb externals: Only after they are moved to working on the main workstation primary drive and copies to the server back up, or only when they are put into the archive? 

Bill Bogle, Jr.

BTW,
Logged

www.BoglePhoto.com, blog at www.BoglePhoto.blogspot.com
PC AMD 64 x2 dual core 5200+ 2.6Ghz, Vista 32bit SP1,PS CS3, Lightroom 2, IIP, EM2, Dell Studio 2.0 Ghz Core 2 Duo, Vista 64 bit
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #3 on: June 18, 2009, 09:54:54 PM »

Bill,
Right, I suggest mirroring the primary data to the backup, not a two-way sync.

I do a direct mirror to the online backup, and then do a periodic mirror to the offline drive when I bring it back online.

As to cleaning out the ingestion backups, I do it as the drives fill and I need the room.

Peter
Logged
Kevin Forsmo
Newbie
*
Posts: 31



View Profile
« Reply #4 on: June 24, 2009, 03:58:23 PM »

Peter,

I was ideally looking for something that was able to handle syncing, as I'm trying to explore a setup where i can make changes to either data set and have it reflected in the other.  In the book I know you somewhat recommend against this, but it seems that the primary reason is the time it takes to copy my 18 GB of LR previews back and forth...which I'm aware of and am considering living with.

Regardless of the intent or situation, I think the question at the root (and one that I'm potentially more interested in) is whether any of the sync or backup utilities can handle the following situation cleanly.  I realize that syncback is a file-level utility, but in my mind the following shouldn't be impossible.

Let's say I have a recurring sync set up between two directory structures, say C:\ and D:\.
If I move 10 GB of files around on C:\, the operation is relatively quick, as the only thing written is the allocation table or whatever, I'm not actually copying data.  Ideally a sync or backup utility would somehow realize this and simply update the allocation table on D:\ rather than deleting 10 GB of data on D:\ then copying 10 GB of data from C:\ to D:\.  Getting up and running with the DAM thing has made me rethink all my backup, not just pictures, and up to this point I've kind of planned to revamp a sync scheme, but if it happens that every time I move something, it extends the backup time because it gets copied anew, I'll have to rethink.

Kevin
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #5 on: June 25, 2009, 01:50:45 PM »

Kevin,
This is the kind of situation that ZFS is designed to handle. Check Wikipedia to read about that file system. Unfortunately, you won't be finding that kind of capability in anything currently available. This would be more possible on the Mac, where there is a breadcrumb trail left in the filesystem whenever you rename or move files, but I don't think Windows has that. I do not think I would trust such a system in the near term in any case.

The best approach will probably be to rethink your workflow so that unchanged files are moved around a bit less if the network traffic (or time consumed) is too great with the current implementation.

Peter

Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!