Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3
Problems copying catalogs
The DAM Forum
Welcome, Guest. Please login or register.
August 14, 2020, 07:03:02 AM

Login with username, password and session length
Search:     Advanced search
28033 Posts in 5147 Topics by 2904 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  Software Discussions
| |-+  Media Pro & Expression Media
| | |-+  Problems copying catalogs
« previous next »
Pages: [1] Print
Author Topic: Problems copying catalogs  (Read 6530 times)
ralphpaonessa
Full Member
***
Posts: 108


View Profile WWW
« on: March 10, 2008, 03:12:31 PM »

Realizing that EM 2 will still not support catalogs larger than ca. 1.8 GB, I set out to split my current catalog (1.6 GB, but getting slow). I did this by making 4 copies of the catalog file, and then opening each one in IVMP and deleting everything I didn't want to keep in that catalog. My main catalog is still about 1 GB, but I'm unwilling to split it further.

I've seen Peter's advice in other posts to copy everything from one catalog into a new, blank catalog, with an eye towards creating a more stable catalog. So I've been trying this with my 1 GB catalog and encountering problems.

Problem 1: If I select all 37,000 thumbnails in the catalog and click on Edit > Copy, the program crashes repeatedly. Both EM and IVMP do this.

Problem 2: If I copied and pasted from a catalog where images were missing from the disk (deleted on disk but not removed from catalog), IVMP created a very messed up catalog. Generally, the missing images appeared as hundreds of "orphans" in the file list of one file each, showing "Drive F" but no folder. I solved this problem by searching for and deleting from the catalog any missing images and starting again.

At this point, I decided to try using Expressions Media instead of IVMP, in the hopes that it might work better. It still couldn't copy all 37,000 iamges at once (Problem 1).

But I was able to select portions of the catalog (4,000-8,000 images at a time) and paste them to a single new EM catalog.

Problem 3: Sometimes this paste operation took a very long time, (e.g. 15 minutes) and other times it took only a minute. I tried it a few times. Inevitably, the first paste of a few thousand images was "quick." Subsequent pastes were very slow. What surprised me was that, at times, one of these pastes was still fairly quick, even though it still involved several thousand files. Not sure why this happened.

Problem 4: I have a custom field called Species, which contains perhaps a hundred entries. Most of the 37,000 inature images are assigned to a species (Moose, Robin, Anna's Hummingbird, etc.). In the new catalog, there were two custom "Species" fields, each with slightly different corrupted names. (The new names were "Species" followedd by one or two "non-standard" characters, e.g. an "E" with an accent.)

As far as I can tell, images where a Species was assigned recently were put into the "new" species fields (which retained their correct names), and older entries stayed in the "older" species fields. In some cased, a given Species appeared in both lists, which simply meant there were older and newer assignments.

I manually selected each of the species in the "new" list and then assigned them to the proper species in the old list (which I sometimes had to recreate).

As far as I can tell, everything was either in one species list or the other, no duplicatdes and nothing missing.

I finally deleted the new Species custom field from the catalog. Everything seems OK, except EM insists on renaming "Species" by adding an odd character to the end.

Any idea what caused these problems with the custom field? I am now using EM, but could switch back to IVMP.

Thanks for any advice.

Ralph Paonessa
Logged

Ralph Paonessa
RPPhoto.com
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #1 on: March 10, 2008, 03:18:54 PM »

Ralph,
Rather than Copy and Paste, I generally drag from one window to another.  You could also try to "Import Catalog".

The odd character in the custom field is a known bug.  I think the best workaround would be to make a catalog set for each value of the custom field, assign the appropriate images, and then delete the custom field.  Reverse on the receiving end.

Peter
Logged
ralphpaonessa
Full Member
***
Posts: 108


View Profile WWW
« Reply #2 on: March 10, 2008, 03:31:00 PM »

When I try to drag and drop a large number of thumbnails, IVMP (Windows) always "hesitates." By this I mean that the screen interface doesn't keep up. E.g., if I want to assign a keyword to 1,000 thumbnails and try to select them all and drag them to the keyword, it appears that nothing is happening. Eventually, the program works in the background and the display catches up some seconds later.

When I tried to drag and drop several thousand (or all) images, nothing seemed to happen for "a very long time." Finally I got tired of waiting and tried the Copy and Paste routine. In that case, the new thumnails start to appear immediately in the new catalog. If I try and click on the catalog, Vista says EM is "Not Responding" and the little "going-around-in-circles" icon appears. Most of the time, it was there for minutes, but finally it went away and the transfer was complete.

I believe that if I tried to select ALL the 37,000 thumnails and drag them, the program repeatedly crashed (both IVMP and EM). In fact, if I tried to select them all and do ANYTHING, e.g. click somewhere, it crashed. Are you able to select and drag large catalogs all at once?

Ralph Paonessa
Logged

Ralph Paonessa
RPPhoto.com
ralphpaonessa
Full Member
***
Posts: 108


View Profile WWW
« Reply #3 on: March 10, 2008, 03:46:52 PM »

Peter,

Does that "Custom Field character bug" only show up in this sort of copying of one catalog to another? Or in all custom fields in EM?

I could follow your suggestion of copying after replicating all the custom field entries as catalog sets -- but there are hundred of them and it would be very laborious unless I could write a script.

Are you generally reporting your experiences on Mac or Windows?

Thanks for your advice!
Logged

Ralph Paonessa
RPPhoto.com
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #4 on: March 11, 2008, 05:39:45 AM »

The delay is to be expected when selecting lots of images - it's writing a temp file to the disk that may be 1 GB.

As to drag-to-keyword, yes, that's my experience. When I have a large number, I select, and then right-click on the keyword and "Add Term".

Most of my experience with migration is with Mac.  My other experience in Vista.  Almost no XP.

The Custom Fields bug is, I think, related to iView catalogs migrated to EM.  If the custom field is created in EM, then I don't think it shows up.  Hopefully they will get that fixed without you having to resort to the workaround.  I'd wait on this, if you can.

Peter
Logged
ralphpaonessa
Full Member
***
Posts: 108


View Profile WWW
« Reply #5 on: March 11, 2008, 02:53:00 PM »

Peter,

Can I assume the custom field bug just has to do with the odd characters in the custom field name, but that otherwise the custom fields function as intended? My custom fields (categorizing about 40,000 bird images by species) are the heart of my catalog; it would be a disaster if I lost those.

I think I can write a script to copy those both to keywords and to catalog sets. Perhaps I'll do that as a way of backing up that information. I actually would like to have those added to keywords; and having them as catalog sets probably increase my options.

BTW, I tried to Import into a blank EM catalog. The system cranked for over an hour; the progress meter continuously showed negative percentages. But ultimately it created a new catalog. This time, there were again two Species custom fields where there should have been one; but one was empty, and the other had all the species.

Looking forward to the release of EM 2 ...
Logged

Ralph Paonessa
RPPhoto.com
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #6 on: March 11, 2008, 07:56:28 PM »

Ralph,
Why leave iView?
Peter
Logged
ralphpaonessa
Full Member
***
Posts: 108


View Profile WWW
« Reply #7 on: March 12, 2008, 12:07:36 AM »

Why leave iView?
Hmmm ...

First, let me ask: Can I switch back and forth between IVMP and EM on the same catalog? (I wouldn't do this routinely; but perhaps I could try EM for a while and then go back to IVMP if it suited me.)

As to why leave iView:

Perhaps right now, I shouldn't. My biggest frustration with iView is the 1.8-2 GB catalog limit. I've also become frustrated with the slow response times in my 1.6 GB catalog, which are slightly better after splitting it down to 1 GB.

It does seem like iView users will eventually have to switch to EM -- because that's where the bugs will (hopefully) get fixed, that's where new cameras will be supported (not that I have one), that's where program improvements will occur.

I have a love/hate relationship with IVMP. It still seems to be conceptually the best DAM software for large, "serious" collection. It has also had a lot of bugs, and the interface is sometimes very clunky and annoying. (But it does work.)

But your question forces me to concede that, right now, perhaps there is no reason to switch to EM. In practice, I just became interested in EM (after a few years with IVMP) because I thought the new release of EM 2 would support larger catalogs. In fact it won't, and I am disappointed MS chose not to until EM 3 (although it appears that strategically they felt forced to wait until the catalog structure was radically revamped in EM 3 to make this improvement).

Then, I set about splitting my 1.6 GB catalog into smaller ones using IVMP. I found this process so buggy and crash-prone that I decided, what the hell, let's try EM -- but I suppose EM has been just as buggy if not more so!

So maybe I will just revert to IVMP for now, and live with the DAM(N) catalog size limitation.

So, that begs the question: Do you have any comments on whether the improvements planned for EM 2 (now in beta) will merit switching from IVMP to EM 2? The catalog size limit will not be overcome. But perhaps it will just be a smoother, less buggy evolution of IVMP?

As always, your comments are appreciated.
Logged

Ralph Paonessa
RPPhoto.com
ralphpaonessa
Full Member
***
Posts: 108


View Profile WWW
« Reply #8 on: March 12, 2008, 12:12:22 AM »

P.S. I have to note that, as the last four threads in this EM forum were started by me, and no others have appeared in the past week, that may be a pretty good indication that converting to EM has not been a wildly popular move with members of this forum. LOLĀ  Cheesy
Logged

Ralph Paonessa
RPPhoto.com
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #9 on: March 12, 2008, 04:41:32 AM »

Ralph,
There are some reasons to switch - some people have iView problems that are fixed by EM.

Yes, you can move catalogs back and forth, but I'm not sure if there are any implications if you do it a lot.

>In fact it won't, and I am disappointed MS chose not to until EM 3 (although it appears that
> strategically they felt forced to wait until the catalog structure
>was radically revamped in EM 3 to make this improvement).

This is the kind of tough call involved with software development.  Would it be better to make a temporary change to the database structure, and then change it again - or would it be better to only change it once?  It's not an easy call.

Peter
Logged
ralphpaonessa
Full Member
***
Posts: 108


View Profile WWW
« Reply #10 on: March 12, 2008, 12:38:11 PM »

There are some reasons to switch - some people have iView problems that are fixed by EM.
Can you briefly list what these are?

Is there an informal list somewhere of bugs in iView that are fixed in EM, and new bugs in EM? I haven't been able to find that in these forums.

I believe MS did a lot of work "under the hood" on iView and then EM, but I know that EM was not universally embraced as better than iView. Is it just a case of choosing which bugs you prefer?
Logged

Ralph Paonessa
RPPhoto.com
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #11 on: March 12, 2008, 01:12:26 PM »

Ralph,
I don't remember what bugs were closed or introduced with each. I would say that for iView users, the question i, "is there a reason to switch" rather than"is there a reason to stay with iView?"

Hence the phrasing of my question, why leave iView?
Peter
Logged
andris
Full Member
***
Posts: 149


View Profile WWW Email
« Reply #12 on: March 12, 2008, 01:29:54 PM »

Hi guys,

Sorry to be chiming in so late...but I just wanted to note that I've seen your 'problem 2' from your original post.  That is,

Quote
Problem 2: If I copied and pasted from a catalog where images were missing from the disk (deleted on disk but not removed from catalog), IVMP created a very messed up catalog. Generally, the missing images appeared as hundreds of "orphans" in the file list of one file each, showing "Drive F" but no folder. I solved this problem by searching for and deleting from the catalog any missing images and starting again.

I think in my case it stemmed from having created the catalog on one computer, saving it to a server, and working on it on a third.  I think it also may have something to do with the fact that the drive letter assigned to the drive where the images originally lived when cataloged on computer 1 was assigned to a completely different device on computer 2.  I was able to fix the problem by attaching the missing archive drive, right clicking on the top level folder containing images on the drive, and using the 'reset folder path' option to help iView sort out the drive letter confusion properly.  After that, I was able to copy files into a new catalog without orphans.

Either way, filesystem problems like these 'orphans' really creep me out.  We do a lot of cataloging on one machine and working in the catalog in another.  I'm hoping EM2 resolves this bug, since they say they're supposed to be better at the multi-user thing.

Thanks,

Andris
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #13 on: March 13, 2008, 04:40:13 AM »

Andris,
Agreed.  This seems to be a Windows-only issue, and is related to that drive letter thing (which I'm still getting used to.)  EM is trying to resolve for missing files in the background, and getting confused.
Peter
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!