The DAM Forum
Welcome, Guest. Please login or register.
July 25, 2014, 10:39:46 AM

Login with username, password and session length
Search:     Advanced search
28016 Posts in 5137 Topics by 2912 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  Software Discussions
| |-+  Lightroom
| | |-+  Converting Catalog Sets and Keywords from iView to Lightroom
« previous next »
Pages: [1] 2 3 4 Print
Author Topic: Converting Catalog Sets and Keywords from iView to Lightroom  (Read 32125 times)
Marc Rochkind
Hero Member
*****
Posts: 1136


View Profile WWW Email
« on: February 02, 2007, 11:58:12 AM »

This is something I've been thinking about. I thought I'd share my thoughts so that anyone who is ahead of me on this, or wants to catch up, can join in.

To frame this issue: It's about converting iView keywords to LR keywords and iView Catalog Sets to LR Collections. (Collections are new in LR 1 and did not appear in any Beta.)

iView has the ability to export Catalog Sets and Keywords both to a DNG and to a sidecar. It may even write them into other formats... I'm not sure of the details. LR can read keywords from a DNG or a sidecar, but I don't know about Collections. Even when I export metadata from LR, I don't see anything in the generated file about Collections. I'm thinking that Collections are inside the LR database and nowhere else. (Am I right?)

So, here are four proposed methods to convert that occur to me, in order from "very painful" to "not bad at all."

Method 1

1. In iView, sync all metadata to the files.
2. In LR, import files. This will get you the keywords (I've tested it).
3. Re-establish Collections manually to mirror what you had in Catalog Sets. Shouldn't take more than a month or two of steady work.

Method 2

1. In iView, create keywords to reflect Catalog Sets. That is, the images in the Catalog Set Family/Outings/LaborDay2006 all get the keyword Family_Outings_LaborDay2006.
2 - 4. Perform Method 1.
5. In LR, for each keyword generated from a Catalog Set, create a Collection. Maybe could be done in a week.

Method 3

1. Same as #1 in Method 2, but done with an iView script, yet to be developed.
2 - 5. Same as Method 3.

Method 4

1 - 4. Same as Method 4.
5. Create the Collections automatically either by running a script inside LR or by manipulating the LR SQLite3 database. To my knowledge, LR doesn't have a script system, but I could be wrong.

OK, so, what I really want to know is whether anyone has been thinking about this and whether anyone knows of any current activity along the lines of Methods 3 and 4. Also, whether there's a completely different way of doing it that I don't know about.

--Marc
Logged

johnbeardy
Administrator
Hero Member
*****
Posts: 1813


View Profile WWW
« Reply #1 on: February 02, 2007, 12:20:30 PM »

Yes, Marc, I had been thinking along similar lines. I've not tried anything yet, but those sorts of thoughts are what lies behind my interest in the LR database.

You are right about collections - they never get into the XMP. One little point is there is a Windows bug in 3.1.2 where it can fail to read XMP sidecars - it seems to be fixed in 3.1.3 rc10.

I'm mainly thinking about DNGs here, but I'd focus on the Catalog Sets and the Collections - the keywords roundtrip nicely. As for the methods, I'm thinking along the lines of 4 and it would have to be based on updating the back end database. Scripting is promised but it won't come soon enough and I wouldn't be surprised if it's inadequate (eg just JS, not VB or AS).

I dismiss 3 because of bugs in scripting sets on Windows. 1 and 2 are too manual, and I don't like the idea of abusing keywords to carry other data.

One thought is that beta 4 included a concept called binders which is used for transporting and merging libraries. It has been pulled from v1 due to the move from shoots to folders, and the welcome abandonment of the managed folders area. IIRC it did allow Collections to be passed between libraries. And who knows, it may be revived before long...

John
Logged
Marc Rochkind
Hero Member
*****
Posts: 1136


View Profile WWW Email
« Reply #2 on: February 02, 2007, 12:38:30 PM »

John--

I think you have a script to build keywords from a path, so perhaps its a short step to keywords from Catalog Sets. I share your concern about using keywords to carry other data. An alternative is to pull the Catalog Set info from an exported XMP and then put it into the LR database. I think an app to do that wouldn't be too complicated. Maybe even easier than the keyword approach.

On the other hand, once nice thing about the keyword-from-Catalog Set approach is that the Collections don't have to be built in LR right away. The keywords continue to be useful for identifying the image collections. Maybe some of the collections never have to be built.

Anyway, I'm going to play around with iView scripts and writing into the LR database to see what's feasible.

--Marc

--Marc
Logged

johnbeardy
Administrator
Hero Member
*****
Posts: 1813


View Profile WWW
« Reply #3 on: February 02, 2007, 12:52:02 PM »

Marc

I don't know if you know this, but the images are assigned to Collections in the table AglibraryTagImage - look at the items where tagKind = AgCollectionTagKind. The names of the Collections are in AglibraryTag. The table ImageFiles represents the physical images, and the related table Images includes both the image records and any virtual copies. But you've probably gone beyond that already.

John
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #4 on: February 02, 2007, 12:56:18 PM »

Marc,
While - as John points out - LR has pulled the functionality out to transfer collections, I would not consider this final.
I'll be disappointed if we do not get the ability to transfer collection data to an XMP namespace.  If that does show up, then the transfer of Catalog Set data to Collections would be pretty straightforward, in terms of a back-end script or application that can do a little text editing to the XMP data.
This is part of what seemed to be so interesting about the idea that you would jump into DNG/XMP editing. There's lots of room for very useful utilities here, and I don't see many people out there who seem inclined to take on the task.

I guess one point of all this is that it might make sense to wait a while before trying to do much in terms of data migration.  While I'd hope to see the Collection data defined in a .x release, there's no way to know at this point when it might show up.
Hope this makes sense.
Peter
Logged
Marc Rochkind
Hero Member
*****
Posts: 1136


View Profile WWW Email
« Reply #5 on: February 02, 2007, 01:10:45 PM »

I've found a script already built to convert Catalog Sets to keywords:

http://infomatters.com/im/freestuff.html#iview

I tried it and it works. The only problem is that a hierarchy like A/B/C shows up as the 3 keywords A, B, C. Not as A-B-C. So, each level (expecially the lowest) has to be unique.

That aside, since I have only a 100 or so catalog sets, I think I might actually try this. What I'll do is take my iView Library, run the script (after making sure the leaf nodes are unique), sync everything, and then import the whole thing into LR. I'll let you know what happens.

[John: I can run the script, but I can't get it to appear in the iView drop-down for scripts. Do you know how to do that? What they say in the manual -- that it happens automatically -- seems wrong.]

--Marc
Logged

johnbeardy
Administrator
Hero Member
*****
Posts: 1813


View Profile WWW
« Reply #6 on: February 02, 2007, 01:11:25 PM »

Marc / Peter

I'm not a hard core coder - I stop at the likes of VB but don't get into C++ and think Lua will fall on the wrong side of that wall. But I can automate the likes of Excel, iView etc and a bit more, and I guess what I'd really love to see is a generic automation object for Windows that lets me read/write the XMP values in sidecars or documented file formats.

I wouldn't be at all surprised if Collections do make it into the XMP before long, so wiring up sets and collections via the XMP might be less dangerous than hitting the database. That would also bypass the Catalog Sets scripting bug on Windows.

John
Logged
johnbeardy
Administrator
Hero Member
*****
Posts: 1813


View Profile WWW
« Reply #7 on: February 02, 2007, 01:15:04 PM »

Marc

Put "the script" (which one, my sets to keywords?) in C:\Documents and Settings\User\Application Data\iView\Plug-ins\Scripts and restart iView.

C:\Documents and Settings\User\Application Data\iView\Plug-ins\Toolbar is also handy - put exe files in there and they show up under a toolbar icon.

John
Logged
Rick McCleary
Full Member
***
Posts: 240


View Profile WWW Email
« Reply #8 on: February 02, 2007, 03:04:32 PM »

For the most part, you guys are talking way over my head.  And I'm glad you're having this conversation.  It goes to the core of Peter's mantra of making sure your catalg is portable from one DAM solution to another.  The release of LR is really the first time us iView users are having to consider all the implications.

But one thing that has always concerned me is how to migrate to another DAM solution from iView (when the time comes...probably not yet) and maintain the integrity of the Catalog Set info.  For me, the Catalog Set information is the most critical.  It's the basis of the entire organization of the catalog.  I don't see how a catalog set-to-keyword script would be of any use.  I'd just end up with a bunch of files that have keywords, but no organization.  As Marc pointed out, "The only problem is that a hierarchy like A/B/C shows up as the 3 keywords A, B, C. Not as A-B-C."  That, for me, is more than a problem; it's a disqualifier.
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #9 on: February 02, 2007, 03:51:38 PM »

Rick,
You're right on the money.
The issue will be migrating catalog set info, and exactly how to do that.  The first question will be what are you going to migrate to?  At the moment, I'd say nothing: just wait a bit.

I would suggest that it will probably not be time to migrate until there is a way to embed the hierarchical information into the file.  It's there in the form of Catalog Sets in iView, and it could be there in the form of hierarchical keywords, or collections - once there is a standard way to write that information to the file.  Unfortunately, Adobe has not created a standardized way to write that information to the file yet. 

The solution for the portability of this information will be an XMP namespace, which is a syntax definition of how to write the information to the file.  Once the information can be defined in XMP, then manipulating the data will be pretty easy. (RaopidFixer works by editing the Camera Raw XMP data with a simple text editor).

So the short answer is to sit tight. We should see a more robust version of this kind of data handling before too long, I hope.
Peter
Logged
Rick McCleary
Full Member
***
Posts: 240


View Profile WWW Email
« Reply #10 on: February 02, 2007, 04:39:34 PM »

Peter -

I take your advice and will sit tight; I'm just wondering out loud about possible future moves.  C1 is going to be releasing their DNG-compatible update soon (I hope.)  That will certainly give me lots to think about.  Things will get complicated enough; I'm happy to stick with iView.

But the catalog set/keyword hierarchy thing is a major issue.  I look forward to some sort of standardization on that front.  When you smart guys get it figured out, it will be a great day!  I'll continue to eavesdrop on your conversations...
Logged
Marc Rochkind
Hero Member
*****
Posts: 1136


View Profile WWW Email
« Reply #11 on: February 03, 2007, 11:03:04 AM »

Here's how I converted my iView MediaPro catalog sets to keywords so I could import my images into Lightroom without losing their organization. I had only about 9,000 images and about 50 catalog sets. Whether my method will work for you I don't know. I expect the number of images wouldn't be a limit, but if you have hundreds of catalog sets you might find what I did to be too time consuming.

I won't discuss here WHY you might want to convert from iView to Lightroom or WHETHER, given your requirements, you can convert. I'm writing only about the mechanics of conversion.

For me, the benefits of conversion are that I no longer have to go from iView to ACR to adjust raws and then to Photoshop to print. If Lightroom meets my cataloging needs, and I won't know until I try it whether it does (and, even if it does, it might not meet yours), then I can do 99% of everything I do in Lightroom, going to Photoshop only occasionally.

The whole job only took about an hour of my time; the bulk of the time was spent with iView and Lightroom syncing and importing in the background.

The following method is essentially what I called Method 3 in my original post on this thread. (Not mentioned in the steps is that I did iView saves from time-to-time.)

1. I made a backup copy of my iView catalog and all my images.

2. I renamed about half of my catalog sets so that each node would be unique; the other half were 1-level catalog sets that I was in effect using as keywords. For example, the hierarchies Birthday/2005/John and Sports/Football/John should be renamed something like Birthday/Birthday-2005/Birthday-2005-John and Sports/Sports-Football/Sports-Football-John, because in the conversion of catalog sets to keywords, the script I used doesn't incorporate the levels of the hierarchy into the new keywords. Without this renaming step, both the Birthday and Football pictures of John would have the keyword "John," which loses part of the structure.

3. I then used an iView script from http://infomatters.com/im/freestuff.html#iview to convert the catalog sets to keywords. I think this script could be modified to incorporated all the levels into the keywords, thus obviating my step #1. However, for me, step #1 was easier than working on the script.

4. I synced all my images to their external image files. VERY IMPORTANT!!!! This took most of an hour. (This step is also important to sync any other annotations that might be in iView so that Lightroom can import them.)

5. Leaving the images where they were (in a set of folders built originally by ImageIngesterPro and my scanner), I then imported all 9000+ into Lightroom.

6. When I get around to it, I'll select images for the various keywords generated in step #3 and assign them to Lightroom collections, or hierarchical keywords, or both. But, there's no rush to do that, as they're nicely tagged by keywords already. I'll probably make a collection when I need the features of a collection (e.g., a slideshow or web page).

I haven't yet worked with my images in Lightroom very much, but so far the import seems to have gone very well. I'll post more here if I see anything amiss over the next few days.

--Marc
« Last Edit: February 03, 2007, 11:38:11 AM by Marc Rochkind » Logged

peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #12 on: February 03, 2007, 11:17:51 AM »

Marc,
Great illustration of how to get all your work out of iView.
I *do* think this will get easier before too long.
Peter
Logged
Marc Rochkind
Hero Member
*****
Posts: 1136


View Profile WWW Email
« Reply #13 on: February 03, 2007, 11:39:49 AM »

Peter--

I hope it will be easier... I wanted to do it NOW, however, because I'm too excited about the promise of Lightroom to want to wait. My iView catalog is still alive a well in case I need to go back to it.

--Marc
Logged

Chris Bishop
Sr. Member
****
Posts: 472


View Profile
« Reply #14 on: February 04, 2007, 04:07:00 AM »

Great thread, but I assume it's between people in the LR testing group, as LR v1 isn't actually out yet is it?
Is v1 so good it is going to make iVMP obsolete--I only bought it a week ago!!!
Chris Bishop
Logged
Pages: [1] 2 3 4 Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!