It appears that the presence of any sidecar blocks IIP from entering any metadata.  It will still convert the file to DNG but that is all.  Marc says this is behavior as designed.

I shoot tethered to a laptop.  My clients and I review the images during the shoot in Bridge and sometimes experiment with ACR crops, slight color corrections, star ratings etc.  This creates an ACR metadata entry in the XMP file that doesn't exist in files that are untouched.

It appears that if my CR2 files have any ACR changes made to the files (even something as small as adding a star for the best pick), IIP3 won't apply ANY of the metadata settings I have established for importing the images and converting to DNG.  It also appears that even if something as incidental as adding a star and then removing it, ACR creates a metadata entry that doesn't leave after the star is removed in ACR.

So it seems ACR changes made on CR2 files prior to injestion block IIP3 from doing one of its primary tasks.

Is this a bug, or is this behavior as designed?

 

I adopted DNG as soon as it came out and due to paranoia, I've always embedded a copy of the original CR2 file in the DNG.  Recently I've been playing with the idea of dropping that policy.  A couple of tests confirm that my converted DNG files can be up to about 20% SMALLER than the origial RAW file.  What is happening to the data to reduce the file size?   Is this some sort of lossless compression?

I recently downloaded the beta software from the Plus  website  www.useplus.org that is used to embed licensing information into images.  The software has a mechanism that first makes a copy of the original file and appends the words   "_original" before the file tag on that file and then applies the metadata describing the license to the true copy of the file which ends up with the original file name.  Consequently, the folder containing the file(s) to be appended ends up with two identical sets of images, one with no licensing information and the word "original" appended to it and a second image bearing the original file name with the licensing information embedded.

I'm curious to hear comments on how this might affect your workflow.  I can somewhat understand the desire to protect the original by making a copy of it in order to append the licensing metadata, but if total protection of the original is the reason supporting the policy, why make any changes at all to the original file?

I can see this policy creating a lot of unnecessary work and a significant potential for error

1) To use the PLUS embedder in a Kroghiean DAM environment, one would be required to make a copy of an archived master and move it out of the protected archive to the production environment before embedding a license in order to avoid turning the archive environment into a production environment.

2) The production environment would end up with two sets of files in the same folder only one of which would be useful.  That would require one to delete the unnecessary duplicate (appended with "_original") to end up with the proper set of files for client delivery.

3)  Currently the PLUS embedder does not support .DNG (or any other form of RAW file)  I'm not sure this is a major issue for me since I never deliver raw files to clients, but those with different delivery policies and more far ranging visions regarding the future of parametric image editing may see difficulties.

Any other comments or potential pitfalls?

Bruce

Well that turned out to be a huge waste of time...   

I let the catalog finish building while I was out at a meeting and got back to a new catalog that performs identically to the last two I've created.  So going back to your questions Peter:


Yes on both counts.  Across the network there is a significant lag, but why should the software be constantly checking on resourses even when no demands are placed on it, and why should the software functions be constantly bogged down by this OCD behavior?  I'm about ready to dust off Portfolio 8 and see what it will do .

..sigh...

P.S.  One other thing...the spinning beachball in the lower right conrer never stops moving even when no network activity is ais indicated

Hey Peter!

While you didn't hit on the exact solution, between the input you and Robert gave, I may have pieced together enough information to find a way to the solution I hadn't considered before.  Your last post was the final piece.

Before I purchased my new Vista Machine I had been building an archive bucket by bucket on local drives.  After I had set up the network and moved the first group of Derived and Raw folders onto their own dedicated drives on the server, I may have tried to save time creating the new catalogs by importing the "top level folder" which by accident turned out to be the individual drives themselves.  I think it's possible that Windows became confused by this very logical but misguided approach.  (Huge irony and sarcasm inserted)

After your last post, I've gone back and started the process of importing the individual buckets one-by-one.  So far at the completion of every import the entire system seems to be behaving more like I remembered it when it was all on a single machine.  I've begun selecting groups of 5 buckets to import at one time and still it seems to accept them and behave. 

If this turns out well when I'm finished, the lesson learned is to never EVER choose a drive to import thinking all the buckets will follow.  EM2 will do it, and act quite normally while it does,  but it will go insane after it has finished.

Thanks to you both for your help. 

And the hits just keep on coming   

As an idea toward increasing performance, I decided to try updating the database with 1024 x medium quality previews so I could disconnect the archive until I really need to access the originals.  In the past, I've rarely used the preview tab since generally when I find what I'm looking for, I just open the archive at that location with Bridge.

What a surprise to find that after several hours of building previews, (which increased the database size by a factor of over 10) I discovered that even with the color match checkbox selected in the media rendering,  EM2 doesn't appear to behave consistently as a profile aware piece of software. 

So now I have an ugly workaround that addresses (sort of) the performance issues but I'm stuck with a different problem. The previews are a mish-mash of different renderings.  A rendered 16-bit TIF file and its un-rendered (layered) parent.PSD file both profiled in ProPhoto RGB color space, yield vastly different previews generated by EM2 on import, yet when the media tab is directed at these original files in the archive, (when I make the archive live and pick my navel fuzz while waiting) no such variation is visible.

Does anyone know of a Microsoft location for professional support of this product?

Bruce

Interestingly, my experience doesn't seem to support this.  Recall that when I shut down the file server machine containing the original files, EM2 performance returns to an acceptable level.  This would suggest that there is something inside the software that is forcing it to constantly ping the original files when the archive is live, even though it doesn't need to do so.

This seems to suggest that when the archive is not live, EM2 relies on the database, but when the archive is live it begins acting like a file browser.  (Actually worse since  virtually every function including using the menus at the top is drastically affected by the performance slowdown. eg: with a thumbnail selected one can switch from Organize to Info with no lag, but re-selecting Organize and there is a 5-7 second lag before anything happens.) 

I had to get back to the studio to be certain, but yes.  I'm running SP2 in EM2 and all OS software is up to date. 

This is really slow when the server is running.  The bite is that I'm usually trying to retrieve Derived files when I invoke this particular database so unmounting the drive isn't really much of an option except for creating new sorts and groups.  I still have to fire up the networked computer to get at the files for printing so I may be stuck in the mud for good on this.  It bites doubly hard since the lag is the same on the second drive dedicated to the RAW Archive with only just over 10K images in a similar configuration.

I remember hearing from another IT guy regarding a different issue when I was creating this linkage that Vista and XP don't always play well together over a network.  Does anyone have an anecdotal evidence to support this observation?  And why does Bridge access the same files so well over the same network?

Hi Peter,

I rebuilt the catalog overnight and this morning the performance is identical.  Turning off the file server machine (which essentially unmounts the drive) seems to eliminates the lag but I won't have time 'till later today to see about manipulating the database in a lot of different ways.  A couple of different types of searches and filters worked well and catalog sets seem to swap very quickly now.

You said: " Sometimes the catalog spends too much time checking on the status of the files, and this causes bad slowdown. "  This is beginning to appear to be the issue here.  I am noticing constant hard drive activity and data transfer through the gigabit switch when the Server is on and the slowdown is happening.  This is beginning to feel almost exactly like  EM2 is behaving as a file browser in that situation and not a database application.  Could there be a setting I've overlooked that causes EM2 to constantly have to seek the archive drive instead of it's own database?

Does performance get better if the archive drive is not mounted?

I'm not exactly certain what you mean by this question, Peter.  The archive drive is one of two internal drives on the P4 that is networked to the Vista machine.  How would it be "unmounted"?

Are there many folders of hierarchy?

There are only about 60 buckets and each one typically has a folder hierarchy that rarely exceedes a depth of 3 folders, but each bucket may have up to about 4 of these types of hierarchies within it.

How many images in the catalog?      About 5,000

IS there a huge build-out of keywords, or any other metadata field?

Not what I would call a huge keyword buildout.  Less than 100 unique keywords.  No unusual metadata use.  All the typical stuff you put in in IIP plus I've been using your  Rank & File Lite plugin

If you make a new catalog and reimport all images, does the problem show up in the new catalog?     I'll let you know after that has run its cycle That will likely be tomorrow


Bruce

P.S.  This reply window is buggy as heck.  I get to the bottom and can't see what I'm typing most of the time 'cause the scrollbar keeps poping up and hiding the line I'm typing.

There is something really wrong here and I'm trying to figure out what the deal is.

currently I'm running Expression Media 2 on a New I7 Quad core  with 6 meg RAM running Vista Business

My archive resides on an old P4 Running XP Pro networked to the Visa machine via gigbit lan, and is basically a dedicated file sever a-la Edition one of the DAM book.  I'm creating a catalog of derivative files that is currently only about 200 Gigs in size and the Catalog of these files itself is only about 50 Megs. The catalog resides on the Vista machine.

The trouble is that Expression Media 2 performance is almost laughably poor in response time.  At the most basic level, just trying to navigate the data structure in the Catalog Folders window takes 10-15 seconds just to expand one level of any bucket and the software locks and won't respond 'till it has looked up the info and displayed it.  This is nearly consistent everywhere in the program.  Moving from one Catalog set to another can take as long as 5-6 seconds.  The software almost acts almost like it's never seen the data before and has to digest it before it can do anything with it.

Ironically, I can point Bridge 4 (which of course is a file browser and not a database) at the same data structure across the same network and navigate the data structure with nearly no hesitation at all including sections of the archive Bridge has never looked at and therefore doesn't have stored in its main Cache.

Everything in EM2 seems to run like it's stuck in glue and I can't seem to understand why.

Bruce

Can't be there so I'll ask from here...

New edition of the book?  Publication date?  Is there going to be a new edition?

Inquiring minds want to know!   

Bruce

Is there a setting that will allow me to hide the app during an injestion?  I'd rather it didn't take up screen space during an injestion.

Thanks!

Yeah Marc, that's sort of what I figured and it does make some sense.  I'm still making adjustments.  Rebuilding automated workflows is TEDIOUS!!!  I'm re-building both my tethered and card capture workflows around IIP.  When it's done it will be well worth the effort.

(I could save a bunch of manual work if IIP would honor the original subfolders at the destination, but hey, we can't have everything can we!) 

Bruce