Expression Media 2 Stuck in the Mud

[Bruce Van Inwegen]

There is something really wrong here and I'm trying to figure out what the deal is.

currently I'm running Expression Media 2 on a New I7 Quad core  with 6 meg RAM running Vista Business

My archive resides on an old P4 Running XP Pro networked to the Visa machine via gigbit lan, and is basically a dedicated file sever a-la Edition one of the DAM book.  I'm creating a catalog of derivative files that is currently only about 200 Gigs in size and the Catalog of these files itself is only about 50 Megs. The catalog resides on the Vista machine.

The trouble is that Expression Media 2 performance is almost laughably poor in response time.  At the most basic level, just trying to navigate the data structure in the Catalog Folders window takes 10-15 seconds just to expand one level of any bucket and the software locks and won't respond 'till it has looked up the info and displayed it.  This is nearly consistent everywhere in the program.  Moving from one Catalog set to another can take as long as 5-6 seconds.  The software almost acts almost like it's never seen the data before and has to digest it before it can do anything with it.

Ironically, I can point Bridge 4 (which of course is a file browser and not a database) at the same data structure across the same network and navigate the data structure with nearly no hesitation at all including sections of the archive Bridge has never looked at and therefore doesn't have stored in its main Cache.

Everything in EM2 seems to run like it's stuck in glue and I can't seem to understand why.

Bruce

[peterkrogh]

Bruce,
Does performance get better if the archive drive is not mounted?
Are there many folders of hierarchy?

How many images in the catalog?

IS there a huge build-out of keywords, or any other metadata field?

If you make a new catalog and reimport all images, does the problem show up in the new catalog?

Peter

[Bruce Van Inwegen]

Does performance get better if the archive drive is not mounted?

I'm not exactly certain what you mean by this question, Peter.  The archive drive is one of two internal drives on the P4 that is networked to the Vista machine.  How would it be "unmounted"?

Are there many folders of hierarchy?

There are only about 60 buckets and each one typically has a folder hierarchy that rarely exceedes a depth of 3 folders, but each bucket may have up to about 4 of these types of hierarchies within it.

How many images in the catalog?      About 5,000

IS there a huge build-out of keywords, or any other metadata field?

Not what I would call a huge keyword buildout.  Less than 100 unique keywords.  No unusual metadata use.  All the typical stuff you put in in IIP plus I've been using your  Rank & File Lite plugin

If you make a new catalog and reimport all images, does the problem show up in the new catalog?     I'll let you know after that has run its cycle That will likely be tomorrow


Bruce

P.S.  This reply window is buggy as heck.  I get to the bottom and can't see what I'm typing most of the time 'cause the scrollbar keeps poping up and hiding the line I'm typing.

[peterkrogh]

Bruce,
As to the first question - Can you disconnect the image storage from the machine that is running the catalog?  Sometimes the catalog spends too much time checking on the status of the files, and this causes bad slowdown.  IF the storage drive is inaccessible, then it won't try to check.

You could also try that catalog on another machine, if you have one. That can help isolate the problem.

Other stuff does not sound like an issue - try the rebuilding and see what happens.

Peter

[Bruce Van Inwegen]

Hi Peter,

I rebuilt the catalog overnight and this morning the performance is identical.  Turning off the file server machine (which essentially unmounts the drive) seems to eliminates the lag but I won't have time 'till later today to see about manipulating the database in a lot of different ways.  A couple of different types of searches and filters worked well and catalog sets seem to swap very quickly now.

You said: " Sometimes the catalog spends too much time checking on the status of the files, and this causes bad slowdown. "  This is beginning to appear to be the issue here.  I am noticing constant hard drive activity and data transfer through the gigabit switch when the Server is on and the slowdown is happening.  This is beginning to feel almost exactly like  EM2 is behaving as a file browser in that situation and not a database application.  Could there be a setting I've overlooked that causes EM2 to constantly have to seek the archive drive instead of it's own database?

[peterkrogh]

Bruce,
There is/was a bug regarding Network attached storage. I thought that had been resolved, but I've never experienced it personally, so I can't remember details. Are you using SP2?
Peter

[Bruce Van Inwegen]

I had to get back to the studio to be certain, but yes.  I'm running SP2 in EM2 and all OS software is up to date. 

This is really slow when the server is running.  The bite is that I'm usually trying to retrieve Derived files when I invoke this particular database so unmounting the drive isn't really much of an option except for creating new sorts and groups.  I still have to fire up the networked computer to get at the files for printing so I may be stuck in the mud for good on this.  It bites doubly hard since the lag is the same on the second drive dedicated to the RAW Archive with only just over 10K images in a similar configuration.

I remember hearing from another IT guy regarding a different issue when I was creating this linkage that Vista and XP don't always play well together over a network.  Does anyone have an anecdotal evidence to support this observation?  And why does Bridge access the same files so well over the same network?

[peterkrogh]

Bruce,
As I recall, the issue was some kind of connection mismatch - too many pending requests jammed up communication or something.  I thought this had mostly been cleared up, and a search of the forum does not produce a clear record of what's going on.

Maybe Robert or John knows what's up. (IIRC, Robert had a good handle on this).

Peter

[roberte]

Hi,

Peter the network bug in xMedia on Windows has been fixed. It was an over-zealous security issue. I'm using xMedia 2 SP2 on XP over a network regularly and it works as well as the Mac version.

The only thing that annoys me is the new rendering for JPEGs where you see a small preview then the full image pops up. This "feature" was put in SP2 especially for those accessing images over a network. The idea being at least you see a small image as you scroll through previews. Personally I find it annoying.

Bruce that doesn't help you... But at least you know it isn't xMedia. So I guess there is an issue with the network itself. The reason Bridge and Lightroom "feel" faster is they build a cache of thumbnails and previews. xMedia and Photo Mechanic read and display the actual image. On a local disk they are far quicker than their Adobe equivalents. Over a network they can suffer.

-- Robert.

[Bruce Van Inwegen]

Bruce that doesn't help you... But at least you know it isn't xMedia. So I guess there is an issue with the network itself.

Interestingly, my experience doesn't seem to support this.  Recall that when I shut down the file server machine containing the original files, EM2 performance returns to an acceptable level.  This would suggest that there is something inside the software that is forcing it to constantly ping the original files when the archive is live, even though it doesn't need to do so.

This seems to suggest that when the archive is not live, EM2 relies on the database, but when the archive is live it begins acting like a file browser.  (Actually worse since  virtually every function including using the menus at the top is drastically affected by the performance slowdown. eg: with a thumbnail selected one can switch from Organize to Info with no lag, but re-selecting Organize and there is a 5-7 second lag before anything happens.) 

[peterkrogh]

Bruce,
It's not acting as a file browser, but rather it's doing a lot of checking on the resources.

Try this - make a catalog with one image on a local drive.
Double-click to see the image full size

Then add a similar image from the network drive
Double-click to see the full image

IS there a difference?
Is there a similar difference between launching an image into Camera Raw at full size between the two?

Peter

[Bruce Van Inwegen]

Hey Peter!

While you didn't hit on the exact solution, between the input you and Robert gave, I may have pieced together enough information to find a way to the solution I hadn't considered before.  Your last post was the final piece.

Before I purchased my new Vista Machine I had been building an archive bucket by bucket on local drives.  After I had set up the network and moved the first group of Derived and Raw folders onto their own dedicated drives on the server, I may have tried to save time creating the new catalogs by importing the "top level folder" which by accident turned out to be the individual drives themselves.  I think it's possible that Windows became confused by this very logical but misguided approach.  (Huge irony and sarcasm inserted)

After your last post, I've gone back and started the process of importing the individual buckets one-by-one.  So far at the completion of every import the entire system seems to be behaving more like I remembered it when it was all on a single machine.  I've begun selecting groups of 5 buckets to import at one time and still it seems to accept them and behave. 

If this turns out well when I'm finished, the lesson learned is to never EVER choose a drive to import thinking all the buckets will follow.  EM2 will do it, and act quite normally while it does,  but it will go insane after it has finished.

Thanks to you both for your help. 

[Bruce Van Inwegen]

Well that turned out to be a huge waste of time...   

I let the catalog finish building while I was out at a meeting and got back to a new catalog that performs identically to the last two I've created.  So going back to your questions Peter:

Bruce,
It's not acting as a file browser, but rather it's doing a lot of checking on the resources.

Try this - make a catalog with one image on a local drive.
Double-click to see the image full size

Then add a similar image from the network drive
Double-click to see the full image

IS there a difference? 
Is there a similar difference between launching an image into Camera Raw at full size between the two?

Peter

Yes on both counts.  Across the network there is a significant lag, but why should the software be constantly checking on resourses even when no demands are placed on it, and why should the software functions be constantly bogged down by this OCD behavior?  I'm about ready to dust off Portfolio 8 and see what it will do .

..sigh...

P.S.  One other thing...the spinning beachball in the lower right conrer never stops moving even when no network activity is ais indicated

[peterkrogh]

Bruce,
I'm guessing that there are some collisions as Expression checks on the files over the network. Like Lightroom, Expression checks on remote resources in the background when the remote drive is mounted. That's the best explanation I can give.

As to Portfolio, unfortunately, it is not well-suited to managing a raw file arcihve.
Peter