Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /home/content/60/9972860/html/smf/Sources/Load.php(225) : runtime-created function on line 3
Expression Media / iView bug: sluggishness due to increasing folder numbers
The DAM Forum
Welcome, Guest. Please login or register.
November 22, 2019, 12:16:34 AM

Login with username, password and session length
Search:     Advanced search
28033 Posts in 5147 Topics by 2904 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  Software Discussions
| |-+  Media Pro & Expression Media
| | |-+  Expression Media / iView bug: sluggishness due to increasing folder numbers
« previous next »
Pages: [1] Print
Author Topic: Expression Media / iView bug: sluggishness due to increasing folder numbers  (Read 9463 times)
Arlen
Newbie
*
Posts: 31


View Profile
« on: October 07, 2007, 02:48:53 PM »

It seems that a large number of iView (and now Expression Media) users have experienced sluggish performance with the program, judging from the number of posts on the topic both in the forums here and those on iView's website. I started using iView late last year after buying Peter's "The DAM Book" and following the advice there and on this website. I didn't personally experience any of these "slowdown" issues until recently. The behavior I reference is a general "freezing" or unresponsiveness for several seconds after clicking on a "green button" beside a folder in the Organize panel; trying to move a scroll bar; switching between List, Thumbnail and Media tabs; or other operations where a screen update is called for. Basically, the program acts busy for several seconds until it gets around to responding to your mouse click or arrow key operations. The slowdown can become crippling to the work flow of a busy photographer dealing with large numbers of images.

Recently I imported a sizable number of legacy images into one of my catalogs, and noticed this type of slowdown. Investigation showed that the sluggishness occurred in this catalog (let's call it Cat1), which had grown from 245MB / 4300 image files, to 577MB / 12,600 images; and in a second catalog (Cat2) that was 1.1GB and contained 24,100 images; but NOT in a third catalog (Cat3) that was 324MB / 7200 images in size.

So I decided to experiment with various approaches to curing this problem. For clarity, let me say that I am using a Windows XP-Pro system; 2.40 gigahertz AMD Pentium 4 dual processor; large, fast, local (not networked), defragmented drives with plenty of free space; properly set up Windows paging parameters; and have not experienced any speed/responsiveness issues with programs other than iView and Expression Media.

I initially looked back at all the threads on iView/XMedia sluggishness/slowness in this forum and in iView's support forum. I tried virtually everything that had been suggested at one time or another, that had worked for some people in some cases, to address similar problems; but none of them worked for me. (The things I tried included rebuilding the file, and dragging the media items into a new catalog file.) I saw that there were a large number of posters who also were apparently never able to solve the sluggishness issue.

Since the slowness seemed correlated with file size among my 3 catalogs, I played with that. Cat1 had started out responding fine at 245MB/4300 files, but ground to a crawl when enough folders & files were added to bring it to 577MB/12,600 files. So I progressively removed file-containing folders from that catalog until I got back down close to the original size. Results:  each major size reduction gave improved performance, until it was fine again as it neared the original size.

Many users, however, seem not to be having responsiveness problems, even with catalog file sizes and image numbers in the range of my larger catalogs. So what gives?

I'm not positive that all the reported slowness problems have the same cause, but I found a post on the iView support forum that seems to get at the root of mine, and probably those of many others. An iView user posted that his experiments showed that the slowdown correlates more specifically with the number of FOLDERS in the catalog, rather than directly with the number of IMAGE FILES. So two users with the same number of images in their catalog may have different experiences in program responsiveness if one of them has the images in just a few folders, while the second one has the images spread among many folders. In response to that user's query, an iView Technical Support person did a series of experiments in December of last year on a Windows machine and replicated the problem, and showed that the slowdown was a result of an increase in folders within the catalog. He stated that he planned to run the test on a Mac next; but if he ever did, those results were not reported. You can read the thread on the iView support forum here:  http://forum.iview-multimedia.com/viewtopic.php?p=27700#27700.

So, there appears to be an underlying bug related to folder numbers, at least on Windows machines. I have tested and found that it persists in Expression Media, even after the recent release of Service Pack 1. A catalog file that is slow in iView is just as slow in XMedia.

I don't know what a good workaround for this problem is. I have lots of folders and subfolders:  Originals and Derivative Folders; Bucket Folders; many Job Folders; often with Sub-Job Folders. It's not easy for me to reduce the number of folders, especially retrospectively, without a lot of work and loss of organization. In at least one critical case, I need to keep all the files relating to one subject (of insects, taken over a number of years) in one catalog, because of an extensive use of Catalog Sets to characterize them and a need to see all of the files in a Set at once. If you have comments or similar experiences, please chime in. In particular, if anyone can confirm or refute that this issue is a problem on a Mac, that would be very valuable.

I hope we can convince Microsoft that this is a serious enough issue that they should give it a priority in their next bug fix release. I've filed a report on the Feedback section of their website, and others from the iView support forum are adding their comments there. If this is an issue that affects you, or you are concerned that it might, please add your own comments here:  https://connect.microsoft.com/Expression/feedback/ViewFeedback.aspx?FeedbackID=303134. Or if you have an inside track to the development team at Microsoft, a word there would be appreciated.

Arlen



« Last Edit: October 07, 2007, 06:08:34 PM by Arlen » Logged
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #1 on: November 28, 2007, 10:45:53 AM »

After a month and a half, this post hasn't generated any responses, even from Peter. So I guess that must mean that no one here has experienced a similar problem to the one described? No slow-downs attributable to folder numbers? (Or maybe my description was just too darned long to read the whole thing.  Grin)

 I'm still haviing this problem, and don't see any way around it except a huge effort to reduce folder numbers by consolidation and loss of some useful organization.

Arlen
Logged
andris
Full Member
***
Posts: 149


View Profile WWW Email
« Reply #2 on: November 28, 2007, 10:58:59 AM »

Hey, sorry I didn't catch your post the first time around.  This issue is definitely impacting me as well.  I manage an archive of digital photographs for a photography studio, and we have a series of archive drives of RAWs from shoots and a separate archive drive containing all the derivative files from that same time period.  Each drive of originals gets its own iView catalog, and performance on those catalogs is good.  The derivatives drive has its own separate iView catalog, in which performance is abysmal.  As you've indicated, the cause of the slowdown appears to be number of folders/subfolders. 

On the drives of originals, each folder contains a large number of files but there aren't that many folders.  Structure goes something like:

job name -> shot number

The derivatives archive drive, however, contains many more subfolders.  Each shoot only generates between 1 and 30 derivatives, and over time this structure creates subfolders at a much faster rate than on the archive drives.

I came to the same conclusion as you that the only workaround would be a massive restructuring of folder hierarchies....something I don't have the time or the desire to do.

I'd love to hear it if anyone has a different way to address this problem.

Andris
Logged
billseymour
Sr. Member
****
Posts: 308


View Profile
« Reply #3 on: November 28, 2007, 03:10:02 PM »

Reading these posts, a few thoughts:

1. How many folders are you talking about?

2. Can you use keywording to essentially group a 'folder' of images? So that, if you now have:
folder_01, folder_02... folder_100

Might things not work better if you put all the images into one (or more) 'bucket' folders, with the following additional keywording:

images from folder_01 get keyword: 'folder_01'  or  'folder: 01'
images from folder_02 get.....etc.

You can now bring up your 'folder_01' images, and can then work with them, or send copies of them to wherever they need to go.

I have not encountered the described problem, but then my images are only in 3-6 folders/buckets. This problem may be partly a software issue, and partly an organizational one (a zillion folders runs contrary to the organizational metaphor of the DAM book). Anyhow, I'll watch this thread with interest.
Bill
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #4 on: November 28, 2007, 04:24:01 PM »

Andris,
There is an iView script that can replicate your entire folder structure to catalog sets, so that you can flatten the folder structure without losing information, or making much work for yourself.

I have a mac version of the script.

Peter
Logged
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #5 on: November 28, 2007, 05:09:31 PM »

Peter, do you know if there is a PC version of the script? That sounds like it would be the best work-around I can think of, without having to go through too much work.

Newly created subfolders can be kept to a minimum and instead utilize iView or another cataloging program to create categories. The real problem is with legacy files & folders, created before starting to use a database program like iView. In those days many of us used folder directories as our primary organizing tool, so it made sense to break things down into many folders instead of mixing them into a just a few.

What is irksome is that you won't know that iView will start to choke on lots of folders (unless you happen to read a thread like this one), until you have grown your catalog to a substantial size after bringing in your legacy files and folders. By the time you realize there's a problem, you've already put in a lot of time and effort, and it's a little late to easily choose another program. I'm hoping to find a reasonable workaround (like the mentioned script) that will let me stick with iView/Expression Media. I'm also hoping that Microsoft will fix the problem in the next release, but if they don't hear from many users about it, I'm not optimistic that it will be a priority.

Arlen
« Last Edit: November 28, 2007, 05:15:23 PM by Arlen » Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #6 on: November 28, 2007, 06:22:27 PM »

Arlen,
I agree that it is annoying (or sometimes worse) to find out this stuff later.
Keep in mind that as new software comes on line, and collections grow, this kind of problem will continue to be with us.

For instance, I recently ported my 2006 collection to Lightroom to test. it has slowed to a crawl, and is pretty much unusable. (45,000 images).

It turns out that the likely culprit is my use of hierarchical keywords. I ported the iView Catalog Sets to Lightroom Hierarchical Keywords with Beardsworth's script.  There are about 800 keywords total in the catalog, maybe an average of 3 deep in nests.  The best guess from the Lightroom engineers is that this is what is impacting speed.

When programs are written, assumptions are made about how people will use them.  Once they are released into the wild, all kinds of things happen.  People do stuff that was never envisioned.

The folder limitation is not terribly surprising, given that the original iView team really did not envision using the program to manage so many assets in such a complex directory structure.

Once again, I find myself saying to have patience, and figure out how to workaround the limitations of the tools in their present form - things will get better in the future.

Peter
Logged
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #7 on: November 28, 2007, 06:35:17 PM »

I"m hanging in there, Peter. Just hope I'm not left hanging too long.  Grin

Can you tell us the name of the Mac script you mentioned, so we can search to see if there is a PC equivalent?
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #8 on: November 29, 2007, 06:06:08 AM »

I think it's called "Make Sets from Folder Name"
Pretty catchy.
Peter
Logged
dh
Newbie
*
Posts: 2


View Profile WWW Email
« Reply #9 on: November 24, 2008, 11:16:46 AM »

We've recently discovered that in xMedia when there are lots of Catalog Folders with active paths, xMedia attempts to map every path to the corresponding image. One of our marketing catalogs has 20,000 images in many hundreds of folders. Because the files are kept on a drive on the server (which is always online), xMedia is mapping 20,000 images at once - creating extreme slowness. It's a bit quicker when accessing the catalog on the server than the other workstations - but not by much.

However, when we disconnect the drive and use the same catalog of 20,000 images, viewing (and doing anything else in) the catalog is incredibly fast - like it should be. Now, we're doing all searching and catalog organization with a disconnected drive. When we actually need to work with a file we connect the drive for a bit, bring it into Photoshop, and then disconnect the drive again.

It seems that the ideal solution would be for Microsoft to develop an 'offline' viewing option (ideally with a keyboard shortcut).

David
Logged

David Hanks
Digital Asset Manager
Stephanie Rausser Photography
707 | 778-7078  phone
707 | 778-7079  fax
david@stephanierausser.com
www.stephanierausser.com
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #10 on: November 24, 2008, 01:39:55 PM »

David,
Yes, there definitely is a lot of overhead with keeping live updates of status.  In the iView EM structure, lots of folders definitely triggers the problem.

Your solution is a variant of one John, Robert and I have been talking about for Lightroom.  As databases get larger, and are accesssed by multiple computers, this gets even more important.
Peter
Logged
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #11 on: December 06, 2008, 10:48:00 AM »

David, thanks so much for posting your experience with disconnecting the drive containing the folders. I tried that, and it is just amazing how much faster you can work in iView with the source drive disconnected. I hope everyone will go to the Expression Media site and post a request for the next version to allow "offline viewing", as you suggest, at the very least. This issue is my number one problem with iView/xMedia, without a doubt. High numbers of images in lots of different folders make the program almost unusable without disconnecting the source drive.

Arlen

(You can see the bug report and add your own comments to the Microsoft team here: https://connect.microsoft.com/Expression/feedback/ViewFeedback.aspx?FeedbackID=303134)
« Last Edit: December 06, 2008, 11:13:11 AM by Arlen » Logged
GilesKS
Newbie
*
Posts: 20


View Profile Email
« Reply #12 on: December 16, 2008, 05:30:28 AM »

There is a workaround for this problem (in iView at least, I haven't tried Expression Media) which provides "offline viewing" functionality. In the "Catalog Folders" pane, reset the folder path to an incorrect path (e.g. to the desktop). This causes all the photos in that folder to register missing (?) and forces iView to use saved previews rather than accessing the original media files. You can do this for either a superfolder containing all the cataloged photos, or a subfolder such as a single DNG bucket.

I only discovered this since reading this thread recently, but it's much much faster for viewing a slideshow of DNG files. Obviously it doesn't work if you don't have previews saved within the catalog, and it has the disadvantage that you can't then open individual files without resetting each folder to its correct path. But, it's useful for flicking through multiple photos, marking/rating them or putting them into catalog sets etc., and easier than physically disconnecting/reconnecting physical drives to achieve the same end. I agree though that it would be preferable to have an "offline viewing" button on the toolbar to do the same thing, or a menu preferences option to access previews by default (when available) instead of original files.

Giles
Logged
Arlen
Newbie
*
Posts: 31


View Profile
« Reply #13 on: January 24, 2009, 01:54:17 PM »

I was swamped with a big, long-term project and wasn't able to try out version 2, SP1 of Expression Meda until recently. I'm happy to report that the problem of extreme sluggishness with large numbers of folders has been greatly alleviated, so now I am able to use the program again. Hooray!

For those who have not tried using the new Hierarchical Keywords feature, you need to take care however to avoid falling victim to a painful bug in this version. Microsoft is now aware of it, and have said there will be a fix in the next bug fix release. (See the thread "Hierarchical Keyword Bug" in this forum for details.)

Arlen
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!