The DAM Forum
Welcome, Guest. Please login or register.
December 19, 2014, 08:46:36 AM

Login with username, password and session length
Search:     Advanced search
28021 Posts in 5141 Topics by 2910 Members
Latest Member: kbroch
* Home Help Search Login Register
+  The DAM Forum
|-+  Workflow Discussions
| |-+  Multi-User Configurations
| | |-+  Server for multi-user DAM
« previous next »
Pages: [1] Print
Author Topic: Server for multi-user DAM  (Read 9000 times)
andris
Full Member
***
Posts: 149


View Profile WWW Email
« on: July 24, 2007, 10:01:38 PM »

Not much action in this multi-user forum (perhaps because iView ain't so great at the multi-user?)...but here goes.

I just took a position as digital asset manager for a lifestyle photographer.  Including the photographer herself, there is a workgroup of four of us who need access to the collection on our Windows workstations.  The photographer does the editing/rating, I share the retouching and cataloging work with another person, and the studio manager occasionally responds to image requests by pulling images from the catalog.  Their current workflow includes a complex system of shuffling around removable drives from workstation to workstation.  All of this shuffling adds a lot of room for human error, and also makes it difficult to split up the work on large shoots with short deadlines.

Obviously the ideal solution would include a network server, and this is the direction I'm hoping to take them.  The ideal workflow I envision looks something like this:

1)  Return from shoot with hard drive full of raws, run image ingester on the server to rename, import, and apply basic metadata
2)  Photographer (sitting at her networked workstation, working on the network copy of the files) does the rating either with Bridge or PhotoMechanic.
3)  I (sitting at my networked workstation, working via the network) perform basic raw adjustments, build proofs, and upload to our website
4)  Client makes selects via the website
5)  The other digital assistant and I make master files, build the deliverable tiffs, and upload to the website
6)  I run the Adobe DNG converter on the server, and convert the entire shoot to DNGs
7)  Files are cataloged into iView/EM, the catalog permanently resides on the server (which is backed up nightly)
7)  DNGs live on the live and local server, eventually 'aging off' onto the primary and backup removable drives which constitute our archive as space limitations on the server dictate. 

I have some concerns as to whether this will be possible to implement given real-world imperfections.  Our photographer shoots 17 megapixel raws (tethered), and shoots them prolifically.  It's not unusual for one day of shooting to generate 100+ GB.  Even working locally, CS2 bridge takes quite a while to build caches for these large numbers of raws and CS3 bridge is painfully slow.  I'm scared to see what adding a network to the mix will do.  Our wired network is gigabit ethernet, and I'm looking at servers that support dual NICs and load balancing.   Does anyone have any thoughts on whether doing all of these operations (rating, camera raw adjustments, and master file creation) over the network are likely to cause us problems?  If necessary, the raws destined to be master files could be copied locally for work since there aren't that many of them, but if the rating and batch adjustments cannot be done over the network it sort of defeats the purpose. 

Originally, I'd thought to build a Linux server for this purpose...but given the pace of things, I don't have the time.  Also, since I'm hoping to do batch processing like importing and DNG conversion on the server to free up the image workstations, I think I'm pinned to Windows.  Does anyone know anything about the MS Storage Server OS?  Just how dumbed down is it?  Could it run the Adobe DNG converter, or do I need to stick to Windows XP?  Also, I don't have much experience with optimizing servers...where should I spend and where can I cut costs?  I'm thinking I want hardware RAID, but what about memory and processing power?  My feeling is that I don't need large amounts of either.  Any other thoughts on how best to optimize a machine for this purpose?

Finally, I know iView's strong points aren't multi-user or networked configurations.  Extensis Portfolio would probably be the way to go, but given the high cost of Portfolio (about $6,000, if I understand correctly) and the fact that we're already invested in iView, I think we're going to stick with it.  I suppose we'll just have to be careful that only one of us is making changes to the catalog at a time.  Has anyone used iView in this kind of set up before?  How did it go?

Thanks in advance,

Andris
Logged
peterkrogh
Administrator
Hero Member
*****
Posts: 5682


View Profile Email
« Reply #1 on: July 25, 2007, 01:06:38 PM »

Andris,
This kind of configuration should be able to work, but there is a bunch of tuning to do along the way to make it work right.  It's beyond my ability to address in this ofrum. (Possible consulting, if you're interested).

A few thoughts:
1. Steps 2 and 3 might be speeded up by means of a Lightroom catalog for the shoot, which can be traded back and forth, including a previews file. THis might be fastr than Bridge, once the previews are built.

2. Portfolio has multi-user, but it's not well-designed for a Raw or DNG workflow.  If you can make it work with Bridge/ACR/Lightroom for the next year, there might be some interesting and cost effective solutions that enable multi-user capability without al the dancing.
Peter
Logged
andris
Full Member
***
Posts: 149


View Profile WWW Email
« Reply #2 on: July 27, 2007, 12:05:33 PM »

Thanks for the input Peter, I'll definitely think through some creative ways of sharing Lightroom catalogs of shoots and such.

So what about all you hardware guys out there?  Anyone have any input on the hardware side of this...beyond dual NICs and hardware RAID where should I spend my money?  Memory?  Processing power?  How much do those matter to server performance?

Thanks,

Andris
Logged
Khaled
Newbie
*
Posts: 2

Surf's Up!


View Profile
« Reply #3 on: August 15, 2007, 08:06:28 AM »

Hey andris,

I've had a network setup for a few months now - Raid 5 Array on Dual Xeon P4 server with Hyperthreading enabled running Ubuntu Linux - even though its only two people - myself and wife.
I inherited this machine - it was a linux server in its previous life, so i thought it would make a good windows server, however the problem here is that windows will only see 3 GB of the 6GB available!!! so i decided to bite the bullet and go with linux - Ubuntu has proven to be quite idiot proof! AND it sees all 6Gb of RAM Smiley

One thing i can recommend is pump up your DNG converting machine - that thing will be doing quite a bit of processing - i noticed you said you will run it off the server directly - i am not doing that currently, and i'm not sure it would be the best way forward since it may affect the performance of the server (i guess only during conversions, but if you have loads of conversions taking place, this could be quite some time).

Something i have only read about is 64bit processing - i know that for digital video encoding Quad core 64bit CPUs will scream through an encoding session with 32bit CPUs far behind (i have not tested this personally).

Another thing with 64Bit processing is that 64Bit CPUs can now see a maximum 16 Exabytes of RAM!!! so the limitations of 32Bit computing will seem like a distant memory!

Good luck with your setup mate.

« Last Edit: August 16, 2007, 08:41:07 PM by Khaled » Logged

Khaled.
andris
Full Member
***
Posts: 149


View Profile WWW Email
« Reply #4 on: August 15, 2007, 11:55:31 AM »

Thanks for the response.  How has the network setup worked for you?  One issue I'm sure will be problematic until the service pack for XM is iView's performance over the network.  Cataloging files over the network is painfully slow, so to avoid that I'll catalog the shoots on the server itself.  I haven't done any testing yet on performance of accessing a catalog at a network location when the media is offline.  I'm also concerned I'll run into problems with multiple users trying to access the same catalog with iView.  I'm guessing the solution will be to use the iView reader for most searching, which seems to work fine over the network.

Another area I'm worried about is Bridge's use of a distributed cache over a network.  I'm aware of the fact that bridge will not export a distributed cache to a network location unless you manually force it to.  I'm hoping once the cache is there that bridge will see and use it.

Interesting thought on the performance increase of 64 bit processing on DNG conversions.  I guess I would need a 64 bit OS to take advantage of this?  I'm not sure I want to get into the headache of running Vista 64 bit on our new server...might create more harm than good.  Is there a conclusive answer as to whether DNG converter can take advantage of multiple processors and hyperthreading?  I saw some comments on this a while back, but I didn't see anything conclusive.

Andris
Logged
Khaled
Newbie
*
Posts: 2

Surf's Up!


View Profile
« Reply #5 on: August 16, 2007, 08:35:59 PM »

Hi Andris
Some thoughts...

How has the network setup worked for you? 
It has been good so far - however i have only installed the reader for my wife's machine, she is intersted in only one thing - printing pictures that i take of our family / kids etc... basically looks thru the catalog, notes down which images she wants a jpg of for printing and sends it to me in an email - afterwhich i make available in her network home directory - and she copies to usb and goes off to the print shop - for you it will be different for sure, since you're running a real commercial operation.

Another area I'm worried about is Bridge's use of a distributed cache over a network.  I'm aware of the fact that bridge will not export a distributed cache to a network location unless you manually force it to.  I'm hoping once the cache is there that bridge will see and use it.

I currently export the cache to all the network folders - with the main bridge cache on a 2 disk striped array for maximum speed - for cataloging i set it off before going to bed at night and let it go through the night cataloging away...

Interesting thought on the performance increase of 64 bit processing on DNG conversions.  I guess I would need a 64 bit OS to take advantage of this?  I'm not sure I want to get into the headache of running Vista 64 bit on our new server...might create more harm than good.  Is there a conclusive answer as to whether DNG converter can take advantage of multiple processors and hyperthreading?  I saw some comments on this a while back, but I didn't see anything conclusive.

I would not use Vista for anything at this stage! go with windows server 64 bit and XP on the workstations - or of course you could be adventurous and go Linux 64bit Smiley - which has the added benfit of being free - allowing u to divert $$ to faster CPU or more RAM...
I dont have any experience in testing differences between hyperthreading on or off.
Logged

Khaled.
eightfish
Newbie
*
Posts: 4


View Profile Email
« Reply #6 on: August 31, 2007, 08:56:22 AM »

I'm running into the same issues here - trying to understand how limiting JBOD is versus moving to a server like X-Serve or X_SERVE with Raid. Peter, is there anywhere listed other than your book, a break down of pluses and minuses of the two?

I've followed your book and set up several JBOD enclosures, as eSATA port multiplier boxes connected to my main PC, and that's linked into my main G5. We have several computers all hooked up through a 5 port ethernet router, and recently when we've gone to open folders across the platforms, or from the JBOD boxes, we get the all feared Apple beachball on the screen and the computers freeze for about 10-20 minutes, I'm guessing tryign to resolve themselves.

Is this because I don't have a dedicated server computer which the JBODs runs from? Or now that we're expanding to 4-5 work stations, might we need a more powerful set up for routing data, like an X-Serve  of X-Serve Raid from Apple?

Is there any advice you can give us...?
Anyone have any thoughts or know where I can get more information on this?
Logged
Pages: [1] Print 
« previous next »
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!