Does the DNG format compress the image data?

[Louie Sherwin]

I have noticed a wide variation in file sizes between the raw files and the resulting DNG file. All my raw files are exactly the same size, 13.5 MB but the resulting DNG can vary between about 8 MB and  about 11 MB but never larger than the original raw file. So I am curious about what is going on.

tks, louie

[johnbeardy]

Louie

It depends on the file type. Canon files don't reduce much in size, Nikon ones do. It's a lossless "compression", just more efficient packing of the data, and not a quality-file size trade off like JPEG.

John

[Louie Sherwin]

Thanks John,  I use Olympus and apparently the ORF files benefit quite a bit from the compression.

-louie

[peterkrogh]

Louie,
Much of the compression gains come in relation to the JPEG file that is in both the DNG and the Raw file.  The rest of it is achieved through lossless compression of the raw data itself.
Peter

[ianw]

I have noticed a wide variation in file sizes between the raw files and the resulting DNG file. All my raw files are exactly the same size, 13.5 MB but the resulting DNG can vary between about 8 MB and  about 11 MB but never larger than the original raw file.

Louie,

I'm an Olympus user as well.  I have an E1 that produces ORF files that are all 10.1MB in size.  I've used an E500 were the ORF files were all 13.4MB.  Until the most recent iteration of cameras Olympus did not compress any data in the raw images.  There are pros and cons to this.  An advantage is for working out 'bucket sizes'.  I always put 400 E1 images or 300 E500 images onto a backup DVD, and have a spreadsheet to tell me how many images of each type to do if I used a combination of cameras.  You also know how many images will fit onto a memory card if you shoot raw only - if it says 380 then that's what you'll get.  The disadvantage is that you are using more storage space than you really need to - actually an advantage if you have a laptop with a small hard drive as you can zip ORF files down to around the equivalent DNG size.

I've just got an E510 and now Olympus are using file compression (also in E410 and forthcoming E3).  I've just come back from a week away where I've taken over 4,000 images and they vary in size from 8.13 to 11.9MB.  This variable size will cause me to rethink how I determine when to backup to DVD - I'll have to take image size into account rather than image numbers - not a problem, but shows that you need to have a flexible workflow.  These files don't zip well - resultant file is a similar size, sometimes a bit smaller and sometimes larger.  When converted to DNG the image sizes are larger - and this is due to the full size JPG image in the DNG, rather than the smaller (and lower quality) one that Olympus put in.

Ian

[dmdorn_ctusa]

Ian,

Have you considered using a program, like ARCHIVE CREATOR, that can burn you backups automatically by spanning more than one DVD.  In addition, it can also create, on each disk a thumbnail directory of the entire backup.

Another program, ImageIngester Pro, can now create spanned backups as the images are ingested from the card or camera.

As an accountant, now retired, I try to avoid spreadsheets anytime I can.

David

[ianw]

David,

Your reply was a blast from the past - I had to re-read the whole thread to remember what it was about.

I use Roxio Media Creator to do my backups - it was free on my PC when I got it.  I'm sure its several versions out of date but it does all I need of it.  It can span data onto several disks if required but I've never backed anything up like this.  I only put 400 images (and sidecars) per DVD.  For my old camera this was exactly 4.05GB.  For my new camera, with bigger but compressed images, this is anything from 3.8 - 4.2GB, although usually at the smaller end.  When reading up on DVD storage some years ago I saw some advice that you shouldn't fill a DVD - it writes from the inside to the edge and if full it will have written right to the edge.  The edge of the DVD is the bit most likely to get damaged from handling.  The amount I write to the disk leaves about 5-8mm of 'safe to touch' disk.

I know ImageIngester Pro quite well - I'm a trouble maker on its forum!  The backup feature is brand new and something I've not tried yet.  On my home set up I'm not sure if I'd ever use it.  First, and most important to me, I don't want to keep all my ingested images.  I review them and delete those that would never get used.  I do a lot of work with models and they blink or squint etc.  These are images of zero value to me and I don't want to back them up.  Second, once reviewed Roxio can write to multiple drives at a time so I get it to create 3 copies at once using all my burners.

That said if I was on the road with my laptop then I may use this feature as an extra means of backup.  I'm not sure how much it would slow ingestion down - I need to try it some time.

I hate spreadsheets as well.  However I'm still using one and can't see that changing.  I use it as my tick list to ensure I've done everything for a set of images.  This means ingesting, reviewing, adding non-ingestion Meta data, using RapidFixer, backing up to hard drives and DVD, and verifying those DVDs with DNG converter.  This is loosely based on the checklist in The DAM Book and fits my needs.

Ian

[dmdorn_ctusa]

Ian,

Wow!, another year gone by between replies.  I lost most of last year to due to tearing my right rotator cuff and being right handed spent most of the year recovering. I discovered I am not a good left hand typist. 

Anyway, I see your point about not using Image Ingester to buiild disk backup during ingestion.  If Image Ingester allowed full screen viewing of thumbs (which is does not do for now) I could see culling images on ingestion and using the back up features.

What I am doing now, which someone posted in a forum and whose name I can't recall, is using FastStoneViewer to review images on the card and make my deletions directly from the card...and then ingest the remainder.  So far I have had no problems with my Transcend UMA CF cards doing this.  They reformat in the camera with no problem. 

This approach is helpful to me as once an image is on the drive; I have an emotional (mental?) block about deleting the image.

FastStoneViewer is great for this as it is, as the name implies, fast but more important is the speed of evaluating the image  A double left click on the thumbnail image and it opens full screen...a single left click and the area of the click is at 100% resolution and with the button still held you can move to any area of the image in the 100% view; a last double click and you back to the thumbnail in the browser window. No need to reselect between a magnifier function and and "hand" function like a lot other image browsers.   The icing on the cake is that for personal use FastStoneViewer is free and a license for commercial use is very reasonable.

I am just now implemeting IDImager as my DAM software and have not really tried the card deletion approach with it  But I would like to hear if others cull directly from the card and then ingest.

David

[peterkrogh]

David,
Im personally uncomfortable with the practice of deleting on the card for a couple of reasons.

The first is that you could delet accidentally, and there is no backup.
The second is that mucking around with the card's filesystem seems risky to me, over the long term.

YMMV,
Peter

[george]

Hi David & Peter,

I am just now implemeting IDImager as my DAM software and have not really tried the card deletion approach with it  But I would like to hear if others cull directly from the card and then ingest.

Im personally uncomfortable with the practice of deleting on the card for a couple of reasons.
The first is that you could delet accidentally, and there is no backup.
The second is that mucking around with the card's filesystem seems risky to me, over the long term.

I also use IDimager for DAM, including its downloader.  However, I don't have the downloader delete images from the card, for exactly the reasons Peter mentioned.  Once everything's backed up to my satisfaction, I format the card in the camera.

--George