For my own work, I use one with the diffuser removed. I don't want a piece of dust to show up in a lot of images.
It does mean that you need to make sure the light is well-diffused. I've found the need to triple-diffuse my strobes - A softbox with a baffle, and then a piece of milk plexiglass between the softbox and the film stage a foot or so back).
I have a few questions about you lighting. I have found the lighting set up to be very critical in terms of getting accurate colors, and I wanted to see what exactly you were using? How many watt-seconds?, what size softbox?, set-up how far away?, etc. Any chance you have a photo of your lighting set up?