How raw is raw?

[Kaptain Klevtsov]

The raw image format that we use is allegedly the data straight off the ccd in the DSLR. My question is, is there any point setting ISO, does it make a difference (I believe it does) and what else does the camera adjust in the raw file? I'm thinking Nikon specifically here, but I wouldn't want to limit discussion to just Nikon as all input will be good.

Kaptain Klevtsov

[EA2007]

I think the ISO makes a huge difference.....its the difference between a darker shot and a lighter shot of the same exposure time, however I have noticed especially on Olympus dSLR's that the noise is tremendous once you go past 640 ISO, but thats down to a larger chip.

For speed of processing then Canon and Nikon are brilliant, my Olympus takes ages, for example:

1 min exposure on ISO 800 in RAW takes another 40 seconds to process before the image appears

likewise....

1 min exposure on ISO 800 in RAW with noise reduction on takes another minute to process before the image appears.

Whereas I have seen processing on the Canon's being just a second after the shutter re-opens.

[Kaptain Klevtsov]

Of course, but is the same data collected on the camera chip for a 20 second exposure at ISO200 as if the camera was set to ISO1600? Of course it is. So if the raw data is off the chip, both raw files should be the same.

Only if the ISO scaling is done before the raw file is written would we see a difference. Sure the JPG would have the scaling, but the raw as well?

Kaptain Klevtsov

[Sam]

KK, I just took too raw shots with my camera at different ISO settings and the output raw files are certainly different.

Sam

[Kaptain Klevtsov]

So Canon do that too, I'd guessed that they would. I wonder what else is not as raw as it might be?

Kaptain Klevtsov

[Sam]

KK,

Here the ISO100 one:

(click to enlarge)

Here's the ISO 1600 one:

(click to enlarge)

Both done with the canon 400D at F3.5 and speed 1/160. They were done in RAW and then converted to JPG to post on here - but you get the idea.

Sam

[narrowbandpaul]

hi KK

The ISO term stems from photographic film of days gone by. The term is still used to define sensitivity!

If my theory is correct, then ISO is the biggest lie I have come across, in DSLR imaging.

We will start with the notion of gain...which converts electrons to a Digital Number (DN) which is displayed on your screen.

During your exposure, photons kick electrons in to a higher energy state, where the are free to move. The free electrons are gathered by the pixel and stored, forming your image.

After you finish your exposure the sensor is read out, and the electrons in each pixel are converted first to a voltage, then to a digital number DN. It is this number that you see on your screen...you dont see the number of electrons, you see the number of DN.

The gain of the camera tells you how many electrons correspond to each DN, or 1/gain tells you how many DN correspond to each electron.

For example...a gain of 4e-/DN means each DN corresponds to 4electrons, so a value of 6000DN on your screen means 24000e- in that pixel

similarly 1/gain corresponds to 0.25 DN/e-...ie every DN corresponds to a quarter of an electron...

Heres my theory....

The ISO when applied to a digital sensor, is just the value of the gain...when you have a low ISO, like 100, the value of the gain is high, so that 1/gain is low...ie it takes a lot of electrons to get a 'bright' correctly exposed image...

When you go to a high ISO, the gain is low, so that 1/gain is high. This means that it takes less electrons to get to the same signal level...in DN of course

Thinking about it...

lets say you take 2 exposures...one at ISO 100 and one at ISO 800, both with the same exposure time...

at both times the number of ELECTRONS collected in a pixel will be the same, because the number of photons coming from the object is the same.

Lets say the exposures collect 4000e- in a pixel

Say at ISO100 that the gain is 1..so 1/gain is also one. This means that you will see 4000*1/gain DN in your image...

For ISO 100 this means 4000DN, which is quite faint...

Lets say at ISO800 the value of the gain is 0.1, ie 1/gain =10.

This means the same number of electrons now corresponds to 40,000DN

At ISO 800 in our example is 10x brighter....

I state that this is a lie.

The most important factor in deciding image quality is the Signal to Noise Ratio SNR

This is for the most part gain independant. Half the gain, your image is 2x as bright, but 2* noisier (also observed at high ISO), so the SNR is still the same. The above statement was considering DN, not e- as gain controls the DN you see....

Each pixel in our ISO comparison had the same exposure length, so had the same number of electrons. The SNR of both images is the same....

This follows from the fact that DN units are irrelevant, only e- have any meaning

So ISO makes your image brighter...but not any more detailed, or with higher SNR

the SNR, or quality is ISO independant....

The low gain at high ISO makes the noise more apparant too, as it makes a small number of electrons represent a large DN value....your image looks noisier, but isnt, as it will contain the same number of electrons as an image at a low ISO.

So you may as well work at low ISO since the noise is less apparant...it will need more work for processing, as the data will lie at low DN values at the left of the histogram...

I fully believe the above arguement to be the true.

Hope this helps

Paul

[Sam]

The signal from the chip is modified by the ISO setting prior to the signal being converted to digital rather than a scaling factor applied after the signal has been digitised.

[narrowbandpaul]

The gain is where the signal is digitised...it is analogue up to that point

[Kaptain Klevtsov]

So if the ISO is applied at the ADC stage, I could use ISO 200 all the timje and do the scaling in PS or another photo software application then, as all the data would be there and bright stars wouldn't be clipped?

Or, putting it another way, all the data is in the low ISO raw, but just needs more post processing to boost the signal later. Kind of like the exposure compensation thats available when opening a raw in PS?

Or is there an advantage to using gain at the ADC stage? I can't think of one off the top of my head as its only a scaling factor, the SNR will be the same.

Kaptain klevtsov

[Sam]

KK,

With the two raw images I took earlier, I cranked up the brightness on the ISO100 one to give it the same brightness as the ISO1600 one. Interestingly the brightened 100 showed a heck of a lot more noise than the 1600 - lots of vertical banding. :scratch:

Sam

[EA2007]

Paul, you yet again baffle us with your knowledge........however we all know though that photography is magic and Canon, Kodak, Nikon and most other leading imaging manufacturers are owned by Paul Daniels and Debbie McGee!

Ha ha

[narrowbandpaul]

i baffle or alienate lots of people

photography aint magic, its just quantum mechanics and electrostatics

[EA2007]

I am quite getting into this photography stuff,(not especailly asto-imaging as I ain't got the kit as yet) but always had an interest in it and videoing, love messing around with a camcorder. You should see some of my holiday films they are tragically amusing and very geeky ha ha.

Although my dads the best.......he leaves the camcorder on whilst walking and only films his feet and the road / gravel......we acceptably call this "footage"

[EA2007]

p.s. he doesn't do it on purpose, only by accident (don't want you all thinking hes mad, or Quentin Tarantino, who is actually mad!)

[dph1nm]

So you may as well work at low ISO since the noise is less apparant...it will need more work for processing, as the data will lie at low DN values at the left of the histogram...

I have read that the relative read noise is altered by changing the ISO (i.e. gain), but not by much, so one can probably ignore this effect. Also, some cameras cheat, and get high ISOs by simply doing arithmetic in software!

There is also an argument for setting your ISO so that the gain is ~1 i.e. you get 1 DN for 1 photon, on the grounds that anything lower loses real data. For DSLRs this can be quite high (> ISO1000). Personally I think this is a bit extreme - you just need to ensure your read noise (maybe even just sky background noise?) is sampled adequately, which you can probably do at a lower ISO.

NigelM

[Kaptain Klevtsov]

I have read that the relative read noise is altered by changing the ISO (i.e. gain), but not by much, so one can probably ignore this effect. Also, some cameras cheat, and get high ISOs by simply doing arithmetic in software!

NigelM

How else could they do it? You expose the camera sensor for a period of time and it gets photons, so the rest has to be up to camera software. I'm trying to find out why, or if, the raw data isn't. Whatever the camera can do in software, I can do in Photoshop, so how much does the camera do in making the raw file and what can be done to optimise the "raw" image data is what I'm chasing down.

Kaptain Klevtsov

[dph1nm]

How else could they do it?

As far as I understand it (and I am no expert on digital cameras, so I might be wrong), they electronically alter the amplifier gain before the analogue signal from the CCD goes into the A-to-D converter, so it is a hardware fix not a software one (and this means that RAW data is affected by this). The ones that cheat take the output from the A-to-D converter and run a bit of code that multiplies the numbers by two (or whatever factor is needed).

NigelM

[Sam]

How else could they do it?

As far as I understand it (and I am no expert on digital cameras, so I might be wrong), they electronically alter the amplifier gain before the analogue signal from the CCD goes into the A-to-D converter, so it is a hardware fix not a software one (and this means that RAW data is affected by this). The ones that cheat take the output from the A-to-D converter and run a bit of code that multiplies the numbers by two (or whatever factor is needed).

NigelM

I had a chat with someone at work who knows stuff and the above is what they reckon as well.

So basically RAW isn't really raw!

[Psychobilly]

Various manufacturers have there own take on RAW as well... the nikons have the imfamous star eater mode which you can defeat by switching the camera of straight after finishing the exposure but before the camera gets to work... Some

manufacturers also apply various compression techniques to their raw images...

Still my fav format though ... as you can do so much with the datapost capture......

Billy...