An ISO/RAW problem

[SilverAstro]

4 hours ago, Filroden said:

The signal looks the same in both pictures. You can see the same star trails in both. The only difference between them is levels of noise. If you layered both pictures over each other and subtracted them, the stars would completely disappear and you'd have just noise left.

Yep ! Now you see it / now you dont, well almost ! the one differenced over the other, didnt quite get them aligned in the extremities.

 

[SilverAstro]

3 hours ago, dph1nm said:

To quote: "By default, dcraw writes PGM/PPM/PAM with 8-bit samples, a BT.709 gamma curve, a histogram-based white level". So I suspect your dcraw command is scaling the tiffs to have the same levels and applying a non-linear transform. So I think you need "-4" not "-6" as the option. "-4" gives a linear transform instead of a gamma curve and doesn't vary the white level.

Bingo! Bravo mon brave, mentions Nigel in dispatches That solves that mystery, see pic below, , errr ummm  or does it  , , ,   but if all the info (more or less, give or take!) is there in the 400 ready to be scaled up to the same as 1600 with little difference does this not mean it is all there in the RAW(ishness) in the sensor anyway - just waiting to be (suitably) scaled (by the ISO or whatever) ,,, oh dear now my head is really really beginning to hurt, need to go into a darkened room with an ice pack and think about this !!!

EDIT later,  After a while in the darkened room :- In other words, are we now saying that the 'stuff' (pardon the technical term ) in the sensor RAW is the same for any 60sec at f2.8  (regardless of the ISO, much as I was saying earlier!) , just waiting to be scaled by whatever subsequent parameters are chosen ?   arrrgh, I must stop over-thinking this and just go buy me a Canon to try as well 

[jsmoraes]

Quote

After a while in the darkened room :- In other words, are we now saying that the 'stuff' (pardon the technical term ) in the sensor RAW is the same for any 60sec at f2.8  (regardless of the ISO, much as I was saying earlier!) , just waiting to be scaled by whatever subsequent parameters are chosen ? 

... the 'stuff' (pardon the technical term ) in the sensor RAW is the same for any 60sec ...

Yes, Silver. The sensor is a flat plate receiving the impact of photons. The only work it does, is convert these impact of photons (causing charges in the active electronic elements of sensor) to analog eletric current. The time of exposition will determine the amount  of this convertion; amount of details on the photo.

All internal parameters of camera is to improve and amplify those electrical information. But, attention: amplify isn't create new information. Time of exposition will give you AMOUNT of information. Parameters will change the INTENSITY of information.

[Stub Mandrel]

Amplification (effectively the ISO) rating) is performed before digitisation so it affects the RAW data. RAW data is a number for each pixel without colour correction, debayering or otehr adjustment (in theory, but some cameras DO carry out some tweaks to RAW data).

 

Here's a way of understanding it. The graph shows that changing the ISO effectively makes the output cover a smaller and smaller range of voltages on the sensor.

This remains that ISO 1600 has 16 times as many steps across any given difference in light levels as ISO 100, but over a smaller range of light levels.

So a single properly exposed ISO 1600 will ALWAYS be able to capture more detail than a lower ISO, of course, the catch is that the noise gets magnified as well.

Stacking improves the signal:noise ratio.

Use lower ISO and you WILL need more exposures to find the same level of detail, just as if you had taken shorter subs.

For ordinary single-shot pictures you want to minimise the noise and use the lowest ISO you can get away with.

 

Almost finally, stretching lower ISO to match higher ISO will also stretch the noise PLUS stretching won't put in the missing detail.

 

Finally, there are quirks about dark currents, noise and amplification methods that mean it isn't quite that simple. I have read that ISO800 is the best noise/data balance for my canon 450D.

 

[SilverAstro]

Gosh, a big "Thank You" to everyone for all your contributions to my search for understanding I wont say I understand everything about digital ISO ( I'll blame some hang-ups from my film days ! ) but I am much happier now !!

My project today was to draw a line through alpha and zeta Bootes in my two 'pink' stretched images and extract the luminance along it, plotted in gnuplot and output to a png terminal. The plot lines are a bit thin so I hope they will show ok on the forum.   alpha has saturated at 255 but zeta is displaying a nice sharp plot peaking at 245. The really interesting bit is that the ISO400 trace in green stretched (as @dph1nm pointed out) to the ISO1600 trace is a very close match ?? with even the noise in the grass being quite a good fit, not a lot of data lost it would seem.  I think my next experiment will be to do the same from an ISO100 , , ,

Meanwhile there is a beautiful sky out there and the Moon and Jupiter are calling me , , , ,

 

[StuartJPP]

As mentioned, ISO does affect the RAW file...it isn't a setting like the white balance that is just a "hint" to tell the processing software what the setting was.

It determines the amplification applied before digitisation.

On some Nikon/Sony cameras the RAW files are basically "ISOless" or have "ISO Invariance"...i.e...they basically have a single amplifier and as such ISO is basically just a brightness setting...not so for the majority of cameras and certainly not for Canon at the moment.

[SilverAstro]

5 hours ago, StuartJPP said:

On some Nikon/Sony cameras the RAW files are basically "ISOless"

probably to stop interminable discussions on fora about the effect of ISO on Exp  lol

Today's experiment :-  I removed the two stars from the two data sets, and put the noises into a spreadsheet, calculated the RMS, and got 131.6 and 137.3 for the two ISOs ,, pretty durn close I recon ( durn being not a statistical term )

[ I used   sqrt(average(F1:F347)  where the Fs are (C1*C1)etc  if anyone wants to check over my shoulder considering my earlier prob with DCRAW ! ]

if I put 243 for zetaBoo in to 20log(243/131.6) =5.3dB   and 20log(243/137.3 = 4.9dB   have we got a mathematician in the house ? should that be 10log  ?? (not sure if I am talking power or voltage ratio here )

Anyway, the S/N ratio for the two ISOs are still looking very close ? ( this is all back of envelope on the run and all subject to change in the event of blunders and peer (you lot !) review )

Epilog : this is all in aid of a continuing puzzle of mine : how to evaluate the quality of stacking and how many more subs are needed to improve a stack by a quantifiable 'how much' iyswim )

[Astrofriend]

When using dcraw you must set the parameters to 16 bit linear. No brigthen or stretch. Don't use cameras internal whitebalance.

Then you can compare different iso settings and see the effect of it.

 

Lars