Jump to content

Banner.jpg.b89429c566825f6ab32bcafbada449c9.jpg

The "No EQ" DSO Challenge!


JGM1971

Recommended Posts

The effect of ISO on a RAW ( if any !) and the result in stacks, is the cause of much debate across the interweb ! I think a new topic ex-this EQ topic may be a good idea cos I suspect this will run and run ? ! :)

My experiments so far are with Filroden, ie. any ISO setting is a post RAW developement feature but ymmv depending upon camera manufacturer and if or not a true RAW  ( or true-ish) is being presented from the sensor for the user. Most third-party softwares like DSS and I believe Startools and Pixinsight (but I dont have these latter two) all use Dave Coffin's DCRAW. Thus it would be a good idea to view one's RAWs in DCRAW first before getting hung up on which ISO is best, to eliminate the possibility that DSS etc. are doing things under the covers before showing the "raw".

 

  • Like 1
Link to comment
Share on other sites

7 hours ago, SteveNickolls said:

Hi Mike, Thanks for these posts. I can understand some of the article but floundering elsewhere. I took a look on my camera at the RAW exposures I took a few nights ago with the histogram also showing. I found that the histogram, whether for brightness or RGB channels was showing well off from the LH side about 60-70% along not the 10% in the article. Now this was for 50 seconds exposures at ISO 1600. I presume it means the image is being swamped by light pollution (it's bad here) so I should reduce the exposure time or drop the ISO value? Or does it matter? Unfortunately I have no background in photography to draw upon, just learning from mistakes and successes.

Cheers,
Steve

Sounds like your sky is quite bright.

Generally speaking the correct exposure for your sky conditions is in the range 20 to 40% but always err on longer than shorter.
Personally I would'nt go that high but it does'nt mean you cannot get anything out of it.
Try dropping the ISO and keep the same exposure and see if it clears at least 10%, if it does your good.

  • Like 1
Link to comment
Share on other sites

7 hours ago, Filroden said:

A quick search doesn't give me a clear answer. Your assumption about it being log base 2 looks broadly correct, but based on a test someone did and posted to flickr, it seems the histogram shows more than 5 stops, but with the additional stops bunched at either end of the scale. Not sure me using spaces in this does it correctly but the numbers 0, 4, 8, 16 and 64 line up with the five vertical lines, with the 2 and 32 being halfway between the first set and last set of lines. Note: this is specific to Canon.

0      2      4            8            16    32     64

Hopefully someone much more technical than me has a better answer!

The x-axis on the histogram is effectively exposure value, i.e. f stop, so log to the base 2 would be correct I think.

Ian

Link to comment
Share on other sites

The signal from the CCD/CMOS has to be processed to perform the A2D conversion. ISO affects the gain applied during that conversion and as it can't be zero gain then EVERY digital image has to have some effective ISO rating

Link to comment
Share on other sites

15 hours ago, Stub Mandrel said:

An anyone answer me a question - is the histogram x-scale linear or logarithmic scale?

i.e. if it has say 5 divisions are these:

1          2          3         4        5            -  linear

1          10        100      1000   10000     - log base 10

1          2          4          8       16           - log base 2

My guess is that it's a log base 2, which would fit with things like changing aperture by one f-stop, changing ISO by one step or doubling exposure would move the histogram by one division, this would also explain why a histogram shifted right by a longer exposure doesn't get noticeably wider.

Does anyone know what scale Canon use?

Personally I don't worry about the intricacies of the in camera histogram.
It's just a tool to give you an idea that you have the correct exposure.
It's a histogram of a jpeg, just get the peak to at least 20% and you should be good.
If you want to push it higher, try it but I believe there is no real benefit of going over 50%.

 

  • Like 1
Link to comment
Share on other sites

Hi folks, here is my latest offering taken on the evening of the 14th May 2016 of the Crescent Nebula NGC 6888 in Cygnus. The image was taken using a SkyWatcher Startravel 102mm f/4.9 refractor, Synscan Alt-Az mount and Canon 600D DSLR. Due to the position of the target I was able to employ 60 second exposures. I took 91 sixty second light frames at ISO 1600 and for the first time used stock dark frames (x50) and bias frames (x50). x50 flat frames were taken the following morning. The frames were stacked in DSS and further processed in StarTools. Of the 91 light frames taken DSS was happy to use x67 (just under 74%).

Hope you like the image.

NGC 6888 Crescent Nebula-

NGC 6888Save5.jpg

Cheers,
Steve

Edited by SteveNickolls
  • Like 3
Link to comment
Share on other sites

Having looked at Samir's article, it set we pondering, and this is my take on what goes on!

When we image, the light photons liberate electronic charge in each of the pixels of the sensor; the more photons, the more charge liberated. At the end of the exposure, the charge in each pixel is measured in an analogue way, but for the purposes of producing a digital RAW file it has to be converted into a digital signal. This is the job of the analogue-to-digital converter, or ADC. At 'base' or 'native' ISO, this is a straight conversion of the analogue signal. In order to give a range of sensitivities, as measured on an increasing ISO scale, the analogue signal is amplified before being passed to the ADC, so that weaker signals have a greater amplitude before measurement. There is a downside to this, however, because at some point the ADC will saturate, and this means that pixels which have received a high number of photons, and therefore contain a large liberated charge, will be above the measurement abilities of the ADC. The net affect of this is that the  dynamic range is reduced, more so as the ISO values are increased.

There is a further complication, because manufacturers introduce digital gain as well as analogue gain, at higher ISOs, and are often very guarded about the processes that they use, so it isn't always obvious what the native ISO of the sensor is and at what point they introduce digital gain. We don't want to operate in the digital gain region.

I managed to find this information for my camera from http://www.photonstophotos.net/Charts/RN_ADU.htm

chart.jpeg

Read noise increases linearly with ISO but above ISO1600 it is clear that the operation of the sensor moves into a different regime. Curves for other cameras can be quite different, according to the strategies adopted by the camera manufacturers.

So what about the histogram? The histogram is a graphical display of the distribution of light intensity recorded by the sensor. It is not taken directly from the sensor, but rather from a jpeg processed from the RAW data. In producing the jpeg, a colour calibration curve and a response curve is applied in order to make the linear data recorded by the sensor appear more as the eye would perceive the scene. These curves are specific to camera models. Typically, the x-coordinate of the histogram would be linearly scaled from 0 to 256, so that no signal would be zero, and the maximum signal before saturation, or clipping, occurs would be 256. Some cameras show the histograms for each RGB colour, but others just a combined luminance curve. Because the histogram is taken from the jpeg, it should be clear that the display will vary according to the ISO setting, so that a source giving a single peak near the bottom end would have a peak which shifts progressively to the right as the ISO setting used to image that source is raised. Therefore, any recommendation to use, say 10% or 30%, for the placing of the sky-fog peak seems to me to be somewhat erroneous. I guess the main thing is to ensure that there is a gap between the peak and the zero of the histogram, but not one so wide that the rest of the curve is squashed to the right.

Up until now I've not taken a lot of notice of the histogram, but I've now gone through all my images just to find out what I've been getting. The following are typical:

This is M64, at 1600ISO, single frame of 30s

Histo M64.jpg

 

and this is M42, at 1600ISO, single frame of 15s.

Histo M42.jpg

Notice that in both of these one can make out a small component of the curve at the far right hand end of the histogram, and although this isn't the case for all my images, it is so in by far the majority of cases. So even with an exposure of 15s, I am still at risk of saturating the brightest stars. What I am not sure of though, is whether this really means that the sensor is saturated as well, but if it is, then it is likely that colour information will be lost for these particular objects. With this particular image, I guess I had the option of dropping the ISO to try to recapture some of the dynamic range, perhaps to as low as 400ISO, as there is plenty of room to the left of the sky-fog peak. That is predicated on being able to see the distribution well enough on the camera itself of course, as the histograms presented here are taken from my RAW processor. I could reduce the exposure time, but that implies a reduction in the number of photons captured. That is unacceptable as I want to maximise the signal-to-noise ratio to get the cleanest possible image.

Here endeth the lesson :icon_biggrin:

Ian

  • Like 4
Link to comment
Share on other sites

Interesting read.

What does one of your 90s histograms look like?

It might be that there is always a white out to the far right, or what does the histogram look like from a camera with a greater dynamic range maybe like a 5diii

Edited by happy-kat
Link to comment
Share on other sites

15 minutes ago, happy-kat said:

Interesting read.

What does one of your 90s histograms look like?

It might be that there is always a white out to the far right, or what does the histogram look like from a camera with a greater dynamic range maybe like a 5diii

Thanks happy-kat. The longest exposure I've used is 30s because most of my imaging is of objects towards the south where rotation is at its worst, and because to go beyond that I'd have to buy an intervalometer. I should think the sky-fog peak would be well up the histogram with 3x the exposure.

I'm not sure that there is much further to the right, because you'd expect to see the histogram piling up against the histogram RH edge, though admittedly with these low exposure levels that might be difficult to observe until it was extreme.

I can't say whether a 5diii would be better or by how much. It would depend on the ISO it needed apart from anything else, and the Fuji does seem to have good photon efficiency down into the red. I don't think their performances are too far apart.

Ian

Edited by The Admiral
Link to comment
Share on other sites

3 hours ago, SteveNickolls said:

Hi folks, here is my latest offering taken on the evening of the 14th May 2016 of the Crescent Nebula NGC 6888 in Cygnus. The image was taken using a SkyWatcher Startravel 102mm f/4.9 refractor, Synscan Alt-Az mount and Canon 600D DSLR. Due to the position of the target I was able to employ 60 second exposures. I took 91 sixty second light frames at ISO 1600 and for the first time used stock dark frames (x50) and bias frames (x50). x50 flat frames were taken the following morning. The frames were stacked in DSS and further processed in StarTools. Of the 91 light frames taken DSS was happy to use x67 (just under 74%).

Hope you like the image.

NGC 6888 Crescent Nebula-

 

Beautiful image. Do you think the stock darks and bias had any effect?

I've been quiet for a few days. Although it's been clear here, the moon is getting too bright in the evenings and I've not had the stamina to wait until 2am for it to darken. Also, even though it's often looked clear, there has been a high haze that has reflected more light than usual.

I did take some subs but they were so bright from the moon plus normal background lights that I've not been able to integrate them with earlier subs. However, I did drag myself outside last night at about 2:30 and what a sight! I could almost make out the Milky Way (may have been imagining it but I thought it was there, overhead) and I could definitely make out mag 4.3 stars with direct vision. I was so tempted to set up and take some subs as the North America Nebula was at about 60deg altitude. Antares was quite high and another tempting target, though by 2:30 it was already behind the houses near me. I'm now wondering whether I need to set up and start imaging around 1:30. Though I suspect that now the moon will destroy any attempt other than globulars.

  • Like 1
Link to comment
Share on other sites

1 hour ago, The Admiral said:

Having looked at Samir's article, it set we pondering, and this is my take on what goes on!

When we image, the light photons liberate electronic charge in each of the pixels of the sensor; the more photons, the more charge liberated. At the end of the exposure, the charge in each pixel is measured in an analogue way, but for the purposes of producing a digital RAW file it has to be converted into a digital signal. This is the job of the analogue-to-digital converter, or ADC. At 'base' or 'native' ISO, this is a straight conversion of the analogue signal. In order to give a range of sensitivities, as measured on an increasing ISO scale, the analogue signal is amplified before being passed to the ADC, so that weaker signals have a greater amplitude before measurement. There is a downside to this, however, because at some point the ADC will saturate, and this means that pixels which have received a high number of photons, and therefore contain a large liberated charge, will be above the measurement abilities of the ADC. The net affect of this is that the  dynamic range is reduced, more so as the ISO values are increased.

There is a further complication, because manufacturers introduce digital gain as well as analogue gain, at higher ISOs, and are often very guarded about the processes that they use, so it isn't always obvious what the native ISO of the sensor is and at what point they introduce digital gain. We don't want to operate in the digital gain region.

I managed to find this information for my camera from http://www.photonstophotos.net/Charts/RN_ADU.htm

chart.jpeg

Read noise increases linearly with ISO but above ISO1600 it is clear that the operation of the sensor moves into a different regime. Curves for other cameras can be quite different, according to the strategies adopted by the camera manufacturers.

So what about the histogram? The histogram is a graphical display of the distribution of light intensity recorded by the sensor. It is not taken directly from the sensor, but rather from a jpeg processed from the RAW data. In producing the jpeg, a colour calibration curve and a response curve is applied in order to make the linear data recorded by the sensor appear more as the eye would perceive the scene. These curves are specific to camera models. Typically, the x-coordinate of the histogram would be linearly scaled from 0 to 256, so that no signal would be zero, and the maximum signal before saturation, or clipping, occurs would be 256. Some cameras show the histograms for each RGB colour, but others just a combined luminance curve. Because the histogram is taken from the jpeg, it should be clear that the display will vary according to the ISO setting, so that a source giving a single peak near the bottom end would have a peak which shifts progressively to the right as the ISO setting used to image that source is raised. Therefore, any recommendation to use, say 10% or 30%, for the placing of the sky-fog peak seems to me to be somewhat erroneous. I guess the main thing is to ensure that there is a gap between the peak and the zero of the histogram, but not one so wide that the rest of the curve is squashed to the right.

Up until now I've not taken a lot of notice of the histogram, but I've now gone through all my images just to find out what I've been getting. The following are typical:

This is M64, at 1600ISO, single frame of 30s

Histo M64.jpg

 

and this is M42, at 1600ISO, single frame of 15s.

Histo M42.jpg

Notice that in both of these one can make out a small component of the curve at the far right hand end of the histogram, and although this isn't the case for all my images, it is so in by far the majority of cases. So even with an exposure of 15s, I am still at risk of saturating the brightest stars. What I am not sure of though, is whether this really means that the sensor is saturated as well, but if it is, then it is likely that colour information will be lost for these particular objects. With this particular image, I guess I had the option of dropping the ISO to try to recapture some of the dynamic range, perhaps to as low as 400ISO, as there is plenty of room to the left of the sky-fog peak. That is predicated on being able to see the distribution well enough on the camera itself of course, as the histograms presented here are taken from my RAW processor. I could reduce the exposure time, but that implies a reduction in the number of photons captured. That is unacceptable as I want to maximise the signal-to-noise ratio to get the cleanest possible image.

Here endeth the lesson :icon_biggrin:

Ian

A very good post, but my thought is that the area on far right is occupied by bright stars in MOST images is tiny but we can explore it. Choosing an image of a light-polluted sky (over Walsall's light dome)

Bootes.jpg

Corel Photo Paint has an option to adjust histogram clipping. Its auto setting allows 5% clipping  and the histogram of a rather light polluted sky looks like this, kjsut as it would in Photoshop or on the back of the camera:

histogram 1.jpg

Now look at this histogram with 100% clipping allowed:

 

histogram.jpg

 

In particular note the 'secondary peak' at far right, that's the bright star Arcturus and other brighter stars. The interesting thing is that it doesn't appear to be significantly clipped...

A shame Photoshop doesn't have this great feature of Photo-paint.

 

  • Like 2
Link to comment
Share on other sites

57 minutes ago, Filroden said:

Beautiful image. Do you think the stock darks and bias had any effect?

I've been quiet for a few days. Although it's been clear here, the moon is getting too bright in the evenings and I've not had the stamina to wait until 2am for it to darken. Also, even though it's often looked clear, there has been a high haze that has reflected more light than usual.

I did take some subs but they were so bright from the moon plus normal background lights that I've not been able to integrate them with earlier subs. However, I did drag myself outside last night at about 2:30 and what a sight! I could almost make out the Milky Way (may have been imagining it but I thought it was there, overhead) and I could definitely make out mag 4.3 stars with direct vision. I was so tempted to set up and take some subs as the North America Nebula was at about 60deg altitude. Antares was quite high and another tempting target, though by 2:30 it was already behind the houses near me. I'm now wondering whether I need to set up and start imaging around 1:30. Though I suspect that now the moon will destroy any attempt other than globulars.

Hi Ken,

Thanks for your kind comment. It might be my last effort for this season as the nights are taking longer to darken and I need my sleep unfortunately. Good luck imaging from 1.30 AM it's really what's needed at this time. Yes, the Moon seems to have an agreement with the weather about coinciding with clear skies!

I don't believe using stock frames has done any detriment Ken (but no way to objectively tell really) and I'm getting more confident in StarTools with the mask tool and FILTER module. Still a long way to go but getting there. I'm sure using 60 seconds exposures and doing a fair number of them helped this time though. 

Seeing the Milky Way would have been a great sight, the light pollution affecting your sky sounds similar to that here as I can make out the Mag 4.3 stars in Ursa Minor but seldom the fainter ones. I've only glimpsed the MW once from here and that was about 4 years ago in August if i recollect correctly. I'm waiting until the county council swaps all the remaining sodium street lights for leds and we might get to see the MW again. Fingers crossed anyway.

I would love to be able to view the Southern sky low down, I've only observed Sagittarius and Scorpio when on holiday at a dark site and there's a lot of Messiers I would like to see. There is a newly opened public park nearby with views to the horizon over the Trent Valley so if I can rope in some heavies to come along for security I might get to see Sagittarius and Scorpio later in the year.

Cheers,
Steve

  • Like 1
Link to comment
Share on other sites

Thanks Neil.

I'm not entirely sure what is being shown in your example - I'm not familiar with Paint.

Actually, in answer to happy-kat's question about 90s exposure, I thought I'd simulate that in Capture 1. That 3x exposure is equivalent to 1.7 stops (or EV), so adjusting the exposure slider I get this.

Histo2 M42.jpg

However, this time I enabled 'exposure warning', which will indicate where the value exceeds 255 by the red splotches. As you can see, the centre of M42 and other stars appear to be clipped. When I checked with the x1 setting, I still had clipping. As I'm sure that you are aware, RAW converters can 'fill in' for saturated channels in order to try to make a 'best guess' at what the colour should be, so caution is needed.

Exporting the normal file as a tiff and looking at it in my favoured pixel manipulator, Picture Window Pro, using high histogram expansion, I get this:

Histostretch M42.jpg

As you can see, the data is definitely piling up on the RHS, so I think in my case there is clipping of highlights.

Ian

  • Like 2
Link to comment
Share on other sites

  • 2 weeks later...

Things have been a bit quiet around here; the truth is I find late nights difficult! Anyhow, I've just acquired a 0.79x reducer/flattener and I took the opportunity late last night to try it out and see if I could optimise the sensor spacing. In order to minimize the risk of trailing I used 15 second subs and only took 10 at each setting. That enabled me to find frames with minimal trailing and look at the distortion of the stars near the corners.

Anyhow, as I was looking towards Lyra I also captured the Ring Nebula, so I wondered what I'd be able to get out of those 10 subs. Surprisingly, I thought it worked quite well, considering that the integrated exposure was only 2½ minutes! This is without the reducer/sensor spacing being optimised.

ST1 Lights gen-LR1.jpg

 

Here is M57 enlarged.

ST1 Lights gen-LR1 crp res shp.jpg

 

As usual, the set-up was an Altair Wave 102mm f7 SuperED APO, TS Photoline 2" 0.79x reducer/flattener, Fuji X-T1, Nexstar 6/8SE Alt-Az mount, 28 May 2016. No flats or darks used, though I did use bias frames (~50). Stacked in DSS, processed in StarTools, and finished off in Lightroom and PWPro.

Cheers, Ian

  • Like 6
Link to comment
Share on other sites

I'm partway through moving house so the scopes are all packed up. Typically tonight looks like it might me clear! Anyway, I doubt I'll be Imaging before July. 

Thats a nice M57 but is there a slight pink/red tint to it? Some of the stars look pink whereas they are probably closer to white? Nonetheless, amazing for 150s of data!

That reducer is giving you a similar field of view as my 80mm. Can't wait to see everyone's efforts on M33 in the coming months. 

Link to comment
Share on other sites

Thanks for your comment Ken, and yes you are probably right. I always seem to have trouble with star colours and often get pink stars, and M57 should really go blue-cyan-orange and not blue-cyan-magenta. Might have another go tomorrow when I feel a bit less jaded after last night's extended day. Us olduns can't take these late nights like we used to :icon_biggrin:.

I hope the house move goes smoothly.

Ian

Link to comment
Share on other sites

Hi,

The month of May has certainly provided a greater than usual number of imaging nights this year and on the 28th I was able to image the Cocoon Nebula IC 5146 (C 19). The image used x75 one minute exposures at ISO 800 plus x60 darks, x50 flat frames and x50 bias frames. The frames were stacked in DSS and the master image processed using StarTools. The equipment used was my SkyWatcher Startravel 102mm f/4.9 refractor, Synscan Alt-Az mount and Canon 600D DSLR. x91 light frames were taken and DSS accepted 75 for stacking (82%). The dark nebulosity around IC 5146 and extending above it and apparently trailing to the right is Barnard 168. While the Cocoon nebula is around 12' in angular measurement its true size is around 15 light years, amazing. I hope you like the image.

Cheers,
Steve

IC5146 SGL.jpg

  • Like 5
Link to comment
Share on other sites

4 hours ago, Stub Mandrel said:

It amazes me how some parts of the sky are overflowing with stars like that shot, and others are almost empty - at least until you use a Hubble to spot the zillions of galaxies hidden in the background.

Yes, your eye sees nothing in the light pollution then the camera opens up a vista of stars across unimaginable stretches of space, quite breathtaking.

3 hours ago, The Admiral said:

Good capture Steve, and well done for having the fortitude for imaging at this time of year!

Ian

Hi Ian,

Thanks for the endorsement, I had thought we had seen the last of the clear skies in May but the past few days have made it a bumper month.

 

Cheers,
Steve

Link to comment
Share on other sites

Steve and Ian,  a couple of  great nebula  images  there, I am a fan of the nebula's .

Nice and clear, good stars well done guys.  

Waiting for some clear sky's  now. 

Nige.

  • Like 1
Link to comment
Share on other sites

Its a little quiet here atm so here's an image I wasn't going to post until finished but that could take a while waiting for the right conditions.

 Last night when it finally got dark I wanted to get a short tester of another area of NGC7000, so I took only about 30 minutes or so exposure. 

It's more or less what I expected to see except the 3 or 4 odd smudges, I'm not sure whether they are on the camera or scope but they don't appear to be in other images of ngc 7000 ,  any ideas?

My next late session will be to capture this with at least  1.5 hours exp. This was a quick test to make sure I was getting the right part so rushed through  StarTools too :) 

45 × 30s + 5 × 45s light 12 dark 50 bias. DSS, ST . 150p alt az goto.

Nige.

PSX_20160607_201652.jpg

Link to comment
Share on other sites

Hi Nige,

Good going with NGC 7000, will you add the additional images to what you already have? I'm intrigued how much exposures this object will soak up :-) From what I've experienced with StarTools it likes lots of exposure time, it makes all the subsequent processing steps easier and you find you can actually use some modules to produce nice images. You can find odd colours here and there in an image with little data and in the WIPE module selecting Vignetting can help remove odd colours and gradients. Good luck taking more images.

Cheers,
Steve

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you agree to our Terms of Use.