First LRGB image - strange RGB glows around edges

[symmetal]

Brendan, here's my effort with StarTools . The  'Wipe' coped pretty well with the background, but setting the 'gradient fall off' to 75% helped quite a bit too.

On Autodev after selecting the bulk of M81 as a ROI, setting 'Ignore Fine Detail' to 2.5 pixels helped avoid accentuating the noise.

On 'Color' module using the Star sampling technique gets the colour much closer.

I used the default Noise Reduction settings as it didn't seem to overdo things like it sometimes can, and left enough noise to be more realistic.

I used Photoshop to raise the background to around RGB 23,23,23 and did slight colour adjustments to taste.

All in all, a pretty good result as you didn't have a great number of subs in total. 🙂

In APP, after doing your integration it's bebeficial to show the analytical graphs to see which subs are the worst and can then be removed from another integration.

Right click on the file list at the bottom of the screen and select 'sort on quality'. Then right click again and select 'Create Analytical Graph' and on the next screen again select quality. The resultant graph shows your channels sorted best to worst and bad subs show up lower down the graph. The sub numbers refer to the entry in the ordered list at the bottom and you can untick, remove or even delete poor subs to excluse them from another integration.

Repeat the above  using 'sort on star shape', and then 'sort on SNR' to get graphs corresponding to those traits to see the worst subs again and exclude them if you wish.

Then do another integration to see if the result looks better.

Alan

 

[symmetal]

I forgot to add that your darks and flats look fine and the minimum ADU on your darks is around 40 so offset 50 is fine. Flats are nicely centred on the histogram.

APP did give me the same warning as you had about flat and flat darks not matching but checking the fits headers I can't see what the problem. I though that APT may have been mis-labeling something but they all look fine. It's worth asking on the APP forum why this is happening.

Your optimum sky background ADU for 10 * RN^2  is 1290, Your subs are on average

L   1950 ADU

R  1960 ADU

G  2200 ADU

B  1770 ADU

The L matches the RGB fairly well so 3 x RGB exposure is correct. You can reduce the overall exposures in proportion, but if you go too far you run the risk of blue going below the optimum as blue is the least sensitive. Personally I would leave them as they are. When you've got more experience you can tinker with them a bit if you wish. Don't try using different exposures for RG and B to match the sky ADU better as it becomes a nightmare, as you need different darks for each colour. 😬

Alan

 

[vlaiv]

24 minutes ago, symmetal said:

Your optimum sky background ADU for 10 * RN^2  is 1290, Your subs are on average

L   1950 ADU

R  1960 ADU

G  2200 ADU

B  1770 ADU

I think there are quite a few problems with this calculation.

Let's use x5 rule and see what we will come up with.

At unity gain, read noise of ASI1600 is 1.7e, so x5 larger read noise is 8.5e. We want LP levels to be such that they produce 8.5e of read noise.

With shot noise relationship is simple signal is square of associated noise, so LP signal that will produce 8.5e is 72.25e That is our target.

Since camera is set to unity gain - ADUs should match electrons, but remember, ASI1600 is 12bit camera and ADU values are in fact multiplied by x16 to fill 16bit range.

Resulting LP level needed (in ADU like they are in subs) is 72.26 * 16 = 1156

All subs show larger value than that, so all exposure lengths are good. If need be, they can be reduced, but if not - I would leave them as is.

[BrendanC]

Again, thank you @vlaiv and @symmetal, you've been absolutely brilliant. I've learned a ton of stuff over the past few days, and I also have 'The Astrophotography Manual: A Practical and Scientific Approach to Deep Sky Imaging' on order so that I can get to grips more with the theory as well as the practice.

Edited April 3, 2022 by BrendanC

[symmetal]

30 minutes ago, vlaiv said:

I think there are quite a few problems with this calculation.

There's no hard and fast rule to use. But the difference between your value and the one I use, that seemed to be widely quoted when CMOS cameras came out is not that significant.  1290 ADU vs 1156 ADU.

Using blue, your calculation implies the exposures overall could be reduced to 65% of their current value, while mine implies they could be reduced to 72%. That's not a difference to be concerned about I would have thought, and they are only guidelines after all. If there is some moon about the exposures could be reduced far more to like 25% but that means loads of different sets of darks needed for different sky conditions so is it worth all that hassle. 🙂

Alan

[vlaiv]

31 minutes ago, symmetal said:

There's no hard and fast rule to use. But the difference between your value and the one I use, that seemed to be widely quoted when CMOS cameras came out is not that significant.  1290 ADU vs 1156 ADU.

Quite true, but here is bit that I don't really get:

1 hour ago, symmetal said:

Your optimum sky background ADU for 10 * RN^2  is 1290,

How did you get from 10 * RN^2 = 1290?

This would imply that RN is equal to sqrt(129) = ~11.35 or something?

I can't get numbers to work when I plug in standard RN values for ASI1600

[Padraic M]

Hi @BrendanC I know I'm coming late to this discussion and you've made great progress with the issues already, but just to add some thoughts on light leaking through the PDS OTAs. I recently took on a 150PDS and had issues with gradients and LP. I tested the OTA for leaks (similar to what Vlaid suggested earlier), by putting the scope cover in place; looping 1-second or 2-second captures, and shining a bright torch all over the OTA. I found issues with the primary mirror end, obviously, but I also found significant issues around the focuser draw-tube. I know that you've covered the primary end with a shower-cap, but you will also need to cover the focuser draw-tube somehow. I haven't quite found the best way to do this. I use tin-foil when I'm taking flats, but when taking lights there will need to be some flexibility there to allow for autofocus movement of the draw-tube while still blocking the light.

Bear in mind that there should be very little light-leak at night when taking lights, but a nearby street light, or potentially an LED on some part of your rig, could cause problems over a number of minutes exposure.

However, when your LED panel is lit up to take flats, you will definitely get lots of light leaking into the focuser, unless you have fully masked the LED panel around the scope aperture. This means that the lights could be fine but your flats are contaminated.

Finally, any environmental lighting shining obliquely across the OTA opening may illuminate the inside of the tube slightly and cause problems. I now fit a sizeable dew shield the the top of the scope to protect the opening (actually the dew shield for my C8 SCT fits perfectly).

Environmental light leaks are a pain because they move relative to the OTA/sensor as you track and flip; while sky light pollution and vignetting/reflections in the optical path stay in the same orientation as the image. Consider if you were to track a DSO through a full 90 degrees; your environmental light leak would illuminate your subs on two or even three sides through the duration of the session. No light pollution/gradient removal tool will be able to address this. You may also get LP gradients at different angles in your R, G and B channels when shooting mono as the light ingress angle will change relative to the OTA as the night progresses.

hth

[BrendanC]

Wow. This is brilliant advice, and makes it even more urgent that I do the vlaiv test. I can totally see now how the LED light could leak into the focuser tube and cause all sorts of havoc with the flats. And you're right about how a few minutes' exposure could really show up any local light pollution. Thanks!

 

 

Edited April 3, 2022 by BrendanC

[symmetal]

3 hours ago, vlaiv said:

How did you get from 10 * RN^2 = 1290?

Ah! I thought on your initial post that 1290 was wrong and 1156 was to be preferred which was what confused me.

Read Noise at unity gain I estimated to be 1.75 e- from the ASI1600 graphs.

10 * RN^2 = 10 * 1.75 * 1.75  = 30.6 electrons

Convert electrons to ADU by dividing by the gain in e-/ADU which is 1.0 at unity gain

30.6 / 1.0 = 30.6

Offsets. AFAIK, for the 12 and 14 bit Zwo cameras, an offset increase of  1 increases the ADC output by 1. Certainly true for the ASI1600, ASI071, ASI178, and  ASI224 which I have. For my 16 bit ASI6200 an offset increase of 1 increases the ADC output by 10. Most likely true for the ASI2600 as well.

Add the offset to the above figure to get the actual output from the ASI1600's 12bit ADC for a 10 * RN^2 sky background value.

30.6 + 50 = 80.6 ADU (12 bit)

Convert 12bit ADC output value to the actual 16 bit value which the camera outputs by adding 4 zero value LSBits, equivalent to multiplying by 16.

80.6 * 16 = 1289.6 ADU (16 bit)   TA DA! 😊

If I use the 1.7e- read noise figure that you read from the graph then the figure becomes 1262 ADU

For interest, using a 1.7e- value, working backwards, 1156 = 7.9 * RN^2. On the discussion on the CN forum years ago when the 1600 came out, some advocated 5 * RN^2 as an acceptable value, but 10 seemed to gain most traction. Your value is right between the two. 😁

Alan

Edit: Checking your formula vlaiv you don't seem to take the offset used into account so using offset 50,  then 800 ADU needs to be added to your 1152 ADU making 1952 ADU the figure to aim for in your case. 🤔

Edited April 4, 2022 by symmetal

[tomato]

It’s interesting that in some instances APP displayed a warning during the calibration process, using the default settings and processing each channel separately, I received no warnings with the data.

[BrendanC]

'Interesting' is one word for it! The full story (so far) is here: https://www.astropixelprocessor.com/community/main-forum/critical-warning-flat-field-calibration-2/

[tomato]

The numbers are fixed right, once the frame is captured and saved, so it's really weird that one instance of running APP gives you the warnings, and another using the same version, settings etc, does not? 

[vlaiv]

9 hours ago, symmetal said:

80.6 * 16 = 1289.6 ADU (16 bit)   TA DA! 😊

I see now where confusion comes from, and yes, that is why I said the math must be wrong.

Measurements are usually performed on calibrated subs. Using 50 or whatever offset is very unreliable, and you don't have to do it on properly calibrated subs as only thing that is left is light signal.

We should not be doing multiplication / division with 16 either, so that is a bug in drivers / captures software (whichever is responsible) as there is field in fits header that gives you e/ADU value.

As is, it says that e/ADU is 1 (or rather 1.001somethingsomething) but ADU numbers are x16 larger than that. In perfect world - knowing e/ADU from fits header and read noise (or measuring it from bias stats) and having calibrated light is enough.

[ONIKKINEN]

13 hours ago, Padraic M said:

Hi @BrendanC I know I'm coming late to this discussion and you've made great progress with the issues already, but just to add some thoughts on light leaking through the PDS OTAs. I recently took on a 150PDS and had issues with gradients and LP. I tested the OTA for leaks (similar to what Vlaid suggested earlier), by putting the scope cover in place; looping 1-second or 2-second captures, and shining a bright torch all over the OTA. I found issues with the primary mirror end, obviously, but I also found significant issues around the focuser draw-tube. I know that you've covered the primary end with a shower-cap, but you will also need to cover the focuser draw-tube somehow. I haven't quite found the best way to do this. I use tin-foil when I'm taking flats, but when taking lights there will need to be some flexibility there to allow for autofocus movement of the draw-tube while still blocking the light.

Bear in mind that there should be very little light-leak at night when taking lights, but a nearby street light, or potentially an LED on some part of your rig, could cause problems over a number of minutes exposure.

However, when your LED panel is lit up to take flats, you will definitely get lots of light leaking into the focuser, unless you have fully masked the LED panel around the scope aperture. This means that the lights could be fine but your flats are contaminated.

Finally, any environmental lighting shining obliquely across the OTA opening may illuminate the inside of the tube slightly and cause problems. I now fit a sizeable dew shield the the top of the scope to protect the opening (actually the dew shield for my C8 SCT fits perfectly).

Environmental light leaks are a pain because they move relative to the OTA/sensor as you track and flip; while sky light pollution and vignetting/reflections in the optical path stay in the same orientation as the image. Consider if you were to track a DSO through a full 90 degrees; your environmental light leak would illuminate your subs on two or even three sides through the duration of the session. No light pollution/gradient removal tool will be able to address this. You may also get LP gradients at different angles in your R, G and B channels when shooting mono as the light ingress angle will change relative to the OTA as the night progresses.

hth

@BrendanC Most of these issues above are fixed by flocking the inside of the tube, but of course if there is significant local lighting going on you need to block these somehow. The focuser drawtube light leak issue is quite small and i doubt it ruins your lights or flats unless you shine a light directly to the focuser, but this will definitely ruin your darks. The stock paint inside the tube is hardly black and will reflect some light here and there and may ruin flats, especially the area just behind the secondary mirror will reflect unwanted light back into your camera. Uncovered shiny screwheads and such also should be blackened.

You can buy flocking material from FLO to line the inside of the tube with. I think it was something like 5 pounds per roll (you will probably need 2) so not one of the more expensive investments in the hobby. I used to have problems with flats but after doing many modifications, flocking the tube being one of them, now i really have no issues at all with flats.

[BrendanC]

2 hours ago, tomato said:

The numbers are fixed right, once the frame is captured and saved, so it's really weird that one instance of running APP gives you the warnings, and another using the same version, settings etc, does not? 

In a word, yes! It makes no sense whatsoever. Mind you, the message that APP showed was also factually incorrect and very misleading, and caused me no end of grief going through everything several times before posting. On top of that I also found a mysterious bug in APT while shooting. It's been a nightmare getting all this to work.

[BrendanC]

48 minutes ago, ONIKKINEN said:

Most of these issues above are fixed by flocking the inside of the tube

Thanks, and that's a good call too. I've flocked opposite the secondary, painted the vanes and internal screws black, but not flocked or painted the entire inner OTA. I might just do that.

Thing is, throughout the past two years of DSLR photography, I never had these issues. So, given that it seems my capture technique isn't too bad, and I'm getting the hang of processing LRGB and narrowband, I'm now wondering whether it's a simple case of the ASI1600 being so much more sensitive and picking up light pollution where the DSLR didn't, both in the taking of subs and lights from the flat panel. I had a similar experience when I moved from an AZ to an equatorial mount: suddenly the step up in equipment reveals inadequacies in other areas, techniques, knowledge etc.

[symmetal]

7 hours ago, vlaiv said:

I see now where confusion comes from, and yes, that is why I said the math must be wrong.

Measurements are usually performed on calibrated subs. Using 50 or whatever offset is very unreliable, and you don't have to do it on properly calibrated subs as only thing that is left is light signal.

We should not be doing multiplication / division with 16 either, so that is a bug in drivers / captures software (whichever is responsible) as there is field in fits header that gives you e/ADU value.

As is, it says that e/ADU is 1 (or rather 1.001somethingsomething) but ADU numbers are x16 larger than that. In perfect world - knowing e/ADU from fits header and read noise (or measuring it from bias stats) and having calibrated light is enough.

I'm confused as to where you are saying your calculated 1156 ADU is actually measured. It seems to be on a calibrated sub which is not quick to determine, having to take a sub and then calibrate it to get the background ADU value. This is not available in the capture program.

The method I use just takes the background ADU ( median or mean of sub will be similar for small targets ) from the captured sub itself, readily displayed on any capture program. This is its intention, to be a quick check. This will include the sky background signal, sky background noise, offset, dark noise, and read noise.

The 10 * RN^2 ADU value (which includes offset) is when the sky background noise is at least 10 x read noise, which is the same as  sky background is at least 10 x read noise^2.

There will also be dark noise present in the sky background ADU read, so in reality the sky background measured will actually be greater than 10 x read noise^2. I think 🤔

Read noise is the only one not removed by calibration so as long as the sky background noise is significantly greater than the read noise to make read noise essentially insignificant is what's important.

These are really only figures to aim for as a rough guide to avoid unnecessary under or over exposure.

Flak jacket donned for vlaiv's reply. 😬

Alan

 

Edited April 4, 2022 by symmetal

[vlaiv]

3 minutes ago, symmetal said:

The 10 * RN^2 ADU value (which includes offset) is when the sky background noise is at least 10 x read noise, which is the same as  sky background is at least 10 x read noise^2.

No, that is not the case.

Say that read noise is X. You say you want your LP noise to be at least x10 that of read noise, right? So LP noise is thus 10*X.

Relationship between signal and associated Poisson noise is that signal = square of noise. From this we have that LP signal is (10*X)^2 = 100*X^2

Saying that sky background is at least 10 x read noise^2 is equal to saying that LP noise is sqrt(10) times larger than read noise - or about 3.1622... times larger than read noise.

6 minutes ago, symmetal said:

Read noise is the only one not removed by calibration so as long as the sky background noise is significantly greater than the read noise to make read noise essentially insignificant is what's important.

No noise is removed by calibration - it is only signal that is removed by calibration. Noise remains.

8 minutes ago, symmetal said:

I'm confused as to where you are saying your calculated 1156 ADU is actually measured

Calculate 1156ADU needs to be measured on calibrated sub. You are right - this demands a bit more processing and isn't suitable method for quick "in the field" testing, but it is accurate.

If you want to do "in field" testing - you should also account for dark current signal as it can be significant as well (depending on temperature) and offset used won't always be correct (I've seen offset being missed by few electrons regularly).

Problem with "in field" testing is that you can't just simply scale things like we propose (divide calculated and measured ADU to get increase/decrease coefficient for sub duration)  - because dark current also depends on time and different time will skew ADU values in non trivial way.

 

[BrendanC]

I like kittens. 

[symmetal]

1 hour ago, vlaiv said:

Saying that sky background is at least 10 x read noise^2 is equal to saying that LP noise is sqrt(10) times larger than read noise - or about 3.1622... times larger than read noise.

Looks like I misread the meaning of the original formula,  but it does seem to provide a consistant  difference though less in comparison to your method. I did your calculations on calibrated subs from my ASI6200 and the background ADU seems to correspond to around 4 x RN which is not too far different to your 5 x RN. I think if perhaps tweaked to taste, the method I use does provide a useful indication of what the results will be when calibrated. 🙂

1 hour ago, vlaiv said:

No noise is removed by calibration - it is only signal that is removed by calibration. Noise remains

I was casually treating unwanted signal as noise, but you're right. 

Alan