Jump to content

Narrowband

Walking Noise is coming from my flats!


Recommended Posts

10 hours ago, wimvb said:

One instance where calibration frames increase noise (although not here), is dark subtraction. This subtracts the effect of dark current and amp glow, but leaves the associated noise unaltered. The result is a noisier image (decreased S in SNR).

I agree that calibrating an image can reveal faint noise, but it equally reveals faint signals.

In my examples above the horsehead, which is rather faint, is totally obscured if the corrected flat is applied.

 

10 hours ago, wimvb said:

It would be strange indeed if flats add walking noise, because that is the result of improper calibration AND (proper) star alignment. Calibration frames are never aligned, just stacked, so any hot pixels or fixed pattern noise is also stacked on top if each other.

Thought experiment, imagine a perfect master flat aside from one dark patch:

Subtract this from a series of subs that move slightly between exposures.

The corrected subs will all have a bright spot at the same place in the frame.

Now align and stack the subs, and the 'bright spot' becomes a streak.

This is what I think might be happening.

 

 

I'm going to re-make my flats for a third time, fortunately as the setup is just a lens still on the camera and I don't have any noticeable dust bunnies this should be OK

 

Link to comment
Share on other sites

  • Replies 51
  • Created
  • Last Reply
45 minutes ago, Stub Mandrel said:

This is what I think might be happening

That's correct of course. And to avoid it, it's necessary to optimise the flat calibration routine, and examine the master. Calibration and stacking may be tedious, but very critical.

Link to comment
Share on other sites

I've had a go using a third set of flats. The G & B histograms were a bit far to the left on the first and second sets (modded camera) so I allowed the red to be at about 65% this time, this has put G at about 45% and B at about 50%.

The results look better. Redid the flat darks as well to match.

It does look a bit better, will do a full process later.

 

Link to comment
Share on other sites

  • 2 weeks later...

Hmm I have a bit of M42 data from last week which shows the exact same kind of noise. I wrote it off as too few frames (only 7x4min) because my laptop ran out of juice.

I might have a go again with new calibration frames.

Eager to hear if you find a definite cause for this.

Link to comment
Share on other sites

Looks like my issue came from the master dark I made. I tried to make it scalable, by substracting the bias signal, must have done something wrong.

When I just used all the individual dark frames and let BPP make the master dark, it all worked.

Link to comment
Share on other sites

1 minute ago, carastro said:

I am still sceptical that the defects you are seeing Neil are caused by flats.  I think it is a stacking artifact and more to do with darks.  

Carole 

They are there whether I use darks or not. Something is not quite right and I can't pin down what it is.

Link to comment
Share on other sites

Does this happen on all your images.  I recall having something similar in the early days of imaging and being told it is a stacking artifact, but never got to the bottom of it, and only had it happen a couple of times.

Maybe if you could post up a link to your subs we could have a go at stacking and see whether it happens to us too, then we might be able to narrow it down perhaps to the stacking software rather than your flats. 

Carole 

Link to comment
Share on other sites

On 12/13/2017 at 09:51, Stub Mandrel said:

Thought experiment, imagine a perfect master flat aside from one dark patch:

Subtract this from a series of subs that move slightly between exposures.

The corrected subs will all have a bright spot at the same place in the frame.

Now align and stack the subs, and the 'bright spot' becomes a streak.

This is what I think might be happening.

 

 

I'm going to re-make my flats for a third time, fortunately as the setup is just a lens still on the camera and I don't have any noticeable dust bunnies this should be OK

 

But it should not be subtracted, flat calibration is a division process and should even out light levels.

I was always under the impression that walking noise is more likely the background colour noise mainly associated with a dslr.
Dithering is the best way to get rid of colour noise.

Link to comment
Share on other sites

On 22/12/2017 at 12:15, carastro said:

I am still sceptical that the defects you are seeing Neil are caused by flats.  I think it is a stacking artifact and more to do with darks.  

Carole 

I will be the first to say I know little or nothing about this type of photography but I am learning (slowly). I am not guiding at the moment for a hundred different reasons so I am still doing 1 min and under and getting some pleasing, well for me , results.

I had an image of about 80 lights from M33, all 30sec and it had this brushed background really bad, which up set me a bit. This morning I re-stacked all combinations even down to only lights, which apart from gradient was fine clearly showing it was not the lights. Of the other combinations I only saw the same decent result when Darks were left out. It didn't seem to matter from the point of view of the "Brushing" whether it had bias. flats or dark flats, though with light flats (it get confusing for a beginner) there was no gradient.

So the up shot is I tend to agree with yourself Carole. However using the same darks on 5 other images taken at the same time and temp do not show anything but as I am playing to a degree no other image shot that night was more than 20x30 lights 

Alan

Link to comment
Share on other sites

I did a comparison on some recent data using darks and not using them.

There was more noise with the darks.

The sub length is 5 minutes, cooled at an ambient temperature of 0 to -4 and the darks match but were done in  August with much higher ambient temperature (the cooling was still sub-zero).

Looking the the darks, the amount of noise is TINY and I think my problem is that the overall noise floor is getting so low with extreme cooling that the remaining noisy pixels become more obvious, whether they come from subs, flats or darks.

Link to comment
Share on other sites

This is one of my early images (with a DSLR) where I got background streaks, and never got to the bottom of it.  This was back in 2011.  I think I was advised to start dithering at this time.  I think I deleted the other image, but I never had this problem again.

Bubble%2020th%20and%2021st%20August%2020

 

Link to comment
Share on other sites

23 minutes ago, carastro said:

This is one of my early images (with a DSLR) where I got background streaks, and never got to the bottom of it.  This was back in 2011.  I think I was advised to start dithering at this time.  I think I deleted the other image, but I naver had this problem again.

Bubble%2020th%20and%2021st%20August%2020

 

My experience is the same as yours Carole. Mine all used to look like that, started dithering and never happened again 

Link to comment
Share on other sites

I have combined my November and December data.

This is a stack with no flats and no darks. Two doses of Gradient Exterminator needed which damaged the dynamic range, just the lights and bias frames. Not a single streak in evidence, but you can see I have lost fainter parts of the arms.

5a3ece4d01a2d_noflatnodark.thumb.png.3b64a76f4a027c649ed7996fad9df598.png

This is a stack with flats but no darks, it has had aggressive noise reduction in Astra and shows the streaking (zoom in to the galaxy). The streaks can't be caused by the darks as they weren't used.

5a3ecd816a76b_nodark.thumb.png.d6b6ce0314366e7ddb1db6d91e8afd2b.png

I have to conclude my flats are the main source of the streaks.

This doesn't mean dithering wouldn't get rid of the streaks, but either many more flats or cooled flats might be an easier route to eliminating them.

Link to comment
Share on other sites

As an experiment, I hit the flats, no darks version with 'dust and scratches' and used 'screen' to blend it with the luminosity from the no darks no flats version to bring back the fainter areas. Not perfect, but better. I now need to leave this and return later (when I contain less Hobgoblin).

5a3ed127cd299_Vesrionsblended.thumb.png.e3cfea21a06bc2f19013b25251215fa8.png

 

Link to comment
Share on other sites

5 hours ago, Stub Mandrel said:

I have combined my November and December data.

This is a stack with no flats and no darks. Two doses of Gradient Exterminator needed which damaged the dynamic range, just the lights and bias frames. Not a single streak in evidence, but you can see I have lost fainter parts of the arms.

 

This is a stack with flats but no darks, it has had aggressive noise reduction in Astra and shows the streaking (zoom in to the galaxy). The streaks can't be caused by the darks as they weren't used.

 

I have to conclude my flats are the main source of the streaks.

This doesn't mean dithering wouldn't get rid of the streaks, but either many more flats or cooled flats might be an easier route to eliminating them.

Not quite correct as hot pixels are streaked in both images.
The hot pixel streaks are in the same direction as the walking streaks.

IMO the walking noise is colour noise, you can see the red and blue in Carole's image.

You don't need cooled flats, use a low ISO and a bright enough light source to keep the exposure under a second.

Link to comment
Share on other sites

7 hours ago, carastro said:

Having said that, I have not dithered since I got my Mono CCD camera - which was in 2012, and don't get these sort of problems.  I really would like to understand exactly what happened.

Carole 

Your picture is the exact same effect as I was refering to. One thing is for sure a combination of thing causes it as I do have images taken at the same time with the same scop and camera that seem as clean as a whistle. I use a DSLR Canon 40D. I have restacked affected captures and they don't seem to change apart from when other groups (darks in this case) have been omitted, I wonder if it is just a combination of things that brings it out of the lights data that is there but just difficult to see without something else. What ever it is it is something we are causing as using the 18 inch I don't see any of this in the general area:icon_biggrin:. Merry Christmas one and all.

Link to comment
Share on other sites

4 hours ago, wxsatuser said:

Not quite correct as hot pixels are streaked in both images.
The hot pixel streaks are in the same direction as the walking streaks.

IMO the walking noise is colour noise, you can see the red and blue in Carole's image.

You don't need cooled flats, use a low ISO and a bright enough light source to keep the exposure under a second.

One thing I don't quite understand is why the images 'walk' when I am guiding - I have to conclude it is a combination of field rotation and differential flexure - by definition if guiding is perfect the image can't 'walk'.

My concern (as dithering is impractical for me at the moment although I am investigating solutions) is to find the source of the noise.

As it isn't from my data, it must be coming from control frames.

With 'supercooled' lights I don't need darks but I still need flats and that seems to be the source of the noise, which is then 'walked' across the image by the drift in the lights.

 

I tried blurring a master flat, but  DSS wouldn't accept it.

 

I have always taken my flats at the same ISO as the lights, using diffuse daylight, so short exposures (~1/100 s).

I'm going to try cooled flats at ISO100.

Link to comment
Share on other sites

16 pixels in X which is quite a lot, 2 in Y. Field rotation is <0.1 degrees.

Only looking at the longer of the two sets of data.

My guidescope is light and mounted rigidly, could be the mirror moving or the focuser of the guidescope.

Might swap to my other guidescope which I 3D printed a new focuser body for and is quite tight (they are both cut down cheap 'toy' scopes upgraded with fully coated air spaced achro lenses from Astromedia!)

Link to comment
Share on other sites

Well... the focuser of my guidescope has a lot of play!

I've swapped the back end for the 3D printed version which is smooth but totally shake free.

I'll probably print a new housing for the old focuser tube as well so I have two guiders again.

This won't get rid of noise, but it might stop the walking.

Link to comment
Share on other sites

  • 1 month later...

Over the last 24 hours I have, on and off, been running all sorts of different DSS permutations on my NGC2903 data.

It seems using cooled flats and cooled dark flats both help reduce the noise.

A further improvement is achieved by blurring the master flat.

I finally eliminated the streaky pattern - by not using bias frames AND not setting the dark point to zero.

The final step will be to produce cooled bias frames and see how they work - I never suspected that minimum length exposures could suffer from significant thermal noise, but it seems they might.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you agree to our Terms of Use.