Can't find why my flats are not even and change as I rotate my flat panel

[aleixandrus]

Hi! I have an issue with my flats and, after many-many tries, I can resolve.

I have a ZWO ASI183MM Pro with a Samyang 135mm @ f2.8. I also have a ZWO EFWmini with 6.5nm SHO Baader filters. My lights (both subs and master) seems fine. I recently acquired a circular flat panel (3D printed) with a slightly bigger diameter than the Samyang 135mm. What I found is, as I rotate and change the position of the flat panel, the light pattern of my flats also change orientation.

 

Please, note that this screenshots are low quality as it is a remote connection (the subs SNR are as as good as expected). However, I think the "light to dark" pattern and how it change as I move the flat panel is clearly visible. As I was concerned about the quality of the panel and how even the illumination is, I also use a 10" tablet but the issue is still there. I tried the following:

• Double check camera parameters (gain 111, offset 8, temperature -5ºC)
• Double check the lens hood for evident tilt or other mechanical issues
• Visualize the subs with different software: NINA - PixInsight - ZWO Fits Viewer
• Change the exposure length: 0.1s - 1s - 4s - 10s - 15s
• Change the brightness of the panel/tablet (10% - 40% - 90%)
• Change the position of the histogram peak (20% - 40% - 80%)
• Change the filter: S2 - Ha - O3 - L - No filter
• Change room illumination: on - off - covering  the telescope with a blanket

I always get the same results: as I change the position of the tablet/panel over the lens hood, the light pattern in the flats also change. But for a given position, the pattern remains stable.

When visualized with no histogram stretching, the flat subs look normal with very small vignette. I was concerned that as I'm virtually stretching too much the histogram of individual subs, making small differences too visible. So I stack 30 flats (same panel/tablet orientation) with WBPP and darkflats to check if the differences or if outlier data are averaged, but no, the master flat looks similar to individual subs (but calibrated, cleaner).

Oh, by the way, as the ASI183 has a small sensor, the Samyang 135 has a flat field (support full frame cameras) and I tend to shoot large nebula, my master lights seem to calibrate "well" to my eyes. But background extraction techniques are somehow tricky to me (I use Pixinsight ABE or DBE and GraXpertt) and I'm not experienced enough to confirm the masters have no hidden issues.

I'm ran out of ideas, I don't know what to try now. I suppose the issue is not what I do with the flats processing but the flats itself: why my flats are not even and change as I rotate my flat panel. Any help of hint is very welcome.

Edited December 20, 2023 by aleixandrus

[Ouroboros]

Have you tried taking flats some other way, @aleixandrus ie without the flat panel?  It might be good to establish whether this problem is panel or telescope related? 

Edited December 20, 2023 by Ouroboros

[aleixandrus]

I tried with the flat panel and an old 10" tablet. I'll try with a new tablet (iPad) as soon as I can borrow it and do some sky flats. I can understand the flat panel is wrong... but that tablet? It has no sense to me.

Edited December 20, 2023 by aleixandrus

[Elp]

What's the issue exactly? You shouldn't be moving the light source when taking flats they all have to be taken under exactly the same condition as each other.

[aleixandrus]

I do not move the panel during the flat sequence. The issue is that every time I put the panel to take flats (ie. different night sessions), the light pattern changes so also the flats. What I understand is the flats cannot be dependent on how I put the flat panel.

[Elp]

This is also what I do. As long as you use the flats respectively to their lights per session to calibrate. Then stack and register each master stack for the final image.

Also ensure your flat panel is as dim as you can get it by putting diffusing material between the panel and the lens (I use perspex of reducing darkness/opacities), my lum flats are still quite quick at around 2-3s each, but narrowband flats are always 10s. I use the auto flat routine with the asiair so it calculates the time automatically.

[Elp]

Is your flat panel an LED drawing/tracing tablet?

[bosun21]

Try placing a clean white T shirt over the objective then the light panel. You can then set your light panel to the best illumination. This helped a great deal for me.

[aleixandrus]

I currently use a conventional 10" tablet with a diffuse layer and a 3D printed flat panel with LED lighting and a diffuse layer. I also test different exposures from 0.1s to 15s and different histogram average values. I'll try again with a different tablet and I'll add more diffuse layers.

But the key is not the quality of the flat itself but the different illumination patterns according to the panel orientation.

[Ouroboros]

I’d stick a t-shirt over the telescope and point it at the daytime sky. That’s got to produce uniform illumination as long as you avoid shadows falling on the diffuser. Try not to have too short exposures ie  add more layers to make them longer than a second or two. See what that gives you, 

[Oddsocks]

1 hour ago, Elp said:

What's the issue exactly? You shouldn't be moving the light source when taking flats they all have to be taken under exactly the same condition as each other.

This is not correct, it would only apply if the light source is perfectly homogenous across the full width of the illuminated surface, which the great majority of LED panels used in amateur class astrophotography are not.

The recommended method for creating a master flat with an uneven illumination source is to move/rotate the panel randomly between each sub exposure, or every x number of subs, so that when combined small differences in single subs due to uneven illumination are averaged out in the master flat.

One issue that crops up with old TFT tablets used as a flats source is that the output light is strongly polarised, which can in itself cause gradients in flats and random movement/rotation of the tablet between flats subs is therefore necessary.

In Aleixandrus case he states that he sees the same gradient in flats created with his new led panel, and an old tablet, but we don't know if he is calibrating the flats before examining them for linearity, which is important since any fixed bias gradient in the sensor will show in the flats when stretched.

The gold standard for resolving flats issues is to compare artificial panel flats to pre-dawn or post-sunset sky flats taken with a stationary mount (tracking switched off) pointing approximately 30 degrees above the horizon of the anti-solar point, and no other diffusers in the path. If the calibrated sky flats also show the same gradient then you can rule out the panel as being entirely to blame.

[Elp]

Granted flats are a complicated issue so further examination is needed. But from my experience I never move the flat panel per set. And every session the flats can differ in pattern, especially at F2 if you compare them to each other, and also depending on what filters are being used. I see black areas at the edge of my flats a lot (on histogram stretch preview) but when calibrated with the images it works fine. I've been using the 183MM (cooled and uncooled) and SY135 (two lenses) for years and haven't had an issue, compared to my 294MC which is another matter entirely.

I'm not sure but the light refresh nature of the panel you are using could possibly be a cause. Reading other forums about the 294 some people were using EL panels as their lighting is more even than LED.

If you're absolutely sure the flat panel is sitting flush and square to the lens hood I'd for precaution check the rest of the imaging train for any light leaking or tilt, but movement of the panel which you've tried kind of rules this out.

What does your calibrated stack look like, if it's okay are you concerning yourself for no reason? I'd understand if you'd want to calibrate multiple sessions with one master flat but again from my experience it's better to check your data after each session so to make sure nothing went wrong.

Edited December 20, 2023 by Elp

[Ouroboros]

19 minutes ago, Oddsocks said:

The gold standard for resolving flats issues is to compare artificial panel flats to pre-dawn or post-sunset sky flats taken with a stationary mount (tracking switched off) pointing approximately 30 degrees above the horizon of the anti-solar point, and no other diffusers in the path. If the calibrated sky flats also show the same gradient then you can rule out the panel as being entirely to blame.

My experience is that you don’t need to be that picky. Any old sky with a diffuser ie several layers of t-shirt or whatever over the input end of the telescope will work absolutely fine. I do point the scope away from the sun and avoid shadow falling on the diffuser.  It’s fine as long as the light falling on the t-shirt is reasonably uniform.  The diffuser is so far from infinity (or the far field) the camera doesn’t ‘see’ it as an imagable object, if that makes sense. I’ve done my flats like that for years until more recently when I started using an iPad plus T-shirt diffuser with a small refractor. Again. No problem.  

[aleixandrus]

12 minutes ago, Oddsocks said:

One issue that crops up with old TFT tablets used as a flats source is that the output light is strongly polarised, which can in itself cause gradients in flats and random movement/rotation of the tablet between flats subs is therefore necessary.

This is a very good point I was aware of. This is other reason why I want to test with other tablet.

13 minutes ago, Oddsocks said:

In Aleixandrus case he states that he sees the same gradient in flats created with his new led panel, and an old tablet, but we don't know if he is calibrating the flats before examining them for linearity, which is important since any fixed bias gradient in the sensor will show in the flats when stretched.

The screenshots are a flat sub at it goes from the camera. Zero processing (just virtual histogram stretching). I think I'll need to shoot some sky flats... never done that before.

Could you please share some tips to check for flat linearity? I stick with the same parameters as my usual light subs (gain 111, offset 8, temperature -5º) while keeping the histogram close to recommended ADU value by ZWO (adjusting brightness for 4-10s exposures depending on the filter).

Just now, Elp said:

And every session the flats can differ in pattern, especially at F2 if you compare them to each other, and also depending on what filters are being used. I see black areas at the edge of my flats a lot (on histogram stretch preview) but when calibrated with the images it works fine.

I really appreciate your thoughts as I already know you have a similar setup 😇 So, can you please confirm the light pattern is different each time you shoot flats? Do you think this is normal with all telescopes or may be this a Samyang135 related "issue"?

[Oddsocks]

25 minutes ago, aleixandrus said:

Could you please share some tips to check for flat linearity? I stick with the same parameters as my usual light subs (gain 111, offset 8, temperature -5º) while keeping the histogram close to recommended ADU value by ZWO (adjusting brightness for 4-10s exposures depending on the filter).

The camera parameters for flats should be just the same as for lights, gain, offset, and temperature, while target ADU for the flats should be the same for sky-flats and panel flats so that you can directly compare the two, although you can't adjust the sky brightness when taking sky-flats, you can only adjust the time (or add neutral density absorbers to the beam path).

If you open a sky flat, un-stretched, but bias calibrated, and use the cursor readout mode of your chosen image processing application to read out the ADU value at selected points across the image, say the four corners (but inside any cut-off caused by undersized filters etc) and one point in the centre then you can calculate the approximate gradient across the image in percentage terms.

Carry out the same procedure for your LED panel and the sky-flat.

If the measured ADU values for the sky flat show an even illumination across the frame but the panel flat shows a distinct gradient in ADU terms then you'll know that the panel is to blame but as mentioned above, if you rotate and move the panel between flat subs then the unevenness in illumination will be averaged out in the stacked master.

Depending on your choice of post-processing software you may already have the tools included to allow you to directly evaluate a flat frame without having to manually measure the ADU at specific points across sample panel flats and sky flats.

Below is an example flat frame (un-stretched, from a 100mm f5/6 refractor equipped with a rotor, image distributor, photometer and spectrometer), and its corresponding flat profile, as measured in PixInsight, showing a collimation issue where the heavy (~5Kg) image distributor and spectrometer/photometer mounted on this system is pulling the rotator out of alignment. The important thing to note is that the flat image shows a variation of ~8.82% in ADU from the central beam out to the edges and each contour line represents 0.5% of the total ADU range in this image, but overall there is minimum non-linearity across the whole frame. The light source for this flat was an electroluminescent cover/calibrator panel, not LED. 

Raw flat, un-stretched:

Flat profile (measured in PixInsight):

[Elp]

3 hours ago, aleixandrus said:

please confirm the light pattern is different each time you shoot flats

See below, as you can see its not the same pattern every time, this has always been the case whenever I take flats no matter the optics/camera/filter im using as I setup and breakdown every time so need to make sure my calibration images are okay before teardown, and pattern changes are more evident when I use my C6 Hyperstar F2 setup though you can only notice it in a very high historgram preview, all these images calibrated fine, even the 294 one which does strange things after dynamic background extraction:

 

You still haven't mentioned if this issue is causing you any problems....

From the above sessions plus others this was the final image if you haven't seen it, no problems whatsoever when I calibrated each session's stack with their respective flats, dark flats and darks:

 

[aleixandrus]

4 hours ago, Elp said:

You still haven't mentioned if this issue is causing you any problems....

I'm going to be honest: I don't know. As far as my little experience says, it seems they are just fine. However, I always feel the background extraction is not completely right but I'm not sure if there are my (poor) skills with DBE or my untrained eye. I also find some squared patters if I over-over-stretch the stars image after using StarXTerminator. Not a problem with normal processing but it may be a symptom of something going wrong. What really makes me uncomfortable is having different flat patters depending on the orientation of the light source, given *I do not disassembly my OTA*, I carry everything from the garage to the garden in a single piece, mount included.

 

6 hours ago, Oddsocks said:

Below is an example flat frame (un-stretched, from a 100mm f5/6 refractor equipped with a rotor, image distributor, photometer and spectrometer), and its corresponding flat profile, as measured in PixInsight, showing a collimation issue where the heavy (~5Kg) image distributor and spectrometer/photometer mounted on this system is pulling the rotator out of alignment. The important thing to note is that the flat image shows a variation of ~8.82% in ADU from the central beam out to the edges and each contour line represents 0.5% of the total ADU range in this image, but overall there is minimum non-linearity across the whole frame. The light source for this flat was an electroluminescent cover/calibrator panel, not LED. 

I won't be able to try sky flats for a few days... you know, bad weather 😢 For now, I run some analysis using Pixinsight. What's your opinion?

• Top>Down: Ha - O3 - S2
• Left>Right: master flat - stretched master flat (STF) - FlatContourPlot

I can provide more samples or the flats/masters if this help.

Flats with the 10" LCD old tablet + difusser:

The values of the contour plot ranges for the LCD Tablet are:

• Ha: 0.423 to 0.442
• O3: 0.442 to 0.467
• S2; 0.349 to 0.372

 

 

Flats with the homemade LED Flat Panel + difusser:

The values of the contour plot ranges for the LED Panel are:

• Ha: 0.335 to 0.383
• O3: 0.348 to 0.385
• S2; 0.324 to 0.370

   

[Elp]

What does the calibrated light stack look like with a histogram stretch preview? Before and after DBE is applied, no other processing.

[aleixandrus]

On 20/12/2023 at 21:54, Elp said:

What does the calibrated light stack look like

Well, maybe this is not the most correct approach, but I did a 'manual' calibration of a master light (Ha) of my current project. All images are linear (just STF applied).

• Left: master light with only dark calibration (no flats)
• Right: with master flat calibration with PixelMath (shown in small at the right)

 

• Left: Master light with dark and flat calibration
• Right: ABE applied with default values (extracted background shown in small at the bottom right) > maybe the ABE is messing things up.

 

• Left: Master light with dark and flat calibration
• Right: ABE applied with 1 degree interpolation function, which I feel more exact given the gradient a see in the image (extracted background shown in small at the bottom right)

 

[Elp]

Bit hard to see without massively stretching the images but from the outset they don't look too bad.

I don't use PI but in Siril before applying background extraction you have to crop out any edges due to stacking misalignments (it's not essential but can affect if you use auto grid sample points), then in DBE place manual sample points in the image in blank areas of the image (no stars should be within the sample point square, neither should nebulosity or target detail), less than 20 points across the image should do. Then use an algorithm usually higher than 1 degree (1 degree works well if the lights clearly show a linear only light gradient, it works better if you apply it per image before stacking rather than on the complete stack).

I used to use auto grid sample points, or I only use it for a quick preview process. But I learned with the 294MC not to rely on this and place points manually as this sensor is extremely awkward to work with due to flats and bias signal calibration issues, doing DBE manually is much more reliable. No such issues with my other cameras but I find overall it's better to place the sample points manually.

[aleixandrus]

49 minutes ago, Elp said:

1 degree works well if the lights clearly show a linear only light gradient [...] DBE manually is much more reliable

Yeah, I agree. This is just a quick&dirty test to show the data. To me, the light sub is reasonably flat even without master flat applied. This is a more stretched version of the previous screenshots

Master vs Master+Flat

 

Master+Flat with 1-degree ABE vs. Master+Flat with 4-degree ABE (default values). Extracted background in the small bottom images.

[ollypenrice]

On 20/12/2023 at 09:44, aleixandrus said:

Hi! I have an issue with my flats and, after many-many tries, I can resolve.

I have a ZWO ASI183MM Pro with a Samyang 135mm @ f2.8. I also have a ZWO EFWmini with 6.5nm SHO Baader filters. My lights (both subs and master) seems fine. I recently acquired a circular flat panel (3D printed) with a slightly bigger diameter than the Samyang 135mm. What I found is, as I rotate and change the position of the flat panel, the light pattern of my flats also change orientation.

 

 

 

Whatever these are, they are not flats - by which I mean that they are not images of the illumination your optics' illuminated circle. To my mind, these are not really optical in origin so are probably electronic. Your camera reacting irrationally to your panel. They bear no resemblance to the illumination of a Samyang 135. It's also possible that you are getting strong and asymmetrical reflections.

As Oddsocks suggested, I would just try sky flats. One of my robotic shed clients also has a rig in Spain and has discovered that his panel flats were systematically eating into his data, leaving him with far less signal than expected. Switching to sky flats has transformed his calibrated signal strength.

Olly

[newbie alert]

Your flat panel is unevenly illuminated which will cause issues with creating a flat field

[ollypenrice]

On 23/12/2023 at 10:59, newbie alert said:

Your flat panel is unevenly illuminated which will cause issues with creating a flat field

You base this on rhe flats as posted? I would not be so sure. I think they are so far off the mark as to suggest a more fundamental problem. I doubt that these flats are recording the illumination at all. I think they might be entirely rogue.

Olly

[newbie alert]

3 hours ago, ollypenrice said:

You base this on rhe flats as posted? I would not be so sure. I think they are so far off the mark as to suggest a more fundamental problem. I doubt that these flats are recording the illumination at all. I think they might be entirely rogue.

Olly

No idea 💡 as just read the text, can't remember seeing the images , didn't realise any was there🙈