PixInsight DBE causes black clipping... why?

[MartinFransson]

I have a problem that keeps me from getting anywhere in PI:

When I use Dynamic Background Extraction, no matter how I do it always clips the data, causing black pixels and areas that are too dark. I understand that it has to remove gradients and LP etc. but it causes effects that make the data look bad after stretching.

Gradient XTerminator in Photoshop for example does a much better job at keeping the data.

Am I doing something wrong? I would like a bit of room to the left in the histogram, even after performing DBE. Now, when I stretch the left slider cannot be moved at all to the right without a lot of clipping. This means there are pixels that are virtually black after performing DBE. I can´t seem to salvage this afterwards.

When I´m finished with an image I like my blackpoint at 20-25 (RGB value), not close to 0 as is the result now.

Any tips and suggestions welcome!

[johnrt]

Hmm, I have not experienced this, do you have an example of an image you're working on? 

[MartinFransson]

Well, I´ll try to show what I mean. First, Pixinsight.

What you see here is my stacked image with first a STF applied, second window is STF but without the channels linked. Third image is a 100 crop from the same image.

[MartinFransson]

Next, we have my DBE setting, after that the DBE:d image with STF applied (linked channels) and third is a 100% crop of that image. What I feel is that it cuts too hard in the shadow areas, making the background noisy in a way that I find hard to correct. The background in the uncorrected image appears a lot smoother, even if that might be a trick of the eye since there is also a lot of LP in there...

[MartinFransson]

I just found out that I´ve been using a tolerance setting of 1 or above, to include all my control points in DBE. Raising that value seem to cut more of the background. Maybe I should do several iterations of DBE with lower values instead?

[harry page]

Hi

I would say DBE is doing a decent job of correcting your image ( small improvements may still be possible with setting tweaks )

there are signs of the dreaded rain effects , which could be helped with better dithering etc

but why do you say pixinsight is clipping your Data , you have only used a auto stf to visualize the data if you are not happy with the auto setting manually adjust them

at this stage no data has been altered or lost , only when you use the histogram transformation tool are permanent changes made and all at the users input

 

Harry 

[MartinFransson]

I´m still a beginner with PI but what I mean is that the DBE leaves no room to the left of the histogram. See attached image. The background is just too black, without me even getting a chance to adjust the left slider. (the attached image shows just the Lum part)

The DBE puts my histogram too far to the left.

 

[MartinFransson]

If I bring the original image into Photoshop instead, I can stretch it and remove gradients and still have my histogram a bit to the right, giving me the chance to set my background black level where I want it.

 

[harry page]

Hi

Pixinsight will not clip any black point , what you are doing is stretching the image  , dbe has given you a flat image which

lets you stretch the image a lot , that noise you see is there all the time it was just hidden by all that light pollution

I also do not see that the background is too black , and I would think most would say its stretch slightly too far !!!

 

Harry

[Oddsocks]

I see there is what looks like a stacking artefact or shadow along the upper edge of the image and there are several DBE sample points in that dark shadow (unless this is just a jpg compression artefact in the screen shot). I don't know how relevant this is to how DBE works in the latest PI versions but reading back in the PI forums from the early days of DBE it was suggested that stacking artefacts, or edge shadows, should be cropped out before running the model, or at least, no DBE sample points should be placed in the artefact/shadow areas otherwise an incorrect model is created.

Probably not relevant at all but worth a mention?

 

[MartinFransson]

12 minutes ago, harry page said:

Hi

Pixinsight will not clip any black point , what you are doing is stretching the image  , dbe has given you a flat image which

lets you stretch the image a lot , that noise you see is there all the time it was just hidden by all that light pollution

I also do not see that the background is too black , and I would think most would say its stretch slightly too far !!!

 

Harry

I understand that the noise was there all the time (just hidden in LP) and the image is stretched too far to show the effect. But the problem I have, whether I stretch it hard or not, is that the histogram is situated all the way to the left after DBE. I do not want the background that black and I often see other peoples images having that typical PI-look with the very black background.

[MartinFransson]

18 minutes ago, Oddsocks said:

I see there is what looks like a stacking artefact or shadow along the upper edge of the image and there are several DBE sample points in that dark shadow (unless this is just a jpg compression artefact in the screen shot). I don't know how relevant this is to how DBE works in the latest PI versions but reading back in the PI forums from the early days of DBE it was suggested that stacking artefacts, or edge shadows, should be cropped out before running the model, or at least, no DBE sample points should be placed in the artefact/shadow areas otherwise an incorrect model is created.

Probably not relevant at all but worth a mention?

 

I think it is a compression artefact. I usually perform dynamic crop before DBE.

[harry page]

Hi

Ok lets look at the histogram it is not clipped at all not 1 pixel , also look at your image the background is not black ,

Lets see what other people say

Harry

[MartinFransson]

14 minutes ago, harry page said:

Hi

Ok lets look at the histogram it is not clipped at all not 1 pixel , also look at your image the background is not black ,

Lets see what other people say

Harry

OK, maybe I used the wrong term in my topic. The blacks are not clipped. But You can see that the histogram end right at the very left side. There is no room to move the slider the slightest to the right because then there will be clipping. This means that there are a lot of pixel that are almost entirely black. I would like a situation more as in my Photoshop histogram, where there is a little room to the left. Otherwise the darkest parts of the image are too dark for my taste.

I´m sure DBE is doing what it´s supposed to, I´m just wondering if there is also a way of accomplishing what I´m after.

[johnrt]

Hmm, Ok, Harry is right your histogram is not clipped at all. I often find that PI reports a low black point on the histogram window due to noise in the background. A quick despeckle in photoshop and the black point jumps to the right when loaded back up in to PI.

[MartinFransson]

11 minutes ago, johnrt said:

Hmm, Ok, Harry is right your histogram is not clipped at all. I often find that PI reports a low black point on the histogram window due to noise in the background. A quick despeckle in photoshop and the black point jumps to the right when loaded back up in to PI.

Despeckle sounds interesting, how is that done?

[johnrt]

It's really simple, in CS5 it's just under filter / noise / despeckle, there are no options or sliders you just apply it. 

[wimvb]

Just a few tips from a PI newbie:

The PI histogram shows you do have pixels all the way down to the blackpoint, but according to the statistics, nothing is clipped yet.

Below some of the brightest stars in the 1:1 view, there is a faint "anti-blooming", which may partly cause the above mentioned dark pixels.

Your DBE image shows many sample points, while you have an image with lots of weak nebulosity. Consider placing fewer samples inbetween this nebulosity.

Don't discard the correction model; examine it before accepting the DBE results.

Before stretching the image, do a noise reduction. This makes the histogram narrower, and should clean up any stray black pixels.

Consider doing a masked stretch first, followed by a "normal" histogram or curve stretch. As long as you don't move the black point slider, you won't be clipping any pixels.

 

just my € 0,02

 

Good luck

[MartinFransson]

Thanks a lot for all your tips and info! Turned out quite OK for a DSLR image in the end, right?

 

[wimvb]

Great image. With "lagom" dark background

[Barry-Wilson]

Yes, this is an excellent image from a dslr.

[ollypenrice]

On 7/2/2016 at 17:49, MartinFransson said:

Thanks a lot for all your tips and info! Turned out quite OK for a DSLR image in the end, right?

 

That's about as 'unclipped' as an image gets. Excellent. A phrase to remember is that 'It ain't the histogram that you hang on the wall.'

Top class observations from Harry. I wonder if your subs might benefit from being a bit longer?

Olly

[MartinFransson]

On 2016-02-15 at 21:42, ollypenrice said:

That's about as 'unclipped' as an image gets. Excellent. A phrase to remember is that 'It ain't the histogram that you hang on the wall.'

Top class observations from Harry. I wonder if your subs might benefit from being a bit longer?

Olly

Thanks Olly!

This is what an unstretched file looks like. Straight RAW-conversion to jpeg via Canon DPP. 8 minute sub, could maybe be 10 minutes without blowing more stars? ISO 800, f/4. As you can see the sky background is getting rather bright at these sub lengths... is there a break even point where sky background and true signal make it useless going further?

[ollypenrice]

Experiment is everything. Camera, sky, optics all have their own signature. I have deep well depth CCDs and a dark sky. I know what works for me (long subs) but that's my experience.

Olly

[jimjam11]

On 17/02/2016 at 17:11, MartinFransson said:

Thanks Olly!

This is what an unstretched file looks like. Straight RAW-conversion to jpeg via Canon DPP. 8 minute sub, could maybe be 10 minutes without blowing more stars? ISO 800, f/4. As you can see the sky background is getting rather bright at these sub lengths... is there a break even point where sky background and true signal make it useless going further?

8 mins! I am lucky to get 3 at ISO 800/f4 mins even with an lp filter.