Reduce file size of drizzled image - Pixinsight

[mackiedlm]

I have made a drizzle integration from a stack of ASI 2600mc images. it looks great but the drizzle integration file is 1.2 Gb in size.

Two questions;

1. Is this correct it seems ridiculosly large - have I screwed up somewhere?

2. Assuming it is correct, when and how do you downsample in pixinsight, to make the file a more manageable size but without losing the benefits of the drizzle?

Thanks for any input.

[Adreneline]

2 hours ago, mackiedlm said:

I have made a drizzle integration from a stack of ASI 2600mc images. it looks great but the drizzle integration file is 1.2 Gb in size.

Two questions;

1. Is this correct it seems ridiculosly large - have I screwed up somewhere?

2. Assuming it is correct, when and how do you downsample in pixinsight, to make the file a more manageable size but without losing the benefits of the drizzle?

Thanks for any input.

Sorry, can't answer the first question (well I could but I'd be making it up!).

As for downsampling in PI I use this process:

Set the amount you wish to downsample to achieve a manageable file size.

HTH

Adrian

[mackiedlm]

1 hour ago, Adreneline said:

Sorry, can't answer the first question (well I could but I'd be making it up!).

As for downsampling in PI I use this process:

Set the amount you wish to downsample to achieve a manageable file size.

HTH

Adrian

Thanks for this. I really need it now - I've just done HDRComposition on the image and PI wants to make it 64bit with a file size of 2.5Gb. My computer is about to have a stroke 😂

[vlaiv]

4 hours ago, mackiedlm said:

without losing the benefits of the drizzle

What are those exactly?

[mackiedlm]

14 minutes ago, vlaiv said:

What are those exactly?

Less blocky shaped stars mainly, and possibly some improvement in fine detail. But I am far from smart enough to get into a discussion with @vlaiv about the theoretical and mathematical intricacies of drizzle and or Integer resampling.

I'm doing drizzle integration on the basis of what I've read in the likes of Keller that if your image meets certain criteria in terms of sampling etc, Drizzle can be beneficial in those area. My image meets those requirements so I'm giving it a whirl to see what comes out.

I await re-education.

Edited September 2, 2022 by mackiedlm

[vlaiv]

2 minutes ago, mackiedlm said:

Less blocky shaped stars mainly, and possibly some improvement in fine detail. But I am far from smart enough to get into a discussion with @vlaiv about the mathematical intricacies of drizzle and or Integer resampling.

I'm doing drizzle integration on the basis of what I've read in the likes of Keller that if your image meets certain criteria in terms of sampling etc, Drizzle can be beneficial in those area. My image meets those requirements so I'm giving it a whirl to see what comes out.

I await re-education.

I guess it is fairly easy to do normal integration and see if you loose anything (or gain something).

[mackiedlm]

2 minutes ago, vlaiv said:

I guess it is fairly easy to do normal integration and see if you loose anything (or gain something).

Within Pixinsight (I explain this as you've said regularly you dont use it) to do Drizzle integration you first do a normal integration. So thats done already. And the comparison of the stars in the original and the drizzled version shows a huge improvement in the star shapes. That was the first thing I did and it was on the basis of that comparison that I decided to continue with the drizzled version. Because I could see the benefit.

 

But the file size is huge. Which is why I am asking if there is a way to reduce the file size without losing the actual (perceived?) benefit I have gained from the drizzle. Is there?

[vlaiv]

2 minutes ago, mackiedlm said:

So thats done already. And the comparison of the stars in the original and the drizzled version shows a huge improvement in the star shapes. That was the first thing I did and it was on the basis of that comparison that I decided to continue with the drizzled version. Because I could see the benefit.

Could you please share comparison. I've never seen one do so, and I'd be interested to see the difference.

[Fegato]

I think the drizzle comes in to its own if you are clearly under-sampled. Even at 2.45 arc secs per pixel I didn't generally X2 drizzle, although I might with a severe crop. 

But if you are using OSC, you can do a X1 drizzle - this gives you the benefit of a "Bayer Drizzle" in terms of the colour rendition. No change to file size. I've read a bit about the benefits of this, but refuse to try and explain it, I'd have to leave that to others.

re: 1.2Gb - I think that's probably normal.  I have an ASI2400MC Pro - full frame, but larger pixels, and so a slightly smaller pixel array I think.  I get over 900Mb with a X2 drizzle.  However - I convert to JPG for most presentation after that, and sometimes I even have to reduce the quality of these to get them under 25Mb.

[vlaiv]

17 minutes ago, Fegato said:

But if you are using OSC, you can do a X1 drizzle - this gives you the benefit of a "Bayer Drizzle" in terms of the colour rendition. No change to file size. I've read a bit about the benefits of this, but refuse to try and explain it, I'd have to leave that to others.

Bayer drizzle is regularly used with OSC planetary imaging where sampling is right there at critical sampling - and it works well for that, but yes, for long exposure imaging we are more often over sampled than under sampled (it is actually hard to be under sampled with amateur setups because of F/ratios of optics and pixel sizes that are now common. Only fast optics like F/2 lenses and such stand a chance to be over sampled - but then again, those are not diffraction limited at such high speeds and produce additional blur with optics).

[Fegato]

I've dug out the quote I'd read about Bayer Drizzle (from Juan Conejero on the Pixinsight forum) :

To be more precise, if you want to achieve the best possible result starting with mosaiced raw data (be it DSLR or OSC raw data), and the amount of data available is not marginal, then CFA drizzle is not optional. You must perform it always. The Debayer tool only serves as a previous step for image registration (StarAlignment) and weighing / outlier rejection (ImageIntegration), but de-Bayered frames are just temporary working images. If your data justifies it (both because it is subsampled and because you have enough frames), you can perform drizzle with a scale of 2. Otherwise select a scale of 1 to perform a 'regular' CFA drizzle.

And to be even more precise, in many cases, if not in most of them, drizzle integration is the best option also for non-mosaiced data, that is, for monochrome CCD/CMOS raw data. DrizzleIntegration with a scale of 1 may achieve better results than ImageIntegration because drizzle does not apply pixel interpolation. The final SNR improvement is normally smaller with drizzle, but the total absence of interpolation aliasing artifacts is a wonderful compensation. When there is enough data, I would consider using drizzle x1 on a regular basis. Food for thought

 

 

 

[wimvb]

Juan talks abou drizzle integration without upsampling, ie keeping the original size. This can be done in PI. So, try this and compare with a. the undrizzled image, and b. the 2x drizzled image. 

[vlaiv]

Out of interest - what does it mean to drizzle without upsampling?

I have few issues with quoted text.

36 minutes ago, Fegato said:

DrizzleIntegration with a scale of 1 may achieve better results than ImageIntegration because drizzle does not apply pixel interpolation.

This is very interesting part.

I wonder how on earth would one do sub pixel alignment without interpolation of sorts?

We can choose not to perform interpolation, but instead just perform simple "shift and add". However, such approach will inevitably increase star FWHM significantly (we don't have precision in our mounts to prepare subs in such way that it does not impact FWHM - at least whole pixel dithers aligned with main imaging axis would be needed), and any field rotation due to slight polar misalignment would be visible as no rotation of subs for alignment is performed.

 

[Fegato]

4 minutes ago, vlaiv said:

Out of interest - what does it mean to drizzle without upsampling?

I have few issues with quoted text.

This is very interesting part.

I wonder how on earth would one do sub pixel alignment without interpolation of sorts?

We can choose not to perform interpolation, but instead just perform simple "shift and add". However, such approach will inevitably increase star FWHM significantly (we don't have precision in our mounts to prepare subs in such way that it does not impact FWHM - at least whole pixel dithers aligned with main imaging axis would be needed), and any field rotation due to slight polar misalignment would be visible as no rotation of subs for alignment is performed.

 

yeah sorry, I haven't a clue! (that's why I made sure I quoted it direct from Juan Conejero). 

[wimvb]

Maybe this explains some

https://www.astropixelprocessor.com/community/tutorials-workflows/drizzle-for-mono-cameras/

As well as the linked references in Mabula's post.

[vlaiv]

3 hours ago, wimvb said:

Maybe this explains some

https://www.astropixelprocessor.com/community/tutorials-workflows/drizzle-for-mono-cameras/

As well as the linked references in Mabula's post.

Yes, that is well known resource and origin of the drizzle algorithm (NASA / Hubble team).

Have you seen last post in that discussion - one without answer?

I'll quote:

Quote

A (probably silly) question. I can relatively easily imagine how drizzle works with a scale = 2.0 and droplet = 0.5: each input pixel is kinda split in two and fills two output pixels, which is possible because of dithering. But it's tricky to understand how would droplet = 0.5 and scale = 1 work, because the droplet is smaller than a pixel, so it looks like it will always fall into the same pixel, but with reduced weight (half-value). Or I'm getting it completely wrong?

Indeed - it is very hard to imagine what is drizzle supposed to do if you don't increase resolution - you just "shrink" pixel and place it in its original place - you effectively do nothing as there is no change to pixel.

I want to address one more misconception.

We have read previously in this thread - in one other quote from various posts on other forums - that drizzle does not perform interpolation. But look at following quote from above page:

Quote

Drizzle frequently divides the power from a given input pixel between several
output pixels. As a result, the noise in adjacent pixels will be correlated....

The correlation of adjacent pixels implies that a measurement of the noise in a
drizzled image on the output pixel scale underestimates the noise on larger scales.

This is actually quote from original handbook - but it clearly shows that input pixel values are split over several output pixel values - thus creating pixel to pixel correlation. That in effect is interpolation. If you split droplet to few pixels weighing by surface area - if you don't have rotation that is mathematically equivalent to linear interpolation.

[mackiedlm]

Hmmmm! Ever wish youd not bothered to ask a question?

[sharkmelley]

9 hours ago, vlaiv said:

I want to address one more misconception.

We have read previously in this thread - in one other quote from various posts on other forums - that drizzle does not perform interpolation.

Yes, of course drizzle is performing interpolation to drop the input pixels onto the output grid.  But for Bayer Drizzle the original raw pixels need no demosaicing and hence the usual interpolation during demosaicing is not required.  This means that bayer drizzled one-shot-colour data will give the same image sharpness as drizzled RGB data from a mono camera.

Edited September 3, 2022 by sharkmelley

[vlaiv]

12 hours ago, mackiedlm said:

Hmmmm! Ever wish youd not bothered to ask a question?

Really sorry for making this issue way more technical than it needed to be.

In short, my answer to the question would be:

Don't bother drizzling if you are going to reduce file afterwards. Drizzling will reduce quality of the stack (more noise) in exchange for possibility of improving detail. My view is that detail won't be there to begin with in majority of amateur setups as we are mostly over sampled, so by drizzling - you only loose SNR. In any case, reducing image afterwards is just throwing away that improved detail (if any).

Best to do regular integration which keeps file size smaller in the first place. so there is no need for reduction.

[CraigT82]

Looking at the first document linked to in that APP thread it says this about Drizzle and it’s influence on noise, which seems to contradict that quote from the handbook? I’m probably misunderstanding,  I admit I haven’t read the entirety of either document.

 

 

[vlaiv]

6 minutes ago, CraigT82 said:

Looking at the first document linked to in that APP thread it says this about Drizzle and it’s influence on noise, which seems to contradict that quote from the handbook? I’m probably misunderstanding,  I admit I haven’t read the entirety of either document.

Yes, it does seem to contradict original document - and also common sense

Drizzle takes input pixels and "drizzles" them across larger output image, meaning that not all output pixels will have data from all input pixels for any given sub.

This in turn means that number of averaged samples for stacked output image will be less than number of subs. SNR improvement is equal to square root of averaged samples, so less samples averaged leads to lower SNR improvement over using regular stacking method where input and output samples are matched 1:1 and there is no reduction in number of samples.

[CraigT82]

1 hour ago, vlaiv said:

Yes, it does seem to contradict original document - and also common sense

Well I wouldn’t say it’s common sense but I get your meaning 😂

There is some more in depth explanations further into that paper in section 7 but it’s all Greek to me. Maybe the inventor of the process has embellished it’s capabilities slightly in that paper, wouldn’t be the first time that’s happened! 

Edited September 3, 2022 by CraigT82
Typos

[wimvb]

22 minutes ago, CraigT82 said:

Well I wouldn’t say it’s common sense but I get your meaning 😂

I usually apply the "no free lunch" or "eating and saving the cake" principle. In its crudest form that would be something like this: information that is used to capture detail or enhance detail, cannot also be used to reduce noise.