Is there a Mathematician in the house?

[steppenwolf]

2 minutes ago, ChrisEll said:

Mmm why is the median being used? I see it in DeepSkyStacker as well. I did a lot of stats at uni and never did we consider the median in analyzing results.

Now that, I can't help with!

[Stub Mandrel]

6 hours ago, ChrisEll said:

Mmm why is the median being used? I see it in DeepSkyStacker as well. I did a lot of stats at uni and never did we consider the median in analyzing results.

Median is useful when you have  a proportion of outlier values that are likely to skew the mean, as with small samples of noisy data.

[ChrisEll]

But star eccentricity is not an example of noisy data. Pixel readings from darks and flats I can understand more but the solution is to increase the sample size.

[Stub Mandrel]

1 hour ago, ChrisEll said:

But star eccentricity is not an example of noisy data. Pixel readings from darks and flats I can understand more but the solution is to increase the sample size.

That's why its used in DSS for stacking, I don't know why it is used in this application.

[billyharris72]

Hi Steve:

I've added the "by hand" calculation of the standard deviation for reference in case it's useful. As noted earlier, the one in PI is the same as STDEVP because its summarizing the spread in the image data, not using to make inference from a sample (the image) to a population (the object). The practical difference is that to calculate variance for a sample you would knock one off the sample size before dividing the sum of squared deviations (as in the sheet attached).

I make mean absolute deviation out same as you do, so don't know where the discrepancy comes from - maybe worth asking on a PI forum or support?

As for why median is useful, it's pretty much always useful if the data are not normally (or at least reasonably symmetrically) distributed, and it can be used to help infer whether they are or not (if all is well they will be very similar). With a really lopsided distribution median is often a better indicator of central tendency. That could arise in the kind of data we have here as we have multiple shots; if something changed (guide scope slip, the wind picked up, neighbour's cat gets amorous with the mount) you could end up looking noisy data. That said, the plot underneath is more useful than the median value.

Billy.

Copy of Eccentricity.xlsx

[billyharris72]

This has made me think of a follow up question and it's bugging me:

If we assume that the FWHM eccentricity in these images is all aligned along the same axis, what can we say about the eccentricity of the stacked image? Certainly using sum or mean stacking without rejection it should tend towards the mean eccentricity of the stack, but can we say more than that (e.g. put an error margin on it) just on the basis of the data we have here? Also, can we say what would happen if we changed the stacking algorithm to (for example) median?

I honestly don't know - anyone out there have an answer?

Billy.

[Stub Mandrel]

At what diameter is the eccentricity calculated?

The FWHM?

[andrew s]

39 minutes ago, billyharris72 said:

Certainly using sum or mean stacking without rejection it should tend towards the mean eccentricity of the stack, but can we say more than that (e.g. put an error margin on it) just on the basis of the data we have here?

Yes you can by applying the Central Limit Theorem and as with noise reduction on adding samples it will reduce the estimate in the spread in the eccentricity in the same way.

Regards Andrew

[Stub Mandrel]

As well as the diameter, what is the pixel scale?

Without knowing these it's impossible to know if the effect will be visible or not.

EDIT: I should clarify, the 'egginess' values suggest stars that are about 1 1/4 times larger in one direction than another.

However, this has to be given for a particular scale. If it's the FWHM of a particular star AND the pixel scale is small enough, then the effect may be noticeable ON THAT STAR.

Essentially it quantifies how much the guiding has drifted, but this will be constant for all stars. On smaller stars it will be more noticeable, on larger stars it will be imperceptible.

Without knowing how much that eccentricity translates to in terms of pixels AND the typical size of star images in the image, it is only a measure of the accuracy of guiding, not the quality of the image.

 

[billyharris72]

Thanks Andrew. When I think about it that way it seems obvious. I was trying to come at it on a pixel by pixel basis and wondering how stacking pixels would affect eccentricity. But we can save a whole lot of bother by treating eccentricity itself as the variable, and that ought to work fine.

Except with median stacking - still not sure how that one pans out.

Billy.

[steppenwolf]

3 hours ago, billyharris72 said:

I've added the "by hand" calculation of the standard deviation for reference in case it's useful. As noted earlier, the one in PI is the same as STDEVP because its summarizing the spread in the image data, not using to make inference from a sample (the image) to a population (the object). The practical difference is that to calculate variance for a sample you would knock one off the sample size before dividing the sum of squared deviations (as in the sheet attached).

@billyharris72Thanks for that Billy, for someone as ignorant about statistics as me, it was interesting to note that STDEVP produces the same result as my standard long-hand RMS calculation that I use in a different environment (guiding accuracy) - I hadn't made the connection before!

I am enjoying the discussion!

 

[Ant]

The formula Average(B2:B10) should be Average(B2:B9) I think? I'm on a training course right now so cannot see if that if including a clank cell makes any difference...

 

Ant

[steppenwolf]

You are quite right, Ant, I shouldn't have included it in my description but as it happens it makes no difference to the calculation and I often include a referenced blank cell between the last number and the 'total' calculation so that I can add additional data knowing that it will automatically be included.