Jump to content

SkySurveyBanner.jpg.21855908fce40597655603b6c9af720d.jpg

Is there a Mathematician in the house?

steppenwolf

By steppenwolf
in The Astro Lounge

Recommended Posts

steppenwolf

steppenwolf

Posted
Posted

To exercise may brain (:eek:), I am trying to analyse a plot of data from an imaging run.

The Data and Plot from PixInsight

Plots_1.png.5ff5940544eefccdae5b0294fe458f8f.png

A Screen Grab of my Excel spreadsheet

Excel_screen.png.5cdff0af436c28fd70ea2c525199821f.png

My Excel Spreadsheet

Eccentricity.xlsx

As you can see, I can calculate and agree with the Subframe Median figure of 0.5728. However:-

1. I cannot reconcile the Subframe MeanDeviation figure! How to I do that, I get 0.007853 (0.0079) not 0.007646?

2. Sigma (∑) on the right hand side is a simplified deviation figure but how has this been derived?

Any assistance would be greatly appreciated, and yes, I know, I AM delving into this too deeply but that's the way I roll!! :help:

Thanks in advance.

 

Link to comment
Share on other sites
  • Replies 37
  • Created
  • Last Reply
steppenwolf

steppenwolf

Posted
  • Author
Posted

Possibly Neil but I think that 0.0076nnnn to 0.0078nnnn is too much for a rounding issue but then as I have proved above, mathematics is not my strongest point. I think in this case it is more my ignorance of how the figures are derived! Thanks for the thought though, I should investigate that aspect further.

Link to comment
Share on other sites
steppenwolf

steppenwolf

Posted
  • Author
Posted
5 minutes ago, PhotoGav said:

If you calculate the mean deviation using the Median column in your spreadsheet, i.e. C and not B, you get a value of 0.0076. So it is telling you the Median Mean Deviation, rather than Mean Deviation.

 

Thanks, Gav - hmm the figure is closer (0.007638) but still out and the data table does say specifically ' Subframe MeanDeviation figure ' but you make an interesting observation that I should follow up.

Link to comment
Share on other sites
PhotoGav

PhotoGav

Posted
Posted
5 minutes ago, steppenwolf said:

Thanks, Gav - hmm the figure is closer (0.007638) but still out and the data table does say specifically ' Subframe MeanDeviation figure ' but you make an interesting observation that I should follow up.

That could be to do with the numbers shown in the Eccentricity column of PixInsight being shown only to four decimal places, but perhaps being used to more decimal places in the calculations behind the scenes?

As for your second question about sigma, I am desperately scraping at the base of my skull for the tarnished remnants of A-level mathematics that have been festering there for some 30 odd years and am failing miserably!!!

Link to comment
Share on other sites
steppenwolf

steppenwolf

Posted
  • Author
Posted
10 minutes ago, PhotoGav said:

That could be to do with the numbers shown in the Eccentricity column of PixInsight being shown only to four decimal places, but perhaps being used to more decimal places in the calculations behind the scenes?

As for your second question about sigma, I am desperately scraping at the base of my skull for the tarnished remnants of A-level mathematics that have been festering there for some 30 odd years and am failing miserably!!!

Yes indeed which takes us back to Neil and Louise's suggestion! Now that I would believe but why do PI describe it as they do? I guess the full title could have been 'Subframe Median MeanDeviation' of course. This would make sense (I think) :icon_scratch:

10 minutes ago, PhotoGav said:

As for your second question about sigma, I am desperately scraping at the base of my skull for the tarnished remnants of A-level mathematics that have been festering there for some 30 odd years and am failing miserably!!!

Now there, I do have the advantage - I stopped maths when I got to Calculus just after 'O' Level when I did 'Mathematics for Biologists' as a side course to my main 'A' Level subjects so there is no 'scraping of skull' available for me!

Adding .00002 to some of the figures gets us closer to the target so I think this makes your explanation of Median and the rounding a likely candidate.

Link to comment
Share on other sites
PhotoGav

PhotoGav

Posted
Posted
5 minutes ago, steppenwolf said:

'Mathematics for Biologists'

Sigma should be right up your mathematical street then. Close to the maths A-Level remnants in my head are biology A-level remnants - indeed, I have three A-levels in Biology... a D first time round and then two B's at retake with two different exam boards!! I seem to remember that biological maths was pretty much all statistical deviation from the norm in fruit fly colonies, etc.!

Link to comment
Share on other sites
steppenwolf

steppenwolf

Posted
  • Author
Posted
1 minute ago, PhotoGav said:

Sigma should be right up your mathematical street then. Close to the maths A-Level remnants in my head are biology A-level remnants - indeed, I have three A-levels in Biology... a D first time round and then two B's at retake with two different exam boards!! I seem to remember that biological maths was pretty much all statistical deviation from the norm in fruit fly colonies, etc.!

I much preferred dissection!

Link to comment
Share on other sites
PhotoGav

PhotoGav

Posted
Posted

OK... for the sigma (and I am slightly clutching at straws here, but it sort of works!) if you calculate the Standard Deviation of the 'new' Median Deviation column you obtain a figure of 0.0059867. Then, for each value in the list, calculate how many sigma it is from the norm by dividing the Median Deviation by the Standard Deviation. This gives roughly the right values for each of the eccentricity entries.

I'll get my coat...

Link to comment
Share on other sites
PhotoGav

PhotoGav

Posted
Posted

I have refined my formula for the Sigma plot... If you use:

(Eccentriciy Value - Median)/Standard Deviation of Median Deviation values

you get the correct sigma values for the eccentricity plot in PixInsight.

So the graph is showing how far off the Median each Eccentricity value is in Standard Deviations.

Lovely!

Well, it is raining outside, what else is there to do??!!

Link to comment
Share on other sites
steppenwolf

steppenwolf

Posted
  • Author
Posted
58 minutes ago, PhotoGav said:

OK... for the sigma (and I am slightly clutching at straws here, but it sort of works!) if you calculate the Standard Deviation of the 'new' Median Deviation column you obtain a figure of 0.0059867. Then, for each value in the list, calculate how many sigma it is from the norm by dividing the Median Deviation by the Standard Deviation. This gives roughly the right values for each of the eccentricity entries.

OK, I am getting a figure of 0.0064, NOT 0.0059867 ? My new columns consists of  ABS(Eccentricityvalue - Median of Eccentricityvalues). Is this the correct column, i.e. the Median Deviation column that produces the 0.007638 figure?

Until I get that right, I cannot move on to the final formula, Doh! To calculate the Standard Deviation of that column (right column or not!) I am using the formula =STDEV(Value 1 : Value 8)

Link to comment
Share on other sites
PhotoGav

PhotoGav

Posted
Posted
10 minutes ago, steppenwolf said:

OK, I am getting a figure of 0.0064, NOT 0.0059867 ? My new columns consists of  ABS(Eccentricityvalue - Median of Eccentricityvalues). Is this the correct column, i.e. the Median Deviation column that produces the 0.007638 figure?

Until I get that right, I cannot move on to the final formula, Doh! To calculate the Standard Deviation of that column (right column or not!) I am using the formula =STDEV(Value 1 : Value 8)

Eccentricity-GJ.xlsx

Steve, here's my spreadsheet. You will see that it is column H that is of interest...

Link to comment
Share on other sites
steppenwolf

steppenwolf

Posted
  • Author
Posted

Thanks, Gav, your Standard Deviation function is very different from mine, you have =_xlfn.STDEV.P(E2:E9) whereas I only have STDEV(E2:E9). What is strange is that I can see and use your spreadsheet but if I copy your function and paste it into my spreadsheet, the software fails to recognise the function!

Link to comment
Share on other sites
steppenwolf

steppenwolf

Posted
  • Author
Posted
43 minutes ago, coatesg said:

With a sample size of 8, I'd not worry about being out by ~2.5%... any statistical fluctuations will potentially vary that figure a fair amount..!

Thank you Graeme - this is, of course, a good point. All I really want to do is match up my own calculations to match those of the the PI graph and I am about there now!

Link to comment
Share on other sites
PhotoGav

PhotoGav

Posted
Posted
3 minutes ago, steppenwolf said:

@PhotoGavOK, using STDEVP rather than STDEV gets me there. Thank you so much for your input! I am happy now that I understand what is going on ......

My pleasure Steve, I'm delighted to be able to give my help to you for once! You are still very much in 'help credit'!

I guess this all very much ties up with outlier rejection and the sigma value that you opt for in the software, i.e. how far off the 'standard' are you going to throw away as too far off the 'standard'. Good interesting stuff, thank you for the challenge!

Link to comment
Share on other sites
steppenwolf

steppenwolf

Posted
  • Author
Posted
17 hours ago, PhotoGav said:

My pleasure Steve, I'm delighted to be able to give my help to you for once! You are still very much in 'help credit'!

This is what I love about SGL - there is always someone out there that knows something you don't but is normally happy to divulge. It makes the forum work!

I am now getting consistent results for all parameters generated by PixInsight's SubframeSelector script using the attached spreadsheet.

I've enjoyed the challenge and thank you all for your help.

Updated Spreadsheet

Eccentricity.xlsx

 

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you agree to our Terms of Use.