The magic and mystery of mathematics

[Moonshed]

Imagine you are at a typical country fete and there is a stall where the challenge is to guess how many smarties there are in a large glass jar. You buy a ticket and enter your guess, the person who guesses the closest to the actual number wins the prize.

After all the tickets are in it is possible to calculate the number that has the highest probability of winning. Simply take the mathematical average of all the guesses entered and that will most probably, almost certainly, be the closest number to the actual number. But why? This does rest on the condition that there is a large enough number of guesses made.

Okay, some people will be wildly wrong and guess a number way too high, others way too low and others a little more accurate and yet others even more accurate until a wide spread has been established. It is because of this wide spread that the average will move ever closer to the true number as more and more guesses are made.
However, the question then becomes why is it that a random collection of people all making guesses will result in the average being so close to the true number? I can think of no reason why that should happen. Why does it not by pure chance happen that many more people guess a number that is way too high that results in a “wrong” average number? Probability? Statistics? Standard Deviation? A love of smarties?

I recall watching this on a tv documentary about maths many years ago and it was said that the theory of the mathematical average being very close to the actual number, if not spot on occasionally, has been checked and verified enough times to give it a high degree of confidence. Unfortunately I do not remember if an explanation why this should be so was given.

If any mathematicians out there can explain it to me before I loose a few million more brain cells I would be very grateful. I would also appreciate it if you could find a way to explain it in a way that doesn’t make me look stupid, I understand that is something of a challenge in itself. 🤪

[vlaiv]

7 minutes ago, Moonshed said:

and that will most probably, almost certainly, be the closest number to the actual number.

And that is how you seriously approach mathematical problem

On a serious note - I think that above approach relies on ability of average person to guess "on average" the right number of things in the jar.

Its a bit like astronomy imaging and stacking - if you have enough samples with random error - average of those samples will give you best SNR value - or best value with lowest error - but it depends on two criteria.

1) - error being random

2) - guess on average being right

condition 1 is generally satisfied when people start to guess and there is a lot of them - but condition two might not be - and then you have a bias.

If jar is such that it always leads to over estimating how many items are in it - your average will over estimate exact value and vice verse - if some condition causes under estimation - so will average. Even "silly" things like - asking everyone while they are hungry can lead to over estimation (if cookies are in the jar ).

[iantaylor2uk]

There is a great book on this topic, "The wisdom of crowds" by James Surowiecki.

If you assume there are enough people, and assume their guesses are random, and plot the results on a graph, I'm pretty sure you would get a gaussian distribution due to the central limit theorem in maths.

It is a good question though as to how close you would expect the guess is to the real value. I would think you would get closer as the number of people guessing goes up. 

[Moonshed]

13 minutes ago, vlaiv said:

On a serious note - I think that above approach relies on ability of average person to guess "on average" the right number of things in the jar.

So it boils down to this. If we have a large enough sample of average people they will on average pick a number that is close to the average.

But are you an average enough sort of person who can on average give an answer that on average will be above average? 🤣

[Moonshed]

5 minutes ago, iantaylor2uk said:

If you assume there are enough people, and assume their guesses are random, and plot the results on a graph, I'm pretty sure you would get a gaussian distribution due to the central limit theorem in maths.

I’m also pretty sure you are right about a Gaussian distribution/ probability bell curve, as this will show the same result as the mathematical average but with all the information displayed rather than just the answer.

[vlaiv]

4 minutes ago, iantaylor2uk said:

It is a good question though as to how close you would expect the guess is to the real value. I would think you would get closer as the number of people guessing goes up. 

Only if there is no bias.

If there is bias of some sorts - with increased number of people guessing - you'll be closer to real value + bias rather than real value.

Imagine following scenario - jar is twice as large - but you can't see it as it is not sitting flush on the desk - it is "embedded" in the desk, however, labels on the jar are positioned so that they fool people - so it looks like complete jar is sitting on the desk.

People will make their guesses like they see regular jar, and compared to regular jar - average of their guesses will tend to the same value - but in reality, first jar holds twice as much cookies as the second jar (regular one).

We introduced bias in the mix purposely. People won't suddenly start to give different random answers to account for that what can't be seen, right?

[Moonshed]

4 minutes ago, vlaiv said:

We introduced bias in the mix purposely. People won't suddenly start to give different random answers to account for that what can't be seen, right?

People can only make their guesses on what they can see so your crafty bias trick will work.

Another bias that would affect the outcome, this time a natural one. It could be that the stall is set up in such a way that  people buying a ticket are inadvertently able to see the guesses made by the six previous entrants. This would affect the number they guess thereby skewing the result. There are many ways bias can be introduced either intentionally or unintentionally that will skew the average number guessed. 
When I was studying statistics at college some years ago I was surprised how difficult it can be for example to get accurate opinion polls on any given subject. Picking a representative sample, a large enough sample and applying correct weightings and so on. And I though guessing how many smarties was difficult 😂

[Elp]

Are you trying to win the lottery or something?

[Moonshed]

30 minutes ago, Elp said:

Are you trying to win the lottery or something?

Nah! What would I want with a few million quid? *starts to write comprehensive list.* 😆

[Macavity]

I know just who to ask, if the Jar is Eight (or 24) Dimensional? 😛
https://en.wikipedia.org/wiki/Maryna_Viazovska
Ukrainian mathematician known for her work in sphere packing.
Viazovska's proof for 8 and 24 dimensions is "stunningly simple"!

Edited March 8, 2023 by Macavity

[Moonshed]

18 minutes ago, Macavity said:

I know just who to ask, if the Jar is Eight (or 24) Dimensional? 😛
https://en.wikipedia.org/wiki/Maryna_Viazovska
Ukrainian mathematician known for her work in sphere packing.
Viazovska's proof for 8 and 24 dimensions is "stunningly simple"!

I checked out the link, my goodness what a brilliant mind that women has!

[speckofdust]

I think prior knowledge, however inaccurate it might be, plays a role. The participants would have to have seen the size of the jar and the size of the smarties to come up with a value that is actually close to the truth. If asked to guess the wealth of a random person, most people will say "tens of thousands", but the moment you say that the random person is a banker, the answer will change to millions.  The rest is just Gaussian distribution. 

[Moonshed]

45 minutes ago, speckofdust said:

I think prior knowledge, however inaccurate it might be, plays a role. The participants would have to have seen the size of the jar and the size of the smarties to come up with a value that is actually close to the truth.

Yes, of course, when I set up the example of a jar of smarties and participants being asked to guess how many it contained they could hardly make a guess without seeing it. Have I missed something, a point you are making I have missed?

[Kev M]

On 07/03/2023 at 23:17, vlaiv said:

Only if there is no bias.

If there is bias of some sorts - with increased number of people guessing - you'll be closer to real value + bias rather than real value.

 

This is correct.....there will be a bias due to the lack of the average persons knowledge/experience in estimating the value.

 

If people were told you could fit a certain number of smarties in a 1 Litre jar and then you showed them a jar full of smarties the "wisdom of the crowd" would probably work.

People would actually be estimating the size of the jar and hence working out the number of smarties based on the knowledge given above.

It is likely the average estimation of the size of the jar may be quite accurate.

However it is likely that the average person does not have this knowledge and we would most likely over or underestimate due to common misconception of the value.

 

 

 

 

 

[speckofdust]

On 18/03/2023 at 09:15, Moonshed said:

Yes, of course, when I set up the example of a jar of smarties and participants being asked to guess how many it contained they could hardly make a guess without seeing it. Have I missed something, a point you are making I have missed?

Ah....looks like there was a misunderstanding, but anyway, considering the premise that people have seen the jar and the size of the candies before making the prediction,  then intuition (!!!) tells me that this phenomenon is no longer an incredible one . I guess it is intuition derived from life experience after all, like estimating someone's height, speed of a car, muscle memory when lifting an object  etc that leads to a ball park figure. The more people you throw in to make the guess, the extremely wrong responses are drowned out by the closer to normal ones, ie  resulting in the normal distribution.  Obviously this response does not contain any sound scientific evidence and invokes the nebulous concept of intuition, which nevertheless seems to be a good approximation.........

Edited March 19, 2023 by speckofdust

[Kev M]

On 19/03/2023 at 09:29, speckofdust said:

Ah....looks like there was a misunderstanding, but anyway, considering the premise that people have seen the jar and the size of the candies before making the prediction,  then intuition (!!!) tells me that this phenomenon is no longer an incredible one . I guess it is intuition derived from life experience after all, like estimating someone's height, speed of a car, muscle memory when lifting an object  etc that leads to a ball park figure. The more people you throw in to make the guess, the extremely wrong responses are drowned out by the closer to normal ones, ie  resulting in the normal distribution.  Obviously this response does not contain any sound scientific evidence and invokes the nebulous concept of intuition, which nevertheless seems to be a good approximation.........

People still seem to missing the point regarding bias...

Someone's height....speed of a car and weight of objects are all things we are familiar with and have a great deal of  "life experience" of.

Counting sweets in a jar is not something we do frequently so we have little if any experience of this.

 

 

For example ( without googling it ) how many smarties could you fit in a 1 litre milk bottle ?

 

 

 

 

 

 

 

[Carbon Brush]

This reminds me of something we used to do at work. Back in the days when electronic components were large enough to see!
A supplier always had a stand at shows with a big jar of components. Guess how many capacitors to win whatever it was.
We did not have any clever component counting aids so grew used to estimating handfuls, etc.
It is also possible to estimate the contants of a sweet jar by looking at it for a couple of minutes and getting a feel for width/height.
Often 3 or 4 of us from work would visit the show. We never won.
When the rep visited us in the weeks following, he would bring the scores and always commented on how near we were to the actual number.
But we never let on our technique😄
 

[Moonshed]

1 hour ago, Kev M said:

For example ( without googling it ) how many smarties could you fit in a 1 litre milk bottle ?

1,659…1,660…1,661…give me a minute, I’ll get back to you…1,662….

[Ouroboros]

Just a note in passing of possible historical interest (I don’t think anyone above mentioned this) the Wisdom of Crowds effect was first observed by Sir Francis Galton who, in addition, made a number of fundamental contributions to the subject of statistics.   His reputation has fallen out of favour somewhat because of his beliefs in social Darwinism, eugenics and scientific racism.  Oh well …. a man of his time. 

[saac]

Number of smarties in 1 litre - it's a kind of Feynman problem mentioned in a different thread - focus on order of magnitude, ignore true geometry and the likes of packing efficiency etc.

Make an estimate of a smartie volume , say 12 mm in diameter x 4 mm thick (forget it's an oblate sphere , simplify it to a cylinder) 

so Vol = Area x depth,   3 x (6 x10^-3)^2)  x 4 x 10^-3  =  3 x 36 x10^-6  x 4 x 10 ^-3

= 3 x 144 x 10 ^ -9   , approximates to  432 x 10 ^-9 

let's make it a bit easier and call that 400 x 10 ^-9  m^3

1 litre = 1 thousandth of a m^3

so Number smarties in I litre  =    1 x 10 ^ -3   /  400 x 10 ^ -9

 = 1/400  x 10 ^6

= 2500 smarties !

if you want to bet go for something either side by say 5 % 

 

* of course the smart ass answer is, "depends on how big the smarties are"  

Jim 

 

   

[Kev M]

13 hours ago, saac said:

Number of smarties in 1 litre - it's a kind of Feynman problem mentioned in a different thread - focus on order of magnitude, ignore true geometry and the likes of packing efficiency etc.

Make an estimate of a smartie volume , say 12 mm in diameter x 4 mm thick (forget it's an oblate sphere , simplify it to a cylinder) 

so Vol = Area x depth,   3 x (6 x10^-3)^2)  x 4 x 10^-3  =  3 x 36 x10^-6  x 4 x 10 ^-3

= 3 x 144 x 10 ^ -9   , approximates to  432 x 10 ^-9 

let's make it a bit easier and call that 400 x 10 ^-9  m^3

1 litre = 1 thousandth of a m^3

so Number smarties in I litre  =    1 x 10 ^ -3   /  400 x 10 ^ -9

 = 1/400  x 10 ^6

= 2500 smarties !

if you want to bet go for something either side by say 5 % 

 

* of course the smart ass answer is, "depends on how big the smarties are"  

Jim 

 

   

and we think the average person in a crowd uses this method........ ? 

 

[speckofdust]

I have just googled the picture of smarties and a picture of 1 litre milk bottle ( only used to either 1 pint or 6 pint bottle, thats why) and my guess is about  1200.

[speckofdust]

let's wait to hear from Moonshed.....😀

Quote

1,659…1,660…1,661…give me a minute, I’ll get back to you…1,662….

 

[JOC]

On 07/03/2023 at 22:36, Moonshed said:

Simply take the mathematical average of all the guesses entered and that will most probably, almost certainly, be the closest number to the actual number. But why? This does rest on the condition that there is a large enough number of guesses made.

The underlined bit would not apply if all or even the majority of the guesses are higher or lower. 

If the actual jar contained 1200 smarties, and only two people entered and their guesses were 1250 and 1300 then the average guess would be 1275, but the closest guess would be the 1250 answer.

[Moonshed]

1 hour ago, JOC said:

The underlined bit would not apply if all or even the majority of the guesses are higher or lower. 

If the actual jar contained 1200 smarties, and only two people entered and their guesses were 1250 and 1300 then the average guess would be 1275, but the closest guess would be the 1250 answer.

You are correct in that two people making a guess is nowhere near enough guesses to achieve an average that is close to the actual number. This would be similar to carrying out a survey to predict the winning party at the next election and only asking two people. 🤔 We can ignore that situation as it does not apply.

As you say we could take the situation that given enough guesses are made but the majority of guesses are higher or lower then the result would be an average that was too high or too low. However, the reality of the situation is that when enough people have made their guess the average will always be very close to the actual number, it balances out, the too high guesses are cancelled out by the too low guesses, and this point is the core of the discussion. 
If, with enough participants, the average is not close to the actual number then we can be reasonably certain that a bias has skewed the guesses. I gave an example of bias earlier when I said that the stall could be set up in such a way that when participants where writing down their guesses they were able to see the guesses made by six previous participants, this would affect their guess and skew the result. 

The point is though that it does work, the average will always be close to the actual number. If plotted out on a graph it would reveal a Bell Distribution Curve, damn thing pops up everywhere 😄.

Anyway, back to work, 1,663…1,664…1,665…

Cheers

Keith

 

Edited March 21, 2023 by Moonshed