Introduction
A few years ago, I was learning about the three main averages in school - mean, median and mode - and was struck by how easily just one anomalous value could skew the results. I wondered how the three interacted with each other and how that relationship could be used to potentially improve upon the current methodology of finding an average, which is usually finding a mean, having excluded any anomalies.
I soon realised one of the major differences between the types of average: in some situations, mean may be unrepresentative of the average we would expect to find from a data set, and median and mode could be more applicable. And yet we choose not to use them due to their volatility with providing reliable averages. Have a look at this table, which gives a pretty decent idea of what I'm talking about.
It's immediately apparent here how the £135 product skews the mean in a way that makes it pretty unrepresentative of the data set. Median and mode, on the other hand, are much more accurate, in this case giving the same value (obviously not normal).
So, with this in mind, I asked myself if there was a way to combine these three averages, with the aim of reducing the shortcomings of each while amplifying their positive effects - the answer? The Ultimate Average.
The Concept of the Ultimate Average
My idea was actually quite simple - the ultimate average is the name for a process of stepped averaging. By this, I mean that one finds the three averages (mean, median and mode) of a data set, and then finds the three averages of the averages (which we can describe as the second order averages). The third order averages are then calculated, followed by the fourth, repeated until each median, mode and mean give the same value to a given accuracy (we used two decimal places). Below can be seen averages up to the fourth order, which you can see will lead to an Ultimate Average of 3.00 (as mean is the only variable factor beyond the first order).
You may have realised that mode is very unlikely to occur in anything beyond the first order average - this is because the chances of getting the same number twice in a data set of n=3 are very low, even when considering that they will likely be in the same ballpark. The fact that my example came up with one was an active choice of mine, as I think it helps to show how the mean is altered by the use of the Ultimate Average (which I'm going to call UA from now on because I'm lazy). With more normal data sets, the UA is found pretty fast as you can see in the table below.
The main benefit of the UA can be seen in the above table - in each event, the most important value is given the most weight - in the first, mean's traditional importance overshadows mode and combines with median to provide a more representative average. In the second, mean is again given more weight along with median over mode, in the same vein as the third, in which mode is shunned in favour of median. As we have seen, when median and mode are the same, mean is completely ignored as it is clearly apparent that if it is not the same, it does not represent the data set correctly. If all your values are the same, congratulations, you have found the Ultimate Average! Soon, I'll show an example of the UA in practice with a data set with no common averages (i.e. mean, median and mode are all different, as you would usually find).
But I really ought to mention the execution of my idea - how did my theory become a reality? For this, I have to thank Rahul for the simple reason that he codes - I don't. The only thing holding me back all these years from developing this average was the fact that I didn't have any knowledge of coding, but having Rahul on the team meant I could essentially commission him to write a python code for the Ultimate Average. And so we did. I won't post the raw code on here, but with enough support and interest I may change my mind.
The process works exactly the same as in the above example, finding the mean, median and mode, increasing the order of averages until each the mean, median and mode give the same value. The program then provides a raw value as well as a value to two decimal places for easy reading.
The Ultimate Average In Practice
Here‘s a fairly straightforward example from a maths question about scores in a test marked out of 40.
You can see here how even though the majority of scores are above 30, with a large proportion being 40, the mean is still 30.7, dragged down by scores like 7 (must be Rahul). The advantage of the UA here is patent, bringing the average up by over 3 points, making it more representative of the higher class scores.
Contrastingly, below is a more random data set - the view counts on all OnlyPhysics blog posts at the time of writing! This is more varied, ranging from 2 (oh dear, Rahul) to 76, making the mean slightly less reliable. We also have a mode here, which is always a nice surprise (you’d be surprised - or not - how few data sets contain a mode).
You can see in action here how the mean is not very representative of the data set, with a separation of only 0.5 between the median and mode, while being around 7 from the mean. Therefore, the UA prefers these over the mean, bringing the final value much closer to the median and mode.
Conclusion
And therein lies the beauty of the Ultimate Average: through the utilisation of fairly straightforward, basic maths, one can achieve intelligent, fluid outcomes that respond to data in a way that you would initially expect only humans to be able to - recognising the true distribution of a data set where the archaic traditional averages fall short.
Could we one day see mean replaced by the Ultimate Average? Maybe. There are certainly applications for it in data collection, experiments and in general statistics, and it certainly seems odd that advanced programs and processes still use such fundamental averaging methods as mean and median. The Ultimate Average could one day fill that gap - who knows?
So with all these improvements, where does the UA fall short? Firstly, the methodology is fairly basic, and could potentially be improved with more elements feeding into it, such as for example standard deviation or other types of average. On top of this, we also found that having no mode in a data set (very common as I said) meant that the UA was fairly underwhelming, just finding halfway between the median and the mean - due to the simplicity of the model.
Additionally, the UA is less useful with large data (for example burgers sold at McDonald's in a year) and precise data (exchange rate of GDP to USD to 6dp) than small, simple data (OnlyPhysics blogs per day). This is because the probability of gaining a mode in small, simple data is much higher than with large or complex data points, which means the ultimate average will almost always just be halfway between the mean and the median in large or precise data, which is less useful than data involving a mode. One other point of note is that the actual interface of the code is fairly raw, and could do with some reworking.
So, here's our pledge: if this post hits 1000 views, we‘ll redesign the Ultimate Average to give you the Ultimate-r Average! We’ll attempt to create an actual downloadable program instead of a Python code (well, Rahul will), we’ll research different types of average we could use, and hopefully we will be able to revise input and output methods (allowing you to for example import an excel table or grouped frequency set). If achieved, we could release the code as free software on the website. Write a comment below or share with your mates (would mean the world to us) if you want to see that!
Comentários