Originally posted by JohnSunnyhills
View Post
Announcement
Collapse
No announcement yet.
Stats hard to justify
Collapse
X
-
Originally posted by Khan View PostYes, you are right about that. It needs to change as Uptime confuses everyone since it shows very different numbers when you compare it with your Daily Availability graph which is the true Uptime.
Comment
-
Originally posted by Khan View PostCalculation is done based on this formula
score = SUM/10(
(2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
(2 x Uptime + 2 x Max range + Avg range) of day 2,
...
(2 x Uptime + 2 x Max range + Avg range) of day 30
)
What an individual datafeeder is likely to want to know is :
Number of minutes that I was actually online to FR24, (ie capable of spotting activity if it is there)
the average of of the daily scores over a 30 day period (not the daily scores summed over 30 days)
This means that this then shows availability as well as effective performance.
The inclusion of Average simply reduces the score which is a bit weird when the Max figure is really what it is about, Those further from an airport or flight path benefit with this.
From this I surmise that FR in designing the calculation measure ranking based on:
How close you are to a flight path or an airport (include average)
How much activity is occurring in your area (by using uptime to mean the time when there is actually aircraft activity)
This is likely to measure the level of quality data that a feeder is contributing.
Moving on I am not sure that the the Max Range is being calculated as explained in the ?help on the Add Coverage page.
Monitoring my results it is clear that Max Range is not the maximum range over 30 days, but some more convoluted calculation.
Comment
-
Originally posted by Khan View PostYes, you are right about that. It needs to change as Uptime confuses everyone since it shows very different numbers when you compare it with your Daily Availability graph which is the true Uptime.
This is one thing that I prefer with FR24 stats compared to FA, FR24 does not count the amount of traffic. With FA, you have to live near a big airport to get high ranking, and I feel FR24 system is more fair to all users, but this uptime calc goes a bit against that principle.F-ESDF1, F-ESGG1, F-ESGP1, F-ESNK1, F-ESNV2, F-ESNV3 F-ESSL4, F-ESNZ7, F-LFMN3
T-ESNL1, T-ESNL2, T-ESGR15
P-ESIA, P-ESIB, P-ESGF, P-ESSN, P-EFMA
mrmac (a) fastest.cc
Comment
-
Originally posted by Khan View PostCalculation is done based on this formula
score = SUM/10(
(2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
(2 x Uptime + 2 x Max range + Avg range) of day 2,
...
(2 x Uptime + 2 x Max range + Avg range) of day 30 )
So you are saying that Avg range (also in the table, not only in summary?) is the average of all contacts ? Not the average MAX over the month?
How can the top radars get > 250 nm AVERAGE range, is there no traffic in their vicinity?
(T-feeders set their own position, can we trust it? )
/MF-ESDF1, F-ESGG1, F-ESGP1, F-ESNK1, F-ESNV2, F-ESNV3 F-ESSL4, F-ESNZ7, F-LFMN3
T-ESNL1, T-ESNL2, T-ESGR15
P-ESIA, P-ESIB, P-ESGF, P-ESSN, P-EFMA
mrmac (a) fastest.cc
Comment
-
Originally posted by Patrick Reeves View Post"Range is normally defined as the min-max for two or more values. do you mean max distance?
/MF-ESDF1, F-ESGG1, F-ESGP1, F-ESNK1, F-ESNV2, F-ESNV3 F-ESSL4, F-ESNZ7, F-LFMN3
T-ESNL1, T-ESNL2, T-ESGR15
P-ESIA, P-ESIB, P-ESGF, P-ESSN, P-EFMA
mrmac (a) fastest.cc
Comment
-
Max range is also the average of the max for the last days. All this info will be added in the next release so that there aren't any confusions regarding score calculations anymore. At the moment only upload time is being considered. The idea behind using avg data for Share Statistics page is that each person has an individual stats page where they can see their daily stats e.g Max range, true uptime etc. Some might find this unfair and some otherwise. But this is not set in stone and suggestions for improvements are always welcome.--
Comment
-
The first thing to appreciate is that the Individual Stats page should be the main page used by feeders and that should be the source of information consolidated into the Shared Statistics page. There should be a consistency between these pages so that individual data feeders can review changes occurring and take action etc.
Pertinent elements on the Individual Statistics Page are:
1* Uptime (as a percentage of available time) [This could be expressed as Minutes rather than a percentage]
2* Maximum Distance (nm)
3* Aircraft Seen
4* Positions Reported [These are quality Hits? ie usable by FR24]
Elements that are avalable but not shown clearly on the Individual Page are:
5* Number of Hits [Poor quality position therefore not used by FR24]
6* Activity Time (mins) [Number of minutes during which quality positions reported]
7* Maximum quality range (nm) [Furthest distance of quality positions reported]
8* Average quality range (nm) [Average distance of quality positions reported]
These should also be added to the Individual Statistics page.
When the Individual Statistics page is cleared at zulu midnight it is those values that should be passed through to be shown on the Shared Statistics page (or a selected subset), which eill show data summarised over the preceeding 30 days.
The existing Shared Ststistic page most likely uses; 6*, 7* and 8*. That is why there is this confusion.
[Note If the management of FR24 require this information for operational planning purposes etc, internal reports for this purpose should be developed. The Shared Statistics page is the primary tool of the Datafeeders and should be focussed on consolitating the information from Individual Statistics Pages.]
IMHO the Shared Statistics Page should be summarised information for the previous 30 days.
Individual data elements should be taken from what appears on the Individual Statistics Pages.
I would like to see:
* Uptime The sum of the minutes over the last 30 days using 1* Uptime converted to minutes.
* Max Distance The maximum of the 2* Maximum Distance over the last 30 days.
* Overall Score Anything you like such as sum of Daily (Uptime * Max Distance) [Note Average not included]
* Ranking Based on Overall Score
Could also show:
* Activity Time Using 6*
* Max Quality Range Using 7*
[Note I feel no place for Average Range in my thinking]
Comment
-
I've got poor coverage in the South thru to North-West segment and have been monitoring my stations data to look for improved performance as I change the antenna height etc.
Can someone confirm that the Individual Statistics page is the 'raw' daily 'quality' data and not averaged or summed data processed via some formula ?
And how often is the data on the Individual Statistics page updated/refreshed/re-calculated ? I'm regularly seeing distance data on my local device that is 25-30% above whats appearing on the Polar Plot on my Stat's page - hence my thinking thatsits averaged over 30 days also. I recall someone mentioned in another post that 3 reports are needed somewhere and maybe it applies ( or not ) in this case to eliminate the spurious readings received.
ylis
Comment
-
As reported by FR24's Khan:
All about feeding data to Flightradar24 (The Flightradar24 receiver, Raspberry Pi and Windows feeding software). No discussions about Flightradar24 web or apps.
Calculation is done based on this formula
score = SUM/10(
(2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
(2 x Uptime + 2 x Max range + Avg range) of day 2,
...
(2 x Uptime + 2 x Max range + Avg range) of day 30
)
Or as he once told me in reply to my question:
Can you confirm how my Overall Score is calculated?
Today, for F-EGFE1, the Overall score is 10,590.
The Uptime is 719 (maybe I lost 1 hour in last 30 days) and Max Range = 255 and Average Range = 166.
Let’s assume that’s typical of everyday.
The Stats say the Overall score is calculated:
2 x Uptime + 2 x Max Range + Avg Range summed for the last 30 day.
If I do that calculation for one day:
2 x 719 + 2 x 255 + 166 = 2,114.
Multiply by 30 (assumes each day is the same) = 63,420.
How do you get 10,590?
It's done like this:
(2 x uptime + 2 x max range + avg range) Sum/10
Uptime in this case would become 2x24x60 (calculated per day and taken in minutes).
(2x24x60) + 2 x 255 + 166 = 2880 + 510 + 166 = 3550
3550 x 30 = 106680
106680/10 =10668
The uptime shown on the page is in hours just to keep it simple but calculation is done for each day and the time is taken in minutes.
2 x 24 x 60 = 2 x uptime of 24 hours for that day x 60 minutes
This is calculated for each day and then summed up for the last thirty 30. And is divided by 10 make the number smaller.
Simples
Comment
-
Originally posted by Khan View PostSome might find this unfair and some otherwise. But this is not set in stone and suggestions for improvements are always welcome.
Originally posted by JohnSunnyhills View PostThe inclusion of Average simply reduces the score which is a bit weird when the Max figure is really what it is about, Those further from an airport or flight path benefit with this.
[...]
How much activity is occurring in your area (by using uptime to mean the time when there is actually aircraft activity)
As MrMac hinted at, actual uptime listening (as opposed to time reporting) is a better indicator of usefulness because a station can't report messages when it's not listening/offline. If coverage is what you care about, availability is more important than number of messages relayed or a/c seen. However, I do appreciate that this metric would be too easy to game; you could get a guaranteed minimum even with no antenna connected. Perhaps FR24 took the approach they did to mitigate bad actors.
I disagree with you somewhat about the significance of max distance. The problem with max distance is that you only have to get lucky once (or perhaps a couple of times, see below) to get a figure that is enormously unrepresentative of your actual performance/quality/usefulness (my own station, described below, is currently a good example of that). The mean or median distance would be a better indication of the actual utility of a given station because they better account for obstacles, blind spots and unusual conditions that allow for long, narrow sectors. But both might disadvantage people who live near airports, because they'll have a lot of a/c at short range.
If you look at the distribution of distances in FR24, it probably looks somewhat Poisson, but in any case it'll have a long right tail because it's harder to rx more distant messages. So perhaps a better metric would be the 75th percentile or thereabouts: that way, those with some long range but narrow sectors are not unduly advantaged, and those living near airports are not unduly disadvantaged (because a/c departing and arriving aren't close for very long). That all said, there may be computational/infrastructure reasons why FR24 wouldn't want to do that.
Better still though computationally more expensive, consider the distribution of each individual aircraft's maximum, rather than all positions in aggregate. This compensates for the fact that you could get a large number of messages from a/c along a very narrow corridor, and all those extra data points distort the true angular range distribution.
I can think of several alternatives involving angular distribution and continuity of reporting (as an indicator of quality) that would yield even better yet range metrics.
As you said, it all depends on what you're trying to measure. If you're a Dx-er, then max range is of interest, but this is literally a "mine's longer than yours" contest. I'm unsure how useful that is in terms of measuring quality of coverage.
If you care about quality of coverage, consistency, including the number of samples at a given distance, is probably of more significance than max distance.
Originally posted by JohnSunnyhills View PostThis is likely to measure the level of quality data that a feeder is contributing.
[...]
Monitoring my results it is clear that Max Range is not the maximum range over 30 days, but some more convoluted calculation.
I've only had my SDR for a week and been feeding FR24 for 5 days, and I'm still on the telescopic antenna supplied with the SDR. (I've built some home-brew antennae, an 8.5-element co-lin and a dipole, so I'll find out how they perform in the coming days). I wrote some Munin plugins (similar to jprochazka's adsb receiver project, though much more light-weight) to track a/c count, max and mean (Vincentian) distance, CPU, message count, signal strength and track count in preparation for experimentation with those antennae.
From those data, several things point to poor quality data (for now, anyway): Of tracks seen, dump1090 reports that about 90-95% of them are single-point, and even though my accepted message count (as well as mean and max distances) follow a pronounced diurnal cycle, the number of a/c seen (with and without position) don't vary nearly so much. Assuming each a/c broadcasts at a fairly constant average rate, you'd expect the number of a/c to be somewhat proportional to the number of messages. Also, when writing the plugins I noticed that there are many, many more Mode-S preambles than accepted messages (as in, about a hundred-fold), and about two thirds of Mode-S preambles were marked 'bad'.
Even though my monitoring software reports a max range of 130 nm, FR24 reports a max range 113 nm for the same period. (adsbScope, which has been running longer than the other software, reports a max range of a bit over the 150 nm mark and the best max distance reported by FR24 is 129 nm. This, on a kit telescopic antenna, seems implausible in terms of quality.)
Perhaps FR24 filter out long-distance samples from a/c for which there are fewer than some threshold points.
I don't know how many of these data are available in the AVR format. I suspect not much (and even if they were, as MrMac said, how much can you trust them?), which makes it difficult for FR24 to write generic support for quality monitoring into fr24feed. It's possible that the only data they can reliably get are a/c positions, but it might take more processing power than FR24 are willing to throw at it to ensure the quality of data they accept.
Comment
-
Addendum:
Originally posted by Strix technica View PostEven though my monitoring software reports a max range of 130 nm, FR24 reports a max range 113 nm for the same period. (adsbScope, which has been running longer than the other software, reports a max range of a bit over the 150 nm mark and the best max distance reported by FR24 is 129 nm. This, on a kit telescopic antenna, seems implausible in terms of quality.)
A bug in FR24, perhaps?
Comment
-
Originally posted by Khan View PostCalculation is done based on this formula
score = SUM/10(
(2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
(2 x Uptime + 2 x Max range + Avg range) of day 2,
...
(2 x Uptime + 2 x Max range + Avg range) of day 30)Originally posted by Strix technica View PostAs MrMac hinted at, actual uptime listening (as opposed to time reporting) is a better indicator of usefulness because a station can't report messages when it's not listening/offline. If coverage is what you care about, availability is more important than number of messages relayed or a/c seen.
Most people don't get a max range of more than 250-300 nm, so 720 hours a month weighs the better part of 3:1 max distance. Since Khan pointed out that uptime is counted in minutes (43,200 per month), uptime is worth about 130:1 over the very best max range people get (~325 nm).
Of course, there are a lot of people with uptime > 700, so you still need decent range to get in the top 1000 on the stats table. I haven't draw a distribution, but I would guess it would look very Poisson, with a lot of radars with high uptime packed together.
At this moment some 572 radars have a perfect uptime, including one with a max range of 10 and an average of 1. Yet T-VHHH82 has a rank of 6848, score 8681.
That guy offers the counter-argument that JohnSunnyhills offered that range should count for more than it does, though I'd still argue that average range or, better yet, ~75th percentile would be better than max range.
Comment
-
Strix Technica, you make very good points but as you mentioned, it requires quite a bit of processing doing it that way. It's been under discussion internally to see if we can somehow make it fairer, if that's the right term to use here. As you said, it depends on where you are looking at it from.
At the moment, I can't promise if any short term changes will be made to it since the resources are put into improving individual stats page. But once that's completed then focus can be shifted here.--
Comment
Comment