Announcement

Collapse
No announcement yet.

Stats hard to justify

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by JohnSunnyhills View Post
    Khan, I f think use of the term Uptime is somewhat confusimg on the FR24 website.

    Uptime according to the Radar Statistics page shows up as the time that a Datafeeder is communicating down the line to FR24. My stats almost always show 100% and yet the Uptime in the Add Coverage page shows my radars uptime as being closer to 690. I realise that this is time that aircraft activity is occurring and being collected.

    I suggest the Radar Statistics page should be changed so that the term Uptime there becomes Time On Line or something like that.
    Yes, you are right about that. It needs to change as Uptime confuses everyone since it shows very different numbers when you compare it with your Daily Availability graph which is the true Uptime.
    --

    Comment


    • #17
      Originally posted by Khan View Post
      Yes, you are right about that. It needs to change as Uptime confuses everyone since it shows very different numbers when you compare it with your Daily Availability graph which is the true Uptime.
      The last time I raised this with FR24 was on 8th February 2017. Perhaps FR24 will explain this on the Statistics page so we all understand how it's calculated. It's been confusing us data feeders since Day One. What's the point of showing a score that no one understands?
      Mike


      www.radarspotting.com

      Radarspotting since 2005

      Comment


      • #18
        Originally posted by Khan View Post
        Calculation is done based on this formula

        score = SUM/10(
        (2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
        (2 x Uptime + 2 x Max range + Avg range) of day 2,
        ...
        (2 x Uptime + 2 x Max range + Avg range) of day 30
        )
        Getting back to the calculation, the issue really is for whose benefit is this being calculated.

        What an individual datafeeder is likely to want to know is :
        Number of minutes that I was actually online to FR24, (ie capable of spotting activity if it is there)
        the average of of the daily scores over a 30 day period (not the daily scores summed over 30 days)

        This means that this then shows availability as well as effective performance.
        The inclusion of Average simply reduces the score which is a bit weird when the Max figure is really what it is about, Those further from an airport or flight path benefit with this.

        From this I surmise that FR in designing the calculation measure ranking based on:
        How close you are to a flight path or an airport (include average)
        How much activity is occurring in your area (by using uptime to mean the time when there is actually aircraft activity)

        This is likely to measure the level of quality data that a feeder is contributing.

        Moving on I am not sure that the the Max Range is being calculated as explained in the ?help on the Add Coverage page.
        Monitoring my results it is clear that Max Range is not the maximum range over 30 days, but some more convoluted calculation.

        Comment


        • #19
          Originally posted by Khan View Post
          Yes, you are right about that. It needs to change as Uptime confuses everyone since it shows very different numbers when you compare it with your Daily Availability graph which is the true Uptime.
          It's not so easy to say which is the most fair calculation, "connection" uptime or "received a/c feed" uptime. Some receivers are located in areas where there is no traffic within 250 nm some periods of the day. Should they really be "punished" for this in the stats? Complete coverage depends on users setting up receivers even in areas with little traffic. It is also much harder to find feeders there, while around the big airport there are lots and lots of willing feeders.

          This is one thing that I prefer with FR24 stats compared to FA, FR24 does not count the amount of traffic. With FA, you have to live near a big airport to get high ranking, and I feel FR24 system is more fair to all users, but this uptime calc goes a bit against that principle.
          F-ESDF1, F-ESGG1, F-ESGP1, F-ESNK1, F-ESNV2, F-ESNV3 F-ESSL4, F-ESNZ7, F-LFMN3
          T-ESNL1, T-ESNL2, T-ESGR15
          P-ESIA, P-ESIB, P-ESGF, P-ESSN, P-EFMA
          mrmac (a) fastest.cc

          Comment


          • #20
            Originally posted by Khan View Post
            Calculation is done based on this formula

            score = SUM/10(
            (2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
            (2 x Uptime + 2 x Max range + Avg range) of day 2,
            ...
            (2 x Uptime + 2 x Max range + Avg range) of day 30 )

            So you are saying that Avg range (also in the table, not only in summary?) is the average of all contacts ? Not the average MAX over the month?

            How can the top radars get > 250 nm AVERAGE range, is there no traffic in their vicinity?

            (T-feeders set their own position, can we trust it? )

            /M
            F-ESDF1, F-ESGG1, F-ESGP1, F-ESNK1, F-ESNV2, F-ESNV3 F-ESSL4, F-ESNZ7, F-LFMN3
            T-ESNL1, T-ESNL2, T-ESGR15
            P-ESIA, P-ESIB, P-ESGF, P-ESSN, P-EFMA
            mrmac (a) fastest.cc

            Comment


            • #21
              "2 x Max range"

              Range is normally defined as the min-max for two or more values. do you mean max distance?
              F-KDAG1

              Comment


              • #22
                Originally posted by Patrick Reeves View Post
                "Range is normally defined as the min-max for two or more values. do you mean max distance?
                "Receiver range" , "Range plot" etc are established expressions for radio communication.

                /M
                F-ESDF1, F-ESGG1, F-ESGP1, F-ESNK1, F-ESNV2, F-ESNV3 F-ESSL4, F-ESNZ7, F-LFMN3
                T-ESNL1, T-ESNL2, T-ESGR15
                P-ESIA, P-ESIB, P-ESGF, P-ESSN, P-EFMA
                mrmac (a) fastest.cc

                Comment


                • #23
                  Max range is also the average of the max for the last days. All this info will be added in the next release so that there aren't any confusions regarding score calculations anymore. At the moment only upload time is being considered. The idea behind using avg data for Share Statistics page is that each person has an individual stats page where they can see their daily stats e.g Max range, true uptime etc. Some might find this unfair and some otherwise. But this is not set in stone and suggestions for improvements are always welcome.
                  --

                  Comment


                  • #24
                    The first thing to appreciate is that the Individual Stats page should be the main page used by feeders and that should be the source of information consolidated into the Shared Statistics page. There should be a consistency between these pages so that individual data feeders can review changes occurring and take action etc.

                    Pertinent elements on the Individual Statistics Page are:

                    1* Uptime (as a percentage of available time) [This could be expressed as Minutes rather than a percentage]
                    2* Maximum Distance (nm)
                    3* Aircraft Seen
                    4* Positions Reported [These are quality Hits? ie usable by FR24]

                    Elements that are avalable but not shown clearly on the Individual Page are:

                    5* Number of Hits [Poor quality position therefore not used by FR24]
                    6* Activity Time (mins) [Number of minutes during which quality positions reported]
                    7* Maximum quality range (nm) [Furthest distance of quality positions reported]
                    8* Average quality range (nm) [Average distance of quality positions reported]

                    These should also be added to the Individual Statistics page.

                    When the Individual Statistics page is cleared at zulu midnight it is those values that should be passed through to be shown on the Shared Statistics page (or a selected subset), which eill show data summarised over the preceeding 30 days.

                    The existing Shared Ststistic page most likely uses; 6*, 7* and 8*. That is why there is this confusion.

                    [Note If the management of FR24 require this information for operational planning purposes etc, internal reports for this purpose should be developed. The Shared Statistics page is the primary tool of the Datafeeders and should be focussed on consolitating the information from Individual Statistics Pages.]

                    IMHO the Shared Statistics Page should be summarised information for the previous 30 days.
                    Individual data elements should be taken from what appears on the Individual Statistics Pages.

                    I would like to see:

                    * Uptime The sum of the minutes over the last 30 days using 1* Uptime converted to minutes.
                    * Max Distance The maximum of the 2* Maximum Distance over the last 30 days.
                    * Overall Score Anything you like such as sum of Daily (Uptime * Max Distance) [Note Average not included]
                    * Ranking Based on Overall Score

                    Could also show:
                    * Activity Time Using 6*
                    * Max Quality Range Using 7*

                    [Note I feel no place for Average Range in my thinking]

                    Comment


                    • #25
                      I've got poor coverage in the South thru to North-West segment and have been monitoring my stations data to look for improved performance as I change the antenna height etc.

                      Can someone confirm that the Individual Statistics page is the 'raw' daily 'quality' data and not averaged or summed data processed via some formula ?

                      And how often is the data on the Individual Statistics page updated/refreshed/re-calculated ? I'm regularly seeing distance data on my local device that is 25-30% above whats appearing on the Polar Plot on my Stat's page - hence my thinking thatsits averaged over 30 days also. I recall someone mentioned in another post that 3 reports are needed somewhere and maybe it applies ( or not ) in this case to eliminate the spurious readings received.

                      ylis

                      Comment


                      • #26
                        As reported by FR24's Khan:

                        All about feeding data to Flightradar24 (The Flightradar24 receiver, Raspberry Pi and Windows feeding software). No discussions about Flightradar24 web or apps.


                        Calculation is done based on this formula

                        score = SUM/10(
                        (2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
                        (2 x Uptime + 2 x Max range + Avg range) of day 2,
                        ...
                        (2 x Uptime + 2 x Max range + Avg range) of day 30
                        )


                        Or as he once told me in reply to my question:

                        Can you confirm how my Overall Score is calculated?

                        Today, for F-EGFE1, the Overall score is 10,590.

                        The Uptime is 719 (maybe I lost 1 hour in last 30 days) and Max Range = 255 and Average Range = 166.

                        Let’s assume that’s typical of everyday.

                        The Stats say the Overall score is calculated:

                        2 x Uptime + 2 x Max Range + Avg Range summed for the last 30 day.

                        If I do that calculation for one day:

                        2 x 719 + 2 x 255 + 166 = 2,114.

                        Multiply by 30 (assumes each day is the same) = 63,420.

                        How do you get 10,590?



                        It's done like this:

                        (2 x uptime + 2 x max range + avg range) Sum/10

                        Uptime in this case would become 2x24x60 (calculated per day and taken in minutes).

                        (2x24x60) + 2 x 255 + 166 = 2880 + 510 + 166 = 3550

                        3550 x 30 = 106680

                        106680/10 =10668

                        The uptime shown on the page is in hours just to keep it simple but calculation is done for each day and the time is taken in minutes.

                        2 x 24 x 60 = 2 x uptime of 24 hours for that day x 60 minutes

                        This is calculated for each day and then summed up for the last thirty 30. And is divided by 10 make the number smaller.


                        Simples
                        Last edited by Anmer; 2017-04-18, 11:11. Reason: Added text
                        Mike


                        www.radarspotting.com

                        Radarspotting since 2005

                        Comment


                        • #27
                          Originally posted by Khan View Post
                          Some might find this unfair and some otherwise. But this is not set in stone and suggestions for improvements are always welcome.
                          Khan (hope you see this!), if you'd like me to go into any of the ideas that follow in more detail, let me know. My professional background may be of some use to you in this matter. (Amongst other things, I'm responsible for maintaining some minor infrastructure and I'm nearing completion of PhD in computer science.)

                          Originally posted by JohnSunnyhills View Post
                          The inclusion of Average simply reduces the score which is a bit weird when the Max figure is really what it is about, Those further from an airport or flight path benefit with this.

                          [...]

                          How much activity is occurring in your area (by using uptime to mean the time when there is actually aircraft activity)
                          I have my misgivings about the calculation, too, but as you said it's really a question of what the rank is for and to whose benefit. Were I designing the ranking calculation, I'd weight it differently and I'd calculate uptime differently.

                          As MrMac hinted at, actual uptime listening (as opposed to time reporting) is a better indicator of usefulness because a station can't report messages when it's not listening/offline. If coverage is what you care about, availability is more important than number of messages relayed or a/c seen. However, I do appreciate that this metric would be too easy to game; you could get a guaranteed minimum even with no antenna connected. Perhaps FR24 took the approach they did to mitigate bad actors.

                          I disagree with you somewhat about the significance of max distance. The problem with max distance is that you only have to get lucky once (or perhaps a couple of times, see below) to get a figure that is enormously unrepresentative of your actual performance/quality/usefulness (my own station, described below, is currently a good example of that). The mean or median distance would be a better indication of the actual utility of a given station because they better account for obstacles, blind spots and unusual conditions that allow for long, narrow sectors. But both might disadvantage people who live near airports, because they'll have a lot of a/c at short range.

                          If you look at the distribution of distances in FR24, it probably looks somewhat Poisson, but in any case it'll have a long right tail because it's harder to rx more distant messages. So perhaps a better metric would be the 75th percentile or thereabouts: that way, those with some long range but narrow sectors are not unduly advantaged, and those living near airports are not unduly disadvantaged (because a/c departing and arriving aren't close for very long). That all said, there may be computational/infrastructure reasons why FR24 wouldn't want to do that.

                          Better still though computationally more expensive, consider the distribution of each individual aircraft's maximum, rather than all positions in aggregate. This compensates for the fact that you could get a large number of messages from a/c along a very narrow corridor, and all those extra data points distort the true angular range distribution.

                          I can think of several alternatives involving angular distribution and continuity of reporting (as an indicator of quality) that would yield even better yet range metrics.

                          As you said, it all depends on what you're trying to measure. If you're a Dx-er, then max range is of interest, but this is literally a "mine's longer than yours" contest. I'm unsure how useful that is in terms of measuring quality of coverage.

                          If you care about quality of coverage, consistency, including the number of samples at a given distance, is probably of more significance than max distance.

                          Originally posted by JohnSunnyhills View Post
                          This is likely to measure the level of quality data that a feeder is contributing.

                          [...]

                          Monitoring my results it is clear that Max Range is not the maximum range over 30 days, but some more convoluted calculation.
                          Quality of data is, or ought, to be the first priority.

                          I've only had my SDR for a week and been feeding FR24 for 5 days, and I'm still on the telescopic antenna supplied with the SDR. (I've built some home-brew antennae, an 8.5-element co-lin and a dipole, so I'll find out how they perform in the coming days). I wrote some Munin plugins (similar to jprochazka's adsb receiver project, though much more light-weight) to track a/c count, max and mean (Vincentian) distance, CPU, message count, signal strength and track count in preparation for experimentation with those antennae.

                          From those data, several things point to poor quality data (for now, anyway): Of tracks seen, dump1090 reports that about 90-95% of them are single-point, and even though my accepted message count (as well as mean and max distances) follow a pronounced diurnal cycle, the number of a/c seen (with and without position) don't vary nearly so much. Assuming each a/c broadcasts at a fairly constant average rate, you'd expect the number of a/c to be somewhat proportional to the number of messages. Also, when writing the plugins I noticed that there are many, many more Mode-S preambles than accepted messages (as in, about a hundred-fold), and about two thirds of Mode-S preambles were marked 'bad'.

                          Even though my monitoring software reports a max range of 130 nm, FR24 reports a max range 113 nm for the same period. (adsbScope, which has been running longer than the other software, reports a max range of a bit over the 150 nm mark and the best max distance reported by FR24 is 129 nm. This, on a kit telescopic antenna, seems implausible in terms of quality.)

                          Perhaps FR24 filter out long-distance samples from a/c for which there are fewer than some threshold points.

                          I don't know how many of these data are available in the AVR format. I suspect not much (and even if they were, as MrMac said, how much can you trust them?), which makes it difficult for FR24 to write generic support for quality monitoring into fr24feed. It's possible that the only data they can reliably get are a/c positions, but it might take more processing power than FR24 are willing to throw at it to ensure the quality of data they accept.

                          Comment


                          • #28
                            Addendum:

                            Originally posted by Strix technica View Post
                            Even though my monitoring software reports a max range of 130 nm, FR24 reports a max range 113 nm for the same period. (adsbScope, which has been running longer than the other software, reports a max range of a bit over the 150 nm mark and the best max distance reported by FR24 is 129 nm. This, on a kit telescopic antenna, seems implausible in terms of quality.)
                            Just now, FR24 reports for me for today a max distance of 213 nm, even though my local measurements report a max of 135 nm. Unless FR24's figure is via mlat (for which my local measurements don't account) and my contribution is being attributed, I'm quite sure that's a mistake.

                            A bug in FR24, perhaps?

                            Comment


                            • #29
                              Originally posted by Khan View Post
                              Calculation is done based on this formula

                              score = SUM/10(
                              (2 x Uptime (minutes) + 2 x Max range + Avg range) of day 1,
                              (2 x Uptime + 2 x Max range + Avg range) of day 2,
                              ...
                              (2 x Uptime + 2 x Max range + Avg range) of day 30)
                              Originally posted by Strix technica View Post
                              As MrMac hinted at, actual uptime listening (as opposed to time reporting) is a better indicator of usefulness because a station can't report messages when it's not listening/offline. If coverage is what you care about, availability is more important than number of messages relayed or a/c seen.
                              Just in case it's not obvious (and, being too focused on range, the significance of this only just dawned on me): uptime is already the dominant factor in rank — by a significant margin — even if you count in hours rather than minutes.

                              Most people don't get a max range of more than 250-300 nm, so 720 hours a month weighs the better part of 3:1 max distance. Since Khan pointed out that uptime is counted in minutes (43,200 per month), uptime is worth about 130:1 over the very best max range people get (~325 nm).

                              Of course, there are a lot of people with uptime > 700, so you still need decent range to get in the top 1000 on the stats table. I haven't draw a distribution, but I would guess it would look very Poisson, with a lot of radars with high uptime packed together.

                              At this moment some 572 radars have a perfect uptime, including one with a max range of 10 and an average of 1. Yet T-VHHH82 has a rank of 6848, score 8681.

                              That guy offers the counter-argument that JohnSunnyhills offered that range should count for more than it does, though I'd still argue that average range or, better yet, ~75th percentile would be better than max range.

                              Comment


                              • #30
                                Strix Technica, you make very good points but as you mentioned, it requires quite a bit of processing doing it that way. It's been under discussion internally to see if we can somehow make it fairer, if that's the right term to use here. As you said, it depends on where you are looking at it from.

                                At the moment, I can't promise if any short term changes will be made to it since the resources are put into improving individual stats page. But once that's completed then focus can be shifted here.
                                --

                                Comment

                                Working...
                                X