Archived - Beta test MLAT software for Raspberry Pi

[bhaal]

vinnyspb: Why does FR24's feeding software require a special proprietary version of dump1090? Why can't you use the Mode-S data that comes out of mutability etc? Surely if you need modified timestamps you could do this translation at the server end? I do understand that back at the beginning of the year you launched a complete suite of fr24feed + proprietary dump1090 in order to try and iron out inconsistent data, but I would have thought that data inconsistency could be eliminated by correlation between a number of feeders seeing the same data.

I feel like you have limited your feeder pool by being so restrictive. I see this especially with the RTLSDR MLAT data.. What I see on the FR24 website seems to be wildly inaccurate and very limited at the best of times. This is as opposed to the MLAT data I am seeing on my local VRS which gets feedback from FA's MLAT client, the data is consistent with planes which fly directly over my house, and in some directions I am seeing MLAT positions down to as low as 500ft..

Can some consideration please be put in to using Mode-S data from standard versions of dump1090 for MLAT data, with timestamp modification being done either in the fr24feed client or at the server end. That makes a whole lot more sense to me..

[Oblivian]

Don't believe its proprietary - seems to be the -MR Malcolm Robb updated Dump1090 (last original 1090 update was ages ago) with better reception and processing rate (along with some HTTP server data etc) and I think youll find Piaware uses a fork of that too going by some docs out there.

https://github.com/sjthespian/dump1090_mr/blob/master/README.md

The joys of open source. They all have advantages and disadvantages

[helios]

Surely if you need modified timestamps you could do this translation at the server end?

Timestamps need to be done on the client. The networking path would cause too much jitter (1ms=300km)

[bhaal]

Don't believe its proprietary - seems to be the -MR Malcolm Robb updated Dump1090 (last original 1090 update was ages ago) with better reception and processing rate (along with some HTTP server data etc) and I think youll find Piaware uses a fork of that too going by some docs out there.

https://github.com/sjthespian/dump1090_mr/blob/master/README.md

The joys of open source. They all have advantages and disadvantages

Can you please link me to the source for FR24's fork of MR's fork of the original dump1090 please? As for Piaware, it's not restricted to the MR version of dump1090 for MLAT data, I think the latest .deb for dump1090 produced for PiAware might even be the Mutability fork of dump1090..

Timestamps need to be done on the client. The networking path would cause too much jitter (1ms=300km)

Ok, firstly let me say, prior to my previous post I had not investigated fully how the FR version of Pi MLAT worked. I still haven't, but I think I have a good idea now...

"Take Mode-S only packet attach wildly inaccurate timestamp based on NTP which at best can only be considered accurate to the second, send modified packet to MLAT server, and cross fingers..."

Is that it? Is that how FR are doing this? If so, how many feeders does it take to get a position which could be considered even vaguely correct?

vinnyspb, I am trying to give constructive criticism, while asking these questions so please do not take personal offence :/

Ok... So, NTP lets start there... My nearest stratum 1 NTP server (which can only be used by request [I have not requested it]) is on average 1.847ms (and 7 hops) away from the Linux desktop I type this forum post from (I have "fibre to the premises" internet). It is 2.735ms away from the Pi B+ that runs dump1090, so already right there is a HUGE amount of jitter, around 1ms of which caused by the lag inside the Pi itself... everything runs over that 1 USB host, all the SDR packets as well as the network traffic, then we have the CPU load causing extra grief as well (cpu load which is not helped by the cycle hungry FR24 version of dump1090)..

So, setting an accurate time is going to be extremely difficult.. You might suggest setting up a stratum 1 NTP server ON the Pi itself using a GPS transponder... Wrong, fr24feed uses exotic intercontinental NTP servers...

The main reason why MLAT positioning was originally tossed overboard with regards to RTL-SDR receivers is because of the USB bus lag... THAT still exists regardless of how accurate a clock source is that exists. I can only see 1 way around that... 2 dongles using the same clock crystal, and have dump1090 read data from both dongles, and any identical Mode-S packets it doesn't receive at the same time from both dongles: Toss them overboard.. Any that it does, timestamp them with the onboard GPS clock and send to the MLAT server... Wait... Did I just come up with a cheap alternative to a Mode-S Beast??? I am not proficient in C, so trying to code a soft-Beast is out of my depth, but it's food for thought..

So, working towards what I understand fr24feed to be doing: Tack an inaccurate timestamp to a Mode-S packet which has a huge amount of lag generated by the USB bus and send it to MLAT servers, and hope like crazy that it works out... At this point I stop wondering why the Dash-8 I saw this afternoon on my VRS with FlightAware MLAT positions fed in, as well as seeing the plane with my own eyes, fly directly overhead, appear on the FR24 website about 20km east of my position.

Advantages to doing RTL-SDR MLAT positioning this way: no reliance on any thing else... Can't think of any other advantages?

Disadvantages, wildly inaccurate positions.. Large number of feeders required to make the data more accurate before it's displayed ONLY on the FR24 website, not feedback data for the feeders so they can use that data on their own display system... Also, adding all these timestamps chews a hefty amount of CPU cycles..

Moving forward, the Mutability way of doing things (Which is what FA is using):

dump1090 receives Mode-S packets and ADS-B packets at the same time, for packets which arrive at exactly the same time, send them to the MLAT server. There is no need to add exotic timestamps etc, just send the raw information.

Advantages: Considerably accurate MLAT positions, positions are also calculated quickly, from the normal bare minimum 3 feeder locations to generate an MLAT position, (however the more the merrier in case of poorly configured geographic co-ordinates of feeders).. Less cpu required as data is not being transposed onto existing packets, the packets are just being forwarded..

Disadvantages: MLAT positions cannot be generated for a Mode-S only aircraft if there is no ADS-B enabled aircraft visible to at least 3 feeders at the same time, they also have to receive the same ADS-B and Mode-S packets in the same order as each other, otherwise the data is discarded. (this is occuring right now, a Mode-S only capable rescue helicopter is about to fly near by, I can only tell this by signal strength as there are no ADS-B enabled aircraft within range of my receiver right now)

I may have some or all of this wrong.. Please enlighten me if I have as this is what I am trying to do, understand... I am not posting all this to try and upset people etc.. I am trying to educate myself by generating a bit more discussion, and also hopefully help get better data for FR24!

[Guest]

Hi bhaal,

Thanks for the constructive criticism, but you described exactly the algorithm we've implemented here. There is no rely on NTP-only timestamps. We sync those timestamps across ADS-B traffic server-side, simultaneously visible between 3 Raspberry Pi receivers and 1 FR24 receiver that has exact precise GPS time aligned with Mode-S signal. Positions uncertainty is of course a problem, and we will definitely work on improving the quality. I understand your frustration about vendor-lock for dump1090 - that might be improved in future.
You asked for a link to sources: https://github.com/flightradar24/dump1090

[cbgsy]

Hi everyone,

T-EGJB6 now back online with MLAT here in St Peter Port, Guernsey using an RPi 2 with a PPS-coupled GPS receiver feeding NTP, so should be nice and stable.
Hoping to improve coverage for the Trislanders and inter-island flights which are typically below 2000ft as well as the Dash-8 and ATR aircraft which provide most of our local flights and require MLAT.

Flightradar24 Feeder/Decoder
Linux/generic/armv7l/1.0.14-7
Updated: 17:25:59 GMT+0100 (BST)

FR24 Link: Connected via UDP
FR24 Radar Code: T-EGJB6
Aircraft Tracked (ModeS & ADS-B): 135
Aircraft Uploaded: 102
Receiver: dvbt, Connected
MLAT running: YES

pi@raspberrypi ~ $ ntpq -crv -pn
associd=0 status=011d leap_none, sync_pps, 1 event, kern,
version="ntpd 4.2.6p5@1.2349-o Mon Jul 27 15:39:38 UTC 2015 (1)",
processor="armv7l", system="Linux/3.18.11-v7+", leap=00, stratum=1,
precision=-20, rootdelay=0.000, rootdisp=0.419, refid=PPS,
reftime=d96a1556.10db60db  Mon, Aug  3 2015 17:29:10.065,
clock=d96a1562.690440e5  Mon, Aug  3 2015 17:29:22.410, peer=19256, tc=4,
mintc=3, offset=-0.005, frequency=-1.441, sys_jitter=0.001,
clk_jitter=0.000, clk_wander=0.001
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
[B]o127.127.22.0    .PPS.            0 l   12   16  377    0.000   -0.005   0.001[/B]
*127.127.28.0    .GPS.            0 l   11   16  377    0.000    1.801   1.119
-217.114.59.66   157.44.176.4     2 u  155 1024  377   57.604    7.965   1.916
-217.114.59.3    158.43.192.66    2 u  330 1024  377   57.414    8.811   2.478
+162.13.167.224  161.62.36.7      2 u  266 1024  377   28.485   -3.185   2.732
+185.53.93.157   195.66.241.10    2 u  396 1024  377   27.643   -2.823  69.697

[Kpin]

So vinnyspb, is the following true for this to work?:

1) Minimum 1 FR24 receiver and 3 RPi receivers must see the S-mode aircraft in question.
2) All 4 receivers must at the same time also see an ADS-B aircraft for time reference.

[t-kclt2]

Not sure if this is the right thread.. and hope I haven't missed a conversation about this already, but after updating everything to latest stable (via apt-get), two problems:

1) I am somehow missing MR dump1090... the service is trying to start it from /usr/lib/fr24 but the only thing there is dump1090 (mr-dump1090 nowhere to be found).. is the path to executable configurable somewhere?

2) My feed got renamed... and I don't like the new name. I ran the --signup but provided existing sharing key... (which is still in the .ini file) but got a new name.. why? how do I go back to the old one

[Guest]

So vinnyspb, is the following true for this to work?:

1) Minimum 1 FR24 receiver and 3 RPi receivers must see the S-mode aircraft in question.
2) All 4 receivers must at the same time also see an ADS-B aircraft for time reference.

Exactly.

Not sure if this is the right thread.. and hope I haven't missed a conversation about this already, but after updating everything to latest stable (via apt-get), two problems:

1) I am somehow missing MR dump1090... the service is trying to start it from /usr/lib/fr24 but the only thing there is dump1090 (mr-dump1090 nowhere to be found).. is the path to executable configurable somewhere?

2) My feed got renamed... and I don't like the new name. I ran the --signup but provided existing sharing key... (which is still in the .ini file) but got a new name.. why? how do I go back to the old one

/usr/lib/fr24/dump1090 is a fork off mr-dump1090, you can use all it's features. No need to reconfigure the path.
Feed rename happened most likely due to the different coordinates selected during previous registration.
Please contact FR24 support to change radar id back.

[Lorsde]

New Version with MLAT works fine for me

[ ok ] FR24 Feeder/Decoder Process: running.
[ ok ] FR24 Stats Timestamp: 2015-08-05 08:24:53.
[ ok ] FR24 Link: connected [UDP].
[ ok ] FR24 Radar: T-EDDH14.
[ ok ] FR24 Tracked AC: 50.
[ ok ] Receiver: connected (12532 MSGS/0 SYNC).
[ ok ] FR24 MLAT: ok [UDP].
[ ok ] FR24 MLAT AC seen: 51.


Thanks a lot, great work!

[obj]

Hi bhaal,

Thanks for the constructive criticism, but you described exactly the algorithm we've implemented here. There is no rely on NTP-only timestamps. We sync those timestamps across ADS-B traffic server-side, simultaneously visible between 3 Raspberry Pi receivers and 1 FR24 receiver that has exact precise GPS time aligned with Mode-S signal. Positions uncertainty is of course a problem, and we will definitely work on improving the quality. I understand your frustration about vendor-lock for dump1090 - that might be improved in future.
You asked for a link to sources: https://github.com/flightradar24/dump1090

If you're using ADS-B as a reference beacon anyway, why do you need to modify the 12MHz timestamp into Radarcape-style seconds-since-midnight format? You don't care about the absolute time of arrival.
By doing this conversion using the system clock, you're introducting potentially large errors because you have no guarantee that buffers of samples arrive at regular intervals as measured by the system clock.

(This is why mlat-server and FA's implementation which is built on it are unlikely to ever support this type of converted timestamp)

[oroblraM]

Hi vinnyspb

I'm new to FR24feed, but "listening" here since a long time...

A little suggestion: Is it possible to show version informations like shown below
(or anyhow you think)

pi@raspberrypi ~ $ sudo service fr24feed status
[ INFO ] running Version: 1.0.14 Build: 5 ____ newest Version: 1.0.14 Build: 7
[AutoUpdate] disabled
[ UpTime ] 7h 5m 12s
[ ok ] FR24 Feeder/Decoder Process: running.
[ ok ] FR24 Stats Timestamp: 2015-08-01 19:48:46.
[ ok ] FR24 Link: connected [UDP].
[ ok ] FR24 Radar: T-EDMS6
[ ok ] FR24 Tracked AC: 70.
[ ok ] Receiver: connected (55402 MSGS/0 SYNC).
[ ok ] FR24 MLAT: ok [UDP].
[ ok ] FR24 MLAT AC seen: 69.
pi@raspberrypi ~ $

Greets out of Bavaria

Stefan

btw: What's trying to tell me: " 0 SYNC" ?????

no reply to my suggestion or my btw-question??? surprises even me.......

[terra]

Hi,

I have a RPi2 running since March 2015 and everything was fine. 3 days ago I started the installation for beta testing phase of MLAT. Since then I have a problem. I started the installation as vinnyspb stated on page 1 of this thread. After installation I signed up with my sharing key and restarted the RPI2. The result was as below.

$ sudo service fr24feed status
[ ok ] FR24 Feeder/Decoder Process: running.
[ ok ] FR24 Stats Timestamp: 2015-08-05 08:47:30.
[ ok ] FR24 Link: connected [UDP].
[ ok ] FR24 Radar: T-EDWI8.
[ ok ] FR24 Tracked AC:.
[FAIL] Receiver: down ... failed!
[ ok ] FR24 MLAT: ok [UDP].
[ ok ] FR24 MLAT AC seen: 0.

The logfile says this:

2015-08-05 08:34:46 | [main][i]FR24 Feeder/Decoder [0x02117000]
2015-08-05 08:34:46 | [main][i]Version: 1.0.14-7/generic
2015-08-05 08:34:46 | [main][i]Built on 20150731-1055 (r:master-5e970cb.git/Linux/armv7l)
2015-08-05 08:34:46 | [main][i]Copyright 2008-2015 (c) Piotr Pawluczuk
2015-08-05 08:34:46 | [main][i]Flightradar24 AB
2015-08-05 08:34:46 | [main][i]DNS mode: LIBC
2015-08-05 08:34:46 | ERROR
2015-08-05 08:34:46 | [httpd][i]Server started, listening on 0.0.0.0:8754
2015-08-05 08:34:46 | [main][i]Reader 0 started
2015-08-05 08:34:46 | [reader][i][0]Initializing reader
2015-08-05 08:34:46 | [main][i]MLAT data feed started
2015-08-05 08:34:46 | [reader][i][0]Connecting to Generic receiver via (exe:///usr/lib/fr24/dump1090 --raw --mlat)
2015-08-05 08:34:46 | [master][i]Starting processing thread
2015-08-05 08:34:46 | [reader][i][0]Connected to the receiver, authenticating
2015-08-05 08:34:46 | [mlat][i]Waiting for MLAT configuration
2015-08-05 08:34:46 | [reader][i][0]Authenticated, processing messages
2015-08-05 08:34:46 | [reader][i][0]Connection terminated
2015-08-05 08:34:46 | [main][i]Terminating child process 19522 with SIGETERM
2015-08-05 08:34:47 | [time][i]Synchronizing time via NTP
2015-08-05 08:34:50 | [time][i]Time synchronized correctly, offset +0.0001 seconds
2015-08-05 08:34:50 | [main][i]Feed Network client started
2015-08-05 08:34:50 | [feed][i]Downloading configuration
2015-08-05 08:34:50 | [main][i]RAW data server started
2015-08-05 08:34:50 | [raw][i]Initializing server
2015-08-05 08:34:50 | [raw][i]Starting server on 0.0.0.0:30002
2015-08-05 08:34:50 | [raw][e]Could not bind socket to *:30002, errno=98
2015-08-05 08:34:51 | [feed][c]Interval: 5s
2015-08-05 08:34:51 | [feed][c]Latitude: 53.5073
2015-08-05 08:34:51 | [feed][c]Longitude: 8.0524
2015-08-05 08:34:51 | [feed][c]GND: YES
2015-08-05 08:34:51 | [feed][c]NonADSB: YES
2015-08-05 08:34:51 | [feed][c]Timestamps: optional
2015-08-05 08:34:51 | [feed][c]Max range AIR: 350.0nm
2015-08-05 08:34:51 | [feed][c]Max range GND: 100.0nm
2015-08-05 08:34:51 | [feed][i]defined 1 server
2015-08-05 08:34:51 | [stats][i]Stats thread started
2015-08-05 08:34:51 | [feed][n]EDWI8@83.140.21.66:8099/UDP
2015-08-05 08:34:51 | [feed][n]connecting
2015-08-05 08:34:51 | [mlat][i]MLAT configuration received, service ENABLED
2015-08-05 08:34:51 | [mlat][i]Starting MLAT with preconfigured position: 53.51,8.05,72.0
2015-08-05 08:34:51 | [mlat][i]MLAT bandwidth reduction active, level 1
2015-08-05 08:34:51 | [mlat][i]Configuring UDP connection udp://mlat-1.fr24.com:19788
2015-08-05 08:34:51 | [mlat][i]Registering MLAT station
2015-08-05 08:34:51 | [mlat][i]Registering MLAT station: SUCCESS
2015-08-05 08:34:52 | [reader][i][0]Connecting to Generic receiver via (exe:///usr/lib/fr24/dump1090 --raw --mlat)
2015-08-05 08:34:52 | [reader][i][0]Connected to the receiver, authenticating
2015-08-05 08:34:52 | [reader][i][0]Authenticated, processing messages
2015-08-05 08:34:52 | [reader][i][0]Connection terminated
2015-08-05 08:34:52 | [main][i]Terminating child process 19526 with SIGETERM
2015-08-05 08:34:57 | [reader][i][0]Connecting to Generic receiver via (exe:///usr/lib/fr24/dump1090 --raw --mlat)
2015-08-05 08:34:57 | [reader][i][0]Connected to the receiver, authenticating
2015-08-05 08:34:57 | [reader][i][0]Authenticated, processing messages
2015-08-05 08:34:57 | [reader][i][0]Connection terminated
2015-08-05 08:34:57 | [main][i]Terminating child process 19527 with SIGETERM
2015-08-05 08:35:03 | [reader][i][0]Connecting to Generic receiver via (exe:///usr/lib/fr24/dump1090 --raw --mlat)
2015-08-05 08:35:03 | [reader][i][0]Connected to the receiver, authenticating
2015-08-05 08:35:03 | [reader][i][0]Authenticated, processing messages
2015-08-05 08:35:03 | [reader][i][0]Connection terminated
2015-08-05 08:35:03 | [main][i]Terminating child process 19528 with SIGETERM
2015-08-05 08:35:07 | [mlat][i]Pinging the server
2015-08-05 08:35:07 | [mlat][i]Stats 15675087/2339
2015-08-05 08:35:08 | [reader][i][0]Connecting to Generic receiver via (exe:///usr/lib/fr24/dump1090 --raw --mlat)
2015-08-05 08:35:08 | [reader][i][0]Connected to the receiver, authenticating
2015-08-05 08:35:08 | [reader][i][0]Authenticated, processing messages
2015-08-05 08:35:09 | [reader][i][0]Connection terminated
2015-08-05 08:35:09 | [main][i]Terminating child process 19529 with SIGETERM
2015-08-05 08:35:11 | [feed][n]connected via TCP
2015-08-05 08:35:11 | [feed][n]switching to UDP
2015-08-05 08:35:11 | [feed][n]working
2015-08-05 08:35:14 | [reader][i][0]Connecting to Generic receiver via (exe:///usr/lib/fr24/dump1090 --raw --mlat)

As a result, I did not get the RPI running. I have signed up at least 5 times with automatic / individual configuration, editing the fr24.ini file etc. Nothing changed.
The dump1090 Webinterface shows all adsb and non adsb aircraft as usuall. It seems to be a failure of connecting to fr24 server.

Can anybody help me to find a solution for the problem?

Thank you

problem has been solved !!

[bhaal]

Hi bhaal,

Thanks for the constructive criticism, but you described exactly the algorithm we've implemented here. There is no rely on NTP-only timestamps. We sync those timestamps across ADS-B traffic server-side, simultaneously visible between 3 Raspberry Pi receivers and 1 FR24 receiver that has exact precise GPS time aligned with Mode-S signal. Positions uncertainty is of course a problem, and we will definitely work on improving the quality. I understand your frustration about vendor-lock for dump1090 - that might be improved in future.
You asked for a link to sources: https://github.com/flightradar24/dump1090

Alright, I guess the whole NTP sync thing is what put me on the wrong track with the way the FR MLAT system worked, and as with what obj has asked.. Why modifying timestamps? As I have said before, if you want to modify timestamps do it at the server end rather than trying to reinvent the wheel, also the knock-on effect of doing it client side is as you have seen others mention: it increases cpu load dramatically.. I think the only way you are going to get more accurate MLAT data is to bring the future forward faster and move to non-vendor locked dump1090.. This will bring more ADS-B feeders into your MLAT circle..

[lcsneil]

If it is any help here is the start of my log

2015-07-30 18:50:05 | [main][i]FR24 Feeder/Decoder [0x02117000]
2015-07-30 18:50:05 | [main][i]Version: 1.0.14-5/generic
2015-07-30 18:50:05 | [main][i]Built on 20150717-1545 (r:master-e0237e1.git/Linux/armv7l)
2015-07-30 18:50:05 | [main][i]Copyright 2008-2015 (c) Piotr Pawluczuk
2015-07-30 18:50:05 | [main][i]Flightradar24 AB(http://flightradar24.com)
2015-07-30 18:50:05 | [main][i]DNS mode: LIBC
2015-07-30 18:50:06 | ERROR
2015-07-30 18:50:06 | [w]Detected --net argument for dump1090, disabling internal RAW feed!
2015-07-30 18:50:06 | [w]Detected --net argument for dump1090, disabling internal BS feed!
2015-07-30 18:50:06 | [httpd][i]Server started, listening on 0.0.0.0:8754
2015-07-30 18:50:06 | [main][i]Reader 0 started
2015-07-30 18:50:06 | [master][i]Starting processing thread
2015-07-30 18:50:06 | [main][i]MLAT data feed started
2015-07-30 18:50:06 | [reader][i][0]Initializing reader
2015-07-30 18:50:06 | [mlat][i]Waiting for MLAT configuration
2015-07-30 18:50:06 | [reader][i][0]Connecting to Generic receiver via (exe:///usr/lib/fr24/dump1090 --net --gain -10 --net-http-port 8081 --ppm 12 --raw --mlat)
2015-07-30 18:50:06 | [reader][i][0]Connected to the receiver, authenticating
2015-07-30 18:50:06 | [reader][i][0]Authenticated, processing messages
2015-07-30 18:50:06 | [reader][w][0]Setting new UTC offset: 0!
2015-07-30 18:50:07 | [time][i]Synchronizing time via NTP
2015-07-30 18:50:15 | [time][i]Time synchronized correctly, offset -0.0001 seconds
2015-07-30 18:50:15 | [main][i]Feed Network client started
2015-07-30 18:50:15 | [feed][i]Downloading configuration

What parameters are you passing to dump1090. Maybe that is your issue (note I dont use the standard port hence the 8081).

But my parameters (in etc\fr24feed.ini) are

receiver="dvbt"
fr24key="your own sharing key goes here"
path="/usr/lib/fr24/dump1090"
bs="yes"
raw="yes"
logmode="2"
procargs="--net --gain -10 --net-http-port 8081 --ppm 12"
mlat="yes"
mlat-without-gps="yes"

lcs