Announcement

Collapse
No announcement yet.

dump1090 collecting messages; flightradar24.com reporting Online - No data

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • bimmerdriver
    replied
    Since my last post, fr24feed has been fairly consistently stopping in the late evening and restarting in the early afternoon the next day (local time). It's down for typically around 11-13 hours per day, with few exceptions.

    For example, no uploads from 0800-2000, on 2/29 and 3/1, 0800-2100, on 3/2, 0700-1100, with another brief outage around 2100, on 3/3, 0900-2000, on 3/4, 0800-1800, on 3/5.

    During the time it's not uploading, it's collecting data, which I can see using graphs1090.

    I reported this to support. They suggested switching to beast-mode and disabling IPv6, neither of which made any difference. Then they said it might have something to do with there being fewer messages starting from around 2200-2300 local time, due to reduced air traffic, but the number of messages increases again at 0500-0600 the next day, when the air traffic resumes, so if it was only due to the number of messages, it should only be not uploading for 5-6 hours, not double that, although, since fr24feed does connect, it should be able to resume uploading once there is more traffic.

    Even when fr24feed isn't uploading, it says it's tracking and uploading aircraft, which I find points to a problem. If it's not uploading aircraft, it should not be reporting that it's uploading aircraft.

    When it's not uploading, I've tried restarting fr24feed, restarting dump1090 and fr24feed and even rebooting the VM. It doesn't appear that any of those actions consistently make any difference. It stops uploading and resumes uploading for no apparent reason, with no indicative messages in the log.

    I'm unconvinced this is caused by WSL or because I'm not running the software on an RPi, because the pattern is so consistent, so I decided to install PiAware and try feeding to FlightAware to see if it would work properly.

    Yesterday afternoon, I installed the software and within minutes, it was working, including MLAT. It has worked continuously since. This makes me more convinced that the problem lies with fr24feed and the FlightRadar server. I'm going to keep trying to get support to investigate what's happening, but I don't have a lot of confidence they will do anything. My impression is that they know there is a problem, but they can't be bothered to investigate it. I will report back if there are any developments.

    Leave a comment:


  • bimmerdriver
    replied
    And just like that, it started working again:

    Code:
    eb 22 12:05:31 kaveri fr24feed[133332]: [feed][n]syncing stream: timeout
    Feb 22 12:05:32 kaveri fr24feed[133332]: [feed][n]disconnected
    Feb 22 12:05:32 kaveri fr24feed[133332]: [I] [feed][n]waiting 10 seconds
    Feb 22 12:05:42 kaveri fr24feed[133332]: [feed][n]CYNJ45@185.218.24.24:8099/UDP
    Feb 22 12:05:42 kaveri fr24feed[133332]: [feed][n]connecting
    Feb 22 12:05:42 kaveri fr24feed[133332]: [I] Network thread connecting to 185.218.24.24:8099 for feed CYNJ45
    Feb 22 12:05:43 kaveri fr24feed[133332]: [feed][n]connected via UDP (fd 20)
    Feb 22 12:05:43 kaveri fr24feed[133332]: [feed][n]working
    Feb 22 12:06:09 kaveri fr24feed[133332]: [I] [stats]sent 16 bytes
    Feb 22 12:07:17 kaveri fr24feed[133332]: [feed][n]syncing stream async: 1
    Feb 22 12:07:18 kaveri fr24feed[133332]: [feed][n]syncing stream result: 1
    ...
    Feb 22 12:15:04 kaveri fr24feed[133332]: [feed][n]syncing stream: timeout
    Feb 22 12:16:10 kaveri fr24feed[133332]: [I] [stats]sent 16 bytes
    Feb 22 12:16:22 kaveri fr24feed[133332]: [time][i]Synchronizing time via NTP
    Feb 22 12:16:22 kaveri fr24feed[133332]: [time][e]Time synchronized, offset -37.374 seconds, drift -0.007 seconds/minute
    Feb 22 12:16:37 kaveri fr24feed[133332]: [feed][n]syncing stream async: 1
    Feb 22 12:16:13 kaveri fr24feed[133332]: [time][e]Time jumped backward by -37.38s, resynchronizing
    Feb 22 12:16:13 kaveri fr24feed[133332]: [time][i]Synchronizing time via NTP
    Feb 22 12:16:13 kaveri fr24feed[133332]: [time][e]Time synchronized, offset -0.005 seconds, drift +0.463 seconds/minute
    Feb 22 12:16:16 kaveri fr24feed[133332]: [feed][i]sent 4,0 AC
    ...
    Feb 22 12:16:44 kaveri fr24feed[133332]: [feed][i]sent 4,0 AC
    Feb 22 12:16:45 kaveri fr24feed[133332]: [feed][n]syncing stream: timeout
    Feb 22 12:16:46 kaveri fr24feed[133332]: [feed][n]disconnected
    Feb 22 12:16:46 kaveri fr24feed[133332]: [I] [feed][n]waiting 15 seconds
    Feb 22 12:17:01 kaveri fr24feed[133332]: [feed][n]CYNJ45@185.218.24.22:8099/UDP
    Feb 22 12:17:01 kaveri fr24feed[133332]: [I] Network thread connecting to 185.218.24.22:8099 for feed CYNJ45
    Feb 22 12:17:01 kaveri fr24feed[133332]: [feed][n]connecting
    Feb 22 12:17:01 kaveri fr24feed[133332]: [feed][n]connected via UDP (fd 20)
    Feb 22 12:17:01 kaveri fr24feed[133332]: [feed][n]working
    Feb 22 12:17:01 kaveri fr24feed[133332]: [feed][i]sent 1,0 AC
    ...
    Feb 22 12:17:17 kaveri fr24feed[133332]: [feed][i]sent 5,0 AC
    Feb 22 12:11:58 kaveri fr24feed[133332]: [feed][n]syncing stream async: 1
    Feb 22 12:12:43 kaveri fr24feed[133332]: [feed][n]syncing stream: timeout
    ...
    Feb 22 12:16:10 kaveri fr24feed[133332]: [I] [stats]sent 16 bytes
    Feb 22 12:16:22 kaveri fr24feed[133332]: [time][i]Synchronizing time via NTP
    Feb 22 12:16:22 kaveri fr24feed[133332]: [time][e]Time synchronized, offset -37.374 seconds, drift -0.007 seconds/minute
    Feb 22 12:16:37 kaveri fr24feed[133332]: [feed][n]syncing stream async: 1
    Feb 22 12:16:13 kaveri fr24feed[133332]: [time][e]Time jumped backward by -37.38s, resynchronizing
    Feb 22 12:16:13 kaveri fr24feed[133332]: [time][i]Synchronizing time via NTP
    Feb 22 12:16:13 kaveri fr24feed[133332]: [time][e]Time synchronized, offset -0.005 seconds, drift +0.463 seconds/minute
    Feb 22 12:16:16 kaveri fr24feed[133332]: [feed][i]sent 4,0 AC
    Feb 22 12:16:21 kaveri fr24feed[133332]: [feed][i]sent 4,0 AC
    Feb 22 12:16:27 kaveri fr24feed[133332]: [feed][i]sent 4,0 AC​

    Leave a comment:


  • bimmerdriver
    replied
    Overnight, fr24feed quit feeding, as has been the practice.

    I created a rule in my main firewall to log traffic to 185.218.24.22:8099, 185.218.24.23:8099 and 185.218.24.24:8099. I'm seeing some traffic, but due to the time stamps not being consistent, it's not possible to directly correlate messages in the fr24feed log with the firewall log. My next step will be to configure a firewall in linux to log the packets, but since they don't have the same port number, I have to figure out what socket to monitor.

    As far as my main firewall is concerned, it can see the traffic.

    I also installed graphs1090 and it's showing that my system is continuously receiving messages. The big mystery is why fr24feed is not sending them.

    Leave a comment:


  • bimmerdriver
    replied
    It's been running since this afternoon and I'm trying to monitor the network traffic so I can see if it disappears when it stops working.

    Running netstat -aucpv, I get the following traffic:

    Code:
    Active Internet connections (servers and established)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
    udp        0      0 localhost:323           0.0.0.0:*                           -
    udp        0      0 0.0.0.0:33976           0.0.0.0:*                           25944/fr24feed
    udp        0      0 127.0.0.53:domain       0.0.0.0:*                           97/systemd-resolved
    udp6       0      0 ip6-localhost:323       [::]:*                              -​
    The only item that seems possible is this one:

    Code:
    udp        0      0 0.0.0.0:33976           0.0.0.0:*                           25944/fr24feed
    I tried lsof -i udp and I get the following traffic:

    Code:
    COMMAND     PID            USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
    systemd-r    97 systemd-resolve   13u  IPv4    19561      0t0  UDP 127.0.0.53:domain
    fr24feed  25944            fr24   20u  IPv4 48296958      0t0  UDP *:33976​
    This confirms what I got from netstat.

    I was expecting to find udp traffic to port 8099, because of the log:

    Code:
    Feb 18 14:59:47 kaveri fr24feed[722]: [I] Network thread connecting to 185.218.24.23:8099 for feed ****
    Does anyone know if the address in the log is the actual address to which the datagrams are being sent?
    Last edited by bimmerdriver; 2024-02-22, 05:05.

    Leave a comment:


  • bimmerdriver
    replied
    Originally posted by abcd567 View Post
    Code:
    sudo journalctl -u fr24feed > fr24feed.log
    
    ## OR
    
    sudo journalctl -u fr24feed >> fr24feed.log
    The “>” is the output redirection operator which will overwrite file fr24feed.log

    The “>>” is the output redirection operator as well, but it appends (i.e. adds to existing data) of file fr24feed.log
    Thanks for the suggestion. I forgot about that.

    Code:
    $ journalctl -xeu fr24feed | wc -l
    1000​
    It's not the entire log, but it gives me a 10X greater opportunity to capture some useful information.

    Leave a comment:


  • abcd567
    replied
    Code:
    sudo journalctl -u fr24feed > fr24feed.log
    
    ## OR
    
    sudo journalctl -u fr24feed >> fr24feed.log
    The “>” is the output redirection operator which will overwrite file fr24feed.log

    The “>>” is the output redirection operator as well, but it appends (i.e. adds to existing data) of file fr24feed.log

    Leave a comment:


  • bimmerdriver
    replied
    Since the last time I posted, it worked for around several hours yesterday, then by this morning, it had stopped again. Everything else shows no sign of a problem.

    Since fr24feed says it's connected using UDP, I'm sceptical that there is anything wrong with UDP. If there was an error when the datagram was sent, it should be in the log. If there wasn't an error there should be a corresponding message in the log. Since fr24feed-status is reporting messages, where are they going?

    Of course, I have never been able to capture the fr24feed log at the time it stops or starts.

    I tried adding the following into fr24feed.ini and restarting but it made no difference.

    Code:
    logmode="1"
    logpath="#var#log#fr24feed"
    How do I get the output from fr24feed to be logged into a file?
    Last edited by bimmerdriver; 2024-02-21, 21:45.

    Leave a comment:


  • bimmerdriver
    replied
    Originally posted by abcd567 View Post
    For your infotmation, I have installed "Oracle Virtual Machine" on Windows 11 computer, and have installed on it Debian 12, Debian 13, Ubuntu 22 and Ubuntu 24,. All these OS are amd64

    ​​​​​On all these OS, I have installed dump1090-fa, fr24feed, and piaware. On all these I often face problem that the dongle is not passed-through (or improperly passwd-through) from Windows to Linux. As a result dump1090-fa does not produce any data.

    When this occurs, I fix the issue by ejecting the dongle, shuting down Linux in VM, replug dongle, and restart the Linux OS.
    If you're referring to Virtual Box, I used to use it before the hyper-v environment became available and I encountered the same problem. It works very well, except it has more overhead than hyper-v and especially compared to WSL. It's different from WSL and hyper-v, because they don't have native USB support, which is a long-standing gripe that people have with both environments. Support for USB comes from usbipd-win, which is open-source. It works fine, except that it doesn't automatically start upon boot, which causes problems with dump1090-* and fr24feed, because they expect the receiver to be online when they start. I have to manually connect the USB, then restart the software. Generally that works, but since I installed dump1090-fa, fr24feed has never worked properly after a restart. It always takes several hours or more before it spontaneously starts working, even though dump1090-fa is working. (My system is working again, but I haven't been able to capture the log either when it starts working or when it quits working, so I have no idea what it is causing the problem.)

    Leave a comment:


  • abcd567
    replied
    For your infotmation, I have installed "Oracle Virtual Machine" on Windows 11 computer, and have installed on it Debian 12, Debian 13, Ubuntu 22 and Ubuntu 24,. All these OS are amd64

    ​​​​​On all these OS, I have installed dump1090-fa, fr24feed, and piaware. On all these I often face problem that the dongle is not passed-through (or improperly passwd-through) from Windows to Linux. As a result dump1090-fa does not produce any data.

    When this occurs, I fix the issue by ejecting the dongle, shuting down Linux in VM, replug dongle, and restart the Linux OS.

    Leave a comment:


  • bimmerdriver
    replied
    Originally posted by abcd567 View Post

    Please note Oblivian is NOT Flightradar24 Staff. He is member of feeder community like you. He is not bound to provide you help or guidance. If he tries to help you, it is purely voluntary. Your such remarks about a person voluntarily trying to help you will result in him and others stop providing voluntary help.
    I'm trying to get this working on WSL (and eventually on Hyper-V) because I don't want be forced to purchase an RPi so I can feed data to a for-profit company, especially when I already have enough (too many) computers. (My son used to use this receiver and antenna on Windows, before support for that platform was discontinued, so I know the hardware works.) I understand others would also like to be able to feed data without being forced to purchase an RPi, so if I can get this working, it will be for the benefit of the community. I'm aware that others have gotten this working using docker, so I disagree with the idea that that it won't work in principle. Rather, I think it's not working for a reason and I'm trying to find out what the reason is.

    I understand Oblivian is a volunteer and he is not obligated to help. It's his prerogative to help or not. Based on his response to my post, he apparently doesn't like being called out for making erroneous statements. That says more about him than me.​

    Leave a comment:


  • abcd567
    replied
    Originally posted by bimmerdriver View Post

    Are you kidding?!?

    You make this claim about millisecond message rate and you're misreading the timestamps. Why not admit you made a mistake instead of acting like a child?

    You didn't respond to a single thing I said in response to your previous posts.
    Please note Oblivian is NOT Flightradar24 Staff. He is member of feeder community like you. He is not bound to provide you help or guidance. If he tries to help you, it is purely voluntary. Your such remarks about a person voluntarily trying to help you will result in him and others stop providing voluntary help.

    Leave a comment:


  • bimmerdriver
    replied
    Originally posted by Oblivian View Post
    I don't know what you expect. You are claiming I don't know what I'm talking about. And that you have apparently reported issues.

    Yet back them up showing you don't have any.

    You're feeding..or you aren't.

    It obviously is. Albeit chunked/badly if you're not happy with it. With no indication why other than you're the only one using WSL layers

    On your on.. I'm out.
    ​​​​​
    Are you kidding?!?

    You make this claim about millisecond message rate and you're misreading the timestamps. Why not admit you made a mistake instead of acting like a child?

    You didn't respond to a single thing I said in response to your previous posts.

    Leave a comment:


  • Oblivian
    replied
    I don't know what you expect. You are claiming I don't know what I'm talking about. And that you have apparently reported issues.

    Yet back them up showing you don't have any.

    You're feeding..or you aren't.

    It obviously is. Albeit chunked/badly if you're not happy with it. With no indication why other than you're the only one using WSL layers

    On your on.. I'm out.
    ​​​​​

    Leave a comment:


  • bimmerdriver
    replied
    Originally posted by Oblivian View Post
    HTML Code:
    2024-02-20 21:35:15 | [mlat][i]Stats 7593710/7593710
    2024-02-20 21:35:16 | [feed][i]sent 2,0 AC
    2024-02-20 21:35:16 | 24-02-20 21:35:16.909 [D][:] sent 2 of 2 aircraft, 0 ignored, 0 outdated
    2024-02-20 21:35:24 | [feed][i]sent 2,0 AC
    2024-02-20 21:35:24 | 24-02-20 21:35:24.249 [D][:] sent 2 of 2 aircraft, 0 ignored, 0 outdated
    2024-02-20 21:35:26 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:35:29 | [feed][i]sent 1,0 AC
    2024-02-20 21:35:29 | 24-02-20 21:35:29.309 [D][:] sent 1 of 1 aircraft, 0 ignored, 0 outdated
    2024-02-20 21:35:35 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:35:35 | [mlat][i]Pinging the server
    2024-02-20 21:35:35 | [mlat][i]Stats 7593732/22
    2024-02-20 21:35:39 | [feed][i]sent 2,0 AC
    2024-02-20 21:35:39 | 24-02-20 21:35:39.279 [D][:] sent 2 of 2 aircraft, 0 ignored, 0 outdated
    2024-02-20 21:35:46 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:35:52 | [feed][i]sent 1,0 AC
    2024-02-20 21:35:52 | 24-02-20 21:35:52.769 [D][:] sent 1 of 1 aircraft, 0 ignored, 0 outdated
    2024-02-20 21:35:55 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:35:55 | [mlat][i]Pinging the server
    2024-02-20 21:35:55 | [mlat][i]Stats 7593732/0
    2024-02-20 21:36:00 | [feed][i]sent 1,0 AC
    2024-02-20 21:36:00 | 24-02-20 21:36:00.369 [D][:] sent 1 of 1 aircraft, 0 ignored, 0 outdated
    2024-02-20 21:36:01 | [feed][i]filtering out 1 overlapping AC (saving bandwidth)
    2024-02-20 21:36:06 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:36:06 | [feed][i]sent 0,1 AC
    2024-02-20 21:36:06 | 24-02-20 21:36:06.629 [D][:] sent 0 of 1 aircraft, 1 ignored, 0 outdated
    
    2024-02-20 21:36:19 | [feed][i]sent 0,1 AC
    2024-02-20 21:36:19 | 24-02-20 21:36:19.309 [D][:] sent 0 of 1 aircraft, 1 ignored, 0 outdated
    2024-02-20 21:36:26 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:36:35 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:36:35 | [mlat][i]Pinging the server
    2024-02-20 21:36:35 | [mlat][i]Stats 7593732/0
    2024-02-20 21:36:41 | [feed][i]sent 0,1 AC
    2024-02-20 21:36:41 | [feed][n]syncing stream async: 1
    2024-02-20 21:36:41 | 24-02-20 21:36:41.639 [D][:] sent 0 of 1 aircraft, 1 ignored, 0 outdated
    2024-02-20 21:36:42 | [feed][n]syncing stream result: 1
    2024-02-20 21:36:46 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:36:55 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:36:55 | [mlat][i]Pinging the server
    2024-02-20 21:36:55 | [mlat][i]Stats 7593732/0
    2024-02-20 21:36:56 | [feed][i]sent 0,1 AC
    2024-02-20 21:36:56 | 24-02-20 21:36:56.899 [D][:] sent 0 of 1 aircraft, 1 ignored, 0 outdated
    2024-02-20 21:37:04 | [feed][i]sent 0,1 AC
    2024-02-20 21:37:04 | 24-02-20 21:37:04.229 [D][:] sent 0 of 1 aircraft, 1 ignored, 0 outdated
    2024-02-20 21:37:05 | [feed][i]filtering out 1 overlapping AC (saving bandwidth)
    2024-02-20 21:37:06 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:37:15 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:37:15 | [mlat][i]Pinging the server
    2024-02-20 21:37:15 | [mlat][i]Stats 7593732/0
    2024-02-20 21:37:26 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:37:34 | [feed][n]ping 1
    2024-02-20 21:37:35 | [feed][n]syncing stream result: 1
    2024-02-20 21:37:35 | [mlat][i]No ADS-B time reference available (0/0)
    2024-02-20 21:37:35 | [mlat][i]Pinging the server
    2024-02-20 21:37:35 | [mlat][i]Stats 7593732/0
    milisecond updates. (even when there's only 2 out there this time of evening)
    Sorry, but those aren't millisecond updates. You are misreading the time stamps.

    This "2024-02-20 21:35:16" corresponds to YYYY-MM-DD HH:MM:SS.​

    If you ignore the MLAT messages and refer to the [feed][i]sent messages, you are seeing the same rate of messages that I'm seeing, which are multiple seconds apart.

    I would be interested to see your event log with MLAT turned off, so it's possible to see how many of the messages are caused by MLAT.

    For example, is this message caused by MLAT?

    2024-02-20 21:37:04 | 24-02-20 21:37:04.229 [D][:] sent 0 of 1 aircraft, 1 ignored, 0 outdated​

    Leave a comment:


  • bimmerdriver
    replied
    Originally posted by Oblivian View Post
    You still don't have direct connection to the hardware though. Unlike a pi that interfaces directly.

    It's host system to subsystem to software.

    both for the stick, and for data.

    Dump1090 needs to talk directly to the usb interface as a data stream. This is why you see people running rtl_test and losing data or the stick samplerate is effected when on extension cables or if the CPU or ram gets loaded.

    I'm seeing many guides and attempts online that come to the same conclusion. The hoops you jump through to talk to the usb for it's unreliability in WSL doesn't appear worth it.

    Aircraft are sending data at a rate of 2msg/s each. You should be seeing updates every few seconds. Not per minutes. Which makes me think its dumping data between the interface. And I'm pretty sure that 0sync should be filled with more than 0
    With all due respect, there is no such thing as "direct connection to the hardware". When an application sends a message over a socket, it goes through layers of software. Even device drivers are layered.
    tcp.gif
    UDP is a fundamental part of the IP protocol stack. If it wasn't working, the computer would be dead in the water. I wouldn't even be able open a window on it. The flightradar24 website shows that my receiver is online, so clearly the software is talking with the server(s).

    Also, when I compare the live view from flightrader24.com with the view from skyaware, they are consistent. As I said, my antenna is small and it's in a poor location. Not only is my house located on a hill, my neighbour's house is blocking the view of the sky except for about 90 degrees. When/if I get this working reliably, I will get a better antenna, but it's definitely working.

    Also, as I said, yesterday, my receiver received over a million messages. When I look at the skyaware display, it shows that it's receiving messages at much higher rate than what appears in the log.

    Leave a comment:

Working...
X