ModesDeco2 or ModesMixer2 with PiAware - anybody tried this successfully

[EDDG]

Hello,

at the moment i`m playing with ModeSDeco2 & ModesMixer2 - i want to feed the data to flightaware via piaware, but i can find a way to get it feeding data...!? Anybody here with a successfully solution?!

I´m happy for every little help into the right way !

Best regards,
Peter

[Anmer]

at the moment i`m playing with ModeSDeco2 & ModesMixer2 - i want to feed the data to flightaware via piaware

The PiAware app feeds data decoded by dump1090 running on a Raspberry Pi.  FlightAware also has a combined Dump1090 and feed package to install on a Raspberry Pi

As far as I know, there is no stand alone feeder app for FlightAware.  You can either configure direct feeds from Radarcapes and SBS-3s or setup a secondary one using PlanePlotter.

But I may be wrong.

[sergsero]

Hello,

If you use the src code of FlightAware programs, they can be compiled for any Unix platform: https://github.com/flightaware/piaware
I have it working in Ubuntu on x86.

With the option --outServer fatsv:10001 modesmixer2 outputs data in the same incremental format as FlightAware version of dump1090 MR.

Unfortunately, piaware sender uses in the system port 10001 only from fork dump1090 MR. And does not perceive the same data from the same port from other programs. It waits middleware faup1090.

I asked FlightAware helpdesk wondering whether them, that modesmixer2 has opportunity to send data. But they were not interested. And it was left unfinished.

Best regards,
sergsero

[EDDG]

Hey sergsero,

thanks for your answer.

Best regards,
Peter

[dschaper]

I think I may have PiAware working with ModeSMixer. PiAware-status checks for activity on both ports 30005 and 10001, so I set up Mixer to output beast on 30005 and fatsv on 10001 and verified on my FlightAware account that data was being fed and flights were tracked.

Code Select Expand

./modesmixer2 --inConnect 127.0.0.1:31001 --outServer fatsv:10001 --outServer beast:30005 --log-level 4

And for reference, here is my deco command line

Code Select Expand

./modesdeco2 --gain 49.6 --freq-correction 18 --rbs --location ##:## --beast 31001 --avrmlat 33003 --msg 7777 --web 8088

And my page at FA shows good activity http://flightaware.com/adsb/stats/user/dschaper

Code Select Expand

[2015-05-19 21:31 PDT] ADS-B data program 'modesmixer2' is listening on port 30005, so far so good
[2015-05-19 21:31 PDT] i see modesmixer2 serving on port 10001
[2015-05-19 21:31 PDT] connecting to modesmixer2 on port 10001...
[2015-05-19 21:31 PDT] piaware is connected to modesmixer2 on port 10001
[2015-05-19 21:31 PDT] modesmixer2 is listening for connections on FA-style port 10001
[2015-05-19 21:32 PDT] 14 msgs recv'd from modesmixer2; 14 msgs sent to FlightAware

[grezz]

I confirm that piaware works fine with modesmixer2 (without dump1090) if you have beast output on port 30005. The release notes for piaware v 1.9 (a few versions back) state "piaware now figures out whatever program is serving beast data on port 30005 and is cool with it".

I've been using modesmixer2 to read directly from my Mode-s beast and output to port 30005 for piaware for months now, and never had any issues.

[dfroula]

Piaware has recently added the capability of doing MLAT processing, I decided to see if I could get this working with my modesmixer2 setup that is receiving serial over USB data from a Mode-S Beast receiver rather than than a dongle using dump1090. That setup is working very well, feeding data to the PlanePlotter servers via modesmixer2 and ppup1090 on port 30005.

The basic flow of the new Piaware MLAT system is here:

http://flightaware.com/adsb/piaware/about

My current modesmixer2 invocation is with:

Code Select Expand

modesmixer2 --inSerial /dev/ttyUSB0:3000000:hardware --outServer beast:30005 --web 8080 --location yy.yyyyyy:-xx.xxxxxx

This simply receives serial data (already decoded) in Beast binary format on USB serial port ttyUSB0 and translates it to an output on port 30005. I also start the web server and provide my location for modesmixer web page plotting of my coverage and stats and for more efficient position decoding.

Piaware used to have problems receiving data for normal position reporting on port 30005 from modesmixer2, even though that was the same port used by the expected dump1090 decoder application. However, that issue seems to have been resolved with the latest MLAT-capable version of Piaware. piaware_2.1-2_armhf.deb.

Here are the relevant piaware log messages showing things are starting up just fine with modesmixer2:

Code Select Expand

07/27/2015 01:36:41 ADS-B data program 'modesmixer2' is listening on port 30005, so far so good
07/27/2015 01:36:41 Starting faup1090: /usr/bin/faup1090 --net-bo-ipaddr localhost --net-bo-port 30005 --lat 41.911 --lon -88.357
07/27/2015 01:36:41 Started faup1090 (pid 2311) to connect to modesmixer2
07/27/2015 01:36:41 logged in to FlightAware as user donf99
07/27/2015 01:36:41 multilateration support enabled (use piaware-config to disable)
07/27/2015 01:36:41 multilateration data requested, enabling mlat client
07/27/2015 01:36:41 Starting multilateration client: /usr/lib/fa-mlat-client/fa-mlat-client --input-connect localhost:30005 --udp-transport 70.42.6.203:6288:4111205693
07/27/2015 01:36:45 piaware received a message from modesmixer2!
07/27/2015 01:36:45 piaware has successfully sent several msgs to FlightAware!
07/27/2015 01:36:46 mlat(2313): fa-mlat-client 0.2.3 starting up
07/27/2015 01:36:46 mlat(2313): Using UDP transport to 70.42.6.203:6288
07/27/2015 01:36:46 mlat(2313): Input connected to localhost:30005
07/27/2015 01:37:11 141 msgs recv'd from modesmixer2; 141 msgs sent to FlightAware
07/27/2015 01:37:59 server is sending alive messages; we will expect them
07/27/2015 01:42:11 1972 msgs recv'd from modesmixer2 (1831 in last 5m); 1972 msgs sent to FlightAware
07/27/2015 01:47:11 3874 msgs recv'd from modesmixer2 (1902 in last 5m); 3874 msgs sent to FlightAware
07/27/2015 01:51:46 mlat(2313): Receiver connection: ready
07/27/2015 01:51:46 mlat(2313): Server connection:   ready
07/27/2015 01:51:46 mlat(2313): Receiver:  942.3 msg/s received     15.7kB/s from receiver
07/27/2015 01:51:46 mlat(2313): Server:      2.7 kB/s from server    0.0kB/s TCP to server  10.0kB/s UDP to server
07/27/2015 01:51:46 mlat(2313): Results:  1259.9 positions/minute
07/27/2015 01:51:46 mlat(2313): Aircraft: 140 known, 132 requested by server
07/27/2015 01:52:11 5746 msgs recv'd from modesmixer2 (1872 in last 5m); 5746 msgs sent to FlightAware

If you examine the FlightAware flow disgram, you will see that normal sharing data is received from modesmixer2 on port 30005, the same port used by PlanePlotter and ppup1090 for autonomous uploading to PlanePlotter servers. However, Piaware also has a back-channel on port 30004 that Piaware normally uses to send MLAT positions back to input port 30004 on dump1090 in Beast binary format. Dump1090 can display the MLAT positions on its own web server with distinct color coding.

MLAT on Piaware can be enabled with or with out the backchannel for reporting Piawares own MLAT positions with the configuration command:

piaware-config -mlat 0/1 (enables/disables MLAT processing. Starts the MLAT process if request received from Flightaware servers.)

piaware-config -mlatResults 0/1 (enables/disables sendin MLAT position data on port 30004.)

piaware-config -mlatResultsFormat formatlist (Poorly documented. Seems to allow afjusting MLAT data on 30004 in other than Beat binary.)

I tried enabling reception of the MLAT data in modesmixer 2 by defining an additional input with this invocation:

Code Select Expand

modesmixer2 --inSerial /dev/ttyUSB0:3000000:hardware --inConnect 127.0.0.1:30004 --outServer beast:30005 --web 8080 --location yy.yyyyyy:-xx.xxxxxx

However, when I enable the MLAT back channel on Piaware, I receive port 30004 "Connction Refused" errors in both the modes2mixer web page log and the Piaware log.

It seems modesmixer2 should be able to detect the secondary position reports on port 30004, as they are in the common Beast binary format, according to documentation on Piaware.

Has anyone come up with a way of combining the Piaware MLAT positions in the output stream of modesmixer2?

Thanks and regards,

Don
WD9DMP

[dfroula]

Well,  "--inServer 30004" did the trick! "--inConnect" was the incorrect modesmixer2 switch.

However, there were some undesirable side-effects.

After setting up the reception of MLAT positions from Piaware on modesmixer2, port 30004, the data was being merged with the serial input from my Mode-S Beast receiver. I viewed this data on PlanePlotter via TCP port 30005 and saw an explosion of additional planes from Piaware MLAT! Apparently, the MLAT system is working quite well. Interestingly, the new planes actually show a signal level in PlanePlotter, even though the positions are not being received by an actual receiver. I have no idea why. MLAT planes were identifiable by the somewhat irregular flight path as plotted by PlanePlotter.

What I am concerned about is that the intermingling of MLAT data with Beast receiver data on port 30005 (which is also feeding ppup1090 to the PlanePlotter servers and providing PlanePlotter MLAT services) will likely make the combined data stream unusable for PlanePlotter MLAT fixes, and perhaps normal position reports as well. I suspect the time tags from the Beast are being completely mangled by the MLAT position reports from Piaware. Although Piaware can filter out its own MLAT data from 30005, ppup1090 has no way of doing so.

So, I returned to my original settings, which provide MLAT service to FlightAware and Planeplotter without corrupting the raw datastream from the Mode-S Beast receiver.

An interesting experiment!

Best regards,

Don
WD9DMP

[dschaper]

PiAware MLAT messages are purposefully altered with a 'magic' timecode that identifies them as MLAT messages to the receivers. The decoder (modesmixer) would need to be able to identify these packets and then display them, but not forward them on to any output ports. (PFClient from PlaneFinder in it's latest beta is set to understand what packets are MLAT's and it silently discards them. Other services are not set up that way yet, if ever.) That's why the ability to disable forwarding was added to dump1090, it was previously sending the MLAT's out on all ports as you were doing with SMixer. So until SMixer is able to handle display MLAT and discard, you would be feeding false data. VRS can handle MLAT streams but that is a work in progress with AVR and Basestation formats.

The MLAT packet format is all opensourced on github, but I don't have the technical capability to get that information to sergsero with any kind of usefulness. If there is interest obj (Oliver - Mutability) who did the works can probably explain the details, it appears rather trivial as PF was able to implement it rather quickly.

[dfroula]

That makes perfect sense, thank-you!

Leaving it set as I had would likely incur the wrath of the PlanePlotter admins, as I would have been feeding FlightAware MLAT positions as normal sharing uploads to the PP servers, as if they were from my own receiver. I can't imagine that would be looked on with favor.

It would be nice if modesmixer2 had the option to selectively decode and disable incoming MLAT pseudo-positions from appearing on the output. Then there would be a way of at least seeing the MLAT traffic in modesmixer2's map display without corrupting the output datastream with unwanted MLAT messages.

Right now, there is no way to see the MLAT positions other than in FlightAware's web pages, at least not while supplying data to PlanePlotter's servers.

Regards,

Don
WD9DMP

[dfroula]

The "Magic MLAT timestamp" is defined here, FYI:

https://github.com/mutability/mlat-client/blob/master/mlat/client/output.py#L282

Don
WD9DMP

[dfroula]

This combination allows feeding the PlanePlotter servers via ppup1090 with normal sharing, as well as servicing MLAT requests, while allowing PlanePlotter to display local receiver positions, PlanePlotter normal sharing positions, PlanePlotter MLAT positions, and Piaware MLAT positions, without corrupting the data stream to ppup1090.

You MUST put PlanePlotter in download-only mode!!!!!!!! You then configure the PlanePlotter TCP connection to point to the output of the second modesmixer2 instance on port 30015. To see normal PlanePlotter only plots, point to port 30005 instead of 30015.

ppup1090 is fed from port 30005 with the stream only from the Beast receiver via the first modesmixer2 instance.

The modesmixer2 web interface displays the feed from the Beast receiver only. If you try to run the web server on the second modesmixer2 instance, it will not display the Piaware MLAT fixes, for some reason.

Quite an amazing display of normally invisible Mode-S planes and traffic on PlanePlotter!

Code:
modesmixer2 #1: modesmixer2 --inSerial /dev/ttyUSB0:3000000:hardware --web 8080  --outServer beast:30005
modesmixer2 #2: modesmixer2 --inConnect localhost:30005 --inServer 30004  --outServer beast:30015


This should not break anything, but strains the Pi v2 to the limit at 1300 Mode-S messages/sec. - near 95% CPU. Seems to work OK, though.

Regards,

Don
WD9DMP

[dfroula]

I was WAY off on my CPU estimates of the rather complicated setup. I was misreading the "top" command output. CPU utilization is peaking at only 25%, 75% idle time!

That is with two instances of modesmixer2, piaware running full bidirectional MLAT, ppup1090, and modesmixer2 web interface with map display running. Oh, also a data feed to PlanePlotter active.

Very nice.

Don
WD9DMP

[EGPD1701]

Well,  "--inServer 30004" did the trick! "--inConnect" was the incorrect modesmixer2 switch.

How did you get the "inServer 30004" to work, very time I try that it throws

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> >'
  what():  bind: Permission denied

And I have another question - how do you keep the connection when there is no data.  It times out and disconnects after a while not seeing anything.

[dfroula]

I have not had the problem with the exception error. I have an /etc/init.d startup script that starts three processes (two modesmixer2 and ppup1090) in sequence, with a 5 second delay between each invocation. These are the commands I use in the script to start each:

modesmixer2 --inSerial /dev/ttyUSB0:3000000:hardware --outServer beast:30005

ppup1090 --quiet --net-pp-ipaddr 192.168.1.130

modesmixer2 --inConnect localhost:30005 --inServer 30004  --outServer beast:30015 --web 8080 --location 41.000000:-88.000000

Piaware starts asynchronously, but connects up fine with both instances of modesmixer2 after a few seconds.

I am using the latest test version of modesmixer2 for the Rpi 2.

I live near Chicago O'Hare airport. With my antenna and preamp, there is ALWAYS incoming data, so timeouts have not been an issue. However, modesmixer2 connected to a Beast in this way will not generate keepalive data, so the Piaware connection may time out after no activity. I think it should reconnect when traffic resumes, but have not tested this.

Best,

Don
WD9DMP