ModeSDeco2 and ModeSMixer2 - console programs for RTLSDR and transcoding

[Conexia]

Hi, Sergsero
From yesterday my modeSmixer2 version v.20170506 is not working as before. I have modeSdeco2+modeSmixer2+PiAware MLAT+FR24Feeder in a Raspberry Pi 3 model B.

The left Highcharts graphics in the Chats page are missing. Also the planes footprint in the map and the distance rings are not shown.

I've cleared cache several times and tested it in Chrome versión 58.0.3029.110 and Microsoft Edge 38.14393.0.0

Maybe the error is in Highcharts.

Greetings


[Attachment deleted by Admin to save file space]

[shinesamuel]

Hi, Sergsero,

   i have tried running modesmixer2_rpi2-3_20170506, I am running it on Raspberry pi3 and its getting crushes.
checked the log seeing

May 31 18:12:18 kernel: [38915.750569] Out of memory: Kill process 14859 (modesmixer2) score 643 or sacrifice child
May 31 18:12:18  kernel: [38915.750600] Killed process 14859 (modesmixer2) total-vm:737732kB, anon-rss:693664kB, file-rss:0kB, shmem-rss:0kB

what is the solution for this error?

[sergsero]

Hi shinesamuel,

As I think, a size of the virtual memory that a process uses is listed as total-vm. Part of it is really mapped into the RAM itself as allocated and used. This is RSS.
Part of this RSS is allocated in real memory blocks and this is anonymous memory (anon-rss). There is also RSS memory blocks that are mapped into devices and files, are listed as file-rss.
Based on the above log, modesmixer2 requested memory allocation in total-vm:737732kB of 1 GB total in RPi3. And the kernel, which does not has such a memory, killed the process.

Please, specify if possible some additional information:

- which set of options you used in command line when starting the program?

- how fast the process was killed? The program works for some time or was stopped immediately after run?

- how many aircraft were at this time in observation?

- do you have connected a database files as basestation.sqb?

- what other programs are running on the same RPi3? It would be nice to see a output of the program top on console.

On the attached screenshot below the loading of RPi3 resources (VIRT MEM = 36812kB) when traffic handling of ~100 aircraft.
I'll additionally checking the lot of memory allocation (probably, never used) when the total-vm would be higher, but no real memory would be used. So the rss values would be low as in the your case.

Best regards,
sergsero

[Attachment deleted by Admin to save file space]

[shinesamuel]

Hi Sergsero

   which set of options you used in command line when starting the program?

/home/pi/XDECO/modesmixer2 --inConnect localhost:30003 --inConnect localhost:30005 --inConnect localhost:30002 --inConnect 192.168.XX.XX:30009 --inConnect 192.168.XX.XX:30015 --inConnect 192.168.XX.XX:30011 --outConnect beast:feed.adsbexchange.com:30005 --outConnect avr:feed.adsbexchange.com:30005 --outConnect avrmlat:feed.adsbexchange.com:30005 --outConnect avr:sdrsharp.com:47806 --outConnect beast:data.avdelphi.com:24999 --inConnect 192.168.XX.XX:30030 --inConnect 192.168.XX.XX:30014 --inConnect 192.168.XX.XX:30005 --outServer avr:30030 --outServer beast:31001 --globes 31008:table2:KRDU --outServer beast:31002 --location XXXXX:XXXXX --metric --web 8001 --outServer beast:30007 --outServer sbs10001:10001 --outServer msg:30014 --localtime --google-key XXXXXXXXXXXXXXXX --silhouettes /home/pi/XDECO/SilhouettesLogos --db /home/pi/XDECO/BaseStation.sqb --frdb /home/pi/XDECO/flightroute/flightroute_w.sqb --filter-expire 20 --filter-count 6 --filter-nocountry off --filter-ic off --filter-dup off --filter-time 60 --flight-expire-time 3000 --log-noconsole

how fast the process was killed? The program works for some time or was stopped immediately after run?


i have moved to different RPi2 and process run for a day and then its get killed.

do you have connected a database files as basestation.sqb?

no

what other programs are running on the same RPi3? It would be nice to see a output of the program top on console?

only dump1090-mut

i restarted the console program and seeing the memory is growing up

top - 10:09:04 up 2 days, 15:24,  1 user,  load average: 1.51, 1.64, 5.44
Tasks: 131 total,   3 running, 128 sleeping,   0 stopped,   0 zombie
%Cpu(s): 28.8 us,  8.3 sy,  0.0 ni, 62.0 id,  0.0 wa,  0.0 hi,  0.8 si,  0.0 st
KiB Mem:    994240 total,   228124 used,   766116 free,    15352 buffers
KiB Swap:   102396 total,    29604 used,    72792 free.    57012 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
5725 root      20   0  149040 112316   7116 R  99.7 11.3   9:39.47 modesmixer2
  480 dump1090  15  -5   28684   9088   1020 S  26.1  0.9   1044:14 dump1090-mutabi
13323 pi        20   0    5240   2588   2144 R   1.0  0.3   0:00.16 top
    7 root      20   0       0      0      0 R   0.3  0.0  11:15.46 rcu_sched
   23 root      rt   0       0      0      0 S   0.3  0.0   0:08.30 migration/3

Please help why the memory is growing up which is causing the console is being killed.




[Attachment deleted by Admin to save file space]

[sergsero]

Hi shinesamuel,

Thank you for your answers.
On your screenshot I see the maximum utilization of CPU, not memory. Probably that you are using the huge number of options for transforms data from one format to another, as well as sharing it via network and have reached the limit for RPi2/3 CPU.

It would be nice to know the answer to a question that was also - how many aircraft you observed in that moment? To estimate the amount of processed data traffic.

Also note that options --filter-nocountry and --filter-ic  do not have arguments and for options: --filter-expire, --filter-count and --filter-time you set their default values and not required. Although it shouldn't cause the such big CPU usage.
And you are using a database files: "--db /home/pi/XDECO/BaseStation.sqb --frdb /home/pi/XDECO/flightroute/flightroute_w.sqb"

If you have an possibility, check the same setup on a computer with Intel processor and Intel version of the program.

Regards,
sergsero

[shinesamuel]

Sergsero,

    how many aircraft you observed in that moment?

they were 133 aircraft at this moment.

[sergsero]

Shinesamuel,
thank you for helping.

It can be assumed that in this situation the number of aircraft does not affect the problem, because their numbers comparable with my tests. It remains to test multiple simultaneous conversion of message formats and input/output network's transactions.

I performed similar tests for the Intel platform. And even on outdated computers with CPU core2duo I don't have the problem.

I'll try to do this stress test for the ARM platform.

Best regards,
sergsero

[Conexia]

@sergsero
When you have a minute, would you be so kind to review my last messages about modeSmixer2 version v.20170506 in this same thread and The version of Modesmixer2 with a configuration file?

If you want me to check anything feel free to contact. I'm attaching a Stats screen.

Greetings

Antonio

[Attachment deleted by Admin to save file space]

[Barrow Wight]

Hello, I just startind using this software and made a couple of discoveries that lead to some questions I have.
My apologies if either of these questions have been asked before.

1. What is the status on updating the Raspberry Pi version to Raspbian stretch. It appears that libcrypto and libssl are now installed as versions 1.0.2 and 1.1, so this software will not work anymore without an update.

2. When I use either modesdeco or modessmixer, the map does not show all of the aircraft that other ads-b software displays. Furthermore, the chart that shows messages received is reporting a much lower number than the messages sent display on rtl1090.

[aldoir]

@sergsero,

I would like to run MM2 behind a HTTPS proxy, would you mind changing the script/references to href="//fonts.googleapis.com" and src="//code.highcharts.com" to let the browser choose the best protocol?

[exCzAF member]

Modesdeco2 latest issue....
have a problem : Database 14 error
My OS:  Win 7 64bit and

After start program shortly running "black window" and it is all.
Me tried run .bat file a here is a stop in dialog window " database 14"

What is a mean???
Thanks for advice and opinions.
Larry - ex CzAF member.
Czech republic - LKPD

[freemantp]

Hi Sergsero

I tried to run modesmixer2 behind nginx (as a reverse proxy). In my configuration ningx exposes a https interface and forwards all http requests to the web-interface of modesmixer2.
This basically works in older browser, however newer versions of chrome, firefox and edge don't display any charts or the map. I got to the bottom of this, and found out that these browsers don't like https and http content to be mixed : https://developer.mozilla.org/en-US/docs/Web/Security/Mixed_content

This is due to the fact that resources from these sources are not requested via https.
http://fonts.googleapis.com/
http://code.highcharts.com

Both are reachable via https, too, so i guess it would be an easy fix. I would have fixed this myself and sent you a pull request, but I guess the source code is not publicly available, right?

Cheers
freemantp

[sergsero]

Hi,

The version 20170924 of ModeSMixer2.

+ now resources from these sources http://fonts.googleapis.com/ and http://code.highcharts.com are requested via https.
+ fixed same bugs

Windows Intel x86: modesmixer2_windows_20170924.zip https://drive.google.com/open?id=0B7NYXizl0U6iSnU3amVldVVfUWc

Raspberry Pi 2/3 jessie: modesmixer2_rpi2-3_20170924.tgz https://drive.google.com/open?id=0B7NYXizl0U6iZlY0MTlhWlRWTEE
Raspberry Pi 2/3 stretch: modesmixer2_rpi2-3_deb9_20170924.tgz https://drive.google.com/open?id=0B7NYXizl0U6iclAyNUl1clRkbU0

Ubuntu 16.04 Intel 64 bit: modesmixer2_x86_64_20170924.tgz https://drive.google.com/open?id=0B7NYXizl0U6iT01YRUFMcFZUVkU
Ubuntu 14.04 Intel 32 bit: modesmixer2_i386_20170924.tgz https://drive.google.com/open?id=1JXON6fEhTxTxSBmMt7jrk33p3xZjRe9l

Regards,
sergsero

[gleven]

Hi Sergsero.

Is there any way of combining two streams into one using modesdeco/modesmixer but filtering out MLAT information from one of the streams?
I currently have one omni antenna to get the traffic around me and i'm planning to have an additional yagi antenna pointing at the local airport so i can get more positions from the planes landing/taking off.

I have tried many combinations of stream types (avr, avrmlat, beast, msg, msgmlat, fatsv, but it seems that the MLAT positions picked up by the yagi antenna gives me the following message in piaware log:

Warning: the timestamps provided by your receiver do not seem to be self-consistent. This can happen if you feed data from multiple receivers to a single mlat-client, which is not supported; use a separate mlat-client for each receiver.

I'm using the latest versions of the executables (modesdeco2_rpi2-3_20170127.tgz / modesmixer2_rpi2-3_20170924.tgz)


P.S. I figured out how to do it.Everything is working fine now.

[Oldfella]

Modesdeco2 latest issue....
have a problem : Database 14 error
My OS:  Win 7 64bit and

After start program shortly running "black window" and it is all.
Me tried run .bat file a here is a stop in dialog window " database 14"

What is a mean???
Thanks for advice and opinions.
Larry - ex CzAF member.
Czech republic - LKPD

G'day Larry, did you ever get this "database 14" error sorted?
Also have this error now occurring after a hdd upgrade. I didn't receive it before hand.

TIA
Oldfella