Saturday, December 9, 2017

Satellite Weather Maps, on a Macbook

There are about 3000 active communications and earth observation satellites flying over our heads at all times (plus about 2000 dead ones).  Many of the earth observation satellites broadcast useful data which anyone can receive, once you acquired the necessary equipment and know-how.  See this https://amsat-uk.org/

 Clouds in the UAE Desert

Weather satellites are generally considered to be the most useful of the lot, since the data is open and not encrypted and the signals are quite strong.  The NOAA operates both geostationary and polar observers.  The geo satellites can only be received if you happen to live in its antenna footprint (North America), while the polar satellites pass overhead twice a day wherever you are.

This article describes how to get an image from one of the NOAA polar satellites, using a cheap ($25) little RTL-SDR radio receiver.  These pictures are interesting, since the weather is always changing.  You also need to make a decent antenna, else you may have more noise than picture.

Interface Specifications

NOAA-15, NOAA-18 and NOAA-19 are probably the easiest to interface to.  All three satellites broadcast using an ancient system termed Automatic Picture Transmission (APT).

The APT signal is 2.4 kHz, frequency modulated, described here http://www.cder.dz/download/Art7-1_1.pdf and here https://www.sigidwiki.com/wiki/Automatic_Picture_Transmission_(APT)

Each facsimilee line starts with a tone burst and the monochrome line is encoded onto a 2.4 kHz tone by frequency modulation at a rate of two scan lines per second.

To decode it, you need a phase locked loop to convert the FM signal into a varying brightness level - simple as that.

Which Computer System To Use

Most ordinary mortals use MS Windows computers.  These are generally good for playing games, writing letters and doing bookkeeping, but they are not very good for engineering use.  The problems are many fold:  The operating system scheduler is not real-time, the USB interface is buggy, scientific software invariably require specialized libraries of specific versions, which sometimes clash with libraries that are already installed - known as DLL Hell.

The result is that if your special program happens to work, then you are in luck.   
If it doesn't work, then you are out of luck, there is nothing you can then do about it and your project is hung - Nuf sed.

A Macintosh system is better, since it is based on FreeBSD, but it suffers from some of the same software library issues when using precompiled (non-Free) software.  However, if you use Free software, then it is much the same as Linux/FreeBSD.  In order to use Free scientific/engineering software, you need Xcode (The C compiler provided by Apple, in the App Store),  Macports https://www.macports.org/ and Homebrew https://brew.sh/.  With these tools, you can compile specialized software, much the same as on FreeBSD/Linux.

Linux and BSD have good real-time performance and gives one full control over everything.   On these UNIX systems, Free software is installed by downloading the source code and compiling it on your machine.  This sorts out all the library dependency issues for your system, with the result that specialized scientific and engineering software generally work much better than on other systems.

Note that the future NOAA software systems will all run on Linux and other operating systems will be supported through Linux virtual machines only, as explained here http://noaasis.noaa.gov/NOAASIS/ml/future.html

So, for Linux users, it is the same idea as in this article.  You need to install rtl_sdr, gpredict and WXtoImg.  All the same, just a bit easier, since the repositories have what you need and you won't need weird paths - everything will be in the usual places.

The Heavens Above

The http://www.heavens-above.com web site is very useful, but the best way to see when a satellite will pass overhead is with gpredict.

Install gpredict from macports:
$ sudo port selfupdate
... long wait...

$ sudo port install gpredict
...even longer wait...

Finally, you can run it:
$ /opt/local/bin/gpredict

Gpredict Satellite Orbit Prediction

You need to select the satellites that you want to track, but it is not immediately obvious how.  There is a tiny down arrow at the top right, select Configure, then scroll down to the NOAA sats.  Enter your own ground station co-ordinates and then if you hover the mouse over a bird, you can see how many minutes are left to reach your position.

There is also command line version called predict.  It can be used to get the pass data for a satellite for use in scripts, in order to automate a weather map system.  Here are some examples:

Passes between two dates (in seconds since UNIX epoch):
$ predict -f NOAA-15 $(date +s) $(expr $(date +s) + 600) -o n15pos.txt

Passes for next 10 minutes
$ predict -f NOAA-15 +10m

Single pass after now (useful from a crontab):
$ predict -p NOAA-15 -o n15pos.txt

Single pass after sunrise on a specific day:
predict -p NOAA-15 $(date +%s -d”02/14/2018 06/00/00”) -o n15pos.txt

It can get very complicated, but the simplest method is likely the most useful:
$ predict -p NOAA-15
1518602108 Wed 14Feb18 09:55:08    0   21  192   49   41   3315   2726 *
1518602207 Wed 14Feb18 09:56:47    6   24  196   44   43   2678   2726 *
1518602305 Wed 14Feb18 09:58:25   15   30  200   38   45   2067   2726 *
1518602400 Wed 14Feb18 10:00:00   26   41  204   32   47   1527   2726 *

...

The first line gives the time to start recording and the last when to stop - or you can simply stop recording after 15 minutes, but note that the time is in UTC, so you have to add your time zone offset.

HOWEVER... After much experimentation, I still could not get predict to give the correct time.  It always differs from gpredict by about an hour.  It seems to predict a completely wrong satellite.   Some people are using it, so it must be my Keppler data and satellite database that is wonky.

Image Rendering

Reputedly, the best program to render the images appears to be WXtoImg, which you can get here http://www.wxtoimg.com, but my experience with this program was not good.  The ancient wxsat program worked, but it is a 32 bit Windows program.

 UAE Desert Colour Image by Wxsat

There are other decoders and renderers for Linux/BSD, but I have not tried them all yet.

The secret seems to be to edit the file after recording it and trim the inaudible start and end and save only the good part in the middle in .wav format.  On a Mac, you can trim a file with Quicktime, but it cannot save the result in .wav - it exports to .m4a. To trim, save and resample a .wav file to 11.025 kHz, you need to get Audacity from Sourceforge, or use sox.

RTL-SDR

The software required for the RTL-SDR radio widget is described here: http://www.aeronetworks.ca/2017/11/software-defined-radios.html

You need CubicSDR and rtl_sdr as described in the above link - or gqrx on Linux.

Frequencies:

You can look at and listen to the satellite data with CubicSDR.
  • NOAA15: 137.62 MHz
  • NOAA18: 137.9125 MHz
  • NOAA19: 137.10 MHz
The actual frequency is 1.9 kHz lower than the above (the upper side band) and the modulation type is Wide Band FM (35 kHz).  Once tuned in correctly, you'll hear the fax lines go cheap-cheap-cheap... at two cheaps per second.  If it doesn't sound good, then it also won't decode properly.

You need to play with the gain setting.  My turnstile antenna is on the roof, hooked to a 15 meter 50 Ohm RG58U co-ax going in a window to the RTL SDR dongle plugged into a little netbook.  I use "-g 49" to for NOAA15 and NOAA18.  The NOAA19 satellite is more powerful and needs a lower gain setting of "-g 35".
The satellites also have other sensors on them and in future there will be other frequencies in the L and X band with much more data, as explained here http://noaasis.noaa.gov/NOAASIS/ml/future.html

Antennas

If you would use a simple dipole antenna, then you would only be able to receive something when the bird is almost directly overhead.  This may be good enough at first.

A better receive antenna that you can build yourself using common commercial items, is described here: http://www.aeronetworks.ca/2017/12/amateur-satcom-rx-antenna-for-2-meter.html

You need to know how to wield a drill, soldering iron and tin snips.  Do wear glasses, so you don't poke an eye out with the rods while working on the thing.

The advantage of this antenna is higher signal gain upwards and less noise from the surroundings.  However, the gain is not so high that you need to track the satellites with a mechanical rotator.  Just point it vertically up at the sky.  A plastic, water filled umbrella base, is all you need to keep it standing up.

Weather Data Capture

Once you figured out when a bird will fly overhead, go outside with your whole kit and kaboodle - you won't receive much indoors, if anything.  It depends on what your roof is made of and you won't get a very interesting image at night either.  So, horror of horrors, you have to get out of your cave in daylight!

The 2400 Hz "cheap-cheap" line data screeches can be received with the rtl_fm program and transcoded to wav format with Sound Exchange (sox), as below.

Note that on my Macbook sox resides in /usr/local/bin and rtl_fm in /opt/local/bin, probably since one was installed with homebrew and the other with macports.   This kind of confusion is one reason I prefer Linux for engineering work.

The dangling dash tells sox to read from stdin - the piped data from rtl_fm.  Once you got the data, down sample it with sox to 11.025 kHz for decoding with wxsat.

For example (NOAA15 137.62 MHz - 1.9 kHz, USB):
$ /opt/local/bin/rtl_fm -g 49 -d 0 -M fm -f 137.62M -s 55k -l 0 | /usr/local/bin/sox -r 55k -t raw -e s -b 16 -c 1 - wxdata-55k.wav
$ /usr/bin/sox wxdata-55k.wav -c 1 -r 11025 wxdata-11k.wav

I got better results by setting the gain with -g 49, than by leaving it to automatic.


Found 1 device(s):
  0:  Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Tuner gain set to automatic.
Tuned to 137618100 Hz.
Oversampling input by: 19x.
Oversampling output by: 1x.
Buffer size: 7.84ms
Exact sample rate is: 1045000.031662 Hz
Sampling at 1045000 S/s.
Output at 55000 Hz.

...long wait...

Press Ctrl-C to stop and close the wxdata.wav file.

You can pipe the signals straight into the wxsat rendering program and get the image in real-time, but making it work the simple way first, is hard enough for starters.

Also note that there are multiple types of weather fax modulation modes used by polar sats, geo sats and HF radio.  Russia also has different satellites.  The above examples are for the older polar NOAA satellites only.

Render the Image

Now you need to trim the audio file to get rid of the noise at the beginning and end and then run wxtoimg, hamfax, wxsat or satsignal and read the audio file.  Hopefully, the result will be better than my first try!

Nice Weather!

With a simple antenna, it will only work if the satellite is passing fairly high, more than 20 degrees above the horizon, otherwise there will be too much noise, distortion and doppler shifting and the first and last 5 minutes are typically unusable, leaving maybe 5 minutes worth in the middle that you can actually use. 

If your antenna didn't blow down in the last storm, 
then it isn't high enough.

The WxToImg program proved to be quite useless to me. I never managed to decode anything useful with it, no matter how good the recording sounded to my ears.

Eventually, in sheer desperation, I used Audacity to convert the recordings to 11.025 kHz sampling rate and decoded a picture (the one at the top) with the old wxsat running on a trusty old Windows XP virtual machine on my Mac.  The better solution is to edit your record script and add another line to down sample with sox (you could do it in a single line if you use the sox effects chain -rate 11025).

Schedule a Weather Fax Recording

Use the at daemon (or crontab) to schedule the execution of a script, so that you don't have to leave your lair during the day time and suffer sunburn:
$ at -f /Users/herman/Hamradio/WeatherSats/n19 now + 1 minute

Note that you will get better results by starting the recording cron job 5 minutes late and ending 5 minutes early, since close to the horizon, there is just too much noise and distortion in the signals.

For example:
$ at -f ./n19 now + 168 minutes
job 6 at Fri Feb  9 13:45:00 2018
$ at -f ./n15 now + 369 minutes
job 7 at Fri Feb  9 17:08:00 2018

$ atq
6    Fri Feb  9 13:45:00 2018
7    Fri Feb  9 17:08:00 2018


If you read man at, it says that the daemon is turned off by default.  If needed,  enable it with: 
$ sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.atrun.plist 

The recording will run forever, unless you stop it with a scheduled kill command!

So you can make a simple killall script like this:
#! .bin/bash
killall n19


and schedule it like this:
$ at -f ./n19stop now + 178 minutes

To capture 10 minutes of data.

Other Software

There are also weather fax relays on HF radio for mariners.  The encoding is somewhat different and fldigi can be used to decode them.

The most comprehensive meteorological tool kit is probably gempack https://www.unidata.ucar.edu/software/gempak/

You can also look into wview http://www.wviewweather.com/

Setting the above up will likely be quite an adventure...

More Information

A detailed guide for tuning WxSat and making false colour pictures by combining the images from two cameras (visible and IR) in the satellites:

Group for Earth Observation (GEO):

Satellite Networked Open Ground Station (SatNOGS):

Digital Weather Satellite Reception:

Articles on decoding Russian Meteor-M2 weather pictures.  This one is much higher resolution than the older NOAA satellites and since it is in an orbit for weather prediction in Moscow, it is also very useful for the rest of Eastern Europe and the UAE.

Hardware

You will need a decent antenna or two:
https://www.aeronetworks.ca/2017/12/parasitic-quadrifilar-helical-antenna.html
https://www.aeronetworks.ca/2017/12/amateur-satcom-rx-antenna-for-2-meter.html

You may also want a masthead preamplifier to dig the picture out of the noise:
http://lna4all.blogspot.com/
http://www.minikits.com.au/electronic-kits/rf-amplifiers/rf-preamplifiers/2m-RX-Preamplifier


Have fun!

Herman

No comments:

Post a Comment

On topic comments are welcome. Junk will be deleted.