Thursday, April 20, 2017

Stochastic Systems Analysis

Engineers will recognize that this post is about trading, others may wonder what is a stochastic system.

Market price data is a time series of mostly random data, a stochastic system.  So a Day Trader is a Stochastic Systems Analyst - now doesn't that sound fancy!

Investopedia Simulator
The one thing I learned over the years is that if you want to make money from trading stocks or futures, then you need to have a simple and well defined system, with easy to calculate entry and exit criteria, otherwise you will eventually lose your shirt.

As Douglas Adams, the famous galactic economist of Bistromatics fame said:  When it is infinitely improbably that something will ever happen, it will happen almost immediately.

Price data is mostly random.  In retrospect it seems to follow a trend and engineers and computer programmers realize that they can easily devise a system to do automated trading based on historical and real-time data with some Digital Signal Processing applied to detect a signal in the noise.  I tried it too, but downloading price data and adjusting for stock splits and companies changing names and ticker symbols is a huge problem and if you then devise an automated trading system it may work for a while, but eventually it will diverge and you can lose your shirt.   

An automated trading system requires constant work, which defeats the original idea!

Mutual Funds

The only people that make money from mutual funds are the people that manage the mutual funds.  In Canada, they make money twice - when you buy and when you sell - a total rip-off.   

So instead of investing in a mutual fund of a bank, rather buy the stock of the bank, pocket the dividends and stop complaining about how they rip people off.

Long Plays

One problem is that any trade also influences the market - it is a feedback loop.  One can get price volatility with a purchase of only a few hundred shares in a blue chip company, if nobody is selling at that nanosecond instant.  Now imagine trying to manage a pension fund and buying and selling tens of thousands of shares.  This is why there is always room for the little guy - us.

To build real wealth, you need to be in the market for the long haul - decades.  The pension funds periodically 'balance their portfolios' - usually in March and October and that can cause wild volatility and crashes.  So a good long strategy is to buy in March and sell in September, then buy in November and sell in February.  If you are in cash, then you can buy the crash.  If it doesn't crash, just buy it all back again a month later.

I just bought four blue chip stocks in my RRSP  (my buy in March entry) and within a few minutes lost $200 - bah, humbug.  If you trade long, don't look at the real-time graphs! 

To control the risk, buy at least 5 stocks for your RRSP and set stop loss orders 5% to 7% down - but you need to keep resetting them, since they expire after a few days.  Effectively, you can run your own mutual fund and keep the management fees.

Day Trading

Day trading is for making a little money to live on now.  Investing is to make a lot of money to retire on in 30 years.  It requires different strategies.

Day trading is also a nice mental exercise to keep your brain from atrophying while you wait for the  market grass to grow.

There are few millionaire day traders.  There are however many people who sell books and training courses on day trading.  Selling books and training courses is clearly more profitable and less risky!

You have to read two or three books (No more than three. Stop and practise instead), to learn the lingo and methods, just don't believe in it as the one true gospel - try to find the idea behind it

Bear in mind that if someone sounds like a second hand car salesman selling a lemon, he probably is. 

Day trading is limited by what a human being can manage.  People can juggle 3 to 11 complicated things at the same time in their head.  The result is that a futures trading system is best limited to a pot of $5000 to $15000 and trades limited to 1 to 5 per day.

As Lazarus Long, the famous American futurologist said: The odds are terrible, but if you don't play, you can't win.

If you trade with E-mini, buy and sell one or two contracts at a time, then you could make $50 to $200 a day.  Buy a contract and sell when it is up by 8 ticks or down by 3 ticks.  On average you should win some - not much, but enough to pay the bills.  If you don't know what you are doing yet, you can loose $50 to $200 a day - so you have to practise in a simulator.

The methods are the same for day trading stocks, but you would need a bigger pot of money - about $25,000 to play with -

In general, don't trade with more than 1% of your pot of money to keep the risk down and test your system in a simulator for a week or two before turning loose on live trades.  If you only make one trade a day, then two weeks of practise is only 10 dummy trades - precious little.  If you cannot make money in a simulator, then real live trading will be even worse and you will join the crowd of suckers at the bottom of the heap.

If you are good, then you should invest half your monthly profit for the long haul in blue chips.

Trading Software

Trading software programs now are a dime a dozen - there is no need to write your own anymore.  Ninjatrader is popular for futures trading, but your own bank probably has a system that is good enough - a fancy expensive program will not magically make you a better trader if you only do one trade per day.

TD Bank for example has four trading systems that I know of, the default Webbroker system, a more advanced Java based Advanced Dashboard program that you can download from the Webbroker site which provides real-time data and also ThinkOrSwim and the related TD Ameritrade -

The lowest cost reputable broker that I know of is Interactive Brokers.  If you do one trade per day of a few hundred stocks/futures, then they may be the best, but figuring out exactly how much it will cost you is difficult, since they have a complex pricing schedule -

Depending on how much you trade and how much you have in your account, a trading platform will cost about $60 per month.  So the cost is not ridiculous if you are dedicated to trading and not just doing it for fun.  Also write down the help phone number of your trading system, so that you can call them when you have a computer or network problem and cancel your trades.

Historical Analysis vs Reality

If you find an elevator stock that goes up and down, then you can trade it and make money every few hours or days simply by sitting and staring at the real-time price graph, but eventually something will change.  There will be a war, a plane crash, a court case, a dishonest CFO... and you may loose your shirt.

It is therefore important to use simple statistics and stop loss orders to control the risks and identify opportunities, but you should steer clear from very complex systems, since finely tuned systems will only work in the short term.   

By the time that you buy the book of a successful trader, he is already retired and canoeing in the Caribbean and his lucky streak has long since ended.

You can learn from the past, but the past will never be repeated exactly the same way.

Simple Statistical Analysis

The Greek philosopher Democritus said that a thing is worth whatever someone is prepared to pay for it and the stock market to me resembles a pack of lemmings running over a cliff, with traders trying to outbid each other before the inevitable market crash.

The first book I read on stock trading was clearly written by a trader who was just lucky (or had inside information).  His 'head and shoulders system' was impossible to recreate and everything worked only in retrospect, not with real data.  Successful plays almost always resemble a 'head and shoulders' graph - after the fact.   It is very easy to predict that you should have bought a stock last week.

Much later, I encountered a book by Burton Malkiel,  'A random walk down Wall Street', which rigorously proved with Montecarlo analysis that most hedge and mutual fund traders were just lucky and will blow up eventually.  It is a very dispiriting book, but it proves that one cannot predict the market by looking at price data only - you need to have inside information also.  Ordinary mortals don't have inside information, but big traders and institutions do (the managers sit on multiple company boards and are all golf buddies) and they are the ones leading the pack of lemmings.

The front running big lemmings will never admit to randomness explaining their strings of successful trades, since that will make them look dumb and they will not admit to having inside information, since that is supposed to be illegal.

Successful trading systems for small time investors invariably boil down to a price moving average and a standard deviation calculation, plus a volume indicator.  A Bollinger band is very good to see what the price is doing, while a trading volume indicator (there are many), can identify what the other lemmings are doing and the trick then, is to run with the pack just a little bit and stop, before they fall off the cliff.

The Lemming Effect

Successful trading strategies look not only at price, but also at volume and track price volatility to get a handle on the Lemming Effect.  The big lemmings out in front of the market are the institutional investors with inside information - successful small trading systems calculate how to follow the fat lemmings for a little while.

Bulls make money, and bears make money, but pigs get slaughtered.

When the market is stable (going sideways) or going up (bullish), then you can buy low and sell high and make money, a little bit at a time.

However, when the market is stable or going down (bearish), then you need to reverse things to (short) sell high and buy low.  It is exactly the same operation, just reversed in time.

As Shane, the famous American cowboy said: If god didn't have a sense of humour, then he would be a very lonely man.

Here is a good write-up on Bollinger Band trading -

So how do you know when to buy long or sell short?  Look at the long term trend - where are the lead lemmings going.  Is the market going up, or going down and if you get a losing streak of five dud trades - reverse what you are doing - don't fight the market - flip your strategy around - ping-pong -

Practise Makes Perfect

It all sounds easy until you try to place a trade, then you wonder what amount to enter at the stop buy, what is the ADR and the profit target and what to enter in the stop loss and would the profit be big enough to cover the trading fees - and by the time you figured it all out, the opportunity has passed.  

Therefore you have to make a spread sheet or prepare a programmable calculator with your formulas so that you can enter and place a trade quickly.  You need to keep a spread sheet of your trades anyway, since come tax time, you will need it.

How do you find a stock to day trade?  Well, that is the boring part.  Here are some clues:
  • Daily volume between 500K->25M shares
  • Stock price < $30
  • Range between hi & low to be > 50c
  • Look for a 20c gain on 1000 shares, for a $200 day.
There are not all that many.  You could play with Bank of America in a simulator to get started.    

Keep it simple, and practise in a simulator or for a few weeks before you risk real hard earned moolah.

La voila!


Monday, March 13, 2017

Troubleshooting Video Streaming

Casting About

Video multicast routing usually 'Just Works'.  Occasionally it doesn't work at all, or the video streams for a few minutes only and then one is left scratching one's head...

To find out what is wrong, one has to know how to analyze a playback streaming problem when it happens.

Here is something to read on streaming, multicasting and the Internet Group Management Protocol (IGMP):


Multicast Version

Sometimes is is necessary to use a specific version of IGMP to get things to work. One can force the IGMP version in the /proc/sys/net/ipv4/conf/eth0/force_igmp_version to 0 (auto), 2 or 3.

Multicast Routing

The multicast address subnet is the 224.x.y.z address range.  For routing to work at all, you need a multicast route, set with a command like this:
# route add -net netmask dev eth0

That is if your ethernet port is eth0.  Mine is enp0s26u1u1 at the moment - I'm using a USB ethernet device on this machine, as the RJ45 socket has a solder problem, resulting in a Welsh device name.

$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         UG    1024   0        0 enp0s26u1u1   U     0      0        0 enp0s26u1u1 UGH   1      0        0 enp0s26u1u1       U     0      0        0 enp0s26u1u1

The last line above is the multicast route and that being missing/wrong is 99.999% of the time the actual problem.


Multicast Groups

For multicasting to work, your machine needs to be a member of the multicast group.  This is set at start-up using the IGMP protocol if your route is correct. 

You can list group memberships with netstat:

$ netstat -gn

IPv6/IPv4 Group Memberships
Interface       RefCnt Group
--------------- ------ ---------------------
lo              1
em1             1
wlp3s0          1
enp0s26u1u      1
enp0s26u1u      1
lo              1      ff02::1
lo              1      ff01::1
em1             1      ff02::1
em1             1      ff01::1
wlp3s0          1      ff02::1
wlp3s0          1      ff01::1
enp0s26u1u1     1      ff02::1:ff00:10e
enp0s26u1u1     1      ff02::1
enp0s26u1u1     1      ff01::1

Group Subscription

For streaming to work, the client (video player) has to subscribe to the stream with an IGMP Join message.  The routers listen to the IGMP messages (IGMP snooping) and use the data to send multicast traffic only to members of the group.  Otherwise the LAN will be flooded with streaming data.

The network switches keep track of the MAC addresses used by multicast subscribers and this list expires after 500 to 600 seconds.  The subscription therefore needs to be renewed periodically by  IGMP Query and IGMP Report messages, otherwise the switch will stop forwarding the stream and the video player will stop.

To confirm that the subscription is happening and that the video player is configured right, start tcpdump on the server and then start the video player.  You should see something like this:

# tcpdump -i eth0 igmp
tcpdump: listening on eth0
00:20:09.007094 switch-a.stage > ALL-SYSTEMS.MCAST.NET: igmp query v2 [max resp time 20] [ttl 1]
00:20:09.241946 > igmp v2 report (DF) [tos 0xc0]  [ttl 1]
00:20:10.472159 > igmp v2 report (DF) [tos 0xc0]  [ttl 1]

Multicast Ping

You can also ping all members of the group, if the machines are configured to respond to broadcast/multicast pings:

$ ping
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=2.03 ms
64 bytes from icmp_seq=1 ttl=64 time=2.03 ms (DUP!)
64 bytes from icmp_seq=2 ttl=64 time=2.05 ms
64 bytes from icmp_seq=2 ttl=64 time=2.05 ms (DUP!)

The /proc filesystem variables /proc/net/ipv4/igmp* are useful to see the groups that your host is currently subscribed to. So you can go and dig there yourself if you don't have netstat installed.

The proc File System

Broadcast ping responses are usually turned off, to prevent Denial Of Service attacks.  Ping responses are controlled via /proc as well and can be turned on with:

echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_all

echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts

or on machines with systemd:
sysctl net.ipv4.icmp_echo_ignore_broadcasts=0

Next time you experience a problem, check your IP device and route setup, check the groups and do a ping to see what is going on.

Phun With Streaming

Now let's have some fun with other streaming tools, ffmpeg and gstreamer.

Record Your Laptop PC Camera With ffmpeg

There are many ways to do this, so let's explore one of the roundabout ways and record the camera to a file, while also playing it to see what is going on.

Use ffmpeg to record the camera:
$ ffmpeg -f v4l2 -s 800x600 -i /dev/video0 -r 30 -f matroska -vcodec h264 /tmp/camera.mkv &

Give the recorder a head start, wait a few seconds then play the file:
$ ffplay /tmp/camera.mkv

Now you have a slow mirror - yeah, da...

To stop, run 'killall ffmpeg' and 'killall ffplay'.

Create a Video Test Pattern With Gstreamer

It can be very useful to generate a test pattern, just to get started with gstreamer:
$ gst-launch-0.10 videotestsrc ! ffmpegcolorspace ! autovideosink

You can make a test pattern with gstreamer as explained above, or you can play a video. Here is the jackpot with every test pattern you may ever want and few more: 

Send and Receive Audio with Gstreamer

Streaming audio between two machines can be very useful for an intercom or radio over ethernet.  Here is something to get you started, by sending and receiving audio on the same machine.

Gstreamer transmit in one terminal:
$ gst-launch-0.10 -v audiotestsrc do-timestamp=false ! audioconvert ! audio/x-raw-int,channels=1,depth=16,width=16,rate=8000 ! rtpL16pay ! udpsink host=localhost port=1234

Gstreamer receive in another terminal:
$ gst-launch-0.10 -v udpsrc port=1234 ! "application/x-rtp,media=(string)audio, clock-rate=(int)8050, width=16, depth=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96" ! rtpL16depay ! audioconvert ! alsasink sync=false

Talking to yourself gets boring quickly though.

To ensure that two different machines will work together without an ever increasing time delay due to slight clock differences, make the consumer slightly faster than the producer by using a clock rate of 8050 instead of 8000 Hz.

To send audio between two machines, use the other machine IP address instead of localhost and to make an intercom, you need to make both streams bidirectional - by combining the two examples.

La voila!


Monday, March 6, 2017

Saving the trees - PDF signatures

How to stick a signature graphic into a PDF file, without having to print it first:

A word of warning first

Don't stick your signature into a MS Word document, since anyone with 2 brain cells can alter it.

Also don't use your usual signature - Make a special scrawl for office use.

Print the file to PDF with Foxit Reader

If the file isn't in PDF format already, open your MS Office document with MS Word on Windows.
Print to a file and select PDF format.

If you cannot print to a PDF, install Foxit Reader:

Make a signature graphic with transparent background using Gimp

Make a signature graphic with transparent background:
Take a pen, scrawl on a piece of paper.
Scan it.

Use a real computer - Linux, BSD or Mac:
apt install gimp xournal pdfshuffler

Open the scan file with Gimp
Crop it
Select the white background with the eye dropper
Click Layer, Transparency, Colour to Alpha
Click File, Export, signature.jpg

Annotate the PDF file with Xournal

Open the PDF file with xournal
Click Tools, Image and select the signature.jpg file
Click where you want the signature.
Resize it with the mouse
Click File, Export to PDF and select the original PDF file
Overwrite it.

With xournal, you can annotate a PDF file very nicely.  It is worth learning how to use it.

Sunday, February 26, 2017

Simple OpenBSD File Server

These days, when people think of a file server, they assume that it must support Windows CIFS (a.k.a. SMB or Samba).  A few grizzled sysadmins know that NFS would be much, much simpler to set up and almost nobody would consider using FTP.

Well, that is too bad, since for many situations, anonymous FTP is best and it works purrfectly on my little OpenBSD netbook.

FTP is a very simple protocol, it only talks when it has to and is completely quiet otherwise.  It has none of the chattyness and incessant 'CACA' packets of CIFS.  It is extremely easy to set up and has native support in all operating systems.  Even Windows can do anonymous FTP transparently and can map a FTP server to a drive letter, thus enabling any program to connect to the server directly.

Some will speak up and say that FTP is insecure.  Well, yes, but so is NFS and CIFS.  The difference is that FTP doesn't even pretend to be secure.   BTW, don't use FTP with password authentication - the passwords are sent in the clear making it quite useless.

The Samba manual is about 2 inches thick, while the configuration file for a FTP server is only about a dozen lines.  Need I say more?

When you have a home or office with ten or twenty users, who just need a centralized place to store data that can be regularly backed up and you don't want to waste any time managing it, then an anonymous FTP server could be ideal, since you don't have to waste any time with accounts, passwords and access controls.

Set it and forget it - KISS.

Configuration Frustrations

I installed vsftpd using pkg_add:
# pkg_add vsftpd

Simple as that.

In Linux, the package manager is different for each distribution, otherwise, it is the same idea.

Example Configuration File:
$ cat /etc/vsftpd.conf
ftpd_banner=VsFTPd. Cool, eh?

The problem with setting up a FTP server is that the configuration for the server also depends on the local file system access restrictions.

In an attempt to frustrate a hacker I run the server as an unprivileged user _ftp and make the directory tree owned by a different unprivileged user called ftp.  Ensure that these two users do not have a login shell.  Use /usr/sbin/false instead of bash or ksh.  You can run the script adduser to make these accounts if required.

The chown_upload_mode=0664 is a relatively new parameter. All the other older guides on the web don't show this option and this is the main reason why I wrote this guide.

When you specify:

Then this does nothing, so I set it to 0000 as a reminder:

and you instead need:

That took me a long while to figure out.


Directories and Permissions

I made a tree /ftp/pub/data like this:
# mkdir -p /ftp/ftp  
# mkdir -p /ftp/pub/data
# chown root:ftp /ftp/pub/data
# chmod 1777 /ftp/pub/data

That makes the data directory owned by root:ftp and sticky so that new files will inherit that ownership.

Test, Test, Test

Run the server in one console and log in from another console and transfer little test files.  Then log in from another computer and repeat. 
# vsftpd

$ touch test1
$ touch test2
$ ftp localhost
Login: anonymous
Password: [Enter]
ftp> put test1
ftp> get test1

On BSD, see what is going on with tail - same idea, but a different log system on Linux though:
$ tail -f /var/log/vsftpd.log

It may take a while to work out the fat finger kinks.

Windows Mapping

Use the easy connect wizard in the Windows file browser (File, Map Network Drive, Map as Drive, Connect to a website..., Next, Choose...) to connect to the server using a URL like which will create a shortcut in the left pane which then works like any other network file share.

After installing a 128GB SD card for storage, I now have a little WiFi connected 'NAS' which is normally sleeping peacefully and when I need it, all I need to do is flip open its lid and wait about 5 seconds for it to wake up.

For questions, go to

La voila!


Friday, February 3, 2017

OpenBSD on a Netbook

Recently, I got fed-up with the bloated Linux distributions and wanted to try something that is secure, small and efficient and downloaded OpenBSD 6.0 from Theo De Raadt's server in Calgary.  Since Calgary is actually my old home town - why not?

OpenBSD tries to be the most simple and secure UNIX system out of the box.  It is very much server oriented, but it can do anything and many architectures are supported just for fun. For example, Arm RPi and Beaglebone, Intel 32 and 64 bit and several more.  So OpenBSD is a good choice whether you want to build a server farm, a network router, or a robot.

I have an old little Lenovo S10e netbook that I threatened to toss away numerous times, but it doesn't want to break.  As I feel guilty about tossing something that works perfectly well in a bin, once in a while when I run short of resources, I end up using it again.  Last year, it was pressed into service as a Linux Mirror server to install a bunch of embedded computers.

So I dusted the Netbook off and readied it for a new Olde Skool UNIX experience...

Where to get OpenBSD

The last time I used OpenBSD was about 2004 - for web and mail servers in Calgary.  I certainly know Linux better, but my main machine is a Macbook Pro which runs a kind of BSD and the more things change, the more they stay the same.

Here you go:

When I bought the netbook, I wanted something small that I could carry around easily and although the single core 32 bit Intel processor is slow as molasses with the original Windows 7 OS, it always ran Linux with the XFCE desktop quite well, but I wanted to see whether I can make it fly.

From the above list of files, download the install60.fs file if you want to use a USB memory stick as the install medium.

I made the mistake of not reading the INSTALL.i386 instructions and downloaded the ISO file, then wondered why it would not boot.  So, do go and read the INSTALL file.  Pretty much everything you need to know is in there!

Write the install60.fs file to a memory schtick using dd:


The WiFi adaptor in this netbook never worked with Linux, so I didn't expect it to work with BSD.  I therefore plugged in a trusty little Edimax USB dongle (Ralink chip set) and hoped it would work.  OpenBSD recognized it and loaded the run driver, so the dongle showed up as run0 in ifconfig.

The OpenBSD installer is super simple and OpenBSD is even easier and quicker to install than Linux.  It just takes a few minutes.  So, plug the USB widget in and boot it, follow the very simple instructions and mostly just accept the defaults, till you get to the network configuration.  Be sure to type the correct information in for the WiFi adaptor if you are using one: You need to supply the SSID and password as a minimum.

Of course I fat fingered the password, so it could not connect.  The WiFi setup information is in a file called /etc/hostname.run0 and editing it later presented an interesting challenge, since I am severely vi impeded.  I had to read the vi man page to find out how to delete a character - really.

First boot

OpenBSD is a simple and clean system with no bells and whistles.  None.  Zilch.

When you boot up, you get a nice, self explanatory login prompt:
Login: herman

If you are freaked out by a $ prompt, then you either have to return your Geek Card, or read a UNIX book or three.

At that point, I had to go and fix the WiFi password first and then rebooted to see if it worked properly, but you can simply run startx to get a beautiful FVWM desktop, with a xterm and a clock on it - woohoo.

We have Country AND Western music!

The default install doesn't have much of anything for a laptop machine.  The vi editor, ssh and a ftp client are about it.  No web browser, not even links

Install a Package or Three

In order to make the netbook useful, I need a web browser and an editor that is more to my liking.

Packages are listed here:
(or if you have a better machine!).

You can install the dillo web browser like this:
# pkg_add

Dillo is quite a horrible little browser, but it sure is fast and much less clunky than links.  However, if you want something more full featured, install surf or luakit, which are both based on webkit and work with everything, including yootoob...

To preserve your sanity, add the path to the /root/.profile file:
# echo "export PKG_PATH= >> /root/.profile"

and also export it so it will work immediately:
# export "PKG_PATH="

(If you have a better machine: export "PKG_PATH=

After that you can simply run:
# pkg_add dillo
# pkg_add links 
# pkg_add surf
# pkg_add nano 
# pkg_add abiword
# pkg_add gnumeric
# pkg_add xournal
# pkg_add pdfshuffler
# pkg_add gimp
# pkg_add minicom
# pkg_add putty 
# pkg_add deadbeef

You can make that all one line of course, but I prefer getting error messages for one thing at a time, to preserve my sanity.

Utilities like ftp, ssh, telnet, netcat, tcpdump and more are installed by default, so with the above additions, I can do pretty much anything I would ever want to do on a Netbook.

Stop the Mail Daemon

I don't need the mail daemon on my teenie little netbook.  The mail daemon isn't actually doing much, but I prefer it doing nothing and save every processor cycle and byte of RAM that I can.

Services are controlled via the /etc/rc.conf and /etc/rc.conf.local files, with a utility called rcctl:
# rcctl stop smtpd
# rcctl disable smtpd

The result is:
# cat /etc/rc.conf.local


XFCE Desktop Environment

The FVWM desktop is nice and fast, but it is really only good for masochists.  My favourite light weight desktop is XFCE and installing it is just as simple as any other package.

The package manager pkg_add is quite smart, so you can install XFCE for a better laptop experience by simply running:
# pkg_add xfce 

You can then press Ctrl-Alt-Del to quit X and restart it with:
$ startx /usr/local/bin/startxfce4

Or, you can put exec /usr/local/bin/startxfce4 in ~/.xinitrc and then just run startx as usual and with a nice wintry themed wall paper the little Netbook looks quite cool now:

See this for more details:

Minor Niggles

With OpenBSD, there is no drama and most everything works.  If I close the lid, it sleeps, if I open it, it resumes.  Well, almost - the WiFi dongle didn't come back after a resume, so I needed to figure out how to resuscitate the run device driver and hook it into the resume process somewhere.

The OpenBSD FAQ ( eventually revealed the netstart command, which successfully restores the WiFi connection after a resume:
# sh /etc/netstart

I just needed to figure out where to hook netstart so it would be invoked automatically upon resume.

Advanced Power Management

The problem with the WiFi widget was that apmd was not running.  The Advanced Power Management service controls suspend and resume, processor speed and a few more things.
# rcctl get apmd

Configure and start apmd with:
# rcctl enable apmd
# rcctl start apmd

The result is:
# cat /etc/rc.conf.local

According to the apmd man page the /etc/apm/resume program is run after resuming from standby, so that is the one where I need to put a call to /etc/netstart.

So I tried this:

# mkdir /etc/apm 

# nano /etc/apm/resume

and added the following:
#! /bin/sh

# chmod 755 /etc/apm/resume

Let's see if all is OK:
# rcctl stop apmd
# rcctl start apmd

Let's see if it works with the zzz command: 
# zzz
Suspending system...

and a few seconds later I type:
It resumes from its slumber.

However, it didn't seem to run the resume program.

Let's see what happened:
# tail /var/log/messages
apmd: failed to exec /etc/apm/resume: Exec format error

So, how now brown cow?

Eventually, I did two things to get it to work as explained below.  Don't ask me how I found these tricks, it is just years of experience with obstreperous embedded widgets coming to the rescue and a dogged determination to try various things till the hardware responds the way it should.

Make the netstart script executable, so I don't have to invoke a shell explicitly to run it:

# chmod +x /etc/netstart

Add a delay to the /etc/apm/resume script to give the USB widget time to load its firmware and let the magic fairy dust settle, before trying to configure it:
#! /bin/sh
sleep 1

Now I can make the netbook suspend and resume, the little green lights flash on the WiFi dongle and all is well, the whole universe shook, the BSD daemons sung and flowers fell down from the heavens...

Squash and Square the Web

British Prime Minister Harold Wilson said of the press:
If you can't squash them, square them 
and if you can't square them, squash them.

It is really only a couple hundred companies that are ruining the web for a couple billion people on the planet, so with even the littlest machine it is trivial to disable the culprits.

You can get a good /etc/hosts file to efficiently squash and square the rubbish on the wild wild web here:

My little netbook now zooms like a much faster model, with no Windows, SELinux, systemd, advertisements or spyware to slow it down and now that Uncle Sam decided to ban laptops on board aircraft, this little one becomes the ideal machine to chuck into my luggage since nobody will steal what looks like a clunky old netbook!

Connect to Free WiFi Access Points

My WiFi setup is now automated for home use, but what if I travel and want to connect to free WiFi in McDonalds, or Vienna airport, or Cafe Nero, or.... ???

I solved this with a little script called mcdonalds - duh...
#! /bin/sh
ifconfig run0 nwid mcdonalds
dhclient run0

and ditto for the two or three other places I go to - good enough for me.

Play Internet Radio with deadbeef

On Linux, I use Streamtuner, a wonderful little application, but it doesn't have a BSD equivalent.  The solution is to run dillo and browse to (39,520 radio stations!) however, I didn't have a music player installed yet, so:
# pkg_add deadbeef

Now when I run dillo and click a category on internet-radio, select a station, look at the left and download a playlist .pls file - save as - /home/herman/radio/whatever.pls  - thereafter, in the file browser simply double click the desired .pls file to play the radio station with deadbeef.

There is probably a way to get dillo to spawn deadbeef, but since I only ever listen to two or three radio stations, this is dead simple and now I actually can listen to Country and Western music - Och, my poor bleeding ears...

I then found that the speaker volume keys on the chicklet keyboard actually work too - neat!

OpenBSD on Virtualbox

If you are not quite ready to go bare metal, then you can install OpenBSD in Virtualbox, but since there is no Guest Additions for OBSD, you need to tweak things manually to get full screen operation.   

I actually prefer slightly smaller than full screen, so I can retain access to the host task manager bar more easily.  The easiest way to get the size right, is to take a screen shot by grabbing a rectangle, save the file and open it with a photo editor to see the dimensions.

There are are some weird limitations and  I could only get it to work with 16 bit video on my Lenovo and on my Macbook Pro I had to select 2560x1600x16 and Unscaled High Definition Video in the machine video properties to make it work full screen.

Define a custom screen for the OpenBSD VM like this, sized for my Lenovo Thinkpad:
$ VBoxManage setextradata OpenBSD CustomVideoMode1 1600x868x16

Start the VM and scp the below /etc/X11/xorg.conf file over to the VM. Modify the Depth, DefaultDepth and Modes at the bottom to suit:

Section "ServerLayout"
 Identifier     " Configured"
 Screen      0  "Screen0" 0 0
 InputDevice    "Mouse0" "CorePointer"
 InputDevice    "Keyboard0" "CoreKeyboard"

Section "Files"
 ModulePath   "/usr/X11R6/lib/modules"
 FontPath     "/usr/X11R6/lib/X11/fonts/misc/"
 FontPath     "/usr/X11R6/lib/X11/fonts/TTF/"
 FontPath     "/usr/X11R6/lib/X11/fonts/OTF"
 FontPath     "/usr/X11R6/lib/X11/fonts/Type1/"
 FontPath     "/usr/X11R6/lib/X11/fonts/100dpi/"
 FontPath     "/usr/X11R6/lib/X11/fonts/75dpi/"

Section "Module"
 Load  "dbe"
 Load  "dri"
 Load  "extmod"
 Load  "glx"
 Load  "freetype"

Section "InputDevice"
 Identifier  "Keyboard0"
 Driver      "kbd"

Section "InputDevice"
 Identifier  "Mouse0"
 Driver      "mouse"
 Option     "Protocol" "wsmouse"
 Option     "Device" "/dev/wsmouse"
 Option     "ZAxisMapping" "4 5 6 7"

Section "Monitor"
 Identifier   "Monitor0"
 HorizSync    31-80
 VertRefresh  30-100
 VendorName   "Monitor Vendor"
 ModelName    "Monitor Model"

Section "Device"
 Identifier  "Card0"
 Driver      "vesa"
 VendorName  "InnoTek"
 BoardName   "VirtualBox Graphics Adapter"
 BusID       "PCI:0:2:0"

Section "Screen"
 DefaultDepth  16
 Identifier "Screen0"
 Device     "Card0"
 Monitor    "Monitor0"
 SubSection "Display"
  Viewport   0 0
  Depth     16
  Modes     "1600x868"

When you launch startx, you should get a large window and may need Right-Control F to switch to full-screen mode.

For questions, go to

La voila!


Monday, January 23, 2017

Linux Network Manager Manual Commands

I have a love/hate relationship with the Linux NetworkManager daemon.  It usually works and keeps your laptop PC network connections going smoothly when you move around, but it gets in the way when one does network tests and system integration using a laptop PC.

Each time you plug a cable in, or turn an embedded system off/on, NetworkManager restarts the connection and you can then lose your static IP address setting, which gets tiring really quickly in a lab setup.

Usually, I completely disable NetworkManager and assign a static IP address to my machine on a laboratory bench with a script in /usr/local/bin called static:

#! /bin/bash
echo Configure network for PDLU access

# Disable the Network Manager
systemctl disable NetworkManager.service
systemctl stop NetworkManager.service

# Set static IP address
ifconfig em1 netmask up

# Set multi casting route
route add -net netmask dev em1

# Open up the firewall
iptables -F

# Show setup

and when I get back to a desk, I set things back to normal with a script called dynamic:

#! /bin/bash
echo Configure network normally

# Disable the Network Manager
systemctl enable NetworkManager.service
systemctl restart NetworkManager.service
sleep 1

# Set multi casting route
route add -net netmask dev em1

# Show setup
sleep 1

Sometimes NetworkManager gets confused and the Aplet ends up spinning forever, trying to bring up a non-existent interface.

The way to fix these kind of issues is to invoke the command line program nmcli:

# nmcli connection show active
NAME                UUID                                  DEVICES      DEFAULT  VPN  MASTER-PATH
Wired connection 1  34111952-8271-4f64-a616-a6cd5899bae2  vboxnet0     no       no   --         
Wired connection 2  9dd23ba2-3378-4657-96bb-3b687cfe0180  enp0s29u1u2  yes      no   --       

Disable the errant interface:
# nmcli connection down id "Wired connection 1"

and finally delete it altogether:
# nmcli connection delete id "Wired connection 1"

This way I have a quiet GUI again without the irritating spinning widget in the corner.

La voila!


Saturday, January 21, 2017

Free and Open Systems - Free and Open Minds

The power of marketing, advertising and glossy brochures subtly warp people's minds to wherever advertising Dollars go.  This affects Free and Open embedded operating system software as well.  

Users visit a software company web site that showcases a new software feature and they don't realize that the "new feature" is 30 years old and exists in numerous other systems also

Some people do not understand that there is a large corpus of almost identical operating systems known as UNIX-like or POSIX compatible operating systems and that they can use any one of them and port their software between them, without shedding too many tears.

Embedded OS selection criteria should therefore not concentrate on perceived feature differences, but rather on availability:  Copyrights, Patents, Licensing and Export Permits.

This is especially important in small countries which are subject to the political whims of the big five, who can make you wait indefinitely for an export permit and delay your projects.


The Portable Operating System Interface (POSIX) is a family of standards specified by the IEEE Computer Society for maintaining compatibility between operating systems. 

This corpus of more or less POSIX compatible embedded operating systems include some big names:
Linux, BSD, Darwin, VxWorks, PikeOS, LynxOS, Integrity-178 and several lesser known ones.  

Each of the better known ones are also divided into several lesser known distributions.  For example, there are two types of VxWorks, three types of Apple Mac OS and four or more types of BSD.

Shared Packages

These POSIX operating systems share an enormous library of software.  About 25 000 shared packages are listed in the FreeBSD repositories. 

The actual differences between these operating systems are very subtle and are mostly a matter of who you go to for support, not technicalities, since they all share the same 25 000 packages, while the differences are only the OS kernels.
If any distribution would develop a new security feature or performance improvement, then that idea will get copied/included by everybody else in due course.  

Some features that are now in the news, such as ‘partitioning’ was first developed by BSD, decades ago - called Jails.  

Ditto for hypervisors such as Qemu, KVM, VMware and others which have been around for decades.

All the abovementioned versions of UNIX can do partitioning and hypervisors.  Partitioning isn’t unique to VxWorks or PikeOS - they are just marketing it.  

The PikeOS microkernel isn’t unique either - Apple OSX and Darwin also use the Mach kernel.  It makes these systems a little slower, so microkernels are not popular in general purpose computing. 


Another issue is certification by various authorities, for example DO-178B.  
It is clear that if for example VxWorks, LynxOS, PikeOS or Integrity-178 is certified to a standard, then by extension, BSD also benefits of the software reviews, since they share the same 25 000 base packages.

Regulation-Free Software

When building embedded systems, it is important to avoid using parts and equipment that are encumbered by overbearing regulations such as ITAR. 

VxWorks and PikeOS are both encumbered by licenses and permits that are hard to obtain. 

However, all versions of Linux and BSD are free of encumbrances and are known as Free and Open software. 


Hardware Support

Of all the above UNIX-like operating systems, Linux currently has the widest hardware support and Apple OSX the most limited hardware support. 

Style, is the ability to distinguish quality, 
without having to look at the price tag.

Windriver VxWorks became a popular embedded OS when they added the BSD network stack to VRTX and declared that “VRTX now Works".  

WindRiver also has their own embedded Linux distribution since about 2008. 

Escape From Export Permit Alcatrez

Therefore, it is entirely possible to install an embedded version of BSD or Linux on a CPU card and port software over from an encumbered embedded OS such as VxWorks, LynxOS or Integrity-178, thus avoiding all the licensing and permit issues.

Similarly one could port software from an encumbered version of PikeOS over to Apple's Darwin, which also has a microkernel. 

In Search of Excellence

One day when another OS version has a new feature that is in fashion, then the software can be ported over again.  That is the whole purpose of the POSIX standard.  

The user isn’t captive to any one system or provider and wasting time on obtaining licenses and permits from an unco-operative country isn’t at all necessary.

. -.-. .-. .- ... . --..  .-.. .  .. -. ..-. .- -- .