Saturday, May 13, 2017

WannaCrypt, WannaCry

It is a big Microsoft security waste of time, time again.

The quality of computer software bears no relationship to its price.  
In fact, the best software tends to be free.

I usually tell friends and family that I don't do Windows.  They then assume that I am one of those weird Mac users.  They are correct of course - I am weird and I use a Mac too - amongst many other types of machines, including those most annoying Windows machines...

Long ago, I used to tell people that they can use Microsoft software:
It is rather buggy, but it is cheap.  
Nowadays however, it is still buggy and very expensive,
especially so when you factor in the maintenance cost.

If you are one of those weird people who insists on wasting money on expensive bad quality software, then you should immediately read this:
https://www.theregister.co.uk/2017/05/13/wannacrypt_ransomware_worm/

and this:
https://blogs.technet.microsoft.com/msrc/2017/05/12/customer-guidance-for-wannacrypt-attacks/

Unless you are a medical doctor and it is a matter of life or limb, please don't ask me to help you fix your messed up Windows machine, unless you want OpenBSD or Linux installed on it.  BSD and Linux can do everything that Windows does and more, doesn't cost anything and doesn't break all the time - a good deal if ever there was one.

As O'l Bob Marley said:  
"No Windows, No Cry..."

Note that most Windows malware worms spread through the File and Print server (SMB/CIFS). Therefore the main thing with Windows is to never, ever, ever enable 'File and Printer Sharing', since that starts up a server process and once running, it is a hassle to disable it again. You can disable File and Printer sharing in Network and Sharing Center, but it will *still* run - you need to stop it manually also.

After disabling File and Printer Sharing in the Network Sharing Center, you can stop the server with this command:
c:\> net stop server

Of course, you need to run the Windows command shell as administrator to do the above.

Don't forget to send Thank You letters to Bill Gates, Steve Balmer and Satya Nadella.  They are ultimately responsible for this mess, for creating and aggressively marketing computer software that is not fit for purpose


Cheers,

Herman

Thursday, April 20, 2017

Stochastic Systems Analysis

Engineers will recognize that this post is about trading, others may wonder what is a stochastic system.

Market price data is a time series of mostly random data, a stochastic system.  So a Day Trader is a Stochastic Systems Analyst - now doesn't that sound fancy!

Investopedia Simulator
The one thing I learned over the years is that if you want to make money from trading stocks or futures, then you need to have a simple and well defined system, with easy to calculate entry and exit criteria, otherwise you will eventually lose your shirt.

As Douglas Adams, the famous galactic economist of Bistromatics fame said:  When it is infinitely improbably that something will ever happen, it will happen almost immediately.

Price data is mostly random.  In retrospect it seems to follow a trend and engineers and computer programmers realize that they can easily devise a system to do automated trading based on historical and real-time data with some Digital Signal Processing applied to detect a signal in the noise.  I tried it too, but downloading price data and adjusting for stock splits and companies changing names and ticker symbols is a huge problem and if you then devise an automated trading system it may work for a while, but eventually it will diverge and you can lose your shirt.   

An automated trading system requires constant work, which defeats the original idea!

Mutual Funds

The only people that make money from mutual funds are the people that manage the mutual funds.  In Canada, they make money twice - when you buy and when you sell - a total rip-off.   

So instead of investing in a mutual fund of a bank, rather buy the stock of the bank, pocket the dividends and stop complaining about how they rip people off.

Long Plays

One problem is that any trade also influences the market - it is a feedback loop.  One can get price volatility with a purchase of only a few hundred shares in a blue chip company, if nobody is selling at that nanosecond instant.  Now imagine trying to manage a pension fund and buying and selling tens of thousands of shares.  This is why there is always room for the little guy - us.

To build real wealth, you need to be in the market for the long haul - decades.  The pension funds periodically 'balance their portfolios' - usually in March and October and that can cause wild volatility and crashes.  So a good long strategy is to buy in March and sell in September, then buy in November and sell in February.  If you are in cash, then you can buy the crash.  If it doesn't crash, just buy it all back again a month later.


I just bought four blue chip stocks in my RRSP  (my buy in March entry) and within a few minutes lost $200 - bah, humbug.  If you trade long, don't look at the real-time graphs! 

To control the risk, buy at least 5 stocks for your RRSP and set stop loss orders 5% to 7% down - but you need to keep resetting them, since they expire after a few days.  Effectively, you can run your own mutual fund and keep the management fees.

Day Trading

Day trading is for making a little money to live on now.  Investing is to make a lot of money to retire on in 30 years.  It requires different strategies.

Day trading is also a nice mental exercise to keep your brain from atrophying while you wait for the  market grass to grow.

There are few millionaire day traders.  There are however many people who sell books and training courses on day trading.  Selling books and training courses is clearly more profitable and less risky!

You have to read two or three books (No more than three. Stop and practise instead), to learn the lingo and methods, just don't believe in it as the one true gospel - try to find the idea behind it https://www.rockwelltrading.com/

Bear in mind that if someone sounds like a second hand car salesman selling a lemon, he probably is. 

Day trading is limited by what a human being can manage.  People can juggle 3 to 11 complicated things at the same time in their head.  The result is that a futures trading system is best limited to a pot of $5000 to $15000 and trades limited to 1 to 5 per day.

As Lazarus Long, the famous American futurologist said: The odds are terrible, but if you don't play, you can't win.

If you trade with E-mini, buy and sell one or two contracts at a time, then you could make $50 to $200 a day.  Buy a contract and sell when it is up by 8 ticks or down by 3 ticks.  On average you should win some - not much, but enough to pay the bills.  If you don't know what you are doing yet, you can loose $50 to $200 a day - so you have to practise in a simulator.

The methods are the same for day trading stocks, but you would need a bigger pot of money - about $25,000 to play with - https://www.sec.gov/investor/alerts/daytrading.pdf.

In general, don't trade with more than 1% of your pot of money to keep the risk down and test your system in a simulator for a week or two before turning loose on live trades.  If you only make one trade a day, then two weeks of practise is only 10 dummy trades - precious little.  If you cannot make money in a simulator, then real live trading will be even worse and you will join the crowd of suckers at the bottom of the heap.

If you are good, then you should invest half your monthly profit for the long haul in blue chips.

Trading Software

Trading software programs now are a dime a dozen - there is no need to write your own anymore.  Ninjatrader is popular for futures trading, but your own bank probably has a system that is good enough - a fancy expensive program will not magically make you a better trader if you only do one trade per day.

TD Bank for example has four trading systems that I know of, the default Webbroker https://webbroker.td.com system, a more advanced Java based Advanced Dashboard program that you can download from the Webbroker site which provides real-time data and also ThinkOrSwim and the related TD Ameritrade - https://www.tdameritrade.com/tools-and-platforms.page

The lowest cost reputable broker that I know of is Interactive Brokers.  If you do one trade per day of a few hundred stocks/futures, then they may be the best, but figuring out exactly how much it will cost you is difficult, since they have a complex pricing schedule - https://www.interactivebrokers.com/

Depending on how much you trade and how much you have in your account, a trading platform will cost about $60 per month.  So the cost is not ridiculous if you are dedicated to trading and not just doing it for fun.  Also write down the help phone number of your trading system, so that you can call them when you have a computer or network problem and cancel your trades.

Historical Analysis vs Reality

If you find an elevator stock that goes up and down, then you can trade it and make money every few hours or days simply by sitting and staring at the real-time price graph, but eventually something will change.  There will be a war, a plane crash, a court case, a dishonest CFO... and you may loose your shirt.

It is therefore important to use simple statistics and stop loss orders to control the risks and identify opportunities, but you should steer clear from very complex systems, since finely tuned systems will only work in the short term.   

By the time that you buy the book of a successful trader, he is already retired and canoeing in the Caribbean and his lucky streak has long since ended.

You can learn from the past, but the past will never be repeated exactly the same way.

Simple Statistical Analysis

The Greek philosopher Democritus said that a thing is worth whatever someone is prepared to pay for it and the stock market to me resembles a pack of lemmings running over a cliff, with traders trying to outbid each other before the inevitable market crash.

The first book I read on stock trading was clearly written by a trader who was just lucky (or had inside information).  His 'head and shoulders system' was impossible to recreate and everything worked only in retrospect, not with real data.  Successful plays almost always resemble a 'head and shoulders' graph - after the fact.   It is very easy to predict that you should have bought a stock last week.

Much later, I encountered a book by Burton Malkiel,  'A random walk down Wall Street', which rigorously proved with Montecarlo analysis that most hedge and mutual fund traders were just lucky and will blow up eventually.  It is a very dispiriting book, but it proves that one cannot predict the market by looking at price data only - you need to have inside information also.  Ordinary mortals don't have inside information, but big traders and institutions do (the managers sit on multiple company boards and are all golf buddies) and they are the ones leading the pack of lemmings.

The front running big lemmings will never admit to randomness explaining their strings of successful trades, since that will make them look dumb and they will not admit to having inside information, since that is supposed to be illegal.

Successful trading systems for small time investors invariably boil down to a price moving average and a standard deviation calculation, plus a volume indicator.  A Bollinger band https://www.bollingerbands.com/ is very good to see what the price is doing, while a trading volume indicator (there are many), can identify what the other lemmings are doing and the trick then, is to run with the pack just a little bit and stop, before they fall off the cliff.

The Lemming Effect

Successful trading strategies look not only at price, but also at volume and track price volatility to get a handle on the Lemming Effect.  The big lemmings out in front of the market are the institutional investors with inside information - successful small trading systems calculate how to follow the fat lemmings for a little while.

Bulls make money, and bears make money, but pigs get slaughtered.

When the market is stable (going sideways) or going up (bullish), then you can buy low and sell high and make money, a little bit at a time.

However, when the market is stable or going down (bearish), then you need to reverse things to (short) sell high and buy low.  It is exactly the same operation, just reversed in time.

As Shane, the famous American cowboy said: If god didn't have a sense of humour, then he would be a very lonely man.

Here is a good write-up on Bollinger Band trading - https://www.forbes.com/2007/05/11/bollinger-intel-yahoo-pf-education-in_jd_0511chartroom_inl.html

So how do you know when to buy long or sell short?  Look at the long term trend - where are the lead lemmings going.  Is the market going up, or going down and if you get a losing streak of five dud trades - reverse what you are doing - don't fight the market - flip your strategy around - ping-pong - http://www.traderslog.com/ping-pong-trading-making-money-in-sideways-markets

Practise Makes Perfect

It all sounds easy until you try to place a trade, then you wonder what amount to enter at the stop buy, what is the ADR and the profit target and what to enter in the stop loss and would the profit be big enough to cover the trading fees - and by the time you figured it all out, the opportunity has passed.  

Therefore you have to make a spread sheet or prepare a programmable calculator with your formulas so that you can enter and place a trade quickly.  You need to keep a spread sheet of your trades anyway, since come tax time, you will need it.

How do you find a stock to day trade?  Well, that is the boring part.  Here are some clues:
  • Daily volume between 500K->25M shares
  • Stock price < $30
  • Range between hi & low to be > 50c
  • Look for a 20c gain on 1000 shares, for a $200 day.
There are not all that many.  You could play with Bank of America in a simulator to get started.    

Keep it simple https://www.rockwelltrading.com/tw/book-simple-trading-strategy/, and practise in a simulator http://www.investopedia.com/simulator/ or http://www.infinityfutures.com/ for a few weeks before you risk real hard earned moolah.


La voila!

Herman




Monday, March 13, 2017

Troubleshooting Video Streaming

Casting About

Video multicast routing usually 'Just Works'.  Occasionally it doesn't work at all, or the video streams for a few minutes only and then one is left scratching one's head...

To find out what is wrong, one has to know how to analyze a playback streaming problem when it happens.

Here is something to read on streaming, multicasting and the Internet Group Management Protocol (IGMP):
http://www.tldp.org/HOWTO/text/Multicast-HOWTO

 

Multicast Version

Sometimes is is necessary to use a specific version of IGMP to get things to work. One can force the IGMP version in the /proc/sys/net/ipv4/conf/eth0/force_igmp_version to 0 (auto), 2 or 3.

Multicast Routing

The multicast address subnet is the 224.x.y.z address range.  For routing to work at all, you need a multicast route, set with a command like this:
# route add -net 224.0.0.0 netmask 240.0.0.0 dev eth0

That is if your ethernet port is eth0.  Mine is enp0s26u1u1 at the moment - I'm using a USB ethernet device on this machine, as the RJ45 socket has a solder problem, resulting in a Welsh device name.

$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         172.22.2.1      0.0.0.0         UG    1024   0        0 enp0s26u1u1
172.22.2.0      0.0.0.0         255.255.255.0   U     0      0        0 enp0s26u1u1
172.22.201.21   172.22.2.1      255.255.255.255 UGH   1      0        0 enp0s26u1u1
224.0.0.0       0.0.0.0         240.0.0.0       U     0      0        0 enp0s26u1u1


The last line above is the multicast route and that being missing/wrong is 99.999% of the time the actual problem.

 

Multicast Groups

For multicasting to work, your machine needs to be a member of the multicast group.  This is set at start-up using the IGMP protocol if your route is correct. 

You can list group memberships with netstat:
 

$ netstat -gn

IPv6/IPv4 Group Memberships
Interface       RefCnt Group
--------------- ------ ---------------------
lo              1      224.0.0.1
em1             1      224.0.0.1
wlp3s0          1      224.0.0.1
enp0s26u1u      1      224.0.0.251
enp0s26u1u      1      224.0.0.1
lo              1      ff02::1
lo              1      ff01::1
em1             1      ff02::1
em1             1      ff01::1
wlp3s0          1      ff02::1
wlp3s0          1      ff01::1
enp0s26u1u1     1      ff02::1:ff00:10e
enp0s26u1u1     1      ff02::1
enp0s26u1u1     1      ff01::1


Group Subscription

For streaming to work, the client (video player) has to subscribe to the stream with an IGMP Join message.  The routers listen to the IGMP messages (IGMP snooping) and use the data to send multicast traffic only to members of the group.  Otherwise the LAN will be flooded with streaming data.

The network switches keep track of the MAC addresses used by multicast subscribers and this list expires after 500 to 600 seconds.  The subscription therefore needs to be renewed periodically by  IGMP Query and IGMP Report messages, otherwise the switch will stop forwarding the stream and the video player will stop.

To confirm that the subscription is happening and that the video player is configured right, start tcpdump on the server and then start the video player.  You should see something like this:

# tcpdump -i eth0 igmp
tcpdump: listening on eth0
00:20:09.007094 switch-a.stage > ALL-SYSTEMS.MCAST.NET: igmp query v2 [max resp time 20] [ttl 1]
00:20:09.241946 10.129.22.236 > 232.0.1.10: igmp v2 report 232.0.1.10 (DF) [tos 0xc0]  [ttl 1]
00:20:10.472159 10.129.22.236 > 236.0.1.101: igmp v2 report 236.0.1.101 (DF) [tos 0xc0]  [ttl 1]
 
 

Multicast Ping

You can also ping all members of the group, if the machines are configured to respond to broadcast/multicast pings:

$ ping 224.0.0.1
PING 224.0.0.1 (224.0.0.1) 56(84) bytes of data.
64 bytes from 172.22.2.2: icmp_seq=1 ttl=64 time=2.03 ms
64 bytes from 172.22.2.3: icmp_seq=1 ttl=64 time=2.03 ms (DUP!)
64 bytes from 172.22.2.3: icmp_seq=2 ttl=64 time=2.05 ms
64 bytes from 172.22.2.2: icmp_seq=2 ttl=64 time=2.05 ms (DUP!)


The /proc filesystem variables /proc/net/ipv4/igmp* are useful to see the groups that your host is currently subscribed to. So you can go and dig there yourself if you don't have netstat installed.




The proc File System

Broadcast ping responses are usually turned off, to prevent Denial Of Service attacks.  Ping responses are controlled via /proc as well and can be turned on with:

echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_all

echo 0 >/proc/sys/net/ipv4/icmp_echo_ignore_broadcasts


or on machines with systemd:
sysctl net.ipv4.icmp_echo_ignore_broadcasts=0


Next time you experience a problem, check your IP device and route setup, check the groups and do a ping to see what is going on.



Phun With Streaming

Now let's have some fun with other streaming tools, ffmpeg and gstreamer.

Record Your Laptop PC Camera With ffmpeg

There are many ways to do this, so let's explore one of the roundabout ways and record the camera to a file, while also playing it to see what is going on.

Use ffmpeg to record the camera:
$ ffmpeg -f v4l2 -s 800x600 -i /dev/video0 -r 30 -f matroska -vcodec h264 /tmp/camera.mkv &

Give the recorder a head start, wait a few seconds then play the file:
$ ffplay /tmp/camera.mkv

Now you have a slow mirror - yeah, da...

To stop, run 'killall ffmpeg' and 'killall ffplay'.


Create a Video Test Pattern With Gstreamer

It can be very useful to generate a test pattern, just to get started with gstreamer:
$ gst-launch-0.10 videotestsrc ! ffmpegcolorspace ! autovideosink


You can make a test pattern with gstreamer as explained above, or you can play a video. Here is the jackpot with every test pattern you may ever want and few more: http://www.w6rz.net/ 
 

Send and Receive Audio with Gstreamer

Streaming audio between two machines can be very useful for an intercom or radio over ethernet.  Here is something to get you started, by sending and receiving audio on the same machine.

Gstreamer transmit in one terminal:
$ gst-launch-0.10 -v audiotestsrc do-timestamp=false ! audioconvert ! audio/x-raw-int,channels=1,depth=16,width=16,rate=8000 ! rtpL16pay ! udpsink host=localhost port=1234

Gstreamer receive in another terminal:
$ gst-launch-0.10 -v udpsrc port=1234 ! "application/x-rtp,media=(string)audio, clock-rate=(int)8050, width=16, depth=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96" ! rtpL16depay ! audioconvert ! alsasink sync=false

Talking to yourself gets boring quickly though.


To ensure that two different machines will work together without an ever increasing time delay due to slight clock differences, make the consumer slightly faster than the producer by using a clock rate of 8050 instead of 8000 Hz.

To send audio between two machines, use the other machine IP address instead of localhost and to make an intercom, you need to make both streams bidirectional - by combining the two examples.


La voila!


Herman

Monday, March 6, 2017

Saving the trees - PDF signatures

How to stick a signature graphic into a PDF file, without having to print it first:

A word of warning first

Don't stick your signature into a MS Word document, since anyone with 2 brain cells can alter it.

Also don't use your usual signature - Make a special scrawl for office use.

Print the file to PDF with Foxit Reader

If the file isn't in PDF format already, open your MS Office document with MS Word on Windows.
Print to a file and select PDF format.

If you cannot print to a PDF, install Foxit Reader: https://www.foxitsoftware.com/products/pdf-reader/

Make a signature graphic with transparent background using Gimp

Make a signature graphic with transparent background:
Take a pen, scrawl on a piece of paper.
Scan it.

Use a real computer - Linux, BSD or Mac:
apt install gimp xournal pdfshuffler

Open the scan file with Gimp
Crop it
Select the white background with the eye dropper
Click Layer, Transparency, Colour to Alpha
Click File, Export, signature.jpg

Annotate the PDF file with Xournal

Open the PDF file with xournal
Click Tools, Image and select the signature.jpg file
Click where you want the signature.
Resize it with the mouse
Click File, Export to PDF and select the original PDF file
Overwrite it.


With xournal, you can annotate a PDF file very nicely.  It is worth learning how to use it.


Sunday, February 26, 2017

Simple OpenBSD File Server

These days, when people think of a file server, they assume that it must support Windows CIFS (a.k.a. SMB or Samba).  A few grizzled sysadmins know that NFS would be much, much simpler to set up and almost nobody would consider using FTP.

Well, that is too bad, since for many situations, anonymous FTP is best and it works purrfectly on my little OpenBSD netbook.

FTP is a very simple protocol, it only talks when it has to and is completely quiet otherwise.  It has none of the chattyness and incessant 'CACA' packets of CIFS.  It is extremely easy to set up and has native support in all operating systems.  Even Windows can do anonymous FTP transparently and can map a FTP server to a drive letter, thus enabling any program to connect to the server directly.

Some will speak up and say that FTP is insecure.  Well, yes, but so is NFS and CIFS.  The difference is that FTP doesn't even pretend to be secure.   BTW, don't use FTP with password authentication - the passwords are sent in the clear making it quite useless.

The Samba manual is about 2 inches thick, while the configuration file for a FTP server is only about a dozen lines.  Need I say more?

When you have a home or office with ten or twenty users, who just need a centralized place to store data that can be regularly backed up and you don't want to waste any time managing it, then an anonymous FTP server could be ideal, since you don't have to waste any time with accounts, passwords and access controls.

Set it and forget it - KISS.

Configuration Frustrations

I installed vsftpd using pkg_add:
# pkg_add vsftpd

Simple as that.

In Linux, the package manager is different for each distribution, otherwise, it is the same idea.

Example Configuration File:
$ cat /etc/vsftpd.conf
anonymous_enable=YES
anon_root=/ftp/pub
local_enable=YES
write_enable=YES
anon_other_write_enable=YES
local_umask=0000
anon_upload_enable=YES
anon_mkdir_write_enable=YES
dirmessage_enable=YES
xferlog_enable=YES
connect_from_port_20=YES
chown_uploads=YES
chown_username=ftp
chown_upload_mode=0664
nopriv_user=_ftp
ftpd_banner=VsFTPd. Cool, eh?
chroot_list_enable=YES
chroot_list_file=/etc/ftpchroot
userlist_enable=YES
userlist_file=/etc/ftpusers
secure_chroot_dir=/var/empty
pasv_min_port=49152
pasv_max_port=65535
text_userdb_names=YES
listen=YES


The problem with setting up a FTP server is that the configuration for the server also depends on the local file system access restrictions.

In an attempt to frustrate a hacker I run the server as an unprivileged user _ftp and make the directory tree owned by a different unprivileged user called ftp.  Ensure that these two users do not have a login shell.  Use /usr/sbin/false instead of bash or ksh.  You can run the script adduser to make these accounts if required.

The chown_upload_mode=0664 is a relatively new parameter. All the other older guides on the web don't show this option and this is the main reason why I wrote this guide.

When you specify:
anon_root=/ftp/pub
write_enable=YES
anon_other_write_enable=YES


Then this does nothing, so I set it to 0000 as a reminder:
local_umask=0000


and you instead need:
chown_upload_mode=0664

That took me a long while to figure out.

 

Directories and Permissions

I made a tree /ftp/pub/data like this:
# mkdir -p /ftp/ftp  
# mkdir -p /ftp/pub/data
# chown root:ftp /ftp/pub/data
# chmod 1777 /ftp/pub/data


That makes the data directory owned by root:ftp and sticky so that new files will inherit that ownership.

Test, Test, Test

Run the server in one console and log in from another console and transfer little test files.  Then log in from another computer and repeat. 
# vsftpd
(ok)

$ touch test1
$ touch test2
$ ftp localhost
Login: anonymous
Password: [Enter]
ftp> put test1
ftp> get test1

On BSD, see what is going on with tail - same idea, but a different log system on Linux though:
$ tail -f /var/log/vsftpd.log

It may take a while to work out the fat finger kinks.


Windows Mapping

Use the easy connect wizard in the Windows file browser (File, Map Network Drive, Map as Drive, Connect to a website..., Next, Choose...) to connect to the server using a URL like ftp://192.168.1.10 which will create a shortcut in the left pane which then works like any other network file share.

After installing a 128GB SD card for storage, I now have a little WiFi connected 'NAS' which is normally sleeping peacefully and when I need it, all I need to do is flip open its lid and wait about 5 seconds for it to wake up.


For questions, go to http://daemonforums.org.

La voila!

Herman

Friday, February 3, 2017

OpenBSD on a Netbook

Recently, I got fed-up with the bloated Linux distributions and wanted to try something that is secure, small and efficient and downloaded OpenBSD 6.0 from Theo De Raadt's server in Calgary.  Since Calgary is actually my old home town - why not?


OpenBSD tries to be the most simple and secure UNIX system out of the box.  It is very much server oriented, but it can do anything and many architectures are supported just for fun. For example, Arm RPi and Beaglebone, Intel 32 and 64 bit and several more.  So OpenBSD is a good choice whether you want to build a server farm, a network router, or a robot.

I have an old little Lenovo S10e netbook that I threatened to toss away numerous times, but it doesn't want to break.  As I feel guilty about tossing something that works perfectly well in a bin, once in a while when I run short of resources, I end up using it again.  Last year, it was pressed into service as a Linux Mirror server to install a bunch of embedded computers.

So I dusted the Netbook off and readied it for a new Olde Skool UNIX experience...

Where to get OpenBSD

The last time I used OpenBSD was about 2004 - for web and mail servers in Calgary.  I certainly know Linux better, but my main machine is a Macbook Pro which runs a kind of BSD and the more things change, the more they stay the same.

Here you go: https://ftp.openbsd.org/pub/OpenBSD/6.0/

When I bought the netbook, I wanted something small that I could carry around easily and although the single core 32 bit Intel processor is slow as molasses with the original Windows 7 OS, it always ran Linux with the XFCE desktop quite well, but I wanted to see whether I can make it fly.

From the above list of files, download the install60.fs file if you want to use a USB memory stick as the install medium.

I made the mistake of not reading the INSTALL.i386 instructions and downloaded the ISO file, then wondered why it would not boot.  So, do go and read the INSTALL file.  Pretty much everything you need to know is in there!

Write the install60.fs file to a memory schtick using dd: http://www.aeronetworks.ca/2013/05/using-dd-on-mac-to-copy-iso-file.html

Install

The WiFi adaptor in this netbook never worked with Linux, so I didn't expect it to work with BSD.  I therefore plugged in a trusty little Edimax USB dongle (Ralink chip set) and hoped it would work.  OpenBSD recognized it and loaded the run driver, so the dongle showed up as run0 in ifconfig.

The OpenBSD installer is super simple and OpenBSD is even easier and quicker to install than Linux.  It just takes a few minutes.  So, plug the USB widget in and boot it, follow the very simple instructions and mostly just accept the defaults, till you get to the network configuration.  Be sure to type the correct information in for the WiFi adaptor if you are using one: You need to supply the SSID and password as a minimum.

Of course I fat fingered the password, so it could not connect.  The WiFi setup information is in a file called /etc/hostname.run0 and editing it later presented an interesting challenge, since I am severely vi impeded.  I had to read the vi man page to find out how to delete a character - really.

First boot

OpenBSD is a simple and clean system with no bells and whistles.  None.  Zilch.

When you boot up, you get a nice, self explanatory login prompt:
Login: herman
password
$

If you are freaked out by a $ prompt, then you either have to return your Geek Card, or read a UNIX book or three.

At that point, I had to go and fix the WiFi password first and then rebooted to see if it worked properly, but you can simply run startx to get a beautiful FVWM desktop, with a xterm and a clock on it - woohoo.

We have Country AND Western music!

The default install doesn't have much of anything for a laptop machine.  The vi editor, ssh and a ftp client are about it.  No web browser, not even links

Install a Package or Three

In order to make the netbook useful, I need a web browser and an editor that is more to my liking.

Packages are listed here: https://ftp.openbsd.org/pub/OpenBSD/6.0/packages/i386/
(or https://ftp.openbsd.org/pub/OpenBSD/6.0/packages/amd64/ if you have a better machine!).

You can install the dillo web browser like this:
# pkg_add https://ftp.openbsd.org/pub/OpenBSD/6.0/packages/i386/dillo-3.0.5p0.tgz

Dillo is quite a horrible little browser, but it sure is fast and much less clunky than links.  However, if you want something more full featured, install surf or luakit, which are both based on webkit and work with everything, including yootoob...

To preserve your sanity, add the path to the /root/.profile file:
# echo "export PKG_PATH=https://ftp.openbsd.org/pub/OpenBSD/6.0/packages/i386/ >> /root/.profile"

and also export it so it will work immediately:
# export "PKG_PATH=https://ftp.openbsd.org/pub/OpenBSD/6.0/packages/i386/"

(If you have a better machine: export "PKG_PATH=https://ftp.openbsd.org/pub/OpenBSD/6.0/packages/amd64/)

After that you can simply run:
# pkg_add dillo
# pkg_add links 
# pkg_add surf
# pkg_add nano 
# pkg_add abiword
# pkg_add gnumeric
# pkg_add xournal
# pkg_add pdfshuffler
# pkg_add gimp
# pkg_add minicom
# pkg_add putty 
# pkg_add deadbeef
...

You can make that all one line of course, but I prefer getting error messages for one thing at a time, to preserve my sanity.

Utilities like ftp, ssh, telnet, netcat, tcpdump and more are installed by default, so with the above additions, I can do pretty much anything I would ever want to do on a Netbook.

Stop the Mail Daemon

I don't need the mail daemon on my teenie little netbook.  The mail daemon isn't actually doing much, but I prefer it doing nothing and save every processor cycle and byte of RAM that I can.

Services are controlled via the /etc/rc.conf and /etc/rc.conf.local files, with a utility called rcctl:
# rcctl stop smtpd
# rcctl disable smtpd

The result is:
# cat /etc/rc.conf.local
smtpd_flags=NO

Simple.

XFCE Desktop Environment

The FVWM desktop is nice and fast, but it is really only good for masochists.  My favourite light weight desktop is XFCE and installing it is just as simple as any other package.

The package manager pkg_add is quite smart, so you can install XFCE for a better laptop experience by simply running:
# pkg_add xfce 

You can then press Ctrl-Alt-Del to quit X and restart it with:
$ startx /usr/local/bin/startxfce4

Or, you can put exec /usr/local/bin/startxfce4 in ~/.xinitrc and then just run startx as usual and with a nice wintry themed wall paper the little Netbook looks quite cool now:


See this for more details: https://www.openbsd.org/faq/faq11.html

Minor Niggles

With OpenBSD, there is no drama and most everything works.  If I close the lid, it sleeps, if I open it, it resumes.  Well, almost - the WiFi dongle didn't come back after a resume, so I needed to figure out how to resuscitate the run device driver and hook it into the resume process somewhere.

The OpenBSD FAQ (https://www.openbsd.org/faq/faq6.html#Wireless) eventually revealed the netstart command, which successfully restores the WiFi connection after a resume:
# sh /etc/netstart

I just needed to figure out where to hook netstart so it would be invoked automatically upon resume.

Advanced Power Management

The problem with the WiFi widget was that apmd was not running.  The Advanced Power Management service controls suspend and resume, processor speed and a few more things.
# rcctl get apmd
apmd_flags=NO

Configure and start apmd with:
# rcctl enable apmd
# rcctl start apmd
apmd(ok)

The result is:
# cat /etc/rc.conf.local
apmd_flags=
smtpd_flags=NO

According to the apmd man page the /etc/apm/resume program is run after resuming from standby, so that is the one where I need to put a call to /etc/netstart.

So I tried this:

# mkdir /etc/apm 

# nano /etc/apm/resume

and added the following:
#! /bin/sh
/etc/netstart

# chmod 755 /etc/apm/resume

Let's see if all is OK:
# rcctl stop apmd
# rcctl start apmd
apmd(ok)

Let's see if it works with the zzz command: 
# zzz
Suspending system...

and a few seconds later I type:
zzz
It resumes from its slumber.

However, it didn't seem to run the resume program.

Let's see what happened:
# tail /var/log/messages
apmd: failed to exec /etc/apm/resume: Exec format error

So, how now brown cow?

Eventually, I did two things to get it to work as explained below.  Don't ask me how I found these tricks, it is just years of experience with obstreperous embedded widgets coming to the rescue and a dogged determination to try various things till the hardware responds the way it should.

Make the netstart script executable, so I don't have to invoke a shell explicitly to run it:

# chmod +x /etc/netstart

Add a delay to the /etc/apm/resume script to give the USB widget time to load its firmware and let the magic fairy dust settle, before trying to configure it:
#! /bin/sh
sleep 1
/etc/netstart


Now I can make the netbook suspend and resume, the little green lights flash on the WiFi dongle and all is well, the whole universe shook, the BSD daemons sung and flowers fell down from the heavens...


Squash and Square the Web

British Prime Minister Harold Wilson said of the press:
If you can't squash them, square them 
and if you can't square them, squash them.

It is really only a couple hundred companies that are ruining the web for a couple billion people on the planet, so with even the littlest machine it is trivial to disable the culprits.

You can get a good /etc/hosts file to efficiently squash and square the rubbish on the wild wild web here:
https://github.com/StevenBlack/hosts

My little netbook now zooms like a much faster model, with no Windows, SELinux, systemd, advertisements or spyware to slow it down and now that Uncle Sam decided to ban laptops on board aircraft, this little one becomes the ideal machine to chuck into my luggage since nobody will steal what looks like a clunky old netbook!

Connect to Free WiFi Access Points

My WiFi setup is now automated for home use, but what if I travel and want to connect to free WiFi in McDonalds, or Vienna airport, or Cafe Nero, or.... ???

I solved this with a little script called mcdonalds - duh...
#! /bin/sh
ifconfig run0 nwid mcdonalds
dhclient run0

and ditto for the two or three other places I go to - good enough for me.

Play Internet Radio with deadbeef

On Linux, I use Streamtuner, a wonderful little application, but it doesn't have a BSD equivalent.  The solution is to run dillo and browse to http://www.internet-radio.com (39,520 radio stations!) however, I didn't have a music player installed yet, so:
# pkg_add deadbeef

Now when I run dillo and click a category on internet-radio, select a station, look at the left and download a playlist .pls file - save as - /home/herman/radio/whatever.pls  - thereafter, in the file browser simply double click the desired .pls file to play the radio station with deadbeef.

There is probably a way to get dillo to spawn deadbeef, but since I only ever listen to two or three radio stations, this is dead simple and now I actually can listen to Country and Western music - Och, my poor bleeding ears...

I then found that the speaker volume keys on the chicklet keyboard actually work too - neat!

OpenBSD on Virtualbox

If you are not quite ready to go bare metal, then you can install OpenBSD in Virtualbox, but since there is no Guest Additions for OBSD, you need to tweak things manually to get full screen operation.   

I actually prefer slightly smaller than full screen, so I can retain access to the host task manager bar more easily.  The easiest way to get the size right, is to take a screen shot by grabbing a rectangle, save the file and open it with a photo editor to see the dimensions.

There are are some weird limitations and  I could only get it to work with 16 bit video on my Lenovo and on my Macbook Pro I had to select 2560x1600x16 and Unscaled High Definition Video in the machine video properties to make it work full screen.

Define a custom screen for the OpenBSD VM like this, sized for my Lenovo Thinkpad:
$ VBoxManage setextradata OpenBSD CustomVideoMode1 1600x868x16

Start the VM and scp the below /etc/X11/xorg.conf file over to the VM. Modify the Depth, DefaultDepth and Modes at the bottom to suit:


Section "ServerLayout"
 Identifier     "X.org Configured"
 Screen      0  "Screen0" 0 0
 InputDevice    "Mouse0" "CorePointer"
 InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
 ModulePath   "/usr/X11R6/lib/modules"
 FontPath     "/usr/X11R6/lib/X11/fonts/misc/"
 FontPath     "/usr/X11R6/lib/X11/fonts/TTF/"
 FontPath     "/usr/X11R6/lib/X11/fonts/OTF"
 FontPath     "/usr/X11R6/lib/X11/fonts/Type1/"
 FontPath     "/usr/X11R6/lib/X11/fonts/100dpi/"
 FontPath     "/usr/X11R6/lib/X11/fonts/75dpi/"
EndSection

Section "Module"
 Load  "dbe"
 Load  "dri"
 Load  "extmod"
 Load  "glx"
 Load  "freetype"
EndSection

Section "InputDevice"
 Identifier  "Keyboard0"
 Driver      "kbd"
EndSection

Section "InputDevice"
 Identifier  "Mouse0"
 Driver      "mouse"
 Option     "Protocol" "wsmouse"
 Option     "Device" "/dev/wsmouse"
 Option     "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
 Identifier   "Monitor0"
 HorizSync    31-80
 VertRefresh  30-100
 VendorName   "Monitor Vendor"
 ModelName    "Monitor Model"
EndSection

Section "Device"
 Identifier  "Card0"
 Driver      "vesa"
 VendorName  "InnoTek"
 BoardName   "VirtualBox Graphics Adapter"
 BusID       "PCI:0:2:0"
EndSection

Section "Screen"
 DefaultDepth  16
 Identifier "Screen0"
 Device     "Card0"
 Monitor    "Monitor0"
 SubSection "Display"
  Viewport   0 0
  Depth     16
  Modes     "1600x868"
 EndSubSection
EndSection

When you launch startx, you should get a large window and may need Right-Control F to switch to full-screen mode.


For questions, go to http://daemonforums.org.

La voila!

Herman


Monday, January 23, 2017

Linux Network Manager Manual Commands

I have a love/hate relationship with the Linux NetworkManager daemon.  It usually works and keeps your laptop PC network connections going smoothly when you move around, but it gets in the way when one does network tests and system integration using a laptop PC.

Each time you plug a cable in, or turn an embedded system off/on, NetworkManager restarts the connection and you can then lose your static IP address setting, which gets tiring really quickly in a lab setup.

Usually, I completely disable NetworkManager and assign a static IP address to my machine on a laboratory bench with a script in /usr/local/bin called static:

#! /bin/bash
echo Configure network for PDLU access

# Disable the Network Manager
systemctl disable NetworkManager.service
systemctl stop NetworkManager.service

# Set static IP address
ifconfig em1 192.168.111.1 netmask 255.255.255.0 up

# Set multi casting route
route add -net 224.0.0.0 netmask 240.0.0.0 dev em1

# Open up the firewall
iptables -F

# Show setup
ifconfig
route


and when I get back to a desk, I set things back to normal with a script called dynamic:

#! /bin/bash
echo Configure network normally

# Disable the Network Manager
systemctl enable NetworkManager.service
systemctl restart NetworkManager.service
sleep 1

# Set multi casting route
route add -net 224.0.0.0 netmask 240.0.0.0 dev em1

# Show setup
sleep 1
ifconfig
route


Sometimes NetworkManager gets confused and the Aplet ends up spinning forever, trying to bring up a non-existent interface.

The way to fix these kind of issues is to invoke the command line program nmcli:

# nmcli connection show active
NAME                UUID                                  DEVICES      DEFAULT  VPN  MASTER-PATH
Wired connection 1  34111952-8271-4f64-a616-a6cd5899bae2  vboxnet0     no       no   --         
Wired connection 2  9dd23ba2-3378-4657-96bb-3b687cfe0180  enp0s29u1u2  yes      no   --       


Disable the errant interface:
# nmcli connection down id "Wired connection 1"

and finally delete it altogether:
# nmcli connection delete id "Wired connection 1"

This way I have a quiet GUI again without the irritating spinning widget in the corner.

La voila!

Herman