Monday, September 21, 2015

Zenity Progress Dialogue

Zenity is a nice and simple way to prettify Bash scripts, similar to kdialog, xdialog and a few others, but on the whole I think the best of the lot.  Though there is nothing preventing you from mixing them up if one has a feature that would be more useful for something.

I started using Zenity years ago, when there wasn't much documentation and consequently didn't use the progress dialogue the right way.  The result being that some scripts eventually broke.  For example, there was no information on how to make the progress bar progress - so I only used the whirr-whirr pulsate effect.

No Progress

In the past, I simply started a pulsater, pushed it into the background, saved the PID and later killed it when necessary, like this:

zenity --progress &
echo "Do something"
kill $PID

That used to worked fine, but recently on Fedora 22, it just sits there and does nothing.  It won't even blink.  Dead as a doornail.  Bah, humbug...

When All Else Fails...

Eventually, I found the documentation and RTFMed:

The problem is that the progress dialog has a new bug and it expects to receive something over stdin in order to get going.  If it never gets anything, it does nothing - in previous versions it worked fine in the background, but no-more.  When it is pushed into the background with &, it just sits there and sulks.

The correct way to make a progress pulsater that must do something and exit based on a condition, at which point one has to stop the pulsater, is like this:
while TRUE, do
  echo "This Will Do Nothing"
  echo "#Change The Text"
  sleep 5
  let "CNT+=1"

  if [ "$CNT" -eq 10 ]; then
    # Close the progress dialog with 100%
    echo "100"
    exit 0

  echo "Continue doing nothing"
) | zenity --progress \
--width=350 \
--title="The Title"
--text="The Default Text" \
--no-cancel \
--auto-close \

The --auto-close parameter will cause the progress dialogue to stop when you echo "100" percent.

An echo statement that starts with a # will change the text and one without, will be ignored.  The --no-cancel will suppress the Cancel button, since it doesn't make sense.

La voila!

Sunday, September 20, 2015

Version Control For Scripts

Running a full blown version control system such as subversion or CVS for Bash scripts is a bit of overkill.  Most people don't use any version control at all on scripts and revert back to the ancient way of saving multiple copies of the files.

The inevitable result is a mess of files called script1, script2, script.ok, script.bad... and then after a few days, you cannot remember which is which.

Fortunately, there is an ancient, light weight versioning system that is very easy to set up and which is perfect for the single user case: RCS.

Setup RCS

It is probably already installed (try whereis rcs), otherwise do:
# yum install rcs

In your script directory, make a directory to keep the archives:
$ mkdir RCS

Edit each script and add the $Id$ keyword to the top, right underneath the familiar #! /bin/bash command:
#! /bin/bash

That will substitute your name, date, time and version number of the file upon checkout, so you always know which version of the script you got.

Check In

As soon as you made an important change, check the file in, with ci:
$ ci scriptname

Type a proper description of the change, it is for your own benefit, when you have to look at a problem, possibly years later.

The file will disappear!

Check Out

Check it out again with co:
$ co scriptname 

By default you get the latest version.  You can also check out older versions using the -r parameter and switch between versions to see which one works best, without losing anything, since they are all in the archive file.

Important Tricks

If you are the only person working on the scripts, then remove strict locking.  This will make your life much easier:
$ rcs -U *

Now you can check files in with the -u option, so they don't disappear and gets checked out again immediately:
$ ci -u scriptname
You can also set an alias (add to the bottom of ~/.bashrc), to save you some more typing, so that you never have to use the co command again, just ci filename:
$ alias ci='ci -u'

That is pretty much all there is to it.

Do read the man pages of ci, co and rcs for details on what to do when the inevitable happens and you want to look at the logs and revert to an older version of a script.

Friday, September 11, 2015

If It Is Free, Take Two... Aspirin

Updates - Untracking Tools

I was wondering when privacy enhancing utilities would be come available. Here is the first one:

the second one:

and the third one:

I still think that the only secure way to use Windows 7, 8 and 10 is in a virtual machine with networking disabled.  The trouble is that you cannot stop encrypted tunnels with deep packet inspection in a router, so you have to unplug the network cable so to speak.

The only real solution is UNIX - buy a Mac, install Linux Fedora or PC-BSD.  Anything else is futile.

Free Candy

There are many an old proverb about free stuff, but the latest action by Microsoft takes the cake.

Even people who declined the free Windows 10 upgrade, will now also have this bloated piggy forced down their throats:

It is one thing sending out free security patches and feature upgrades, but a DVD sized download will cause lots of trouble for thousands of users.  Many a hard disk doesn't have that much free space and many users are on bandwidth limited networks and may get hefty overage bills, for something they did not ask for.

On the other hand, what MS may be finally admitting to, is that Windows 8 is so bad, that the only solution is to replace the whole rotten thing - for free.

Free Insecurity

To add insult to injury, Windows 10 has very serious security issues, which I have investigated a bit with tcpdump and listed in another post:

At this point in time, I cannot recommend to anyone to use Win10.  It is not worth the security risk.

Also, there are complaints that some of the new spyware features are also being pushed out to Windows 7 and 8.

Eventually, there will be some trustworthy third party firewall programs that will make it easy to strangle it all, but until then, Windows 7, 8 and 10 should only be used behind a separate draconian firewall device.  Using a vanilla Win10 laptop PC in a coffee shop or airport would be a disaster waiting to happen, like WinXP SP1 was.

A major problem with 3rd party firewalls though, is that I have never seen a proper review done by someone who actually tested it with a packet sniffer to see if it really works, since most reviewers are ordinary journalists with little or no technical knowledge, who don't know the first thing about tcpdump or wireshark.

The Real Problem: Dirty Tricks

I am not really worried about marketing data sent to Microsoft.  I am worried about what else gets sent in and out and who else is listening while the packets travel half way around the world.  Any government security agency with half a clue can plant incriminating data on Windows computer systems.  If MS can download a whole DVD worth of junkware to half the desktop machines in the world, then how hard could it be for the NSA or FSB to plant child porn, terrorism related junk, or classified data on a machine and then hit the owner with a long jail sentence?

If you were wondering about dirty tricks, have a look at what is happening to Hillary Clinton.  The US authorities has retroactively classified data that wasn't classified at the time.  Now, they can charge her and the previous administrator of her mail server and put both of them in the clink.  The only thing saving Hillary, is the fact that her husband is an ex president, but if I were her mail server administrator, then I'd move to another country, far, far away.

From Soft In The Middle, To Soft In The Head

What on earth were MS thinking?

-. --- / .-- .. -. -.. --- .-- ... --..-- / -. --- / -.-. .-. -.--



Sunday, September 6, 2015

UAE Visa Renewal and Car Selling Red Tape

Latest Visa Renewal Process in Al Ain - 2020

Go to SEHA Immigrant Screening Centre opposite Al Ain Hospital.
Hours: 7 am till 3 pm, Sun - Thu

You have three choices:
  • Go with the enormous crowd gathered under the palm trees...
  • Take the Fast Track - limited numbers - be there early
  • VIP service

VIP Medical Service

  • Go in the main front entrance and go upstairs. 
  • Walk through the Fast Track men's waiting room to the VIP waiting section.  
  • There is a tiny little VIP sign above the door.
  • There will be almost nobody, so the service is immediate.
  • You'll be done in about 20 minutes.
  • You need: Passport and passport copies (first 4 pages plus visa)
  • Cost: Dh500 (that is why there is almost nobody, but it is worth it)
  • In before 10h00, get results by 2 pm the same day (someone else may pick up with the receipt)
  • You need to get a blood test (no fasting) and a chest X-ray
  • They will send you a SMS when they are done around 1 pm - be there at 2 pm.
More info here:

Typing Office - Emirates ID

  • There is a Typing Office (Registry shop) to the East of Jimmy Mall, also in the Municipal building.
  • Renew your Emirates ID for Dh370
  • You will get another SMS if it worked in a few minutes
  • Take the form to the Emirates ID building and have it stamped at reception - they don't even check it.

New Visa

  • Now your employer has 10 days to get you a new visa, after which you can collect you new ID at the Post Office.
  • You have to give your employer your medical report, stamped Emirates ID form (Emiratis love stamps), a copy of your medical insurance card, four photos and your passport, and hope you get it back... 


Selling Your Old/Cheap Car - 2020

Most expats buy a cheap car, since they don't know how long they will stay in the UAE.  I ended up staying 11 years, but all good things eventually comes to an end and then you sit with one or two cars, that you don't quite know what to do with.  Some people abandon their car at the airport and fly off, but that will result in a 3000 Dirham fine that will hit you next time you fly through Dubai - so don't do that.

If you happen to live in Al Ain and you want to get your old/cheap car sold with the least amount of hassle, give Munaver Jaman a call at 055 499 4045.

The big difference is that he will come to you.

The only thing you need to do is go to the traffic police and tell them you want to sell your car and sign the transfer form.

After that, Munaver does everything between the seller, buyer and traffic police, online with his phone (some police confirmation codes are exchanged between everyone's smart phones to make it work) and he charges the buyer a small fee for the whole rigmarole.

So I can recommend this guy, he is quiet and sharp and knows what he is doing.

Closing Your Etisalat Accounts

Dealing with Etisalat (like any telecom company the world over!) is a total nightmare.  The only thing worse, is trying to call Revenue Canada from overseas.  Etisalat has numerous kiosks/shops in every major town, with a large crowd of people patiently waiting outside.  There is a ticket system, but you need to get in first, to get the ticket...
You can call them (101 for voice response, or 800 6665 for a real agent), but then you end up in an interminable queue with boring music on hold.  The queue can time out after an hour or two while you are on hold, so then you have to start all over again.  Whichever way you do it, it will be a day in your life that will never be refunded.

So, as with all telecom companies, the lesson is to never, never, never, set up automatic bill payment.  It is better to pay the bill manually every month, or better yet, pre-pay.  That way, when you want to leave, you can just leave and don't have to cancel anything - they will eventually figure out that the account is inactive - maybe.

Closing Your Bank Account

Your bank will have a minimum balance indicated in your terms of service.  For FAB, the minimum is 3000 Dirhams.  When the account reaches zero, the bank will automatically close the account.

So there are two ways to go about it:
  • Use your smartphone and transfer everything you can to your home country, but then Dh3000 ($817) would likely end up as a gift to the bank - ouch...
  • Go to the bank and wait in line to talk to an agent, transfer every last drop and close the account.

It will help if you don't have a credit card, or any automatic bill payments, otherwise you may lose another day of your life.

La voila!


Friday, September 4, 2015

Audio Networking with Sox and Netcat

Sox + Netcat = VoIP

The idea is to hook the sound devices on two machines together so that there is a bidirectional intercom between them.  This way, you can have an amateur SSB radio hooked to a distant computer in your radio shack and a headset on a local computer in your living room and do VoIP with the remote radio transceiver over your LAN.

I noticed that there is a steady stream of people reading this post.  Please note that Sox with CVSD and netcat will work fine and consume very little processing time on a tiny ARM embedded system, but gstreamer with raw audio over UDP may be a better choice if you have a half decent system on both ends of the link and want the best quality audio.  Therefore, do look into gstreamer also.

For a remote ham radio, the main thing missing, is the PTT switch, which one can do with another netcat proxy to the serial control port of the transceiver and radio control software running locally to set the channel and key the radio.

For streaming, one needs a headerless, self synchronizing CODEC protocol, to connect to a running stream.  Examples are ADPCM, CVSD and LPC10.

Sox gotchas

Sound Exchange has three programs: sox, rec and play.  The difference is that if you run sox, it will glom onto the sound device for both read and write, but it can only do one thing at a time, so it can either record or play, but not both at the same time.

The rec and play programs however, can run independently and concurrently, so those can be used to make a bidirectional intercom.

UNIX systems have many different sound systems: OSS, ALSA, Coreaudio, Pulseaudio, Jack...  Sox knows how to handle them, if you let it use the default settings.  So get the default to work first with a simple mixer application such as aumix, rec and play, before you try something complicated.

Sox also has internal buffering that is extremely large: 8 kilobytes.  The result is that sox by default has a huge delay.  When you use compression, then the more you compress, the longer the delay gets, because the slower stream takes longer to fill the buffers.  You can reduce that with the --buffer parameter.

In general, a simpler sound system such as ALSA or OSS will have smaller buffers and lower delays than the complex pulseaudio or coreaudio, so stick to the basics for best results.

The general syntax is: sox inputspec inputfile outputspec outputfile

A lone "-" instead of a file, tell sox to use stdio or a pipe, so read the examples carefully.  The data flow is from left to right in all the below examples.

Default setup

First get sox to work on the default sound device:

Make a noise:
$ cat /dev/urandom | sox -traw -r44100 -b16 -eunsigned-integer - -d

Play a tone:
$ play -n synth 10 sin 500

$ rec -r8000 -tcvsd file

$ play -r8000 -tcvsd file

Don't bother with the below if you cannot get the above to work.

Hook sox to netcat on two computers:

Assuming IP addresses and

With CVSD, be sure to set the sample rate and reduce the buffer size to reduce the delays.  The resulting network bandwidth is about 1 kilobyte per second and the end to end delay is about 1 second.  CVSD is a very simple CODEC, so it works well on an embedded processor.

On the first computer:
$ rec —buffer 32 -tcvsd -r8000 - | nc -u -l 5555 | play —buffer 32 -tcvsd -r8000 -

and on the second computer:
$ rec —buffer 32 -tcvsd -r8000 - | nc -u 5555 | play —buffer 32 -tcvsd -r8000 -

Just two one liners!

The above works and I had it running between a Macbook Pro with OSX (Mac sox is available from Homebrew) and a Fedora Linux laptop.  I also experimented on a BSD laptop and it behaves the same.

The main problem is with delays due to buffering and the more one compresses the data stream, the longer it takes to fill the pipes, hence longer delays.  The LPC10 CODEC will reduce the network bandwidth further, but the delay will be ridiculous.

To make it robust, you may want to put a while loop with a sleep delay around the above, so that if sox or netcat would exit for whatever reason, it would start again, after waiting a little bit for the dust to settle:
$ while true; do rec —buffer 32 -tcvsd -r8000 - | nc -u 5555 | play —buffer 32 -tcvsd -r8000 -; sleep 1; done

...and it is still a one liner!  OK, technically, it is three.

It took me about 3 days to figure the above out, so when all else fails, RTFM and try again.

It is certainly much better than two paper cups and string...

La voila!