Sunday, May 15, 2016

Whole Disk Encryption

Many people, even card carrying computer geeks, do not understand why a computer hard disk must be encrypted and why a computer must be shut down for the disk encryption to be effective.

This applies, whether you are using Bitlocker, Filevault, PGP, GPG or LUKS.

Why Encrypt Your Disks?

If the disk is not encrypted, then a miscreant can boot the computer with a USB stick or CD and read everything on the disk, or plant incriminating data on your disk, and then call the police, or wait for you to go through a border post where your machine may get searched - then watch you end up in the slammer.

Also, if the disk drive controller would fail and you replace the disk and chuck it in the trash, then the data is still accessible, if someone would replace the drive controller from an identical disk bought on Ebay.

If your laptop PC gets stolen, then it can end up on Ebay, with all your data and the buyer can empty your bank account, order a bunch of credit cards in your name, or sell your house for you and run away with the money.

Even if you are a broke student with no money, someone can still order a credit card in your name and use it.

Therefore, if you don't want to incur thousands of dollars in losses, a ruined credit rating and huge amounts in legal fees to sort out the resulting mess, then you have to encrypt your data.

Protecting Data at Rest

When the machine is powered on and running, the encryption keys are kept in RAM and the data is accessible to you, and an attacker.  So obviously, you should never leave a running computer alone.

When the machine is shut down, the RAM gradually loses power, the keys get lost and your data is safe and 'at rest' - or that is the idea anyway.

On a server, the memory has error correcting capabilities and always powers up in a zero state.  On a cheap laptop computer, the memory doesn't have error correction and the RAM can retain data for several minutes after the machine was powered down.

Therefore, it is possible for a miscreant to quickly boot a recently abandoned laptop PC and read the memory using a special tool (, recover the passwords and keys, then dump the disk and go away and analyse it at his leisure.

Suspend (save in RAM) / Hibernate (save on disk)

When you slam the lid of your laptop, it typically suspends by writing CPU data to RAM and then partially powers down, leaving the memory powered up to preserve the data.  When the battery gets low, it will wake up, write the data to the hard disk and then power down completely.  When you power up again, it will restore itself and get going again.

If the machine suspended, then it can be (quickly) rebooted from a USB stick or CD and the contents of RAM can be read.  The data in RAM is in plain text and a search through the raw data will reveal your passwords and encryption keys, the last files you worked on, even the last PGP encrypted email you sent may have a plain text copy in RAM.  Therefore, suspend is risky and should be avoided.

If the machine hibernated, then the hibernate file can be read from disk, by booting from a USB stick or CD and a search through the raw data could reveal your passwords and encryption keys.  However, if the disk is encrypted, then the hibernate file will be unreadable. Therefore hibernate is potentially secure, if the encryption is done right.

BIOS Boot Options and Password

It doesn't help locking the front door of your house and putting the key under the carpet, or leaving the back door open.  Disk encryption must be done properly and all loopholes must be closed, else it is ineffective.  As shown above, suspending a machine to RAM and leaving it on your desk, is like leaving a slowly vanishing key under the carpet.

It is therefore important to buy yourself time - 10 minutes or more - and make it hard for an attacker to bypass the encryption, by making it difficult to boot from a TFTP server, CD or USB stick.

Therefore, in the BIOS, change the boot order so that the machine will not boot from a network server or removable media and set a BIOS password, so that an attacker cannot easily change the boot order.

Another interesting factoid, is that enabling a BIOS RAM check option - if available - would also help to destroy the RAM data while rebooting.

A determined attacker can open the PC, and reset the BIOS memory chip using a link, or by removing the battery, but that is hard to do on a laptop and it means that the perp needs to do multiple attacks - first to change the BIOS settings and later, to recover your keys and dump the disk.  Therefore setting a BIOS password makes it much more likely that he'll get caught.

Also, if dragoons are bashing your door down, or if you have to cross a border post, it may be a good idea to pull the battery from your laptop PC.

Is This For Real?

I just tried it on my Windows 7 laptop PC with PGP encrypted disk using the McGrew CD to boot and a memory stick for the data and it worked - nuff sed.

Écrasez l'infâme,


Sunday, April 24, 2016

Bash Ctrl-C Cleanup

Most scripts are pretty simple things, but sometimes one writes a monster and it may create several temporary files, lock files and other detritus while running.  Sometimes you will be nice and delete the junk before the script exits.  

However, if the user would terminate the script forcefully with Ctrl-C or kill, then the garbage will not be removed.

Here is the proper way to handle that in Bash:

 # Trap keyboard interrupt (control-c)

 # Control-C Press
   exit $?

   rm /var/lock/mylockfile
   return $?

La voila!


Sunday, April 17, 2016

A Few Pros and Cons of UNIX

I don't like pros and cons discussions on operating systems, since the people who need to read it won't, but as the old sage said: Why do you have a mind, if you cannot change it?

USB Support

The Windows device names of USB serial ports keep on changing.  I have seen a machine in our lab using COM57:.  That means that half a hundred times, some poor engineer sat in front of that machine and wondered why on earth his serial port wasn't working.

Trying to deploy a system that uses USB serial ports on Windows is a nightmare, since each and every machine has a different setup and it changes at a whim, so you need to open up low level configuration to the end user and try to explain to him how to change it in the manual and have a technician on standby to support all the users who won't read the manuals.

IT services also like to disable USB mass storage support, since Windows suffers from a 25 year old security bug that Microsoft is unable to fix.  So whenever an engineer captures data on an oscilloscope/network analyzer/ spectrum analyzer/camera system on a USB memory stick, he has no way to view the data on the PC and has to go and find a Linux PC.

Now multiply all that wasted engineering time by all the other Windows machines in your lab.

File Systems

UNIX is a Rosetta Stone of file systems.  It has specialized storage solutions for everything: Large block flash chips, small block flash chips, spinning metal, big files, small files, redundancy, logical volumes, RAID, compression, encryption, snapshots, deduplication...

Windows has only two file systems, NTFS and ReFS.  No, that other one isn't really a file system.

With deduplication, if you copy a file, it doesn't use any more disk space.  Just think about that one for a second, then go and read up on snapshots and then hold a moment of silence for the Windows users.

Real-Time Operation

Linux has been a real-time OS for more than ten years - since 2002 - fixed by Prof Ingo Molnar in Calgary, Canada.  On single processor machines, the spin locks compile away.  On multi-processor machines, spin locks are pre-emptable.  The Completely Fair Scheduler excels at distributing the work load over the machine cores and it consistently reacts to interrupts in a few microseconds.  BSD and OSX are not bad either.

Windows Vista and 7 have an old fashioned priority scheduler with coarse time slices of 15 ms and slow, non-preemptible interrupt handlers. The result is that whereas an interrupt is usually handled in 10 us, it could sometimes take 500 ms.

Some programs, notably .Net and Chrome, change the Windows 7 time slice to 1 ms, same as on Windows 8 and 10, but it doesn't help significantly.  The Windows debugger and latency measurement tools do not provide an accurate picture, which indicates to me that the tools are starved too and therefore cannot measure time accurately.  This is probably due to contention and non pre-emptable spin locks.  

The Windows scheduler is strictly priority based.  As long as there are processes with a higher priority than yours, then your process will get scheduled ZERO processing time.

Contention and deadlock is handled very inefficiently on Windows.  Processes that are blocked, will periodically have their priority raised to a very high level and will get a big time slice.  This can result in very irregular execution of other time sensitive processes.

Windows does not handle multiprocessing properly.  Threads stick to the processor core that it first ran on.  When many processes start up and some shut down after a while, the remaining ones will not be redistributed to even the work load.

If your medium priority process is stuck on a core with high priority tasks, then your process will not run, despite there being idle cores, or cores loaded with low priority tasks.

The whole mess is described here:

In order to maintain smooth video playback on Windows, you need to maintain very large buffers of 1.5 to 3 seconds.  Therefore, Windows can play streaming television, but it cannot play video from a real-time observation camera with a hand controller properly.

Windows doesn't scale.  Due to the bad scheduler, adding processor cores and memory, does not make Windows significantly faster.

Link Aggregation

Windows desktop OS doesn't support more than one ethernet port properly.  To do link aggregation, or any 'advanced' routing, you need a server version.

LAGG and advanced routing is of course a native part of any UNIX/Linux/Mac OS and many (most?) fancy switches and routers actually run Linux.


Well, I got to mention one, otherwise I have to change the title.

UNIX isn't unfriendly.  It is just very choosy about who exactly its friends are.

UNIX can be rather complicated.   You may even have to read a book or three.  I first studied UNIX postgrad in 1985.  It really hasn't changed much since from a user point of view, so the time spent reading books and man pages was a good investment.

. -.-. .-. .- ... . --..  .-.. .  .. -. ..-. .- -- .


Friday, March 18, 2016

Debian Installation for Control Freaks

Installation of embedded systems present a unique circumstance, because one usually wants to create a system that can be replicated identically.  You may also have to save the whole repository in a Configuration Management system in order to keep strict control over the file versions, so you can do an update once a year or three.

The Debian/Ubuntu installer typically starts from an ISO image on a CD or USB memory stick, but thereafter wants to go online to consult a mirror server somewhere else in the world.  The moment that happens, you lose control over what exactly is installed on your machine.

Make a Bootable USB Stick

For the last couple years, Ubuntu ISO files are dual mode - bootable on CD and USB, same as Red Hat.

Download a server ISO from a mirror server, e.g. Yandex:

Plug the stick in and check the device name with dmesg!
# dmesg

Write it to a USB stick with Data Definition:
# dd if=filename.iso of=/dev/sdb bs=1M

We are Paranoid Control Freaks right?  Get the MD5SUM file from a different mirror server and run the check on the downloaded ISO files:
# md5sum -c MD5SUMS

...and if you are wearing a tin-foil or armadillo hat, use the SHA256 sums also.

Please Sir, can I have more?

So, after you performed a minimal install and the embedded system is running, how can you install more things from the CD image without going online?

The best solution is to create a private mirror server, but one can also install offline using the original ISO images only, by making a few tweaks in the /etc/apt/sources.list file.

If a system has a large hard disk, then one can copy all the ISO files to the HDD and then permanently loop mount them in a fixed place in /mnt using the fstab file.  After that, a suitable file:// entry in the sources list will cause apt, dpkg and aptitude to refer to them instead of going online all the time.

USB Stick Repository

If the distribution ISO files are on a USB memory stick, then they are not always available, but you can ensure that they will always have the same path if you give the stick a proper volume label using gparted, such as USBISO for argument's sake.  The stick will then always mount with the path /run/media/username/USBISO, which should be good enough that you can make a script of the following.

This works better with Debian, for which you can get the complete repository as a series of ISO files.

Make a mount point:
mkdir /mnt/mountpoint

Mount the downloaded ISO file:
mount -t iso9660 -o loop /media/run/username/volumelabel/distrofile.iso /mnt/mountpoint

Add the following line to the top of /etc/apt/sources.list and comment out everything else:
deb file://mnt/mountpoint distroname main restricted

(Of course, use your own username, volumelabel, distrofile, mountpoint and distroname)

From now on, provided that you can find the little stick again (possibly just leave it plugged in forever), you can mount it and when you run apt-get, it will get things from the ISO image file, not some obscure server elsewhere in the world.

Mirror, Mirror on The Wall

Once you get your own web server running with lighttpd and mounted an ISO file with the whole distribution, then simply edit the /etc/apt/sources.list file again and add a suitable deb http://yourmirroripaddress/pathtofiles distroname main restricted entry, same as the other examples in that file.

Just be sure to comment out everything else that doesn't apply and from then on, apt-get will pull files off your web server (More information on mirroring at the bottom of this page here:

Debian/Ubuntu/Fedora *nix Confusion

My  mirror server happens to be a Fedora Linux machine and it is also used to serve the preseed auto-response file and .deb packages for an automated Ubuntu LTS installation, so the descriptions below are a little mixed up - dnf is the Fedora install program and apt-get is the Debian/Ubuntu install program.  Kickstart is for Redhat and Preseed is for Debian.

Vive la difference.

Configure lighttpd on Fedora

For automated installation, a web server is required to serve the Debian preseed file and the .deb packages.

# dnf install -y lighttpd

Edit /etc/lighttpd/lighttpd.conf:

var.server_root = "/var/www"

Make a directory:

#mkdir /var/www/htdocs

Make the file /var/www/htdocs/index.html

Start the server:

# systemctl enable lighttpd.service
systemctl start lighttpd.service

Test the server:

$ firefox http://localhost

Configure dhcpd on Fedora

A DHCP server is necessary to provide an IP address to the new system.  It can also be used to provide the file server name and path to the preseed file, but that level of automation can be a little dangerous, since one could then accidentally re-install a system.


# dnf install -y dhcpd-server

Edit file /etc/dhcp/dhcpd.conf:
default-lease-time 600;
max-lease-time 7200;
option subnet-mask;
option broadcast-address;
option routers;
option domain-name "";
option domain-name-servers;

subnet netmask {

Start the server:

# systemctl enable dhcpd.service
systemctl start dhcpd.service

Configure Standard Ethernet Device Names on Fedora

Fedora uses weird device names for the ethernet ports which break my scripts.  Force Fedora to name the ports eth0..n with a small change in the grub configuration file.

Edit file /etc/default/grub:

Add to the end of GRUB_CMDLINE_LINUX, net.ifnames=0

Configure grub:
# grub2-mkconfig -o /boot/grub2/grub.cfg

Edit /etc/sysonfig/ifcfg-ewhatever:

# cd /etc/sysconfig/network-scripts/

nano ifcfg-whatever

Change the name of whatever to eth0

# reboot

Disable The Fedora Firewall

To let web server requests through, flush the iptables rules:

# iptables -F
# systemctl stop firewalld.service
# systemctl disable firewalld.service

Mount The ISO Files on Fedora

Copy the server, desktop and source ISO files to /home/herman/ISO, then make mount points in /var/www/htdocs:

# mkdir /var/www/htdocs/server

Add a mount line to /etc/fstab for each ISO file:

/home/herman/ISO/ubuntu-14.04.4-server-amd64+mac.iso /var/www/htdocs/server iso9660 loop 0 0

Mount them all:

# mount -a

Now the web server can serve the contents of the ISO files!

(Note that Fedora runs SELinux and if you would put a symlink to an area outside /var/www, then you need to add a rule and relabel with semanager, else you will get perplexing 403 Forbidden errors.)

Auto-install An Ubuntu Embedded System Or Server

Put the preseed file in the /var/www/htdocs root of the web server and as a minimum, replace the "file=..." in the boot prompt of the new computer to be installed with "auto url=http://webserveripaddress/preseedfilename".  Which is a bit easier said than done.

Two Boot Prompts - Cannot Find Kernel

The problem with preseeding, is that it only reads the file after the network setup is done, with the result that the first bunch of entries in the preseed file regarding the keyboard, screen, hostname, domainname and ethernet port, need to be specified on the boot command line (or you have to answer the prompts interactively).

The other problem is that the system has two different boot prompts. Yes, you heard that right.  Somebody deserves seven lashes with a wet noodle for this one.

Set the BIOS so the thing can boot off USB and stick an Ubuntu server ISO thingy in.  Boot up and wait for the language selection screenDo not press Esc during boot.  The blank boot prompt that you will get early on after pressing escape is the wrong one.  

No matter what you type in the blank boot prompt, it will always result in a Cannot find kernel whatever errorThat was a serious time waster and prompted me to write all this. 

Wait For The Language Screen

Once at the language screen, wait a little bit for good measure, then press Esc. You will then get back to the install menu.  Press F6 and then Esc to get a boot prompt with a default string in it looking like this:
Boot: file=/cdrom/preseed/ubuntu-server.seed vga=788 initrd=/install/initrd.gz quiet --

Edit that string to look like this (backspace deletes):
Boot: auto url= locale=en_US console-setup/ask_detect=false keyboard-configuration/layoutcode=us interface=eth0 hostname=test initrd=/install/initrd.gz quiet --

Now press Enter and it should start with only a few remaining questions and eventually read the preseed.txt file and execute it.

Run tcpdump -nlX -i eth0 on the server to see what is going on.

Preseed File Example For An Ubuntu Embedded System Or Server

This is a minimal server install.  Change the username and password.  You can add more packages at the very bottom.

#### Contents of the preconfiguration file (for squeeze)
### Localization
# Preseeding only locale sets language, country and locale.
d-i debian-installer/locale string en_US

# The values can also be preseeded individually for greater flexibility.
#d-i debian-installer/language string en
#d-i debian-installer/country string NL
#d-i debian-installer/locale string en_GB.UTF-8
# Optionally specify additional locales to be generated.
#d-i localechooser/supported-locales en_US.UTF-8, nl_NL.UTF-8

# Keyboard selection.
# Disable automatic (interactive) keymap detection.
d-i console-setup/ask_detect boolean false
#d-i keyboard-configuration/modelcode string pc105
d-i keyboard-configuration/layoutcode string us
# To select a variant of the selected layout (if you leave this out, the
# basic form of the layout will be used):
#d-i keyboard-configuration/variantcode string dvorak

### Network configuration
# Disable network configuration entirely. This is useful for cdrom
# installations on non-networked devices where the network questions,
# warning and long timeouts are a nuisance.
#d-i netcfg/enable boolean false

# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
#d-i netcfg/choose_interface select auto

# To pick a particular interface instead:
d-i netcfg/choose_interface select eth0

# If you have a slow dhcp server and the installer times out waiting for
# it, this might be useful.
d-i netcfg/dhcp_timeout string 60

# If you prefer to configure the network manually, uncomment this line and
# the static network configuration below.
#d-i netcfg/disable_autoconfig boolean true

# If you want the preconfiguration file to work on systems both with and
# without a dhcp server, uncomment these lines and the static network
# configuration below.
d-i netcfg/dhcp_failed note
d-i netcfg/dhcp_options select Configure network manually

# Static network configuration.
d-i netcfg/get_nameservers string
d-i netcfg/get_ipaddress string
d-i netcfg/get_netmask string
d-i netcfg/get_gateway string
d-i netcfg/confirm_static boolean true

# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string gcu
d-i netcfg/get_domain string

# Disable that annoying WEP key dialog.
d-i netcfg/wireless_wep string
# The wacky dhcp hostname that some ISPs use as a password of sorts.
#d-i netcfg/dhcp_hostname string radish

# If non-free firmware is needed for the network or other hardware, you can
# configure the installer to always try to load it, without prompting. Or
# change to false to disable asking.
#d-i hw-detect/load_firmware boolean true

### Network console
# Use the following settings if you wish to make use of the network-console
# component for remote installation over SSH. This only makes sense if you
# intend to perform the remainder of the installation manually.
#d-i anna/choose_modules string network-console
#d-i network-console/password password r00tme
#d-i network-console/password-again password r00tme
# Use this instead if you prefer to use key-based authentication
#d-i network-console/authorized_keys_url http://host/authorized_keys

### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string
d-i mirror/http/directory string /server
d-i mirror/http/proxy ""

# Alternatively: by default, the installer uses where
# CC is the ISO-3166-2 code for the selected country. You can preseed this
# so that it does so without asking.
#d-i mirror/http/mirror select

# Suite to install.
#d-i mirror/suite string squeeze
# Suite to use for loading installer components (optional).
#d-i mirror/udeb/suite string squeeze
# Components to use for loading installer components (optional).
#d-i mirror/udeb/components multiselect main, restricted

### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true

# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string Asia/Dubai

# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean false
# NTP server to use. The default is almost always fine here.
#d-i clock-setup/ntp-server string

### Partitioning
## Partitioning example
# If the system has free space you can choose to only partition that space.
# This is only honoured if partman-auto/method (below) is not set.
# Alternatives: custom, some_device, some_device_crypto, some_device_lvm.
#d-i partman-auto/init_automatically_partition select biggest_free

# Alternatively, you may specify a disk to partition. If the system has only
# one disk the installer will default to using that, but otherwise the device
# name must be given in traditional, non-devfs format (so e.g. /dev/hda or
# /dev/sda, and not e.g. /dev/discs/disc0/disc).
# For example, to use the first SCSI/SATA hard disk:
d-i partman-auto/disk string /dev/sda
# In addition, you'll need to specify the method to use.
# The presently available methods are:
# - regular: use the usual partition types for your architecture
# - lvm:     use LVM to partition the disk
# - crypto:  use LVM within an encrypted partition
d-i partman-auto/method string regular

# If one of the disks that are going to be automatically partitioned
# contains an old LVM configuration, the user will normally receive a
# warning. This can be preseeded away...
d-i partman-lvm/device_remove_lvm boolean true
# The same applies to pre-existing software RAID array:
d-i partman-md/device_remove_md boolean true
# And the same goes for the confirmation to write the lvm partitions.
d-i partman-lvm/confirm boolean true

# For LVM partitioning, you can select how much of the volume group to use
# for logical volumes.
#d-i partman-auto-lvm/guided_size string max
#d-i partman-auto-lvm/guided_size string 10GB
#d-i partman-auto-lvm/guided_size string 50%

# You can choose one of the three predefined partitioning recipes:
# - atomic: all files in one partition
# - home:   separate /home partition
# - multi:  separate /home, /usr, /var, and /tmp partitions
d-i partman-auto/choose_recipe select atomic

# Or provide a recipe of your own...
# If you have a way to get a recipe file into the d-i environment, you can
# just point at it.
#d-i partman-auto/expert_recipe_file string /hd-media/recipe

# If not, you can put an entire recipe into the preconfiguration file in one
# (logical) line. This example creates a small /boot partition, suitable
# swap, and uses the rest of the space for the root partition:
#d-i partman-auto/expert_recipe string                         \
#      boot-root ::                                            \
#              40 50 100 ext3                                  \
#                      $primary{ } $bootable{ }                \
#                      method{ format } format{ }              \
#                      use_filesystem{ } filesystem{ ext3 }    \
#                      mountpoint{ /boot }                     \
#              .                                               \
#              500 10000 1000000000 ext3                       \
#                      method{ format } format{ }              \
#                      use_filesystem{ } filesystem{ ext3 }    \
#                      mountpoint{ / }                         \
#              .                                               \
#              64 512 300% linux-swap                          \
#                      method{ swap } format{ }                \
#              .

# If you just want to change the default filesystem from ext3 to something
# else, you can do that without providing a full recipe.
d-i partman/default_filesystem string ext4

# The full recipe format is documented in the file partman-auto-recipe.txt
# included in the 'debian-installer' package or available from D-I source
# repository. This also documents how to specify settings such as file
# system labels, volume group names and which physical devices to include
# in a volume group.

# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

## Partitioning using RAID
# The method should be set to "raid".
#d-i partman-auto/method string raid
# Specify the disks to be partitioned. They will all get the same layout,
# so this will only work if the disks are the same size.
#d-i partman-auto/disk string /dev/sda /dev/sdb

# Next you need to specify the physical partitions that will be used.
#d-i partman-auto/expert_recipe string \
#      multiraid ::                                         \
#              1000 5000 4000 raid                          \
#                      $primary{ } method{ raid }           \
#              .                                            \
#              64 512 300% raid                             \
#                      method{ raid }                       \
#              .                                            \
#              500 10000 1000000000 raid                    \
#                      method{ raid }                       \
#              .

# Last you need to specify how the previously defined partitions will be
# used in the RAID setup. Remember to use the correct partition numbers
# for logical partitions. RAID levels 0, 1, 5, 6 and 10 are supported;
# devices are separated using "#".
# Parameters are:
# <raidtype> <devcount> <sparecount> <fstype> <mountpoint> \
#          <devices> <sparedevices>

#d-i partman-auto-raid/recipe string \
#    1 2 0 ext3 /                    \
#          /dev/sda1#/dev/sdb1       \
#    .                               \
#    1 2 0 swap -                    \
#          /dev/sda5#/dev/sdb5       \
#    .                               \
#    0 2 0 ext3 /home                \
#          /dev/sda6#/dev/sdb6       \
#    .

# For additional information see the file partman-auto-raid-recipe.txt
# included in the 'debian-installer' package or available from D-I source
# repository.

# This makes partman automatically partition without confirmation.
d-i partman-md/confirm boolean true
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

## Controlling how partitions are mounted
# The default is to mount by UUID, but you can also choose "traditional" to
# use traditional device names, or "label" to try filesystem labels before
# falling back to UUIDs.
d-i partman/mount_style select label

### Base system installation
# Configure APT to not install recommended packages by default. Use of this
# option can result in an incomplete system and should only be used by very
# experienced users.
#d-i base-installer/install-recommends boolean false

# The kernel image (meta) package to be installed; "none" can be used if no
# kernel is to be installed.
d-i base-installer/kernel/image string linux-generic

### Account setup
# Skip creation of a root account (normal user account will be able to
# use sudo). The default is false; preseed this to true if you want to set
# a root password.
#d-i passwd/root-login boolean false
# Alternatively, to skip creation of a normal user account.
#d-i passwd/make-user boolean false

# Root password, either in clear text
#d-i passwd/root-password password r00tme
#d-i passwd/root-password-again password r00tme
# or encrypted using an MD5 hash.
#d-i passwd/root-password-crypted password [MD5 hash]

# To create a normal user account.
d-i passwd/user-fullname string JoePlumber
d-i passwd/username string joeplumber
# Normal user's password, either in clear text
d-i passwd/user-password password r00tme
d-i passwd/user-password-again password r00tme
# or encrypted using an MD5 hash.
#d-i passwd/user-password-crypted password [MD5 hash]
# Create the first user with the specified UID instead of the default.
#d-i passwd/user-uid string 1010
# The installer will warn about weak passwords. If you are sure you know
# what you're doing and want to override it, uncomment this.
d-i user-setup/allow-password-weak boolean true

# The user account will be added to some standard initial groups. To
# override that, use this.
d-i passwd/user-default-groups string audio cdrom video dialout usb

# Set to true if you want to encrypt the first user's home directory.
d-i user-setup/encrypt-home boolean false

### Apt setup
# You can choose to install restricted and universe software, or to install
# software from the backports repository.
d-i apt-setup/restricted boolean true
d-i apt-setup/universe boolean true
#d-i apt-setup/backports boolean true
# Uncomment this if you don't want to use a network mirror.
#d-i apt-setup/use_mirror boolean false
# Select which update services to use; define the mirrors to be used.
# Values shown below are the normal defaults.
#d-i apt-setup/services-select multiselect security
#d-i apt-setup/security_host string
#d-i apt-setup/security_path string /ubuntu

# Additional repositories, local[0-9] available
d-i apt-setup/local0/repository string \ trusty main restricted
#d-i apt-setup/local0/comment string local server
# Enable deb-src lines
#d-i apt-setup/local0/source boolean true
# URL to the public key of the local repository; you must provide a key or
# apt will complain about the unauthenticated repository and so the
# sources.list line will be left commented out
#d-i apt-setup/local0/key string http://local.server/key

# By default the installer requires that repositories be authenticated
# using a known gpg key. This setting can be used to disable that
# authentication. Warning: Insecure, not recommended.
d-i debian-installer/allow_unauthenticated boolean true

### Package selection
tasksel tasksel/first multiselect ubuntu-server
#tasksel tasksel/first multiselect lamp-server, print-server
#tasksel tasksel/first multiselect kubuntu-desktop

# Individual additional packages to install
d-i pkgsel/include string openssh-server build-essential
# Whether to upgrade packages after debootstrap.
# Allowed values: none, safe-upgrade, full-upgrade
d-i pkgsel/upgrade select none

# Language pack selection
d-i pkgsel/language-packs multiselect en

# Policy for applying updates. May be "none" (no automatic updates),
# "unattended-upgrades" (install security updates automatically), or
# "landscape" (manage system with Landscape).
d-i pkgsel/update-policy select none

# Some versions of the installer can report back on what software you have
# installed, and what software you use. The default is not to report back,
# but sending reports helps the project determine what software is most
# popular and include it on CDs.
popularity-contest popularity-contest/participate boolean false

# By default, the system's locate database will be updated after the
# installer has finished installing most packages. This may take a while, so
# if you don't want it, you can set this to "false" to turn it off.
d-i pkgsel/updatedb boolean true

### Boot loader installation
# Grub is the default boot loader (for x86). If you want lilo installed
# instead, uncomment this:
#d-i grub-installer/skip boolean true
# To also skip installing lilo, and install no bootloader, uncomment this
# too:
#d-i lilo-installer/skip boolean true

# With a few exceptions for unusual partitioning setups, GRUB 2 is now the
# default. If you need GRUB Legacy for some particular reason, then
# uncomment this:
#d-i grub-installer/grub2_instead_of_grub_legacy boolean false

# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true

# This one makes grub-installer install to the MBR if it also finds some other
# OS, which is less safe as it might not be able to boot that other OS.
d-i grub-installer/with_other_os boolean false

# Alternatively, if you want to install to a location other than the mbr,
# uncomment and edit these lines:
#d-i grub-installer/only_debian boolean false
#d-i grub-installer/with_other_os boolean false
#d-i grub-installer/bootdev  string (hd0,0)
# To install grub to multiple disks:
#d-i grub-installer/bootdev  string (hd0,0) (hd1,0) (hd2,0)

# Optional password for grub, either in clear text
d-i grub-installer/password password r00tme
d-i grub-installer/password-again password r00tme
# or encrypted using an MD5 hash, see grub-md5-crypt(8).
#d-i grub-installer/password-crypted password [MD5 hash]

# Use the following option to add additional boot parameters for the
# installed system (if supported by the bootloader installer).
# Note: options passed to the installer will be added automatically.
#d-i debian-installer/add-kernel-opts string nousb

### Finishing up the installation
# During installations from serial console, the regular virtual consoles
# (VT1-VT6) are normally disabled in /etc/inittab. Uncomment the next
# line to prevent this.
#d-i finish-install/keep-consoles boolean true

# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note

# This will prevent the installer from ejecting the CD during the reboot,
# which is useful in some situations.
#d-i cdrom-detect/eject boolean false

# This is how to make the installer shutdown when finished, but not
# reboot into the installed system.d-i debian-installer/exit/halt boolean true
# This will power off the machine instead of just halting it.
#d-i debian-installer/exit/poweroff boolean true

### X configuration
# X can detect the right driver for some cards, but if you're preseeding,
# you override whatever it chooses. Still, vesa will work most places.
#xserver-xorg xserver-xorg/config/device/driver select vesa

# A caveat with mouse autodetection is that if it fails, X will retry it
# over and over. So if it's preseeded to be done, there is a possibility of
# an infinite loop if the mouse is not autodetected.
#xserver-xorg xserver-xorg/autodetect_mouse boolean true

# Monitor autodetection is recommended.
xserver-xorg xserver-xorg/autodetect_monitor boolean true
# Uncomment if you have an LCD display.
xserver-xorg xserver-xorg/config/monitor/lcd boolean true
# X has three configuration paths for the monitor. Here's how to preseed
# the "medium" path, which is always available. The "simple" path may not
# be available, and the "advanced" path asks too many questions.
xserver-xorg xserver-xorg/config/monitor/selection-method \
       select medium
xserver-xorg xserver-xorg/config/monitor/mode-list \
       select 1024x768 @ 60 Hz

### Preseeding other packages
# Depending on what software you choose to install, or if things go wrong
# during the installation process, it's possible that other questions may
# be asked. You can preseed those too, of course. To get a list of every
# possible question that could be asked during an install, do an
# installation, and then run these commands:
#   debconf-get-selections --installer > file
#   debconf-get-selections >> file

#### Advanced options
### Running custom commands during the installation
# d-i preseeding is inherently not secure. Nothing in the installer checks
# for attempts at buffer overflows or other exploits of the values of a
# preconfiguration file like this one. Only use preconfiguration files from
# trusted locations! To drive that home, and because it's generally useful,
# here's a way to run any shell command you'd like inside the installer,
# automatically.

# This first command is run as early as possible, just after
# preseeding is read.
#d-i preseed/early_command string anna-install some-udeb
# This command is run immediately before the partitioner starts. It may be
# useful to apply dynamic partitioner preseeding that depends on the state
# of the disks (which may not be visible when preseed/early_command runs).
#d-i partman/early_command \
#       string debconf-set partman-auto/disk "$(list-devices disk | head -n1)"
# This command is run just before the install finishes, but when there is
# still a usable /target directory. You can chroot to /target and use it
# directly, or use the apt-install and in-target commands to easily install
# packages and run commands in the target system.
#d-i preseed/late_command string apt-install zsh; in-target chsh -s /bin/zsh
d-i preseed/late_command string apt-install alsa-utils

d-i preseed/late_command string apt-install gstreamer
d-i preseed/late_command string apt-install minicom

As easy as borscht...

La voila!


Saturday, February 27, 2016

Tax Season

Canada Tax Planning for the Middle Class

Working class people need not do tax planning - they don’t have anything and don’t pay anything.  Rich people hire accountants to keep track of it all.  The rest of us, need to do tax planning.

As Joe Walsh put it:

I have a mansion, forget the price
I've never been there, they tell me it's nice
I live in hotels, I stare at the walls
I have accountants who pay for it all. 

The general solution to tax problems is to use corporate law to your advantage.  Corporate law changes very slowly and is fairly uniform across the world.  Large companies do not like it when governments touch corporate law, so politicians mostly keep their hands off it, since it can be political suicide.  Don't rely on Trusts.  Trust law is very hazy and therefore dangerous and changed for the worse recently, causing many tax problems. The worst thing is personal tax law - it changes with every budget, so it is hopeless to do long term planning for personal taxes.

I started to read up on tax and corporate law during the dotcom boom, when I suddenly got hit with a $25,000 extra tax bill and could not figure out how to reduce it.  That could have been a nice new car or a down payment on another little apartment.  Instead, I had to get a loan to pay my tax - bah, humbug.  Ever since then, I have been trying my best to claw my 25k back little by little.  One year, I paid only $11 personal tax.

The downside was that I also had to eat a lot of macaroni and cheese, because I did not earn a salary and my money stayed in my company.  The last few years, I am blissfully non-resident, living in the UAE, but then I ran into the non-resident rental income tax problem and had to cough up another unexpected few thousand, so one needs to stay vigilant and clued up and I had to start reading tax laws again.

Medical treatment keeps improving.  For middle class people, the average life span now, is 85 years.  Most people live ten years longer than they thought they would.  So, if you don’t want to die of hunger when you are 80, then you got to do a little thinking now.

The conventional wisdom is to get into the real estate rat race as soon as possible and then keep upgrading, and one day when you want to retire, downgrade to free up some capital and slowly consume it.

However, keeping all your eggs in one basket is risky and the above only works if the population is growing.  What if everyone is old, wants to sell and prices are down?  Then you either have to take the loss, or wait another five to ten years, while on a macaroni and cheese diet and once you consumed your capital, you are doomed.

Different Strategy - Cash Flow

I suggest a slightly modified strategy:  Don’t build Capital.  Build Cash Flow.

A big diamond is a capital investment.  It may be fantastic looking, but you cannot eat it.  To do anything useful with it, you have to sell it and diamonds seem to halve in value every ten years, so diamonds are a particularly bad investment actually, but it makes the point.

In contrast to useless fixed capital, if you have cash flow and you need more money - then you just wait a few weeks and then you have more money.

That first little apartment you bought?  Don’t sell it when you upgrade to a house, keep it, mortgage it, use it as collateral, let it.  Let the renter pay the mortgage.  Build your own small private real estate business starting on day one.

If you want to live somewhere else, then you can either buy or rent.  Depending on your citizenship, you can more easily buy property in certain countries, but you can rent anywhere in the world.  If you own five apartments in Canada, then you can rent one on a Greek island and live happily ever after.  You don’t have to buy an apartment in Greece.

OK, so you didn’t do all that and now want to sell your mini mansion?  Go ahead, but immediately buy four or five little apartments and let them.  Now, you have cash flow.  You also still have the capital, so if you want to buy something else, then you have collateral and cash flow and can negotiate a very low interest rate on your new mortgage, because you are a low risk lender with a profitable real estate business.


The only certainties in life are death and taxes.  It is important to manage your tax liability from day one, since at 15%, it is your biggest single expense.

Did I say 15%?  You are paying 33%?   The way to get 15% is to register a private company.

That first little apartment?  Register a company and transfer the apartment to the company before you start letting it.  If you really have to sell the apartment - don’t!  Sell the shares in the company instead.  That way, you only pay about 12% tax effectively while you rent it out and you don’t incur capital gains tax when you sell it, due to the lifetime capital gains exemption (about $800,000) on selling shares in small companies. (The apartment belongs to the company and doesn’t actually get sold - it still belongs to the same company - so no real-estate registration fees and taxes).

Non-resident Status

There is another problem with taxes for non-residents.  One day when you are old and grumpy, you may want to live somewhere else with a milder climate, which could make you a Non-Resident Canadian.   The effective tax rate for non-residents is higher than for residents.  The way to side step that to some degree, is to register a private company in BC (one of the provinces where a non-resident citizen is allowed to own a private company).  Then transfer whatever you owned to the company.  If you own multiple properties, register one company for each property.

To become non-resident, first, don’t have an available residence.  If you do own a house, rent it to someone else, not a family member and don't let it sit empty either - because then it is 'available'.   The best solution is to transfer the house to a company and let the company rent it.  (Note that in Canada, a non-resident citizen is allowed to have a RRSP, but can only get further contribution headroom from Canadian earnings.  So when you leave Canada, you don't have to wind up your RRSP and you don't have to sell your house to become non-resident.  Though in some cases, you may want to.)

Then, get on a plane, go somewhere far away, rent a little place on a beach and stay there.  Preferably do this in late December - it makes the taxes easier, since you need not prorate.  Don’t go back.  You are now non-resident - don’t ruin it.  Next year, file a non-resident tax return.  Simple as that.

There is a special form NR74, which one can use to ask Revenue Canada to rule on your tax resident status.  That is not a good idea.  What if they rule against you?  It is likely best to assume that you have non-resident status and simply file the non-resident tax return.   Then, if they want to dispute your assumed tax status, they have to prove their case, which would be impossible for them to do after you already lived overseas for a year or two.

If you are non-resident, you have to pay 25% tax on gross rent received - no expense deductions allowed.  A small company pays about 15% on the net profit after expenses, so about 12% on average.  So registering a company to handle your real estate is a no brainer if ever there was one.

When you need to take money out of the company, pay yourself a dividend, not a salary and file a T5.  A dividend, up to about $48,000, is not taxed again in your hands.

Why do small companies pay so little tax?

Small companies create employment.  Large companies destroy employment - that is called increasing productivity.  The government doesn’t want unemployed youths rioting in the streets.  Your little real estate company will employ young people to fix the plumbing, service the furnaces, replace the carpets, paint the walls, collect the rent, do the contracts and file your tax return, while you are in your canoe, fishing.  That is why.

See the nice beach volley ball court behind my car?  That is what a front yard lawn looks like in the UAE and those little bonzai trees are five years old already.  Al Ain has the widest beaches in the world.  We have 150 km of sand between us and the water.  It isn't quite walking or portage distance - so I got to stick my Bic canoe (Yes, of ballpoint pen fame - I'm a writer, eh?) on the car.

Why would you want to be a Non-resident Canadian?

Non-residents pay higher tax in Canada than residents, they don’t get the personal exemption of $11,000 and they don’t get free health care, so for some people, it is not a good idea to be non-resident.  If you or your wife has dual citizenship, then it may be a good idea though, since you may get better health care elsewhere. 

If you live and work in a low/tax free country for an extended period, then it is best to be non-resident Canadian, else you need to pay tax in Canada on your foreign income.

Only taxpayers resident in Canada have to file Form T1135 - Foreign Income Verification Statement.  So one day when you move back, you may have some explaining to do.

Further, if you go and live somewhere else for more than 6 months, then you could become non-resident, whether you wanted to be or not and if you live on a yacht, or always travel between 3 countries, then you may be non-resident everywhere.  So you need to be aware of the rules and it is possible to structure your tax obligations such that it works best no matter where you live and then you won’t get hit with an unexpected tax bill.

Lastly, be careful when you return!  It is best to return in the second half of the year, so that you are deemed non-resident the whole year, otherwise foreign earned income from the first half of the year may become taxable in Canada, saddling you with a bill, while it may have been much nicer to use that money for a holiday until the end of June.


If you have RRSP headroom when you leave Canada (who doesn't?), then you could stuff your rental income in there and thereby defer the 25% tax due, until the previously earned headroom is all used.  However,  one day when you retire and draw money, you may have to pay about 22% tax on the withdrawals, so it won't necessarily make much difference and you would need a lot of headroom, which you may not have.   Registering a company is still better by a good margin and you also avoid an ever growing capital gains tax problem.

If you end up living elsewhere and want to withdraw from your RRSP, then it will cost you 15% to 25%, depending on the tax treaty with your country of residence and whether it is a lump sum or a periodic withdrawal.  It may be a good idea to settle down in the ex-communist Central Europe - many hot springs, nice lakes and rivers, good health care and 15% withdrawal rates.


If you are a masochist/bored, read up and do it all yourself at a registry shop, or find a young lawyer to do it all for you.  The lawyer should be 30 years younger than you, so he/she can still do it for you when you are old and grumpy...

La Voila!


Monday, February 15, 2016

Network Emulator

In my experience, it doesn't help telling developers that a radio data link is inherently slow and unreliable and that it gets worse with increasing distance.  They will always design for the best case - 1 meter of copper wire in a lab - and then be all upset when it doesn't work so well in reality.

The solution is to make them a configurable network emulator from an old laptop PC (with a USB ethernet adaptor for a second port), put it between two of their machines and then stand back at a safe distance from any nerf guns or rubber band rifles and watch the wheels fall off the software.

This network torture tool uses netem and bridge-utils to create a transparent bridge between two ethernet ports.  This cruel script is prettied up with Zenity, so that one can use sliders to vary the delay and packet loss.

Either make the network utilities SUID root, or run the script as root.

#! /bin/bash
# Network Emulator
# Version 0.1, Copyright GPL, Feb 2016, Herman Oosthuysen
# Depends upon: zenity, ebtables, bridge-utils, netem
# SUID root: systemctl, killall, ethtool, ifconfig, brctl, iptables, ebtables, tc

# Configuration
export PORT0="eth0"
export PORT1="eth1"
export BR0="br0"
export SPEED="10"
export DUPLEX="full"
export IP0=""
export IP1=""
export MSK=""
export DELAY="0"
export LOSS="0"
export RETURN="0"

zenity --question \
  --width=350 \
  --title="Network Emulator" \
if [ "$?" == "1" ]; then
  echo "Cancel"
  exit 0

# First of all, disable NetworkManager and dhclient, 
# to prevent arguments over control of the ports.
systemctl stop NetworkManager
systemctl disable NetworkManager
killall dhclient
echo "Create a transparent bridge $BR0"
brctl addbr $BR0
brctl stp $BR0 off
brctl addif $BR0 $PORT0
brctl addif $BR0 $PORT1

echo "Full duplex, $SPEED bps"
ifconfig $PORT0 up
ethtool $PORT0
ethtool -s $PORT0 speed $SPEED duplex $DUPLEX autoneg off

ifconfig $PORT1 up
ethtool $PORT1
ethtool -s $PORT1 speed $SPEED duplex $DUPLEX autoneg off

echo "Enable IP4 forwarding"
ifconfig $PORT0 $IP0 promisc up
ifconfig $PORT1 $IP0 promisc up
echo "1" > /proc/sys/net/ipv4/ip_forward

# Give the bridge a pingable address
echo "Bridge IP = $IP1, Netmask = $MSK"
ifconfig $BR0 $IP1 netmask $MSK up

echo "Open iptables and ebtables to allow everything, INPUT, OUTPUT and FORWARD" 
iptables -F
iptables -P INPUT ACCEPT

ebtables -F
ebtables -P INPUT ACCEPT

# Create an initial rule
tc qdisc add dev $BR0 root netem delay $DELAYms 10ms 25%

while TRUE; do
  # Update the progress bar
  # Get the packet delay in ms
  DELAY=$(zenity --scale \
    --text="Packet Delay milliseconds" \
    --value="0" \
    --min-value="0" \
    --max-value="100" \
  # tc qdisc change dev eth0 root netem delay 100ms 10ms 25%
  tc qdisc change dev $BR0 root netem delay $DELAYms 10ms 25%
  # Update the progress bar
  # Get the packet loss in 1/x%
  LOSS=$(zenity --scale \
    --text="Packet Loss Fraction %" \
    --value="0" \
    --min-value="0" \
    --max-value="100" \
  # tc qdisc change dev eth0 root netem loss 0.3% 25%
  tc qdisc change dev $BR0 root netem loss $LOSS% 25%
  # Update the progress bar
  zenity --question \
    --width=350 \
    --title="Network Emulator" \
  if [ "$?" == "1" ]; then
    echo "Cancel"
    echo "100"
    exit 0
) | zenity --progress \
  --width=350 \
  --title="Network Emulator" \
  --text="Running..." \
  --no-cancel \
  --auto-close \

echo "Done!"
exit 0

La voila!


Friday, February 12, 2016

Mirror, Mirror on the Wall...

A Private Linux Mirror

Debian/Ubuntu mirroring is also described down at the bottom - it is super simple.

A Private Fedora Mirror

If you need to replicate Fedora based machines, then you need to set up your own rpm file mirror.  This allows you to automate the whole install with Kickstart off your own server on a LAN and you can then freeze your server at arbitrary points to facilitate a production run of identical machines.

The installation server can be an old laptop PC with a huge USB disk (reformat the disk with gparted to ext4.  The file system must support UNIX permissions and links).  The file server doesn't have to be very fast.  To do an install, you only need this server machine, a big switch and bunch of target machines with Kickstart ( and do a netboot, using DHCP and a web server such as lighttpd.

To make your own mirror server, you should set up an account with Fedora, so that you can get access to their servers and allow them to transparently redirect your machines to your own server if necessary.

Once you opened an account, you can set up a rsync script to download and save only what you need.  The secret to success with rsync is the EXCLUDES file.  In there, list patterns of directory names and files to avoid.  For example, if you only want Fedora 22, then you do that, by excluding 4, 5, 6... 21 and 23, plus a few other junk things that will show up once you try it.

More details here:

FAS Account

Open an account here: - Without an account, you won't be able to download anything with rsync.

Create a new Site, for example mirrors and set the password to something secure.  Specify the Organization URL if you have one and be sure to select the Private checkbox and then save the site.

Now create a Host with a FQDN of  Set the Country code to US and again make sure the Private checkbox is enabled and save the Host.  Once saved, add a new Site-local Netblock.  Go to and make a Netblock

Once this is saved, still under the Host setting, add a new Category. This will tell the Mirror Manager what categories of software this host carries. Examples include Fedora Linux and add a URL serving the content definition, such as which you need to use in your Lighttpd or Apache web server setup.

Note that if you don't want to do the FAS thing, you can do the same as below with any other Fedora mirror close by at a university or Telco that you trust, but you'll have to research the excludes list and rsync group carefully.

For a full public mirror list, click on something in the matrix here:




The most important thing is the excludes file and nobody ever tells what needs to be in there, which prompted me to write this article.  If this file is not good, then rsync will download everything since the abacus was invented and your disk drive is bound to fill up.  The file excludes.list below will exclude everything but Fedora 22:


The only thing not in there, is 22.  As you can see I also don't want Cloud and Docker schtuff, but I do want Arm, i386 and X64 - you may want to tweak it some more.

You can see all the directories you may need to exclude by trolling up and down the tree of the mirror server here:

Rsync vs Wget

The mirroring is recommended to be done with rsync, which will download a group on the server called fedora-enchelada (which plays on 'the whole enchelada' - everything since the abacus).

I have in the distant past made mirrors using wget, but rsync is more efficient and easier to control.  Rsync will honour the excludes list and will not traverse outside its designated directory, but wget will invariably start to walk across to other directories at the same level, thereby downloading more files than it was supposed to, so you have to keep an eye on it and quit it when you think that it is done with what you want (or set it to one directory deeper than what you actually want - like the example below and then let it be).

If you want to use wget, here is an example:
$ wget --continue --recursive --no-parent --no-clobber \

The above wget script will also download other directories under the Live subdirectory, not only the x86_64 one, so you have to watch it.  Wget also has an exclude directive that doesn't work.  Despite these issues, it does work and can be used to download a mirror that doesn't support rsync, or for which you don't have a download account.

Mirror Script

For testing the rsync script below, I made a mirror directory tree in my home directory ~/mirrors

This tree actually needs to be in the web server root which is usually /var/www to serve the files to Kickstart and dnf.

The rsync script called looks like this:

#! /bin/bash
# Mirror Fedora 22 only - at least, that is the idea
# See the exludes.list file - rsync will download everything, except for the patterns in this file
# See the mirroring wiki for details:

export EXCLUDES="excludes.list"

rsync -vaH --exclude-from=${EXCLUDES} \

 --numeric-ids --delete --delete-after --delay-updates \
 rsync:// ~/mirrors

Make this file executable with chmod 755, run it and see what happens.

The first thing rsync does is to download the files list and build a directory tree in ~/mirrors.  While rsync runs, view this growing tree and make sure that it only includes what you want and that unwanted directories remain empty.

If there is a growing pile of files that you don't want, press Ctrl-C to quit the script, add a pattern to the excludes.list file, delete the junk and try again.  Don't leave the machine alone until you are sure that you get only what you want and no more, or you may end up with a terabyte of useless files.

A Private Ubuntu Mirror

On Ubuntu mirrors, all the files are stored in a single directory called pool.  In there, you find all versions of everything.  The releases are controlled through a system of Master Record Index files that list everything about every file in a release.  These index files are zipped up and kept in a directory tree using the release names like Trusty or whatever.  These files will keep your CM manager as happy as a piggy in a mud bath.

The problem with this system is that you cannot replicate an Ubuntu mirror with rsync, unless you copy everything since Adam invented the Abacus, which is about 800 GB.  To get only the files belonging to a specific release, you need a utility that can parse the index files.  This utility is called apt-mirror and it will download about 100 GB of executables for the Trusty release.

Apt-mirror does exactly what is written on the tin.  It Just Works (TM).

The easiest way to run apt-mirror is to make a server with the same release as what you want to mirror and install the package apt-mirror.  You then only need to change one single line in /etc/apt/mirror.list to point to the place where you want to keep all the files and run apt-mirror (The directory must exist). That is all there is to it.

The most important thing with mirroring is to avoid using a Seagate USB disk that shuts itself down every once in a while.

What I eventually did was to uncomment the lines in mirror.list one by one and save the files on USB sticks.  The first line for main needs about 60 GB, which fits on a modern 64 GB schtick. The following lines of updates and security fixes require about 30 GB storage, about 90 GB total for the compiled code and goodness knows how much for the C-code.

Downloading 100 GB will take about 2 days on a typical 4 Mbps home fibre net, vs weeks on a typical overloaded corporate network.

Note that you can interrupt and restart apt-mirror.  It will figure out what happened and carry on where it left off without a complaint.

Apt-mirror is pretty robust.  I managed to fill up my USB thingies to the max on an Ubuntu VM, then copied them together with rsync -a to a larger SD card, made a raw device file for VirtualBox so it could access the SD card, mounted the SD card in the same path and carried on with apt-mirror and finally copied the SD card to my Fedora mirror server.  All done with nary a hiccup.

GPG - InRelease Clearsigned file isn't valid, got 'NODATA'

This error drove me up the wall.  The InRelease file seems to be somehow corrupted by apt-mirror. The solution is to delete the InRelease file from the mirror server.

Another one of life's little mysteries...

Seagate USB Disks

Those infernal Seagate USB disks need something like this command to stay awake:
$ sudo sdparm --clear STANDBY -6 /dev/sdb -S

In addition, if you are running a Virtualbox virtual machine, do not use a USB3 port (yellow) for an external disk.

La voila!

Happy mirroring...