Skip to main content

NSA Snoop and Error Proof Your Archives

An Englishman's Home is His Castle:

In 1760, William Pitt (the Elder) made a famous declaration of this right. "The poorest man may in his cottage bid defiance to all the force of the Crown.  It may be frail, its roof may shake, the wind may blow through it. The rain may enter. The storms may enter.  But the king of England may not enter.  All his forces dare not cross the threshold of the ruined tenement."

There is many a ruined castle in Europe.  You need to defend your castle if you want it to keep its value!

You should also consider the problem of data corruption, since any single bit error in an encrypted archive, will render the whole thing unreadable.

Also remember this: Always tar a directory, never tar a bunch of files. 

 It is extremely annoying when you get an archive from someone, untar it and end up with a million files scattered throughout your home directory...


Encrypting an Archive with GPG

An easy way to secure your data against unwarranted NSA and GCHQ snoops is with gpg.  Always use gpg before uploading archives to a public file server such as DropBox, Copy or Evernote.

Using pipes, you can string the tape archiver tar, bzip2 and gpg together like this:
$ tar -cj test | gpg -c >test.tar.bz2.gpg

(or use gpg -er username for key based encryption)

and to reverse it:

$ gpg -d < test.tar.bz2.gpg | tar -xj

The key configuration of GPG is shown in another post. 

 

Forward Error Correction

A good way to protect your data against corruption is with a Reed Solomon forward error correcting code - it is used on CDROMs.  There is a little known utility available that does it, called rsbep by Guido Fiala and you can get the source using this link: http://www.filewatcher.com/_/?q=rsbep

Then build it the usual way with ./configure; make; sudo make install.  It is part of the BSD and Debian distributions.

Using pipes, you can string the tape archiver tar, bzip2, rsbep and gpg together like this:

$ tar -cj test | gpg -c | rsbep >test.tar.bz2.gpg.rs

(or use gpg -er username for key based encryption)

and to reverse it:

$ rsbep -d < test.tar.bz2.gpg.rs | gpg -d | tar -xj

The Reed Solomon code will protect your archive against error bursts and will help to ensure that you can read the data back from a failing archive system, many years later.

 

Parity Bits

There is also a utility called par2 (and the handy GUI PyPar2) which adds Reed Solomon parity bits in a series of separate files.  These are in the par2cmdline and pypar2 packages.  This can be used to protect any files, but when you copy things around, you got to remember to pass the parity data along too.

Preventive Measures Using Par2

In order to ensure that I get into and stay in the habit of using gpg, I created a couple of scripts, so I can simply do:

$ targpg directory
and
$ untargpg directory

and the scripts will handle the messy details.

I tested the error recovery by corrupting the archive with hexedit - it works like magic!

Make an archive script like this called /usr/local/bin/targpg:
#! /bin/bash
echo Make an encrypted archive of a directory
tar -cj "$1" | gpg -c >"$1.tar.bz2.gpg"
md5sum "$1.tar.bz2.gpg" > " $1.tar.bz2.gpg.md5"
parcreate -n1 "$1.tar.bz2.gpg"
ls -al "$1"*

as well as untargpg:
#! /bin/bash
echo Untar a GPG encrypted archive $1
RESULT=$(md5sum -c "$1.tar.bz2.gpg.md5")

if [ "$RESULT" != "$1.tar.bz2.gpg: OK" ]
then
  echo MD5 error - Attempt a repair
  par2repair $1.tar.bz2.gpg.par2
fi

# Try to decrypt and untar regardless   
# because the md5 and par2 files may be missing
gpg -d < "$1.tar.bz2.gpg" | tar -xj
ls -al "$1"*
 

and then one day when your disk goes south, par2repair may save the day.

La voila!
Labels:

Comments

Post a Comment

On topic comments are welcome. Junk will be deleted.

Popular posts from this blog

Parasitic Quadrifilar Helical Antenna

This article was reprinted in OSCAR News, March 2018:  http://www.amsat-uk.org If you want to receive Satellite Weather Pictures , then you need a decent antenna, otherwise you will receive more noise than picture. For polar orbit satellites, one needs an antenna with a mushroom shaped radiation pattern .  It needs to have strong gain towards the horizon where the satellites are distant, less gain upwards where they are close and as little as possible downwards, which would be wasted and a source of noise.  Most satellites are spin stabilized and therefore the antenna also needs circular polarization, otherwise the received signal will flutter as the antennas rotate through nulls. The helical antenna, first proposed by Kraus in 1948, is the natural solution to circular polarized satellite communications.  It is a simple twisted wire - there seems to be nothing to it.  Various papers have been published on helix antennas, so the operation is pretty well ...
2 comments
Read more

Unlock CRA PDF Forms

Unlock Canada Revenue Agency PDF Forms It appears that there is a relatively new PDF feature to prevent casual copying and saving of a file and that some programs save PDF files with these foolish features active by default.  Many forms from the Canada Revenue Agency are locked in this way, which makes it difficult to do one's taxes, since one can fill the form, but cannot save it.  One can only print the form.  It should be possible to print to a file or export it to a new PDF file, but it is far better to reset the annoying anti-taxpayer flags, since the 'printed' form cannot be edited easily any more and I always manage to make a mistake or three that need to be corrected after review. If there is a Linux (virtual) machine handy, install qpdf and use it to reset the silly flags: $ su - password # dnf update # dnf install qpdf # exit $ qpdf --decrypt lockedfile.pdf unlockedfile.pdf One doesn't need a password to unlock these flags, so the fix is instant. La voila! He...
Post a Comment
Read more

To C or not to C, That is the Question

As most would know, the Kernighan and Ritchie C Programming Language is an improved version of B, which is a simplified version of BCPL, which is derived from ALGOL, which is the Ur computer language that started the whole madness, when Adam needed an operating system for his Abacus, to count Eve's apples in the garden of Eden in Iraq.  The result is that C is my favourite, most hated computer language , which I use for everything. At university, I learned FORTRAN with punch cards on a Sperry-Univac, in order to run SPICE, to simulate an operational amplifier.  Computers rapidly lost their glamour after that era! Nobody taught me C.  I bought the book and figured it out myself. Over time, I wrote a couple of assemblers, a linker-locator, various low level debuggers and schedulers and I even fixed a bug in a C compiler - not because I wanted to, but because I had to, to get the job done!   Much of my software work was down in the weeds with DSP and radio modems...
Post a Comment
Read more