|BTRFS - Snapshot rollback||Monday 25th November 2013|
I was trying to clean up my user directory backup - which I currently rsync and then do periodic snapshots of. I realised that I wasn't deleting old files in my destination and I had a bunch of temp files that I didn't need to include. In my bid to sort out the links I managed to wipe out a lot of data on my backup.|
When I ran rsync again it started trying to move gigs of photos over - not something I fancied doing and also lead me to wonder, if I:
- Create a file
- Snapshot the file
- Delete the original file
- Re-create the original file with an identical copy
Would the 'btrfs' snapshot store the file twice? Unless it was very clever, I suspected it would. A quick drop in on #btrfs on Freenode confirmed my thoughts. By re-running my backups from scratch would make my next snapshot a complete duplication of all the data that was there before. So in addition to taking a long time, I would lose a lot of space. Lucky for me, I have period snapshots that I can roll back to.
Everybody knows that they should keep backups of their data and have disaster recovery and rollback options when it comes to IT. Many people and organisations get the backups but a fewer number test the recovery of that data. If you cannot revert your data then what's the point in the backup? This was my first real-world attempt to rollback a 'btrfs' subvolume.
My first port-of-call was the program that manages my backups - "Snapper". This creates hourly backups and has a built-in undochanges command - which I ran. Everything back to how it was, I kicked of my 'rsync' job again but was confused to see all my files being re-copied. After a bit of investigation it turns out that Snapper does not retain timestamps when it does a rollback (WHY!?!?). A manual rollback is therefore required. I began to brain-freeze when I realised that the snapshots were in a subdirectory of the directory I wanted to rollback. Still, I crossed my fingers and started running commands:
mv mydirectory mydirectory.old
btrfs subvolume snapshot mydirectory.old/.snapshots/20/snapshot mydirectory
mv mydirectory.old/.snapshots mydirectory/
rm -rf mydirectory.old
What this does is:
- Moves the old subvolume out of the way - this should be instant as no data is moved, just the reference
- Takes a snapshot of the snapshot that I want to recover and puts it in place of the original subvolume
- Move all my other snapshots into the new subvolume
- Delete the old subvolume
This all works because snapshots are subvolumes. By moving directories I appear to be essentially creating links to different subvolumes. As each snapshot is exactly that, a snapshot of the entire directory, we can delete the originals without issue.
Useful to know!
|Tweaking SpamAssassin||Tuesday 5th November 2013|
A long time ago I moved my e-mail to my home-server instead of with my hosting provider. One of the objectives of this move was to have centralised spam-filtering. I started thinking of clever ways I could filter mail and such - and then promptly forgot about it.|
The truth of the matter was the junk filtering in Thunderbird was pretty hot and actually I was receiving a lot less spam than when I initially thought about moving my mail. Alas, whilst I've had a few years of peace, this week something has changed and I'm getting an increased level of spam.
I went back to have a look at my spam filtering. At some point I did setup amavisd-new to perform virus-scanning and the like on received e-mails. By default amavis also includes SpamAssassin support out of the box. And, as it turns out SpamAssassin supports a whole range of spam tackling technologies out of the box.
As I was still receiving spam even with these default technologies, clearly something had to be tweaked. SpamAssassin when called from amavisd-new is run as a plugin and not via the command-line, nor via a daemon. So ignore anything on the net talking about things specific to those.
The main things that interested me were Spamhaus, as this is a real-time lookup of dodgy domains and source IP addresses. Again, this is all supported out of the box, and whilst these services were marking the messages as spam, there just wasn't enough weighting to push the messages over the threshold.
Personally, I have taken the approach that these lists are fairly accurate - especially when combined. So the weighting should be increased. To make these tweaks you just need to override the defaults by editing the file /etc/mail/spamassassin/local.cf (your installation may vary). For example:
score URIBL_BLACK 2.5
score URIBL_DBL_SPAM 2.5
In this example I have pushed up a couple of DNS lists, the codes in capitals are the settings and are detailed extensively on the SpamAssassin web-site. The next line addresses language - I don't think anybody has ever e-mailed me in anything apart from English and even if they did, I wouldn't understand it.
This appears to work well but I immediately saw some legitimate mail from my bank get marked as spam (not as a result of the above settings). The all_spam_to will skip all spamming checks against TO addresses. So I have setup an unique e-mail address to be used exclusively by my bank, and I allow this through. There is a still a chance of phising, but it's less likely on an account that only my bank is aware of.