Saturday, December 4, 2010

Part II: Adding SSH Public Keys to DNS

Fairly quick one, you'll need sshfp (available in EPEL for CentOS) installed.

Just need to type:
sshfp -s
which should give the response: IN SSHFP 1 1 c4d55ccbc4fdd2f4304586d6cdc4ad6fca5c743e IN SSHFP 2 1 7b504855c13277490992dea7091537d0f9bfdb1d
You just need to add these lines to your DNS zone for

To get SSH to check these keys, you'll need to modify your /etc/ssh/ssh_config or ~/.ssh/config to include
VerifyHostKeyDNS true

Wednesday, December 1, 2010

Adding GPG public keys to your DNS

Recently I read a couple of interesting posts which outlined how to add GPG public keys to a zone file.

Reasons for wanting GPG key in DNS:

  • It's a nice way for people who wish to get your GPG key to obtain it.
  • It won't typically get blocked by firewalls, had major problems in past getting hkp etc. working through the firewall.
  • I prefer hosting the key myself as opposed to using or similar (lack of control).

There are three methods to do this - I'll go into detail on two:

  1. Use the pka record (TXT-type record) which gives a URI to the location of your key; http/finger etc.
  2. PGP Cert record.
  3. IPGP Cert Record (Won't go into much detail on this).

PKA Record:
This is the quickest/easiest method, however it is just a pointer to the cert, which I'm not keen on.
It can co-exist with the PGP Cert record, so I do it for completeness sake mostly.

Find your GPG key, export the public section & take note of the fingerprint.
gpg --list-keys
gpg --list-keys --fingerprint D4164694
gpg --export --armour D4164694 >
You need to make the .asc file available to the public (http), then construct your dns record:                             IN TXT  "v=pka1;fpr=B830A1A76A1A87C84C95B06C7476F7AFD4164694\;uri="
That's it!

PGP Cert Records:
You need the program 'make-dns-cert', which is not available on Debian/Ubuntu or CentOS, but is a .c program which is distributed with gnupg. I have made it available here (just requires you to do 'gcc make-dns-cert.c -o make-dns-cert').

Assuming you have a copy of your key from the previous step, you just need to type:
make-dns-cert -n -k
This will output a single (very long) line, which you can put directly into your zone file.
Note the above address I used '', when referring to email addresses in DNS zones, the @ is replaced by a period. To stop screwups with copy & paste, would suggest appending the above output into the zone directly & then manually modify the serial.

Once you reload DNS, you can test everything is working using the following comands:
dig +short TXT
dig +short CERT
Note, if you wish your gpg installation to automatically search DNS for GPG keys, you must make the following modification to gpg.conf (typically ~/.gnupg/gpg.conf):
auto-key-locate cert pka
This will ensure it attempts to ge tthe cert first, then fallback to pka, you can also add other methods (such as hkp:// for it to fall further back on.

Quick note about IPGP keys:
Since you can only have one cert record for an email address, and IPGP is similar to the PKA record, in so far as it's a pointer, I prefer going with the easy (TXT Record) PKA & PGP Cert Record.

Read various sites while setting this up, but I found ​ to be the most complete resource for the above.

Sunday, November 21, 2010

Hetzner, upgrading lenny to squeeze

Late last week ended buying a hetzner VM (vServer vq7) mainly out of curiosity.
I'd heard a fair bit of good things about hetzner, and wanted to try them.
Fairly impressed, 30ms - 50ms latency (stable, varies from site-to-site), and they seem to be sanely installed VMs (no custom kernels etc).

Went for Debian 64-bit VM which turned out to be 5.0 (Lenny), again, out of curiosity did the upgrade process to 6.0 (Squeeze - unstable):
Modify sources.list to squeeze
apt-get update
apt-get remove linux-image-2.6-amd64
apt-get install linux-image-2.6-amd64
aptitude full-upgrade

(Note: The above must be done in that order, otherwise you will hit udev/kernel bug).

Sure enough, it came back up, no problems.

Spent last couple days moving all my stats and monitoring over to the hetzner VM, pleased to take some load/work off my atom servers; and have a fourth location for DNS/backups.

Thursday, November 18, 2010

Yaketystats & Jart

I've now switched my stats collection for the third time in the past year, and I'm hoping this will be the last time.

I'm not going to cover configuration or installation here, as their wiki is excellent, I will however post a couple of things which got me, and a couple of changes I made.

Yaketystats Homepage
Five minute demo video (well worth watching)

I really liked collectd,  nice plugins collection - there was a plugin for everything I wanted/cared about.
Written in C, push-based, fast, and regular polling intervals for HD stats (and who doesn't want HD these days :)

The catch is the frontend, they all seem to be severely lacking.

Various areas where CGP lacks was pointed out at my new job, mainly the inability to compare multiple graphs - at a specific point in time, combine values into one graph, and be able to search/combine graphs easily (regex or similar). After much searching for an alternative frontend, I finally came across Jart, which is the frontend for Yaketystats.

Yaketystats 2.1 has been around for over two years (Yaketystats itself has been around for approximately five altogether internally at USG), but it doesn't seem to have gotten the same community attention as munin or collectd, which is a shame.

Yaketystats is all bash/perl/php, and it's all quite simply put together.
That's not to say it's simple -- but it's very easy to modify and add onto.
They have put thought into the backend system, stats are collected on each host, stored (plain text), and a cron runs which posts the stats to a php page on the server. If it cannot contact the server it just keeps queueing up the stats till it can.

Once the stats are submitted, a couple of jobs are run on them, eventually outputting them as RRDs.
I noticed less load from this system than collectd/rrdcached for the same number of hosts.

This is where the real magic starts, Jart is their frontend.
It can draw graphs in three different ways:
1) You draw 'all' the graphs for cpu for a host - this draws a graph containing idle, iowait, user, sys for a system (combined for all CPUs).
2) You select which stats you'd like, specifically, from any number of hosts, which it will then put in one graph
3) Regex!  You can build graphs using Perl's regex -- very, very cool.

You can modify any graph you create, on the fly -- change from using lines, to stacks to area, negative or positive for different values (e.g. received traffic is given in negative, transmitted given in positive values on the graph), change colour etc - you have full control.
You can then save whichever graphs you have open as a playlist - meaning you can re-open these graphs easily, with all your settings.

So far been using this two or three days, and converted a couple other people to it too.

Couple of changes I made for my setups:
patch to add limits tab to graphs.

Remove the info at end of each graph (generation time, most recent rrds etc), modify index.php:
$tmp  = array_merge($this->args,$this->defs,$this->lines,$this->events,$this->comments);
to read
$tmp  = array_merge($this->args,$this->defs,$this->lines,$this->events);

Also, we wanted to sort/break up our server names by the fourth character, and not the first, again requires modifying index.php:
var nl = me.charAt(0);
var nl = me.charAt(3);
Note, found a bug where it breaks completely if you have a username with less characters than it looks for above (4 in this case).

The last change is to remove the capitalisation of the hostnames - comment out the two lines above the charAt line:
mefirst = me.replace(/(.).*/,'$1').toUpperCase();
me      = me.replace(/.(.*)/,mefirst + '$1');

Sunday, November 7, 2010

Collectd - Frontent & performance modifications

Been using collectd a fair while now, one of the largest drawbacks of it the lack of a good frontend.

I recently found Collectd Graph Panel (CGP) which is by far the best frontend I've used for it yet (oddly enough the handiest to setup also!)

It doesn't seem to be under overly active development, but lot of it works, and support for more plugins is getting added to the git repository (which I'd suggest using as opposed to the 0.3 release).

Another modification to collectd I made recently was moving from using the rrd plugin to the rrdcached plugin, which lets the rrdcached service handle the rrd data.
Caching seems to be done better using this service as opposed to within collectd.

By default rrd storage location is different, so to migrate just rsync (assuming defaults) from /var/lib/collectd/ to /var/lib/rrdcached/db/collectd/ and restart collectd (after disabling rrd plugin & enabling rrdcached plugin).

Migrating to nginx from apache (with PHP + CGI support)

Was always aware of nginx, and several people mentioned it, eventually it hit home (especially after reading some interesting LOPSA posts on their mailing list.

For this example, I'll detail my setup for  PHP, CGI, nginx status page and restricting by address (IPv4 & IPv6).

The following is done on Debian Squeeze, I have tested the config with nginx version 0.7 & 0.8.

The following dependencies are required: php-fastcgi fcgiwrap spawn-fcgi php5-cgi

Although not in the Debian repository, I found php-fastcgi packaged, and a friend ended up packaging nginx 0.8 (0.7 is available in the repository).

I went with version 0.8 of nginx as it allows you to restrict access to sites using IPv6 addresses, as opposed to IPv4 only.

Firstly, the /etc/init.d/php-fastcgi wrapper must be modified - by default it listens on a port, and only starts one process - I want to use socket-based communication between nginx & start multiple processes (for me, 3 is enough).


The following is my nginx configs from /etc/nginx/sites-enabled/

server {
        listen; ## listen for ipv4
        listen   [::1]:80;
        server_name  localhost;
        location /nginx_status {
                stub_status on;
                access_log   off;
                allow ::1;
                deny all;
server {
        listen   80;
        listen   [::]:80;
        root   /var/www/stats;
        access_log  /var/log/nginx/;
        location / {
                index  index.html index.php;
                allow ::1;
                deny all;
        location ~ \.php$ {
                gzip off;
                fastcgi_pass unix:/var/run/php-fastcgi.socket;
                fastcgi_index index.php;
                include fastcgi_params;
                fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        location /smokeping/ {
                index smokeping.cgi;
                gzip off;
                if ($uri ~ "/smokeping/smokeping.cgi") {
                        fastcgi_pass unix:/var/run/fcgiwrap.socket;
 I symlinked /usr/share/smokeping/www to /var/www/smokeping & also symlinked /usr/share/smokeping/cgi-bin/smokeping.cgi to /usr/share/smokeping/www/

Thursday, October 21, 2010

New ION2 HTPC build using XBMC

Finally purchased & setup the new HTPC, was a couple of delays along the way, but setup is pretty close to what I had planned in my previous post.

Case -
Antec ISK 300-150 Mini-ITC

Motherboard -
£135 + £24 Shipping

Fans -
2x Noctua NF-R8
€30 + €4 Shipping

Memory: Kingston 1GB PC8500 DD3 SODIMM
€24 + €8 Shipping Kingston 2GB PC8500 DD3 SODIMM
€45 + €3 Shipping

Case same as the low-power always-on box I have, except went from 65W power brick to 150W internal PSU with fan (Motherboard needed 90W max, and only option for power brick would have been picoPSU which I could still go for if wished).
PSU fan is  thankfully as quiet as the Noctua fans, so no need to go for picuPSU.

As previously mentioned, I found the Noctua fans excellent, so went for another pair of the NF-R8s.

Motherboard was the thing I'd been waiting for. Soon as I heard about ION2, been waiting for a suitable board to come out.
Newer generation Atom uses less power, so although I haven't taken any measurements yet, I can only assume the box uses 25W or less.

Using Fedora 13 again for XBMC (SVN) with MPlayer (SVN) for playback, which is the best player I've encountered for VDPAU playback.
Performance is excellent all round, smooth menu navigation and transitions, and no problem with playback.

I did test Debian Squeeze (with an ION1 box), and noticed massive problems.
Lirc and menu navigation was sluggish and juddery.

Outputting video & audio over HDMI from the HTPC to my amp.

The box is near-silent, faint fan noise can only be heard from it when 1m away from it, as I generally sit 3-4m from it, don't hear a thing. It stays reasonably cool, typically at ~40°C.

I'll post up my configs & scripts used soon, and hopefully get some pics up by the weekend.

Saturday, September 25, 2010

Time to complete move to low-power

Finally, ASUS release an ION2 version of their previous boards I've been so pleased with.
AT510NT-I, Fanless Dual-Core 1.8Ghz.

Reckon I'll be upgrading to this fairly soon, and finally put down my ageing Dell Dimension C521 (Current media centre).
A couple problems with my current HTPC setup:

  • Highish power use, for what it's doing
  • Requires separate nVidia graphics card to do VDPAU
  • Both system & GPU have a fan, latter generating quite a bit of noise a times. 

As my HTPC uses XBMC for displaying of directories/files, and then launches mplayer (using VDPAU) for playback, it means there is minimal CPU use; XBMC uses OpenGL for a lot of things, and is quite heavy in that respect.
MPlayer will only use 2-5% CPU when using VDPAU, and ION2 means that VDPAU will also decode Xvid - I'll be very happy with with the above board I believe.

Not yet fully decided on what the new config will be, but I reckon very similar to the first mini-itx box I built using Antec's ISK-300-65

Monday, July 5, 2010

Pics of current setup

Decided I'd throw up some pictures of current/new setup, since already documented power use etc.



Friday, June 25, 2010

New File Server

Finally bought it, it's now sync'ing over files from previous beast.

In the end went with:
  • Lian Li PC-Q08 (
  • 2TB Samsung F3 EcoGreen (
  • 32GB SSD, OCZ Vertex (
  • Nexus Value-430, 430W PSU (
  • Noctua NF-P14 & NF-S12B; 140 & 120mm case fans (
  • Asus AT3IONT-I Fanless & 2x2GB (
  • Startech eSATA & USB 3.5/2.5" Dock (Amazon)
  • 2x2TB Samsung F3 EcoGreen (
Initially went for only one 2TB disk, then got other two later.
Only thing left to get is a PCIe SATA controller (non-RAID). Looking to find one with 4 Internal ports, and one External, however will probably settle for 4 Internal & buy a faceplate with eSATA <-> SATA.

So far very impressed overall; there was more room to work within the case than I thought there would have been - though getting the PSU in, and working with the mass of cables is hellish.
Nice thing Lian Li did though was include a bracket to attack to PSU, which means can aim the extractor fan of the PSU towards the fanless motherboard, and pull more hot air away from it.

CPU seems to be running at about 48C, and the mainboard at 32C, but it's in a fairly small room with the large server beside it; and it has been transferring files for 19hours straight, so would expect some heat buildup.

Not had chance to take power readings yet; will update this & the previous thread once I get around to it; hoping for something fairly low though (so I can justify the cost to myself!)

I tested FreeNAS & Openfiler, and in the end just went for Debian Squeeze.
I couldn't get used to doing everything via web GUI.

WOL works too out of the box, so once I get all my stuff copied over, I'll be scripting shutdowns/wakeups & backups -- will post any particularly interesting scripts.

Last note, noise -- extremely impressed.
The sound of the disks is the loudest thing; and barely hear them. I have the case in the mini-server room (downstairs toilet converted) anyway, so unless actually in there, will never hear a thing.

Wednesday, June 16, 2010

Current Electricity Usage

Got loan of a kill-a-watt type meter.
Did most of the things around the house to get an idea of my usage:

Server, 2x 2Ghz Opteron, 8 disks
On Boot: 280W
Idle (with some reading/writing): 180W

New Atom-based server:
On boot: 28W
Idle: 26W

New Atom-based file server (3x2TB disks & 1x32G SSD)
On boot: 50-60W
Idle/streaming: 45W

Laptop (X61S):
With battery: 30W
Without battery: 20W

TV (Sony 40" W3000)
Standby: 25W
On: 90W

Amp (Sony STR-DA2400ES)
Standby: 40W
On: 60W

HTPC, AMD 64 X2 Dual Core Processor 4400+, SSD, NVidia GT220, Software: XBMC
On-boot: 90W
Idle: 80W
Playing Video 70W

Sub woofer:

Wireless AP (Linksys 3300N): 9W
VOIP Basestation (Gigaset S685IP): 6W
Netgear 5-port gigabit switch (4 ports in use): 5W
DSL Gateway (bridging mode, Zyxel P660-1): 8W

Additional notes:
Nearly all devices used 5W just being plugged in (not in standby, purely plug being in socket)

The meter I used is fairly inaccurate, so the above is more to give myself a rough idea only.

Monday, June 14, 2010

Replacement server[s] + going green.

Looking to replace two machines I've running in my house, one being general server+file server, other being a HTPC.
The server is on 24/7, whereas the HTPC gets suspended when not in use.
Server specs:

  • 2xOpteron 246
  • 3GB memory
  • 7 HDDs (4 RAID, 3 for OS/home/cache -- don't ask how I ended up with 3 for OS etc)
HTPC specs:
  • Dual core Athlon 3400+
  • 4GB memory
  • 32GB SSD
  • NVidia GT220
Replacing the server in two parts.
The first box will be small+quiet, and just do apache, monitoring, logging, pppoe, dhcp, main ssh box etc. I just got this at the end of last week and finished setting it up. It consists of the following:
  • Asus AT3IONT-I Deluxe Fanless Dual Core 1.6GHz
  • 4GB DDR3 1333Mhz Memory
  • 32GB SSD (Corsair X32)
  • Antec ISK-300-65
  • 2x Noctua NF-R8 Case fans
The stock fan with the antec case isn't too quiet, very impressed with the replacement Noctua ones, very quiet. Only one was probably needed.
CPU runs at about 33 Celcius, while motherboard is at 26.
Installed Debian Squeeze, and it's been running since Friday, it's faster than previous server, and runs at about 25W.

Having a separate fileserver so that it doesn't need to stay on all the time - and will still be small/quiet and use relatively low amount of power compared to what I'm using at the moment.
In the late planning stages here, so far what I'm looking at is:
  • Case: Lian-Li PC-Q08
  • Motherboard: Asus AT3IONT-I
  • Memory: 4GB DDR3 1033Mhz
  • Noctua NF-S12B FLX
  • Noctua NF-P14 FLX
  • PSU: Nexus Value 430
  • OCZ Vertex 30GB (SSD)
  • Samsung F3 'Green' 2 TB disk
  • Startech USB/eSATA external 2.5/3.5" dock
The case is fairly new to the market, and will take 6x3.5" HDDs, 1x5.25" Drive, and 1x2.5" SSD.
replacing case fans, with quieter Noctua again.
Will need to pickup a PCIe SATA controller, and also get a SATA -> eSATA card for backups.

Not planning to do RAID, instead keep backups - not sure of details yet, but something along lines of label disk, when it's put into eSATA dock udev picks up label, runs backup script, emailing me on completion (with basic info such as free disk space left). So backups should be fairly painless - once I remember to plug a disk in each week.

Sunday, May 9, 2010

Connecting XBMC to Sony AV AMP STR-(DH800 / DA2400ES)

Sony AMPS (at least the ones list in title), don't seem to add 1080p/60 1080p/24 to their preferred timings, although they do fully support them.

This can be verified by adding
Option "ModeDebug" "1"
to the "Montior" section of xorg.conf, which will allow you to see the the modes which the AMP advertises.

The following are the 1080p/(60|24) Modelines for the above AMPS:

"1920x1080@60p" 148.5 1920 2008 2056 2200 1080 1084 1089 1125 +hsync +vsync
"1920x1080@24p" 74.2 1920 2560 2604 2752 1080 1084 1089 1125 +hsync +vsync

These must be used in conjunction with

Option "ExactModeTimingsDVI" "True"

in the "Screen" section of xorg.conf

Saturday, May 8, 2010

Unbound DNS

Setup Unbound (validating, recursive, and caching DNS resolver) recently, and was surprised by how quick it was to setup.
Quite like it so far, seems fairly lightweight (compared to Bind), and config is straightforward.

After installing, full config is:
trusted-keys-file: "/etc/pki/dnssec-keys//production/reverse/*.conf"
trusted-keys-file: "/etc/pki/dnssec-keys//production/*.conf"
verbosity: 1
statistics-interval: 0
statistics-cumulative: no
extended-statistics: yes
num-threads: 2
interface: ::0
interface-automatic: yes
access-control: allow
access-control: ::1 allow
access-control: ::ffff: allow
access-control: refuse
access-control: ::0/0 refuse
chroot: ""
username: "unbound"
directory: "/etc/unbound"
pidfile: "/var/run/unbound/"
root-hints: "/etc/unbound/named.cache"
harden-glue: yes
harden-dnssec-stripped: yes
harden-referral-path: yes
use-caps-for-id: yes
private-address: fd00::/8
private-address: fe80::/10
private-domain: ""
unwanted-reply-threshold: 10000000
dlv-anchor-file: "/etc/unbound/"
val-clean-additional: yes
val-permissive-mode: no
control-enable: yes
control-interface: ::1
control-port: 953
server-key-file: "/etc/unbound/unbound_server.key"
server-cert-file: "/etc/unbound/unbound_server.pem"
control-key-file: "/etc/unbound/unbound_control.key"
control-cert-file: "/etc/unbound/unbound_control.pem"

Note, be sure to add allowed IP ranges in access-control section.
private-domain refers to domains where RFC1918 addresses are allowed.

Once this is done, you just need to run:
To allow control via unbound-control, then a cron job weekly/monthly to do:
wget -q ftp://FTP.INTERNIC.NET/domain/named.cache -O /etc/unbound/named.cache