Oh, Molly!

I’m sure we all have had “that feeling once”. You patch your desktop or laptop, then type in reboot in a shell in order to boot your computer. And that crucial server you were working on starts shutting down.

But fear not - a solution exists for this and similar problems.

History

Molly-guard was (according to Internet) originally a improvised plexiglass cover shielding the kill switch on an IBM 4341. It was named after a programmers daughter - Molly ... [continue reading]

Stateless OSD servers

When building a Ceph-cluster, it was important for us to plan ahead. Not only does one usually start out with a minimum of ~5 servers, but one should also expect some growth in the cluster. Running the cluster also means patching the operating system and Ceph itself, and with Ceph being a crucial infrastructure component it’s also desirable to have a proper rollback procedure.

Using CI to maintain image

We’ve grown really fond of ram-disk nodes. Using a Jenkins ... [continue reading]

A brief history of the referer header

The poor referer header. Misspelled and misused since its inception.

Its typical use is thus: if I click on a link on a website, the referer header tells the landing page which source page I came from.

Source URL = www.mysite.com/page1 -> Target URL = www.example.com referer = "www.mysite.com/page1" 

It’s heavily used in marketing to analyse where visitors to a website came from, and also very useful for gathering data and statistics about reading habits ... [continue reading]

Job control

How often do you do this:

  • open service_foo.conf,
  • edit,
  • save and close service_foo.conf,
  • restart the service foo,
  • get a syntax error,
  • reopen service_foo.conf,
  • navigate to the same position you were at,
  • edit,
  • save,
  • try restart,
  • etc.

It’s pretty common.

Or:

$ long_running_command # Darn, should've started it in the background instead! CTRL-C $ long_running_command & 

All of these situations can be dealt ... [continue reading]

Thwarting and detecting malware with RPZ and OSSEC

In a recent sysadvent article I described how to configure BIND with a Response Policy Zone. Using an RPZ can efficiently thwart outbound network traffic based on one’s own preferences, and it can be extended to import and/or subscribe to externally provided DNS zones.

Configuring BIND

My local BIND server has been configured with two RPZs. One is maintained manually, mostly for reaching internal resources behind a NAT firewall by their official DNS names. The other one ... [continue reading]

Stress testing with Apache JMeter

Apache JMeter is a nice little tool with tons of functionality for testing web sites. It can be used both for stress testing and functional testing. This tutorial is going to show you how to set it up and get started with some basic stress testing.

Installation and initial setup

First, go to the official Apache JMeter website and download the binary, and unpack it on the machine you will be running the test from (your workstation/laptop should ... [continue reading]

Poor man's VPN via SSH socks proxy

It is late night. You have just arrived at your Grandparents, when the SMS beeper goes off. There is a problem with a SAN controller, and the on-call person know you fixed it the last time. Now, if you only had documented it.

You know you have to fix this yourself, but you have no VPN access. You don’t even have an Internet connection, except your 3G mobile phone, and you really need access to that admin web GUI. There ... [continue reading]

Bash process substitution

In bash, we often use redirects (that is < and > ) to get output from a command to a file, or input from a file to a command. But sometimes, commands takes two or more files as input. Then our ordinary scheme does not work anymore.

Classical problem: Diff output from two commands

Let’s say you want to diff(1) the output of two commands. For example, compare the contents of two directories. ... [continue reading]

Backing up the Rados Object Gateway

Amazon S3 has been around for a while, and it has become increasingly popular to use S3 or S3-like solutions as an object store. In many cases S3 replaces NFS as the chosen type of file system.

And with good reason. Separating application instance from application state is almost always a good decision. And by changing from an architecture that requires low-level access to the host running the application to using a REST interface, we can now deploy the application ... [continue reading]

Introduction to strace

There will come a time when you will find yourself asking “What the heck is that process doing?”. To uncover the mysteries behind the behaviour of a process, we have a tool called strace.

The program strace is very handy when you want to debug the execution of a program. It catches and states all the system calls performed called by a process. It will also catch and state any inter-process signals received by this process.

... [continue reading]