kerneljack's diary

On the first day of the UKUUG Spring Conference I attended the OpenBSD and PF tutorial. I was asked to write a short review of that tutorial for the UKUUG newsletter. I’m posting what I wrote below:

The superiority of OpenBSD when it comes to security is legendary. The OpenBSD community continuously do security audits of their codebase and their website proudly boasts of having only two remote holes in the default install, in more than 10 years!” PF is the default packet filter used in OpenBSD from version 3.0 onwards.

Having setup OpenBSD firewalls using PF in the past, I was interested in expanding my knowledge and this tutorial provided the perfect opportunity for me to do so. The tutorial was given by Peter M. Hansteen, who is a consultant, writer and sysadmin based in Bergen, Norway, and also the author of the excellent “The Book of PF“, published by No Starch Press.

Peter started off by answering some common questions that people might have about PF such as: Can I run it on Linux (Answer: No, but some are trying). He recommended not trusting any GUI tools, and simply using a text editor to edit pf.conf as that is simpler and faster. In addition, some tools claim to automatically convert “other” firewall rules to PF, but he recommended implementing a fresh PF config yourself.

The first firewall I ever tried to configure for my home network used iptables; when I had to implement one in PF, I found it to be a breath of fresh air. I have always found the concept of “chains” that iptables uses confusing. PF doesn’t have any concept of a chain; you simply start your rules by first “blocking everything”, then enabling the things you need, one line after the other. Although in principle this sounds exactly like what you are supposed to do with iptables, in practice the rules you generate are much simpler, and easier to understand.

Leading on from this, he showed examples of how PF should be set up in an environment where you need a “gateway” between 2 networks, and how to deal with problems faced by people who try to use FTP from behind a NAT firewall (ftp-proxy). Tables and filtering by services (http, ftp, etc) were introduced next. A table in PF is basically a list of IP addresses; listing them in a table makes it easier to apply a single rule to a collection of hosts.

Peter then moved on to the subject of dealing with the huge volume of spam that besieges us all. The two main concepts that he focused this section of the tutorial on were “tarpitting” and “greylisting“. In tarpitting, when a blacklisted host connects to you, you send replies to them very very slowly, let’s say around 1 byte at a time. When doing greylisting, you lie to unknown connecting clients using SMTP 45n errors (temporary local error). This usually thwarts spammers, who simply want to quickly connect, deliver their payload and leave. Many spammers don’t attempt to reconnect after seeing this error, while legitimate clients will automatically retry after a short period of time. These legitimate hosts are then added to a whitelist, which means that the next time they try to connect, they will no longer be given a 45n temporary error, their mail will be accepted immediately.

We then got a look at how to thwart SSH bruteforce attacks by using rate-limiting, and a short introduction to wireless networking in OpenBSD. The next thing that Peter talked about, authpf, was quite interesting. Basically users need to authenticate to authpf first; once authenticated, only then is traffic generated by these users allowed to pass through the firewall. Special rules can be setup specifically for authpf users.

The next topic was load balancing and Peter showed how to configure a “web server pool” using PF. Requests to this pool were alternated using a form of round-robin. To solve a common round-robin problem where machines in the pool go down, you can use “hoststated“, which monitors the state (up/down) of the certain specified hosts and compensates accordingly. hoststated has been renamed to relayd in OpenBSD 4.3.

You can tag incoming packets, so you can quickly pass/block packets marked with a certain tag. Setting up a OpenBSD bridge was discussed next. A bridge in this context simply refers to a transparent firewall that sits between 2 or more networks and filters packets at the link level.

Using ALTQ, you can do bandwidth allocation and traffic shaping. You can used class based queues (percent, kilo, or mega bytes), priority based or hierarchical queues. In a class based queue, you can say for example that FTP is only allowed 20% of your bandwidth.

The last major aspect of PF that Peter discussed was CARP (Common Address Redundancy Protocol) and pfsync. Put simply, CARP and pfsync allow you to setup 2 redundant firewalls instead of 1, and in case one firewall fails, everything switches over to the other firewall automatically. pfsync is used to keep the rules between the 2 firewalls in sync.

Overall, I am very pleased that I attended this tutorial. Obviously I was familiar with some of the concepts, but things like authpf, hoststated and CARP were completely new to me. I will definitely use the things I learned here when considering any OpenBSD based firewalling solutions in the future.

I’m a member of the UKUUG, and I just came back from their Spring Conference, which was held in Birmingham this year. It was a great conference! I met lots of other Unix/Linux people and learned a lot simply by chatting with people in the corridors! I will try my best to make it to the next one, which I think will be held sometime later this year.

In any case, I was asked by Alain (UKUUG Chairman) to write a few words about one of the tutorials that I attended on the first day of the conference. It will be published in the next UKUUG newsletter. I will make sure I post a copy of what I write here.

Fedora 7 is released!: I really liked the last Fedora release, but I believe it was slightly plagued by problems with some of it’s package management utilities. I have already installed this release and am quite impressed. Wireless now works with WPA out of the box and their new re-spinning feature is something I will try out someday.

XML Parser benchmarks: I have always had my own suspicions of which XML parser model would be faster (Sax or StaX), but I’m glad to see this benchmark done by the O’Reilly folks.

Fear and loathing at Cupertino: Jeremy Allison’s terrible experience while trying to prepare a talk for his Apple WWDC presentation. Jeremy works on Samba, along with Tridge, who they all call “the smartest man in Australia” Jeremy works at Google now. Smart man.

I’m going to occasionally post links here that I find particularly insightful, interesting or geeky.

Three things that caught my interest today:

PowerTOP: Released by Intel, this utility builds on work done by kernel developers to make the Linux kernel power-efficient. PowerTOP gives you a snapshot of what apps are consuming the most power. Turn off these apps or modify their behavior, and you’ll notice an instant increase in the battery life.

The Linux SLAB Allocator: Traditional heap memory managers suffer from fragmentation, among other issues. The SLAB Allocator in Linux, inspired by a similar implementation for Solaris and various embedded systems, allocates memory as fixed sized objects and uses caches to reduce fragmentation. It also has options to enable hardware cache alignment which allows objects in different caches to share the same cache lines, thus improving performance.

Advanced Linux Programming: After many years of coding mostly Java, I’ve been meaning to brush up on my C, Assembly and general Unix programming skills. I found this excellent book freely available online and it seems to be getting a lot of praise from reviewers on Amazon so I downloaded it. It has a lot of topics that I’m very interested in, like IPC and threads, and it even has a few assembly oriented chapters. I will definitely be reading this one

I helped out yesterday at the Greater London Linux Group’s meeting, held at my old alma mater, the University of Westminster in New Cavendish street. It was a blast going back after so many years, I met one of my best teachers, Sean Tohill who always had an open mind and a keen intellect. The whole event was organised by Simon Morris.
A lot more people than we had anticipated turned up for this meet! There were easily more than 100 people that showed up. Simon had installed SLED 10 on several desktops there and I got Debian running VLC on a big projector screen, playing a movie in a continuous loop.

A few people, including me, volunteered to give short presentations about certain aspects of SLED, and I was giving a short talk and demo of XGL under SLED. I demoed it to about 20 people and they were all quite impressed with it. Along they way I managed to help out several people with their linux problems.

I absolutely loved this meetup and I hope I can attend many more in the future. There are more details and pictures here.

I turned my computer on today to get some work done, started Eclipse and started coding. When I tried to use the command-line though, I got this strange error:


Error: no known VMs. (check for corrupt jvm.cfg file)


I couldn’t run 'java' or 'javac' from the command-line at all! I immediately went to /System/Library/Frameworks/JavaVM.framework and looked for 'jvm.cfg'. I have 3 VMs installed on this machine, 1.3.1, 1.4.2, and 1.5.0. 1.3.1 and 1.4.2 had a proper jvm.cfg file installed but for some reason 1.5.0’s jvm.cfg was a zero-length file. Googling didn’t turn up anything useful except this tip, which wouldn’t work because in my case all my permissions were correct. Fixing permissions using Disk Utility didn’t show any permissions problems at all.

In the end, all I did was copy the 1.4.2 version over to the 1.5.0 directory and all was well. The tip above mentions that Eclipse might have had something to do with this, and there might be some truth to that, as I did update my Eclipse to 3.2 recently, but I have been using it for a week without any problems …

Wow this is unbelievable! Apple has officially released a public beta of Boot Camp which allows anyone to take a Windows XP CD and install it on a Intel Mac complete with dual-boot! I haven’t seen this discussed anywhere today as yet, but in just a short while this is going to be huge. Ars Emporium and Macintouch seem to be the first to have this story. So this is why Apple joined that Windows benchmarking group a while ago. The next version of OSX (Leopard) will also include technology to let you run XP on a Intel Mac. I think Leopard will also include some virtualization technology so you won’t have to reboot your Mac just to use Windows.
From the Boot Camp page:

Boot Camp lets you install Windows XP without moving your Mac data, though you will need to bring your own copy to the table, as Apple Computer does not sell or support Microsoft Windows. Boot Camp will burn a CD of all the required drivers for Windows so you don’t have to scrounge around the Internet looking for them.

Well … I never thought I would see this day. Everyone has known that Apple wouldn’t officially support Windows on Macs and they still won’t offer support for this but it was always assumed that people would have to use hacks to get Windows to run on a Mac. With Boot Camp people will no longer have to rely on hacks and it will also burn you a CD of the required drivers for your XP system. Very very cool indeed.

The motherboard on my server died 2 weeks ago and it’s taken me this long to find a replacement and get the server up and running again. A benefit of changing the motherboard is that now the server is even more quieter than before! It’s all because I took more time to isolate the noisier case fans and to re-install them.

So I haven’t been able to blog for 2 weeks and *so* many things have happened. narf and blanca finally managed to get Windows XP running on a Mac! This is awesome news, especially after disappointing news a while back that Vista will not support EFI either. It seems they have managed to emulate a regular BIOS for the XP on Mac competition. Soon after the announcement, Leo Laporte did a great segment for MacBreak where they install XP on a Mac Mini and that’s the first place where I saw it actually running. A lot of people can now dual-boot their new Intel Macs and have a choice of XP or OSX (and even Linux!). I’ve never seen such a versatile machine! These are interesting times indeed.

A lot of cool stuff has been coming out of Microsoft recently at the PDC. You can watch the webcast of the event here.

Here is a short summary of some of the announcements, linking to a Channel 9 video of each (if possible):

Sparkle – separating visual components and design of an application from the data representation.

Start.com / Gadgets – nice, clean start/home page which can be extended using “gadgets”. and these gadgets can be re-used through inheritance. The windows vista sidebar will also sport gadgets which help people get to commonly used or needed data or tasks.

Microsoft Max – a kind of a cross between iPhoto and iMovie. allows you to create rich interactive photo albums with slick effects and to export these albums so that almost anybody can view them.

LINQ – a really cool idea of trying to remove the impedance mismatch between object and relational databases. From what I have seen, manipulating XML should be easier. It’s all done in C#. Creating and populating objects after using SQL joins should be easier.

WCF or Windows Communication Founcation (formerly Indigo) – Much more than just another web-services framework, it implements a lot of the plumbing that a lot of developers usually have to manage all the way from SOAP to P2P to some other method of app-to-app or pc-to-pc integration. So if you want 2 machines to talk to one another somehow you don’t have to worry too much about the plumbing going on behind all that communication. At least that’s the easiest way I see to summarize it

One the of absolutely coolest things that Jim Allchin showed off was the ability to increase the available memory of a PC running Vista by simply plugging a USB key in. It’s ingenious and I haven’t heard of anyone doing anything like it unless I am mistaken.

WPF/E or Windows Presentation Foundation / Everywhere. WPF was formerly Avalon, a new way to build rich interactive web / client apps using XML. They showed a very cool Netflix (video rental online) demo that they ran on 4 different machines and it scaled, etc flawlessly: a desktop machine, Media Center PC, Tablet PC and a PDA. Everything being vector based makes this a lot easier. Another cool North Face demo here

A lot more stuff than this has been announced of course but it is good to see at least a few cool things and possibly 1 innovation come out of Microsoft so far! They are pretty gung-ho on security nowadays and let’s hope that Vista finally gets security right (by right I mean a lot better than XP by default). Note that I’m usually *not* very pro-Microsoft, but I do believe in competition, and I hope Apple, Google, Linux et al give Microsoft all they’ve got because it makes things a lot more interesting for the consumer and gives us more choice. That is always a good thing.

[Jason Kottke](http://www.kottke.org) has an interesting write-up about how he envisions the future of the OS and the Web. The gist of his post can be summed up in the following points:

1. The OS can be made irrelevant by people writing for another OS, let’s call it WebOS.
2. People now only need to code for one platform (Java, anyone?)
3. A local web server on the client machine will mean that an app can continue on working even if it’s offline (think local app for Gmail, etc).
4. When users are back online, the app will synchronize itself with its online counterpart. Think adding Flickr pictures locally, and then uploading them.

The OS is still there whichever one it is, but it is no longer going to lock-in developers to only developing for that OS. One of the things Jason is saying is that people will no longer need to write for 3 OSes but for just one OS. This sounds just like “people will no longer have to write for 3 OSes but for just one platform: Java”.

Jason has already outlined some of the problems he envisions with this approach, such as web apps accessing local content on your hard drive. Java applets have solved this problem for a long time by using a sandbox model. The only problem is that these applets have to be downloaded and run locally for them to work. A WebOS app, however, will run both locally and remotely, it will live on your local hard drive and perhaps a sandbox model will help there as well. Are vulnerabilities detected in these local apps any more dangerous than vulnerabilities for remote apps? Perhaps there isn’t much different because at the end of the day, a security hole is the same thing whatever app it affects. However, I suppose a local security hole can do a *lot* more damage than a remote one.

Some other suggestions like: “Read newsfeeds from bloglines locally” sound no different to NetNewsWire, FeedDemon or SharpReader, etc. These are all locally accessible feed readers which, when online, update your subscriptions, etc, otherwise they work fine when offline.

I think, as [Paul Graham](http://paulgraham.com/) said, a WebOS will allow totally new kinds of applications to exist and we don’t know what they will look like at all. So this is definitely an interesting space to watch.
Some great examples of new ways of using the web are: [Backpack](http://www.backpackit.com), [Basecamp](http://www.basecamphq.com/), [Gmail](http://www.gmail.com) and [Google Maps](http://maps.google.com). Another up-and-coming app is [Hula](http://hula-project.org/Hula_Server) which has been open-sourced by Novell and is being actively worked on by many GNOME hackers. If you want to see a demo of how the future of web-calendaring might look like, take a look at [this](http://www.nat.org/2005/august/#Hula-Web-Interface) amazing demo on [Nat's](http://www.nat.org) blog.

In other news, I would love to get my hands on one of [these](http://joi.ito.com/archives/2005/08/25/wearing_firefox.html) [Firefox](http://www.getfirefox.com) T-shirts that [Joi](http://joi.ito.com) managed to pick up from the [Mozilla](http://www.mozilla.org) offices. I have ordered quite a few T-shirts from [ThinkGeek.com](http://www.thinkgeek.com) in the past, and I will order one of these if they stock them someday.

A very serious exploit has been discovered on OSX, and everyone using Safari or any other browser on OSX (even FireFox or IE) are urged to follow the instructions here so they can temporarily stop the exploit from working, at least until Apple gets its act together and fixes the problem. The fact that Apple have still not fixed the problem, which they have been aware of since February strikes me as a very stupid thing to do. I don’t know if fixing the exploit would require many in-depth changes to the OS and maybe that’s why it’s taking so long for Apple to fix. One would hope that this is indeed the case. They should have at least warned their users about this a long time ago. Leaving an exploit unpatched like this for such a long time looks really bad for them, especially in enterprise environments where I hear they are trying to make serious inroads.

It seems the HFS+ file system on OSX is quite good at keeping file fragmentation at bay, almost eliminating it altogether in most cases! Amit Singh has a great article about it here.

A colleague at work asked me today how OSX’s Mail.app does it’s spam filtering and this appeared as if just by coincidence

Created by the engineers who bring the Japanese input method and the Speech technologies to you, Mail’s junk mail filters are outstanding. When trained for a sufficient period of time, the filters can reach 98%+ accuracy against spam and are surprisingly painless to use. In fact, this feature alone has convinced many users to switch to Mail.

The first part of that article is also pretty good, a lot of good stuff about spam, picking good email addresses, etc.

Update: These guys have released a program to fix the OSX exploit.

I’ve been playing around a lot with Mac OSX recently and I must say I really love the system. OSX does a lot of things quite differently and that is why most people simply try to use it like Windows and hate it. I tried very hard to understand what Apple tried to do here and I must say I appreciate quite a few things: application bundles, the dock, system defaults, etc. The idea that I can put my whole java app in a directory of its own (including all dependent classes), write a few simple XML files, put images etc into the same directory, set a special ‘bundle’ bit, and that whole directory is now a deployable app is neat!

I must admit, saying it that way it doesn’t sound that different from just putting the java app into a directory and being done with it, but Apple have taken this a bit further than that using special features like the bundle bit and by assuming from the outset that *all* apps will be deployed like this instead of just a few that you develop. This allows you to have multiple versions of the same app in different bundles with different libraries inside them for example (as far as I understand it).

However, customizing features of the OS is not always easy. I needed to control how my app was deployed, so that users could not change the system settings for example and at first I was quite baffled how to do it. Moving the app from one location to another proved futile. OSX always knew where the app was located regardles of which directory I put it in or what I called it I later discovered that if I dig deeper into the bundle and edit the Info.plist xml file and change the BundleName, it will finally no longer run. However, this was not the solution for me since I did need to run it myself sometimes. Eventually after sometime of poking around trying to understand the system and reading hints on the excellent MacOS X Hints website, I discovered that I could simply remove privileges for the user so that he/she could no longer run System Preferences. Simple enough.

Much harder were figuring out how to remove items from the Apple menu bar as I did not want to use one since I was deploying an older app that had never used menus before and many other exotic things like replacing the Finder upon login with my own app. OS X seems to store a lot of things in databases like the NetInfo database and the ‘defaults’ database which it carries over due to it’s NeXT and OpenStep inheritance. This makes it quite convenient to change things, but only once you know where they are! Not unlike the days when I used to memorize Windows registry keys

Weather is extremely hot nowadays, usually around 30-35. The tube is horrible, just horrible, it’s baking hot in there. It’s not unusual for me to come out of it dripping with sweat. Good thing the Mayor has issued a challenge to anyone who comes up with a workable air conditioning system for the tube … Here’s hoping

My parents are coming round in about 2 weeks time, and we are both very happy to be seeing them after such a long time. We haven’t made any travel plans as such, I guess we’ll take each day as it comes …

After being impressed with Mandrake Linux 9.1 as I wrote in the previous post, I decided to install it on my old Compaq laptop (an Armada 1592DT). This is a Pentium 233Mhz machine with 64MB of ram and a 3GB hard disk. I had a feeling this was going to be a mistake and I was right … the latest ‘desktop-based’ distros are quite resource intensive, even if you’re not running much, just X. Of course I didn’t run KDE or GNOME, but even booting the machine and logging in was painful! The whole installation ordeal made my laptop so hot I had to prop it up on some cd cases to cool it down.

Oh well, I used to have a perfect Slackware installation on it running Fluxbox as the window manager and Opera as the browser. Distributions such as Slackware and Debian are really very useful in situations like this, because you can install just the minimum amount necessary and get on with it. So I scrapped Mandrake and installed Debian 3.0r0. No problems so far, I’m using the same fluxbox/opera combination. A very light window manager, and quite a light browser. Opera only takes up about 14% of memory on start up and is taking up about 35% right now, which is fine as browsing is the primary purpose of this machine anyway. Oh well, lesson learned. Shiny new desktop distros just don’t belong on older and slower laptops, even with many services turned off. You’re better off going with a light-weight distro like Slackware or a minimal Debian install.