JRRZZ Blog

Ever since we’ve received fiber to the home in my town I’ve been happily browsing the web using this awesome new 100Mbit/s service. Okay, some of the ISP services have been failing, but the link itself is fine. I’m even watching Discovery HD without any problems. Hell, even the phones work.

I work only a few kilometers from where I live (because I’m a lazy bastard), so obviously we’ve got fiber to the office as well. If only that worked as well as the connection I’ve got at home.

The trouble started when we decided to ask for a business link in stead of a regular link. We’d get 100Mbit/s guaranteed and a 1Gb/s cross link with our other building down the street. There was also some deal with an SIP trunk for a telephony, to enable our regiment of well spoken ladies to chat along freely with the customers. It all sounded pretty nice, and looked like it too. On paper that is.

To sweeten the deal, we’ve rented out a part of our building to the fiber business. We got our fiber for cheap, and they got a nice place for customer service. Whenever we had a problem, we could just walk over to the fiber people and talk nerdy to each other. Maybe even share some IT war stories, who knows.

Four months behind schedule, random people started digging randomly with even more random equipment around our premises. The pavement had to be yanked out, a trench had to be dug and lots of coffee had to be consumed. You couldn’t talk with the digging crew, they all spoke some eastern European language. After the digging crew left, another group of professionals dove into the trenches to fill them with shiny new fiber cables. After they left, the original crew could come back to fill up the trenches with the original soil and put the pavement back where it belonged. This setup sounds great in theory, but these people managed to screw a few things up. The coordination between the two crews wasn’t that well thought out. Trenches would stay open for days, waiting for fiber that just wouldn’t come. The town center looked like a WWI war zone during a weekend. Everybody knows that when you make a hole, put something in it, and put the original soil back, you’ll end up with too much soil. Apparently they didn’t know that. So after these people were finished, you could just follow the narrow stretched speed bumps in the pavement to see where the fiber went into the ground. Anyway, while they were messing around with the orange bundles of joy, we started to wonder why our office didn’t get entrenched. Every other building in the street got their trench, but not us. Did we collaborate with the enemy or something? Why didn’t we get one of those defensive perimeters?

After a while, the eastern European diggers were gone, the opposing factions reached a peace agreement and the trenches vanished from our streets. Minus the pedestrian speed bumps of course, those were still everywhere, causing elderly people to break bones they didn’t even know they had. Everybody in our neighborhood had their orange bundle of fiber sticking out of the pavement, but we didn’t. Neither did our fiber supplying neighbors, who were running the whole show. Somehow they managed to skip themselves and us in their planning.

As (a lot of) time passed by, one by one the naked fiber strings, which were sticking out of the pavement while enduring the worst Dutch weather could throw at them, were warmly tucked away in people’s houses and offices. Even our fiber friends next door managed to get their own fiber connection, illegally adding cabling to our building, while they should’ve connected us instead. We managed the building’s internal gigabit network, on which they could surf the intertubes. So now we had no fiber, no connection and a tenant who does, but doesn’t want to share.

After lots of whining and complaining, the eastern European dudes came back and fixed us some fiber. The complimentary pedestrian speed bump was placed nicely in front of our doorstep. It took even longer for the technicians to hook it all up, but finally we had two fiber connections. What, two? Apparently we needed one for regular internet, and the other one for the link to our other building. The original plan was to put the internet connection at the other building and route the internet connection through the gigabit fiber cross link. We finally had fiber to the office, almost a year behind schedule and twice as many cables as we originally thought we’d get.

But we weren’t there yet. Oh no. People had to activate our connection first. In order to do that they needed to know what kind of connection we had in mind. Somehow something went wrong between sales and the technical people, because now we needed to buy some really expensive Cisco equipment all of a sudden. According to the engineers, that’s the only way to create a sustainable VLAN across their fiber network. We’re a 3com shop, so that would’ve caused quite some havoc in our network unless we added some routers as well. During the process the government telecom watchdog intervened and split the fiber company in two, causing the price to go up. This made the whole ordeal even more expensive than a Cogent multi-gigabit carrier line. Even registering ourselves as an ISP on the (now separated) network, with all the hardware it takes, would’ve been cheaper. So after some debate, we chose to get two regular connections, one of which with 100Mbit/s guaranteed. We only needed one, but it would be a shame not to use the extra fiber. After all that was communicated and done, we got the connections activated. One of these connections was the business line, but they didn’t tell which one. Both the fiber modems were identical and weren’t marked. We couldn’t tell the difference either, because the fiber provider activated both of them as regular consumer lines. Later they told us their activation scheme works using addresses only, so it couldn’t even handle two lines at the same address. So we’ve connected one of the modems to our company router and told the ISP to use that particular modem as a business line. Up until this day we haven’t received a business connection.

After the fiber people cleaned up, the town center continued it’s daily life. People were happily watching HD adult entertainment, calling distant friends and sending spam in unprecedented volumes in their newly fiber connected homes. This digital utopia would only last this long, because a large group of people who can’t even spell the word education decided to start WWII.

Their mission was to turn the peaceful town center into a battlefield again. They brought bigger toys with them as well. While the fiber people could easily manage with a set of spades, these uneducated grunts brought heavy machinery. Slowly it became apparent they were just there to replace the sewage and water systems, the pavement and some shrubberies. After two days of digging they had uncovered almost all of the fiber again, some of it dangerously dangling over deep holes they created to replace the sewage piping.

A few days later we suffered a major brown out. Only our radio kept playing, because it’s built out of an old laptop. One of the half-brained apes in orange clothing managed to cut the power lines with one of his big digging toys. In doing so he blew a bunch of fuses in a central power hub, cutting off half the town. It’s not that bad for an IT company to suffer a brown out, compared to the fact that we had a Perfect Draft next to the fridge, keeping our beer from going bad. We had to drink all of our beer that day. Luckily the power people have been able to fix it quite fast.

Only a short while later the infantile orange dressed mammals did it again, but this time they pulled our precious fiber to bits. Right in front of our door. Luckily the fiber technicians have been able to weld the two pieces back together again. We’ve only suffered minor downtime in comparison to what happened next.

Now these shit-for-brains dirt cowboys have really done it. They’ve completely destroyed an important fiber linkage, causing almost two hundred homes and businesses to lose connection, including us. According to the fiber gurus they can’t fix it this time, without having to replace the whole cable. That means they have to open up the pavement again, all the way back to the junction box, on the other side of the village. This’ll probably take all weekend, so that’s even more downtime.

There are two things that’ll probably happen now after all of this fiber failing. Some lawyers are going to get paid, and paid well. Some nine months after these brown and web outs, we’ll have to welcome lots and lots of new inhabitants in our village.

As requested by some people, another update. The project is progressing nicely. There have been a few minor setbacks and a few more lie ahead. But things are going quite well nonetheless.

Since the last post a lot of timing and connection tests have been thrown at the cluster. I’ve bombarded the poor collection of boxes with ICMP floods, gigabytes of random data and h264 media while torturing it by pulling out random cables and hard disks. The setup withstood everything without real problems. I’d run a nuclear submarine on this system if I had one.

There are a few practical problems though. The firewalls use pfsync to share connections between the machines. This enables them to keep every connection alive when a fail over occurs. The entire IP state table is essentially duplicated constantly between the routing firewalls. If an application node fails though, it’s virtual MAC address (and associated IP addresses) will be transferred to another host, but the active connections cannot. So if you’re streaming some high definition pornography and the application node providing your audiovisual entertainment fails, you’re screwed. Well, that might be a bad choice of words, because the screwing would stop. But anyway, what happens is that the connection will survive from your home box to the cluster. Inside the cluster however, the connection won’t be re-established to the new node.

Right now I’m trying to get VLC to stream over RTP instead of HTTP. RTP (usually) uses UDP, so in theory it should work. I’m still working on understanding the nature of RTP streams, especially now I’m using RTSP. I haven’t got it running just yet, I guess it’s a firewall problem or something. Also, I suspect the RT(S)P states to be stored in a node’s VLC process, which would render this entire idea useless once again. I sure hope I don’t have to write an RTSP sync protocol.

As I’ve written in my previous post about this project, the hardware was kindly donated by my employer. Since the hardware can’t leave the building, I’ve been spending most of my time at my employer’s office instead of where I should be, at M2X. Luckily the hands-on part of building this cluster has been finished, so working remotely would suffice. I’ve been playing around with NX technology lately, so I thought it to be a great idea to put that knowledge to good use. The collective.borg.local cluster is running an X11 box, so that wouldn’t be that much of a problem, right? Since I’m an Open Source kinda guy, I’m using the FreeNX variant. It works, but it’s quite limited. The biggest problem is it’s lack of XDMCP support. With NoMachine’s NX server I could easily select the cluster’s XDM server and set up a compressed X11 session. FreeNX only wants to run local X sessions using XSession or KDE/Gnome scripts. I’ve solved this problem with a nice hack, using OpenSSH’s X11 forwarding. First, I made sure the server running NX could login on the cluster’s X11 box using public key authentication. After enabling X11 forwarding on that machine, an X session running a remote openbox would enable me to mimic XDMCP behaviour. Because the DISPLAY environment variable is set for every child process in the SSH session, every application I start using the window manager will be sent over the same SSH tunnel. The only thing the NX server’s .xsession file contains is exec ssh -X collective openbox. So when I login on the NX server, it will request the cluster to start an openbox process. That process will send it’s X11 data to the NX server, which compresses it and send it to my NX client. Anywhere on the internet. Yes, I could have just used SSH, but X11 is way cooler.

For the time I’ve got left a few really nice projects lie ahead. I’ve got to make this cluster nicely manageable with a web interface and stuff. I think I’m going to use Puppet and Ruby for that purpose. After that’s done some QA software has to be made to ensure the cluster’s streaming quality.

I hope the links are useful enough for you to understand what I’m doing. It’s not that hard, really ;)

For the final stages of my Bachelors degree I’m building a cluster again, just like last year. Clusters are fun. “Imagine a Beowulf cluster of those!” is a fairly common expression amongst the slashdot crowd. This particular cluster is being built to offer non-stop streaming multimedia content. The stream simply can’t fail. Ever. Since I’m doing this for a company, I’m not sure how much of this can become public knowledge. I won’t get into the nifty technical details, but I’ll describe the system globally.

I thought I could just take my old Debian Linux-HA cluster and implement multimedia streaming on top of that. For research’s sake, and a year’s worth of development had just passed, I decided to do some more research first. I’ve been looking into both high availability and manageability. The state of today’s data center management solutions is dreadful at best. Red Hat offers some nice stuff, but makes the huge mistake of requiring an Oracle DBMS. I have been ordered to stay away from any non-free software. Linux-HA is still great, but it’s GUI is broken. The command line interface is ok, but the GUI made the whole thing so much easier to set up. I’m afraid I have too cook up my own solution to get some manageability into the system.

And then there’s latency. In order to maintain a 1080p h264 video stream the connection simply can’t drop for any longer than a second or so. Linux-HA failed miserably during my tests with 5-10 seconds of fail over time. That’s quite obvious, since it’s implementation is entirely in user space. So I needed something faster, something at the routing level. I can’t use any non-free software so Cisco stuff is out of the question. A friend of mine told me about CARP, a free alternative for the patent ridden VRRP. Yes, VRRP should be free, a standard and so on. Think again.

So that brings me to BSD again. I’ve been playing around with BSD for a few years now, but I’ve never really used it for anything beyond my IRC and seed boxes. Since that post I’ve switched over to OpenBSD from NetBSD. Since I’m familiar with OpenBSD and CARP originates from this project, the operating system of choice became obvious.

Setting up this cluster was extremely easy and a lot of fun. Everything is nicely integrated, works as expected and is rock solid. OpenBSD’s packet filter is an extremely powerful tool which I’ve underestimated. It’s easy, fast, feature rich and secure. I can set up my entire network in a single human readable file. From routing and packet filtering to load balancing, it’s all there. Actually, all of the system management tasks are quite easy. I’m sure I’ll be reusing many of OpenBSD’s strengths in the future.

Meet the collective.borg.local test cluster. The hardware is kindly donated by my other employer. The big gray boxes are old AMD Athlon 32-bit machines with MSI VIA based motherboards and quite some RAM. I’ve called them cube-alpha, cube-beta and cube-gamma. Alpha and beta are routing firewalls in fail-over mode. That’s why they’ve got an extra Gig/E NIC connected to the DMZ. I can yank one of those machines out without disturbing the functionality of the network. The third, gamma, is an X11 box used for displaying log output of the other boxes in XTerms. The four black boxes are node1-4. Simple Dell/Intel boxes with a gig of RAM and Core2 or Celeron processors. They do have a nice Intel Gig/E NIC on board, though. These are the application servers. The incoming network connections are balanced equally between the four by the active router. I can unplug three of those and no one will notice. The one remaining will be handling all of the incoming connections, but the cluster won’t go down. All of this is connected using a rather nice 3com switch. All of these machines are running OpenBSD 4.4.

Here’s my work area. On the left is the X11 OpenBSD desktop tailing some log files from the application nodes. On the right my trusty Sun workstation, running Debian, decoding the 1080p h264 stream from the cluster. The movie I’m using is Big Buck Bunny, a movie entirely made using Open Source software. At this point I had killed two nodes and one router. The movie didn’t even hiccup.

There’s lots of work to be done. It’s working technically, but I’ve got to get VLC involved in this. At this moment VLC isn’t able to properly decode h264 over HTTP, so I’m using MPlayer right now. Also, Apache is serving the multimedia. This should become VLC too, using RTSP instead of HTTP. The state of VLC at this moment is very unstable, so I guess I’ve got to fix some things first. I’ve also got to write some custom code to detect errors in the streaming server, which should cause an immediate fail over event. Last but not least, manageability. I’ve got to find a way to both administrate and deploy OpenBSD streaming servers from a central console. That’s going to be a challenge.

I’ve just arrived back home after a weekend of geekery at FOSDEM, Brussels.

I didn’t know almost everybody I know professionally would visit the event. It has been kind of a reunion, heh. Anyways, the highlights I really think everybody should check out for themselves:

• The Debian keynote by Bdale Garbee. Everybody loves Debian. His keynote was a pretty good explanation of what Debian’s all about. Especially the people who are new to Debian should watch this keynote.
• What’s next after Firefox 3.1 by Mike Connor. Two reasons why you should see this, actually. First, Firefox is the most important tool in the struggle for a free web. Mozilla’s efforts at making a better browser should not stay unnoticed. Second, it serves my anti Mac OS X tirades quite well. I hope I’m quoting Mike correctly here: “Mac OS aught to be user friendly”, while trying to get the beamer to work.
• Syslinux and the dynamic x86 boot process by H. Peter Anvin. A very, very interesting talk about syslinux and friends. I’ve been working with syslinux for years to boot custom Linux images, but I didn’t know it could do all that. Syslinux has really become toy worthy. Peter even made Linux boot over an HTTP connection halfway across the globe. Really cool indeed, a must see.
• Emdebian 1.0 release – small & super small Debian by Neil Williams. I’ve been working with Emdebian a while ago at Fontys, which suddenly made embedded software development a lot easier. Release 1.0 features lots of really cool stuff like early support for TDebs and two flavors called “Grip” and “Crush”. Grip is a (way) smaller Debian Lenny. Crush simply gives the embedded software developer even more control over the Debian packages by providing finer grained rules and new packaging tools. I can’t wait to give Crush a spin.

I’m sure I’ve missed some great stuff. Feel free to comment and recommend talks.

During the next months until the summer holidays I’ll be churning out lines of (mainly C) code.

Today I’ve started working on a project that should make VLC cluster aware. I’ll be working with one of the developers, so I’m sure some of my code will end up back into the videolan source tree some day. I’ve got 100, err, 99 days to get the job done. It’s like a Google summer of code thing, but my university will pay me in credits instead of hard cash. This will be my final project for my Bachelor’s degree, so I’ve got to make sure I don’t fuck this up. I’ll update this blog more frequently now in order to keep friends updated, who are all scattered across the globe in the same 100 days deal.

Also, JBISC is going strong. I don’t have that much time to invest in it, but it isn’t dead yet!

Finally I’ve started working on a signal processing framework. This thing will be able to do all sorts of cool stuff. I’m currently aiming for using signal analysis/syntheses techniques to create audio art. The preliminary results are quite impressive. I can’t wait to work this out and rip some more speakers using ultra low frequencies, or getting that “instant headache and nausea” thing back again.

Oh, I’ll be visiting FOSDEM this year, too.

A while ago a colleague of mine advised to update my E71, because it would give me some nice new features. I had never updated the phone since I got it. I was running version 100 dot something, while 200 dot something was the newest version. I must have missed a lot. So I tried updating my E71 today. Here’s the story.

First I went to nokia.com. The site didn’t work. I guess they couldn’t handle the big amount of Nokia fans. So I went to nokia.nl, my local site. This site did work, but just looked funny in Firefox. I tried finding updates for my E71, but I couldn’t find any. I did find a Nokia software update utility, however. So I downloaded that. Apparently I had to run Windows in order to update my phone. Luckily I had a spare XP box collecting dust on my desk. Before the Nokia software updater could finish though, it warned me it had found a newer version of itself. How could that be? I had just downloaded the damn app. It wasn’t just a minor update, either. The software I got from Nokia’s localized site was a full two versions behind. After installing the right Nokia software updater, I had to reboot my machine. Eh, reboot my machine after installing an installer installer? Come on! But okay, I rebooted the computer like I was supposed to. Apparently the application had registered itself as an autorun app of some sort, because it showed itself directly after logging in again. It’s a butt ugly app too, by the way. It’s a mix between Mac OS X buttons and progress bars, Mozilla like green arrows plus Vista window decorations. Anyway, the application warned me I had to run the Nokia PC suite first to make a backup of all of the phone’s data. What? So I need a second app to make a backup. Right, so I installed that PC suite of theirs. It came with all the bells and whistles, including some modem drivers I don’t need. Hooray. Making a backup went smoothly, no problems at all. It did take ages, but that’s okay. I knew the E71 has a slow USB connection. Back to the Nokia software updater. Now it warned me I had to put my profile back to the default setting. I don’t know why they need the phone not to be silent, but if that’s what it takes, I’d gladly re-enable my black metal ringtone. The software updater took five minutes to figure out it was an E71 it was talking to. The PC suite was happily showing a picture of an E71 on the background, but the software updater seemed to be ignorant and wanted to find that out on it’s own. After finally claiming victory over the USB connection, the software updater could continue. It had to download a huge binary blob from one of the slowest servers on the internet. I could almost see the Nokia brand slowly change to Asus. Next, it showed a lot of warnings again. The phone might seem dead during the update, and it might also disconnect every once in a while. Great, thanks for the heads up. During the update the phone’s display turned bright white, like it wanted to make sure the battery would drain before the update could finish. I quickly connected the phone to the power supply, just to make sure it didn’t. After a while the software updater reported that the update was successful.

Now I had upload my backup again. The Nokia PC suite informed me there wasn’t any phone connected. I tried pressing some buttons on the phone in order to wake it up, but it kept mimicking a dead Nokia technician. So I disconnected it and tried to turn it on. It did turn on, for a while. The lights worked, the vibration worked, but where’s my trusty Nokia logo? I reconnected it to the power supply, tried again and it still didn’t work. I tried pulling the battery out and putting it back in again. This did have an effect. After turning on the phone the lights kept burning and the vibration didn’t stop. Great, now my E71 is mimicking a vibrator flashlight combo. I had to pull the damn battery out or the thing would never shut down [sic].

Damn, so I’ve bricked it. I revisited nokia.nl again, looking for some kind of support. Apparently Nokia has built phone hospitals (or morgues) all over the world, called Nokia Care Points. I guessed they’re the Nokia equivalent of the Apple Store, but with well equipped Finnish women instead of gay Apple fanboys. I tried finding a point in Eindhoven. The website that should help me find the Nokia Care Point didn’t work in Firefox. I had an experimental webkit based browser, so I tried that. Hooray, it worked. Apparently there’s a Nokia Care Point in Eindhoven. But it wasn’t what I had expected at all. It’s some kind of business telecom provider, not targeted at the average tech savvy folk like me. I’ll call them using POTS next week or something.

Anyway, if the Nokia Care Point dares to even charge for the repairs I’m going to buy myself a new phone. Not a Nokia this time. What the hell happened to Nokia all of a sudden? They used to produce the best phones in the world, and they still do, but the desktop apps just plain suck. I’ve heard Google has a Linux based phone OS available these days. As long as it runs an SSH client over 3G, I’m a happy hacker.

This whole evolution thing didn’t end up with an operating system we can be proud of. Nature inc can learn a lot from UNIX. I’ll try to explain my opinion using a few examples.

There’s only one kind of user space process in UNIX. Every process has it’s own unique process ID and a scheduler makes sure processes don’t get in the way of each other. Processes can leave messages for one another in well defined uniform ways.

MAN has two kind of processes, male and female. Many processes have the same name which makes it difficult to keep processes apart. Especially the male typed processes have a small name lookup table. There’s no real scheduling. Some processes tend to behave like masters, while most of the other processes run in slave mode. When a master process exits, a state called anarchism arises. A successful call to war(1) spawns a new master process. Processes communicate with a type-specific protocol. The female protocol seems to use more bandwidth than the male equivalent. It’s also quite awkward that the boolean type seems to be inverted. Communication between the two process types is possible, but might randomly call yell(2), cry(2) or even kill(2) in some cases.

UNIX is fairly secure. When configured right UNIX has a very low chance of being infected by some kind of virus. And even if it gets infected, a virus is just another process which can be kill(1)ed off easily. Processes can’t write to each other’s stack unless explicitly defined. Files have owners and access rights defined to them, and every process adheres to these access rights. Memory leaks will cause processes to grow larger, but an out of memory killer will eventually end the process.

Unlike UNIX, MAN isn’t that secure. Every process has a status indicator called health. Viruses aren’t processes but infect individual processes by inter process communication. An infection lowers the value of the health indicator. If health drops below a certain value, which is process specific, a process will cease to function. If it drops to zero, a process will call die(2). Usually processes are able to get rid of an infection themselves. If these attempts are unsuccessful, infected processes are quarantined until either die(2) is called, or an update for the built-in anti virus software solves the problem. Master processes in MAN are able to write to the stacks of slave processes using libmedia or the massively parallel version, libpropaganda. Files are loosely linked to an owner, but that can easily be changed by calling buy(2), steal(2) or find(1). All MAN processes will eventually start leaking memory. But unlike UNIX processes they loose weight until the stack collapses and die(2) is called.

Process structures in UNIX are very simple. A process can fork(2), creating a child process exactly like itself. If the parent process dies without kill(2)ing it’s child processes first, the child processes will turn into zombies. UNIX admins agree zombies should be kill(1)ed on sight.

MAN has very complicated process structures. Only a female process can call fork(2), but needs a male to provide the seed(3) IPC call, since it isn’t present in the female process libraries. But before that can happen a male and a female process have to be able to communicate, which can only happen using the narrow parallel between the two protocols. When a (fragile) line of communication has been formed, the two processes have to call mate(2) simultaneously. Processes can call mate(2) on their own too, but that will just result in a short spike in resource usage. There are theories about unsynchronized calls to mate(2) and effects upon the health indicator, but these are yet to be proven. After both calls have been successful, there’s a once in a month chance fork(2) will be called in the female process. This period, simply called period(4) is the only timer resource that’s quite accurate in the entire MAN system. The down side of period(4) usage is the change in efficiency of the female typed processes. The call itself takes a whopping nine months, during which period(4) is broken, with a huge I/O drain at the end. The process type of the child process, male or female, is determined by a call to rand(3) using the provided seed(3) result. Luckily MAN provides a shorthand to this complicated process, called make(1) love. Unlike UNIX, MAN knows several ways the connection between the parent processes and the child process can break. First, both the parent processes might call die(2). Second, one of the two parent processes can become a resource hog for which the other process calls divorce(2), taking half the stack size of the process for it’s own gain. It’s usually female processes that call divorce(2) by the way, male processes prefer kill(2). The third and final way for the connection to break is when the child process’s stack is large enough to get it’s own address space, instead of using the parent processes indirectly. If however the connection breaks because of one of the first two events, the child becomes an orphan. Within the MAN community kill(1)ing orphans is frowned upon. They are kept running instead until die(2) is called, or they are given to a new set of parent processes that fail at make(1) love.

The final point I’d like to address is the CPU usage by processes. Again, UNIX is fairly simple. A process is sleeping unless directed by the scheduler to do work, or if it’s in a sleep(2) call. This makes CPU usage fairly balanced between processes and the maximum amount of work gets done.

This feature is clearly broken in MAN. A process has to be in a single sleep(2) call for hours on end every day in order to work correctly. I guess this is MAN’s way of avoiding a scheduler. On top of that a MAN process tends to call sleep(2) when it’s supposed to work, and is quite actively calling mate(2) and other useless system calls when the there’s no work to be done.

In short, MAN is way too complex to be a functional system. It’s scheduling is broken and it’s API documentation mentions things that aren’t even on the system (soul_*() system calls, G_SPOT and something called “intelligence”). Just ignore it, it’s a fad. Stick with UNIX.

Tomorrow, the 26th of November, Microsoft will visit my university to “inform” students about it’s “great” offerings in both software and technology.

Of course I’m not such a supporter of Microsoft’s track record. The company’s probably the most evil institution the software developing world has ever seen. I’ve always got my critique ready when another snob, conveniently placed on the Microsoft payroll, openly starts kissing his employer’s ass. But lo and behold: Microsoft has (indirectly) invited me, of all people, to a Q&A session “About Microsoft, with Microsoft”. Of course I humbly accept the invitation.

Because the session will only (surprisingly) last half an hour, I’ve limited my questionnaire barrage to two questions. Maybe some fellow free software developers will join me in this session.

• Why does Microsoft continue to challenge European law, both by abusing it’s monopoly and starting endless appeals in court?
• How does Microsoft think European software patents would encourage innovation, even now the U.S. system clearly shows it doesn’t?

I hope these will be enough to make my willing Microsoft victim, Ruud de Jonge, Director Developer and Platform Enthusiasm, want to develop NetBSD-only console games instead.

Update: according to Microsoft they won’t answer these questions, because it’s not within the scope of this tour. Too bad.

As some of you might know, I’m working on a new programming language.

It sure takes more research than I expected. I’ve already dug into lexical analyzers, compilers, virtual machines and that sort of stuff. It’s a huge amount of new information which I’ve overlooked at the start of this project over a year ago. The low level work that goes into making a programming language actually work is simply overwhelming. But it’s so much fun! It really requires you to take your logic to a next level altogether. Did you think nuclear physics is hard? Try writing a compiler!

With all of this new information stuffed in between my ears I’ve got to tone down my OOP stance even more. It’s still valid, but the everything’s an object idea isn’t that bad at all. It really simplifies both compilers and virtual machines. There’s no real overhead. I really like the idea of extending or inheriting from standard types. It makes the language so much more customizable.

As a part of my journey through the world of programming languages I’ve stumbled upon several really interesting concepts. One of these is aspect oriented programming. I’d heard of it before, but never really got into it. Try writing some Java using AspectJ or watch this Google tech talk if you want some more information about the subject. AOP’s actually very useful when used properly.

The other gem I’ve found is Ruby. What a great language. A lot of the radical ideas for my language are already there, fully functional. I’ve never thought anything of the language, especially when the Rails flood hit. Too bad I’ve ignored it, since it’s an excellent allround language. Ruby also implements closures the way I want them to be implemented, not the Java hack Groovy uses. It uses closures transparently and in a highly compatible way. I had all but abandoned hope for proper closures in a modern non-Lisp language. I think I might actually going to use Ruby as a Bash replacement for day to day programming tasks.

In the next year or so I’m planning to collect even more knowledge about compilers and virtual machines, learn some more languages and maybe have some time off every now and then. I’ve also replaced that horrid Drupal thing with a proper HTML website.

With the rise of internet anti-freedom legislation in lots of countries I might just visit, and the ongoing US lobbyist’s pressure to enforce similar laws here in Europe and the Netherlands, I’ve taken drastic steps to counter these threats. The MAFIAA can eat my 2048-bit AES encrypted shorts!

I’ve already been using SSL/TLS and SSH technologies to encrypt my email while it travels back and forth between the Wasda.nl mailservers and my personal machines. With access to enormous amounts of bandwidth, hardware and IP address space in the Wasda.nl network I’ve done something I’m not quite fond of. I broke a part of the internet to ensure my freedom. Instead of encrypting individual links and protocols, I’ve chosen to encrypt the whole lot by forcing all of my web traffic through OpenVPN-powered tunnels. Now I’m sure I can trust the security of my internet connection, wherever I may roam.

The next step is to securely acquire more trusted exit points. My current exit point is at TransIP’s DCG. It’s network is great but I’ve got to behave. I can’t use IRC nor P2P protocols. It’s not that much of a problem, since my home DSL provider still is relatively trustworthy. I’m looking for cheap places to colocate some cheap hardware to widen this overnet’s reach and possibilities. Some more (Debian) people have shown interest in BitzKrieg.net, which makes this whole endeavor maybe even financially feasible.

The setup of BitzKrieg’s overnet is quite simple, actually. With Debian’s outstanding OpenVPN integration secured internet connections are easy as can be. It uses the 172.16/12 address space and Linux routing to create something resembling a network. Fixed IP- and key sets are created for every single client, ranging from laptops to entire 10/8 and 192.168/16 subnets. 172.27.0/24 is reserved for the VPN itself. Members of this network have to know the CA, yours truly, personally in order to be able to connect.