Turns out that when you are interfacing something to the kernel, it is a REALLY bad idea to log to stderr, ESPECIALLY if it is exceedingly rare, and only in your debug logging code. Grrr.

Just got back from the future shop (evil). While I generally loath them, I have a hard time getting angry at a $99 500G hard drive. They have literally thousands of these things lining the entire store. I cannot imagine how they even manufacture these so cheap.

It used to be (in the old days) that after moving out, the worst a nerdy young sprog had to worry about was . the occasional computer maintenance request from their parents. Times however, are changing. Nowadays parental units are far more likely to have a need for personal webmail, chat, picture management, and even a CMS.

More after the break...

So, the rant server is slowly getting moved over to my new Xen/Enomalism based, s00per chr00ted nginx/fastcgi installation. Everything is going rather well. Safe mode didn't break anything (so far), and smf forum was a breeze to set up again.

One neat thing I started working on was wildcard DNS based subdomains. Rant has a LOAD of different domains, all living in subdomains of the main vhost. The old method was a bunch of different vhost definitions, but since I didn't feel like re-entering them in the new nginx method, I figured I would come up with a better way.

More after the break...

Just a note to self. In the case of a serial console never getting around to the login prompt on a Xen DomU, even though it still displays all of the normal console boot messages, just change /etc/inittab as follows:

from:

1:2345:respawn:/sbin/mingetty tty1

to:

1:2345:respawn:/sbin/mingetty --noclear console

Also note: Why do we still call these things tty(s)? That is SOOO 70s!

Oh Boy! I can hardly wait! A well defined security policy document that will doubtlessly be implemented just as flawlessly as the previous html and css standards.

</sarcasm>

Seriously, I am not giving a hard time to the poor folks at W3C, as much as to the implementations. The W3C standards are VERY complicated, but the implementation from the larger vendors (Micro$oft, I am looking at you) were lackluster for a LONG time. I don't expect any better for the security policy framework, but with potentially much more serious ramifications

W3C Policy Document

So... I just finished writing a cool new application, called ElasticDrive. ElasticDrive is a pluggable virtual storage daemon which can store your files in a virtual block device, which is backed on RAM, Flat Files (like loopback), or as a virtual drive on S3!
(in case you missed it, the S3 part is the coolest).
The reason we wanted to build this was primarily for creating software raid to provide redundancy for the instability inherent in Amazon EC2 instances. We use a lot of EC2 machines, but the only drawback is that when they shut down, they are gone forever.
The interesting thing is that there are all kinds of other uses for this block device. Turns out that people can thing of all kinds of neat ways to store data, and reasons why!
Another cool thing is that the storage engines are pluggable, so adding new storage methods is as easy as writing a well define object that can provide the following interfaces:

• poll(): return pending results
• write(ofs,data,handle): Write data at offset bytes ofs, with handle
• read(ofs,length,handle): duh!
• busy(): Well, are you busy, or can you take another request
• flush(): Flush all pending data to prep for shutdown.

The phantastic Interface ui extensions for jQuery are easy enough to integrate into TurboGears with just a simple static js link, but I though it would be cool to have a widget that supplied the same functionality, so that as new releases of interface came out, all of your TG apps would upgrade simultaneously.

So without further ado:

• tg_interface-1.2_0.01-py2.4.egg
• tg_interface-0.01.tgz

The version numbering is a little confusing. The first digits are the interface release, and the second set are the version of the TG widget. The source release WILL build the 1.2 version, but you can replace the tg_interface/static/interface.js with whatever version you want (I ship the complete library).

NOTE: I cribbed some of the code from the jquery TG widget (for speed of implementation). Sorry if any superfluous code is left over ;)

I think this post is a little on the late side, since I have a feeling that libvirt already won, but I just spent two days learning libvirt, as opposed to the weeks I spent trying to get XenAPI to act in a predictable manner. So here are some thoughts...

Libvirt is a superset of XenAPI

Yep. Libvirt can do everything (important) that XenAPI can do. The funny thing is that it is effectively using XenAPI/XenD to do it, since libvirt just connects to the xend socket to get work done. It is just that libvirt does it in a predictable, and well documented way.

Libvirt is more stable than XenAPI

Libvirt has a well defined and stable implementation. In fact, libvirt's python bindings return useful documentation, whereas the XenAPI being an abstraction of an XMLRPC interface, has no easily usable docs (yes I know there is a PDF file, but compare the methods to the actual XMLRPC stuff, and it is not always obvious how they work). Worse, XenAPI is guaranteed to break in the near future, so you are building on shifting sands.

Libvirt supports more than one virtualization platform

I develop against Xen on a lot of different boxen, and interoperability is one of my worst problems. PV vs HVM is one issue that particularly bothers me. It is impossible for me to test hardware virtualization on my older (but still hella fast) Xeon servers. Libvirt with it's qemu support (which uses the same method calls exactly as the Xen PV and HVM support) gives me the ability to seamlessly use my VM images across different hardware. Better, since there is (as yet untested by me) KVM support, I can develop against whatever platform eventually "wins".

Libvirt is used in production

... by Redhat no less (among a lot of others). That equals a LOT of testing, and smoothing out the messy stuff. XenAPI is basically beta, and nobody is using it. I am sure that XenAPI will mature a lot over the next few months. The Xensource team are VERY smart folks, but libvirt has a large head start, and will probably mature just as much in the same time. Closing the gap will not happen until there isn't much left to fix in libvirt, and their progress slows.

Libvirt is easy to use

So, let's pretend we want to get data about the VM. I already showed how in XenAPI here: XenAPI example... Here are the actual transactions I used to learn the first few things in libvirt that I wanted to do...

import libvirt
conn=libvirt.open('xen:///')
dir(conn)
['__del__', '__doc__', '__init__', '__module__', '_o', 'createLinux', 'createXML', 'defineXML', 
 'getCapabilities', 'getHostname', 'getInfo', 'getMaxVcpus', 'getType', 'getURI', 'listDefinedDomains', 
 'listDefinedNetworks', 'listDomainsID', 'listNetworks', 'lookupByID', 'lookupByName', 'lookupByUUID', 
 'lookupByUUIDString', 'networkDefineXML', 'networkLookupByName', 'networkLookupByUUIDString', 
 'numOfDefinedDomains', 'numOfDefinedNetworks', 'numOfDomains', 'numOfNetworks', 'restore', 'virConnGetLastError', 
 'virConnResetLastError']
#Wow... That was reasonably self explanatory...
foo=conn.lookupByUUIDString('aa753a14-5116-6734-8f0e-475e65db0206') #Yes, I knew the UUID already.
dir(foo)
['ID', 'OSType', 'UUID', 'UUIDString', 'XMLDesc', '__del__', '__doc__', '__init__', '__module__', 
 '_conn', '_o', 'attachDevice', 'autostart', 'connect', 'coreDump', 'create', 'destroy', 'detachDevice', 
 'info', 'maxMemory', 'maxVcpus', 'name', 'pinVcpu', 'reboot', 'resume', 'save', 'setAutostart', 
 'setMaxMemory', 'setMemory', 'setVcpus', 'shutdown', 'suspend', 'undefine']
#Wow. Nice and clean set of obvious API calls. Kind of like feedparser, or boto, or any other well written interface.
#So... how does this work?
help(foo.attachDevice)
 Help on method attachDevice in module libvirt:

 attachDevice(self, xml) method of libvirt.virDomain instance
     Create a virtual device attachment to backend.
#Neat. How about some commands...
foo.info()
 [2, 131072L, 131072L, 1, 3679113315L]
foo.suspend()
 0
foo.info()
 [3, 131072L, 131072L, 1, 3682550185L]
foo.resume()
 0
foo.info()
 [2, 131072L, 131072L, 1, 3682727196L]
#Well that was surprisingly easy...

for day in life:
    pass