Geeks in the West Country

Alt.NET Bristol Beers #1 – Tuesday 21st July 2009

June 29, 2009 · 1 Comment

What: Alt.NET Beers!

When: Tuesday 21st July 2009

Where: Upstairs at The Portcullis, Clifton, Bristol, BS8 4LE

If spending an evening drinking beer and talking Alt.NET is your thing, then come along to the inaugural Alt.NET Bristol beers at the excellent Portcullis tavern in Clifton! Taking inspiration from the Alt.NET London beers, the format will be a mini openconference style session where everybody will get an opportunity to suggest and vote for the topic to be discussed.

If you’ve never attended a session like this before they are a lot of fun and a great way to share knowledge and experience!

The plan for the evening is:

18:00  – 18:45: Socialising and proposal of topics for discussion. Topics can be proposed by placing post-it notes on a topic wall. Before voting we will cluster similar topics together where it appears to make sense.

18:45 – 19:00: Vote for the topic of the day through a show of hands in two rounds.

19:00 – 20: 00 : 60-minute discussion around of the topic of the day. At the end of the discussion we will ask participants to fill in a wiki stating what they hoped to get out of the discussion and what they actually got out.

20:00 – late: Socialising, beers and debate!

The Portcullis has a great selection of beer and cheap food for those of us that require feeding. All of us at Bluewire look forward to seeing you there!

→ 1 CommentCategories: Alt.NET · Events
Tagged: , , ,

Got an invalid XDocument but can’t work out why?

June 15, 2009 · Leave a Comment

This could be the reason…

I uncovered what I suspect may be a .NET Framework bug the other day, when I attempted to validate an XDocument against an XML schema which contained abstract datatypes.  It seems that XDocument.Validate() is unable to resolve derived types properly – it claimed that the concrete type of an XML element was not valid according to its abstract type in the schema.

Only after a lot of checking and self-reassurance that my XDocument really should be valid, and after dragging two colleagues through the problem and having them agree that it should be working too, did I begin to suspect something may be more fundamentally wrong.

I found that Visual Studio’s inbuilt XML document validation did think the document was valid. I then found that if the XDocument were saved out to a string, and validated using another method, there was no validation error.

I explain the problem in depth here. There are some code snippets and a sample Visual Studio project of a simple test case which sandboxes the problem effectively.

→ Leave a CommentCategories: Microsoft
Tagged: , , ,

WPF designer gotcha when setting TextDecorations in triggers

April 16, 2008 · 2 Comments

I just came accross a nasty gotcha with the Visual Studio 2008 WPF designer (Cider). When setting the TextDecorations property for a control using a trigger, the following code will work at runtime, but the designer will complain with “Must specify both Property and Value for Setter. Error at object ‘System.Windows.Setter’, Line 1 Position 737″.

<ControlTemplate.Triggers>
<Trigger Property="IsMouseOver" Value="True">
<Setter TargetName="contentSite" Property="TextDecorations" Value="Underline"/>
</Trigger>
</ControlTemplate.Triggers>

To prevent the designer complaining, you have to use the full syntax for setting TextDecorations, as below:

<ControlTemplate.Triggers>
<Trigger Property="IsMouseOver" Value="True">
<Setter TargetName="contentSite" Property="TextDecorations">
<Setter.Value>
<TextDecorationCollection>
<TextDecoration Location="Underline" />
</TextDecorationCollection>
</Setter.Value>
</Setter>
</Trigger>
</ControlTemplate.Triggers>

This seems to keep the designer happy and behaves the same at runtime as the first example – looks like there’s a bug in the converter for TextDecorations somewhere that’s causing this…

→ 2 CommentsCategories: Microsoft
Tagged: , , , ,

JSUnit MSBuild task

January 30, 2008 · Leave a Comment

This is just a quick post to describe how to use the JSUnit MSBuild task (which you can download here). JSUnit currently ships with JUnit fixtures and ant wrappers to call these – this is essentially an MSBuild task wrapper around the StandaloneTest JUnit fixture (and so requires Java as well). Also there are a few things to bear in mind if you use this in a continuous integration scenario, namely that the process will need to fire up your browser of choice and so will need to interact with the desktop, and that with the current state of JSUnit it is best to access the test runner / pages over http (instead of using file://). Don’t forget you can’t mix and match file: and http: otherwise you’ll get x-site scripting errors in certain browsers.

Anyhoo, to use the task, first include it in your build file like so:

<import project="$(MSBuildJSUnitTasksPath)\MSBuild.JSUnit.Tasks.Targets"></import>

I’ve previously defined the MSBuildJSUnitTasksPath property to point to the directory where the MSBuild.JSUnit.Tasks.Targets and MSBuild.JSUnit.Tasks.dll are located. Having imported the targets you can then call the task using:

<StandaloneTest
JSUnitDir="$(JSUnitDir)"
TestRunnerLocation="http://localhost/JsUnit/TestRunner.html"
TestsPageLocation="http://localhost/TestPages/Foo.html"
LogsDirectory="$(JSUnitLogDir)"
Port="3333"
BrowserFileNames="c:\program files\Internet Explorer\iexplore.exe"
ContinueOnError="true">
<Output TaskParameter="TestsFailed" PropertyName="JSUnit_Failed" />

<Output TaskParameter="TestsPassed" PropertyName="JSUnit_Passed" />
<Output TaskParameter="TestsRun" PropertyName="JSUnit_Run" />
</StandaloneTest>

A few things are going on here. This task will attempt to call the JUnit test provided by JSUnit. This fires up a results server on port 3333 and then loads IE with the specified test runner / test page causing the results to be submitted to the results server and an xml report generated. This report will be placed in the location specified by LogsDirectory along with an HTML report generated by the MSBuild task. The output parameters capture the test statistics if you need to use them later (e.g. for TeamCity reporting :-) ) but remember these only work if the tests succeed (hence the ContinueOnError=”true”). You can then cause the build to fail by looking at the output parameter if you need to.

→ Leave a CommentCategories: JSUnit · Microsoft · msbuild
Tagged:

TeamCity MSBuild tasks

January 24, 2008 · 8 Comments

For our latest product, we’ve switched over from CC.NET to the excellent TeamCity continuous integration server from JetBrains, which, as of version 3, is free for small projects like ours. One of the nice aspects of TC is that it’s very easy to publish data from your build (such as number of unit test passes / failures) so that it gets picked up by the TC server.

To help us along, we’ve written a few simple MSBuild tasks which do just that. We’ve created a sourceforge project for them, but until that’s activated, you can download them here.

There are currently 3 tasks:

MSBuild.TeamCity.Tasks.TeamCityAppendStatusText

Appends text to be displayed in the build status when the build is complete – e.g. “Tests Passed: 3, Tests Failed: 1″

Example usage:

<TeamCityAppendStatusText TeamCityInfoPath="$(teamcity_build_checkoutDir)\teamcity-info.xml" Value="Tests Passed: 3" />
<TeamCityAppendStatusText
TeamCityInfoPath="$(teamcity_build_checkoutDir)\teamcity-info.xml" Value="Tests Failed: 1" />

MSBuild.TeamCity.Tasks.TeamCityAddStatistic

Adds a build statistic that can be picked up for graphing by configuring TeamCity’s main-config.xml (see here).

Example usage:

<TeamCityAddStatistic TeamCityInfoPath="$(teamcity_build_checkoutDir)\teamcity-info.xml" Key="TestsPassed" Value="3" />
<TeamCityAddStatistic
TeamCityInfoPath="$(teamcity_build_checkoutDir)\teamcity-info.xml" Key="TestsFailed" Value="1" />

MSBuild.TeamCity.Tasks.TeamCityAddStatisticList

Variation on the above which allows you to add multiple statistics at once. Each key/ value pair is delimited by ‘;’

Example usage:

<TeamCityAddStatisticList TeamCityInfoPath="$(teamcity_build_checkoutDir)\teamcity-info.xml" KeyValuePairs="TestsPassed=3;TestsFailed=1" />

Thats pretty much it for now as  that’s all we’re using. I’ll post a follow up shortly to show how you can use these to pull in results from MbUnit tests.

If you missed the link above, you can download the tasks here.

→ 8 CommentsCategories: Microsoft
Tagged: , ,

Reverse-proxying with SSL

November 15, 2007 · Leave a Comment

Apache does name-based virtual hosting, and it does proxying, and it does both quite nicely. Problem is, it won’t mix the former with SSL. This, to my mind, is a major failing. We know why it doesn’t do it: SSL is defined per virtual host, but the server must know to which host the client is talking before it can unwrap the SSL-wrapped channel to find out which host… yadda, yadda, open box with crowbar inside.

However, with a wildcard SSL certificate matching all virtual hosts on the server, it is perfectly possible to unwrap the channel before directing the traffic to a specific virtual host. This is an edge case, but a very valuable edge case which MS ISA Server supports just fine.

In short, it would be nice to be able to specify a server-wide SSL wildcard certificate that applied to all SSL virtual hosts and did not prevent name-based virtual hosting. I understand that this may be difficult with Apache’s current architecture (vhost dispatch handled early in the pipeline) but if time allows I may experiment with the source code.

A dedicated HTTP/S reverse-proxying gateway application could be very useful.

→ Leave a CommentCategories: Uncategorized

SVN and Active Directory

November 13, 2007 · 3 Comments

Our old SVN/Trac system is getting a shakeup. We’re constructing a new wiki project for the next version of our product and this seemed like the ideal time to rearrange the furniture. Trac’s getting moved to an actual database (backing up an SQLite DB centrally was getting annoying) and SVN’s finally getting its own server. A properly-configured reverse proxy acting as gateway to all the services should eliminate differences between internal and external URLs for a given service. The whole shebang’s being moved to Ubuntu Linux 7.04 too, since NFSv4 is much nicer for running shared repos and Trac project dirs than Windows Networking, and although we’re not running redundant/load-balanced Trac servers yet it’s probably a good idea to think ahead.

Of course, the reason we used Windows Server originally was that we needed Active Directory integration for Trac and SVN, and the Apache SSPI module is not available on Linux. Group-based, per-directory access control is awkward even with that setup, but there’s a solution: using mod_authz_svn for access control and running a script to synchronise the module’s user file with Active Directory via LDAP. I wasn’t able to find such a script, but hacking one together didn’t take long:


#!/usr/bin/perl
use warnings;
use strict;
use Net::LDAP;
sub resolve_group($);
sub get_member_users($);
sub get_users_in_group($);
# Domain name.
my $ldap_domain = "<your domain name here>";
# DN of the LDAP directory root.
my $ldap_root = "dc=".$ldap_domain;
$ldap_root =~ s/\./,dc=/g;
# Domain Controllers.
my @dcs = ( 'dc1', 'dc2' );
# DN of the LDAP directory base for our queries.
my $ldap_base = "<your base DN here>,$ldap_root";
# Authorisation of guest account used for queries.
my $ldap_username = "<guest AD user DN here>";
my $ldap_password = "<guest AD user's password here>";
#====================================
# Connect and bind to LDAP server.
my $ldap;
foreach (@dcs) {
$ldap = Net::LDAP->new("$_.$ldap_domain") and last;
}
$ldap or die "$@";
$ldap->bind($ldap_username, password => $ldap_password) or die "$@";
# Gets a group by cn.
sub resolve_group($)
{
my($name) = @_;
my $mesg = $ldap->search(
base => $ldap_base,
filter => "(&(cn=$name)(objectClass=group))",
sizelimit => 1
) or die "$@";
return $mesg->entry(0);
}
# Gets sAMAccountName attribute from all member users of a group, descending
# recursively into member groups.
sub get_member_users($)
{
my($group) = @_;
my @users = ();
my $members = $group->get_value('member', asref=>1);
foreach (@{$members}) {
my($mesg, $member, @objectClass);
$mesg = $ldap->search(
base => $_,
filter => "(objectClass=*)",
attrs => [ 'member', 'dn', 'sAMAccountName', 'objectClass' ]
);
$member = $mesg->entry(0);
@objectClass = @{$member->get_value('objectClass', asref=>1)};
if ( grep $_ eq "group", @objectClass ) {
push @users, get_member_users($member);
}
if ( grep $_ eq "user", @objectClass ) {
push @users, @{$member->get_value('sAMAccountName', asref=>1)};
}
}
return @users;
}
sub get_users_in_group($)
{
my($groupName) = @_;
my $group = resolve_group($groupName);
return keys %{{ map { lc $_ => 1 } get_member_users($group) }};
}
my($section, @usernames);
while (readline STDIN) {
if($_ =~ /\[(.*?)\]/) {
$section = $1;
} elsif($section eq "groups" and $_ =~ /^(.*?)\s*=/) {
@usernames = get_users_in_group($1);
my $list = join ', ', @usernames;
$_ = "$1 = $list\n";
}
print $_;
}
$ldap->unbind;

(Yes, Perl’s even nastier when the blog software strips out the indentation and blank lines…)

Download update-groups.pl.

Active Directory will not allow an LDAP client to operate against it anonymously, therefore you will have to provide a user DN and a password in plaintext in the script. Fortunately, the permissions required for this script to operate are minimal. Simply create a new user in the Directory, add it to Domain Guests, set Domain Guests as primary group, and remove from Domain Users. Assuming you haven’t granted extra privileges to Domain Guests, the script will get no more than it needs to operate.

Fill in the fields appropriately for your Active Directory and write a mod_authz_svn user file, eg.:

[groups]
Administrators =
[/]
* = r
@Administrators = rw

When run against this, the Perl script will look up every group name specified in [groups], extract all the member user names from the Active Directory, and write out the new group definition. The script takes input on STDIN and generates output on STDOUT, so run something like the following as a cron job (adjust paths as necessary):

#!/bin/bash
cd /svn
cat security.conf | perl update-groups.pl > security.conf.new && mv security.conf.new security.conf

This will only replace the user file if the new one is generated without errors.

Of course, to authenticate HTTP access to the repo you will have to configure Apache to use LDAP appropriately too. Something like the following works:

<Location />
AuthLDAPURL "ldap://<your domain controllers here>/<full DN for LDAP base here>?sAMAccountName?sub?(objectClass=user)"
AuthLDAPBindDN "<guest AD user's DN here>"
AuthLDAPBindPassword <guest AD user's password here>"
AuthBasicProvider ldap
AuthUserFile /dev/null
AuthType Basic
AuthzLDAPAuthoritative off
AuthName "Repository"
Require valid-user
DAV svn
SVNParentPath /svn
AuthzSVNAccessFile /svn/security.conf
</Location>

As before, fix up paths and values as necessary. Something similar can be used for controlling access to Trac. Don’t forget to add the mod_authz_svn, mod_authnz_ldap and mod_ldap modules to your Apache configuration, depending on Linux distro and version.

→ 3 CommentsCategories: Uncategorized

IIS6 doesn’t like NewSID

August 10, 2007 · 1 Comment

Our PowerShell-based VM building framework is still in use. This is because some of the components for the new system were sent First Class instead of Next Day, and got lost in the mail. In future I’ll be sure to demand Next Day delivery on everything.

So our test server still operates (or not) at the whim of a mess of scripts. This seems to be working out OK so far, although human intervention is required when the VM registration process fails to restart properly, ie. roughly once a week. At least the terminal server I set up lets me fix it from home before I leave for work in the morning, which means the server might actually be built and working by the time I arrive.

The original test server runs Windows XP, which is pretty much the minimum requirement for our product. Recently we ran into a bug that only appeared on Windows 2003, requiring a second test server running that OS. As I mentioned back in May, we use NewSID plus yet more Powershell to deploy images since it suits our system better than Sysprep.

NewSID is a great tool. It generates a new Security ID for the machine, updates the Registry and other system files accordingly, fixes the filesystem’s Access Control Lists to use the new SID, and reboots the machine. Unfortunately, it doesn’t update the IIS Metabase.

On Windows XP this is not an issue for us. IIS versions prior to 6 use only a single application pool and our product’s installer sets up all permissions appropriately anyway. On II6, however, our app pool identity is a member of IIS_WPG, the IIS Worker Process Group, and relies on that group’s permissions. NewSID leaves the old SID in the Metabase and doesn’t add the new one, effectively nuking IIS_WPG’s permissions on all IIS directories.

IIS7 doesn’t have this problem. It uses built-in NT accounts not dependent upon the machine or domain SID, making a clean IIS7 Metabase effectively immune to this sort of corruption. Once you start adding permission sets it’s a different story, but in most cases only a clean system would be the subject of NewSID.

The conventional workaround for the issue is to remove IIS prior to running NewSID and reinstall it afterwards. This is fine for manual setup but falls short in our situation. It is by all means possible to automate the addition and removal of Windows components with the ’sysocmgr’ command, but setting up the necessary answer files is tedious. I wanted a better way.

So I poked around in Powershell, made heavy use of Google, and wrote some exploratory C#. Through Directory Services the Metabase can be edited just like Active Directory. I wrote some code to look through all the Access Control Lists for entries with trustees in a given domain and replace the trustee with its equivalent in another domain. This was all done at the SID level; ‘equivalent’ trustees are considered to be those with the same RID, since NewSID does not change that. Trustees that mapped to known accounts are ignored, since they aren’t broken.

It took me a while to figure out why the ‘patched’ entries were being removed by the original version of the tool. It seems that the AccessControlEntry interface doesn’t like raw SIDs being put into its Trustee property. Resolving the SID to a human-readable account beforehand solved this problem.

The result is a tool called MetabaseACL, which takes a directory path as its first argument:

MetabaseACL "IIS://Localhost"

This will display all ACLs in the Metabase within the given path. To fix SIDs, it can be given the ‘fixup’ command:

MetabaseACL "IIS://Localhost" fixup <old SID> <new SID>

It doesn’t matter if the SIDs are specific NT accounts or domain/machine SIDs; only the domain info will be used.

This integrates nicely with the ‘newsid’ package our VM builder uses. The script which does the work now looks something like this:

if($env:COMPUTERNAME -eq $compName)
{
if(Test-Path "C:\Machine.sid")
{
$id = New-Object System.Security.Principal.NTAccount("Administrator");
$newSid = $id.Translate([System.Security.Principal.SecurityIdentifier]).AccountDomainSid.Value;
$oldSid = Get-Content "C:\Machine.sid";
MetabaseACL "IIS://Localhost" fixup $oldSid $newSid;
Remove-Item "C:\Machine.sid";
}
[We're done here. Call next script in the chain.]
}
else
{
net stop iisadmin /y ;
if(-not (Test-Path HKCU:/SOFTWARE/Sysinternals))
{
New-Item HKCU:/SOFTWARE/Sysinternals
}
# Record old Machine SID.
$id = New-Object System.Security.Principal.NTAccount("Administrator");
$sid = $id.Translate([System.Security.Principal.SecurityIdentifier]).AccountDomainSid.Value;
Set-Content "C:\Machine.sid" $sid;
# Prepare for NewSID.
$key = New-Item HKCU:/SOFTWARE/Sysinternals/NewSID;
$key.SetValue("EulaAccepted", 1);
$p = [diagnostics.process]::Start("newsid.exe", "/a ${compName}");
$p.WaitForExit();
}

This script is run when the VM image boots. First time through it will record its SID and run NewSID. On the second boot it will retrieve the old SID, get the new SID, run MetabaseACL to do the fixup, and call the next script in the image building procedure.

And so ends another adventure with VM image deployment. Judging by the number of forum topics I came across while googling for a solution to the IIS/NewSID problem, this new tool will prove useful to people besides me.

Download MetabaseACL

Zip file includes source code and binaries. Runs on WinXP and Win2003. I haven’t tried it on Vista but it should be fine.

→ 1 CommentCategories: PowerShell

VirtualCloud bears fruit

July 31, 2007 · Leave a Comment

Our testing system is finally coming together. Once the last few parts for the server cluster arrive, integration testing can begin. Meanwhile, one of its components is finding uses outside of its parent project.

Managed.VIM, our C# proxy for the VIM webservice, is available for download from the VirtualCloud Sourceforge site. It is currently in beta (revision 105) and probably not ready for use in a production environment. It doesn’t yet implement all the functionality of VIM either; only the bits we need for VirtualCloud have been written so far.

It’s a nice tool for PowerShell monitoring of VirtualCenter, though. Once the Managed.VIM assembly has been loaded into PowerShell, the proxy object can be created as follows:

$vim = New-Object Managed.VIM.Vim(”https://localhost:8443″, “vc_username”, “vc_password”);

Replace the URL, username and password with values appropriate to your VirtualCenter configuration.

The Vim object exposes collections of VMs, Hosts, Farms, Datastores and Templates. This is the root of the VirtualCenter object hierarchy. Nodes in the hierarchy expose children by type, in the same way as the root. For instance, a Farm can contain Hosts, VM groups and VMs; these are accessible via the Hosts, Groups and VirtualMachines indexed properties respectively, and can be retrieved by name.

The root of the hierarchy also exposes collections of all VMs and Hosts managed by the VirtualCenter service. Hosts are accessible by name, as they are within Farms, but the collection of all VMs is indexed by UUID. This is because you can have multiple VMs with the same name as long as they are in different groups.

The following will get the object corresponding to the host ‘testserver1′ in the server farm ‘TestFarm’:

$testServer = $vim.Farms["TestFarm"].Hosts["testserver1"];

Operations supported by VirtualMachine objects currently include:

  • Start, stop and suspend,
  • Change persistence mode of hard disks (VirtualCenter does not permit this, but VIM does),
  • Get or set device or ISO image used for CD drives,
  • Get CPU count and memory allocated to the VM,
  • Get Host.

Operations supported by Host objects include:

  • Get CPU and memory info,
  • Get accessible datastores,
  • Get hosted VMs,
  • Get Farm.

Operations supported by Template objects include:

  • Get size, memory allocation, datastore and guest OS,
  • Deploy template to Host, with optional customisation of Windows guests.

Customisation of guest operating systems has not actually been tested yet and, like most of this assembly, is not documented yet either. It involves the Managed.VIM.Customisation.WindowsCustomisation class, which exposes some fairly self-explanatory properties and methods. Use at your own risk.

The leaf nodes of the hierarchy (Hosts, VirtualMachines, etc) are heavyweight objects. They are kept synchronised with the VirtualCenter service and therefore generate a certain amount of network traffic. For this reason, each collection also exposes a list of the names of all objects in it, like the Keys property of a dictionary. For example:

$hosts = $vim.Farms["TestFarm"].Hosts.Names;

This is to be preferred when you don’t actually need any of the properties or functions on a Host or VirtualMachine, but merely need to determine which ones are available or where they are.

Datastores are a bit of a sore point at present. Multiple datastores can technically have the same name, although I think this only applies to the default datastore on a host, called ‘local’. Working on this assumption I’ve allowed for multiple ‘local’ datastores, accessed via $vim.Datastores.Local, which is indexed by Host. All other stores are accessed by name (through $vim.Datastores) and must therefore have unique names. Otherwise Managed.VIM’s internals don’t work correctly and will attempt to log lots of errors through log4net. They’ll do this on pretty much every update request too, which will eat lots of CPU and effectively kill the update mechanism. I’m not going to try to fix this until I know it’s a problem, because the fix breaks some other things.

Custom properties on Hosts and VirtualMachines are supported by Managed.VIM. Getting and setting values is possible, but adding new properties requires the VirtualCenter client. This is apparently a limitation of the VIM SDK. This was a bit worrying initially since VirtualCloud will require several custom properties. Fortunately, VirtualCenter doesn’t appear to mind having new properties poked directly into its database, so the VirtualCloud installer (when written) will probably want to know where this database is in order to update it. I’d very much like to know if this is considered acceptable, or whether most VirtualCenter admins want other apps to consider that database inviolate; the alternative is asking the admin to add about twelve properties to VirtualCenter by hand…

By the way, Console is made of pure win. Tabbed PowerShell is extremely useful.

→ Leave a CommentCategories: PowerShell · VMware · VirtualCloud · Virtualisation

Small solution to an Enterprise problem

June 28, 2007 · 1 Comment

Given a set of VMs representing possible client configurations, a set of Selenium test scripts, and a collection of multiprocessor servers, how can the process of running all tests against all VMs be automated and scaled to available processing capacity?

On-demand tests against a developer’s copy of the product would also be useful, as would a means of testing computers for compatibility at the client’s site.

Back in May I wrote about the script-based VM management system I put together in preparation for automated UI testing. While sufficient for building our test server and managing our production builds (which it has been doing very well for over a month now) it’s become apparent that maintaining the scripts necessary to run a testing farm is going to quickly become impossible. Powershell’s great for small stuff, but something this big needs some serious software behind it.

There are existing solutions out there which do what we need, eg. Surgient’s Virtual Lab Management Applications, or VMware’s own Lab Manager, but they tend to be aimed primarily at the larger businesses that have either the infrastructure or budget (usually both) for Enterprise-class solutions. We don’t have the space or staff to run a datacenter, and a managed solution doing the amount of processing we need does not come cheap.

So the plans were laid for building a smaller scale solution to the problem of automating tests across multiple VMs. We stuck with VMware, because our existing infrastructure is based upon their free Server product and the competing products weren’t competitive enough for converting to be worthwhile. The target machines for this system were to be three multi-CPU rack servers, which will later also take on the burdens of the three glorified workstations that currently occupy the server cupboard.

We looked for a way to abstract the VM capacity of the physical servers and treat them as a computing cloud. The original plan was to write some daemons in .NET that would use VMware’s VIX API to talk to the VMware Server instances, but VIX functions often take callbacks and reverse P/Invoke didn’t want to play nicely. While it might’ve been possible to write the daemons in C/C++ instead, it was far cheaper just to fork out for VirtualCenter 1.4 and learn to use the Virtual Infrastructure Management (VIM) SDK from C#. This basically reduced our complex distributed solution to a single service that could use VirtualCenter to do its dirty work.

After a little research it was discovered that switching to ESX Server would cost too much after factoring in VirtualCenter 2 and the necessary hardware (SANs do not come cheap), although it would have let us use version 2 of the VIM SDK, which has proper C# bindings. It was decided that our computing cloud would run a lightweight Linux distro to minimise the overhead of the native OS on our cloud’s nodes. Gentoo was suggested, but I wasn’t inclined to spend eight hours installing it when Ubuntu Server is so much quicker and simpler. We have an evaluation copy of ESX somewhere; at some point I should probably determine exactly how much more efficient a hypervisor would make our computing cloud.

My initial reaction to the version 1.4 VIM SDK was not entirely favourable. I wasn’t happy about the use of HTTP webservices for everything, since this means that the client has to pull updates down from the server via a message loop and there was no C# proxy API provided. Implementing that message loop gave me a headache to begin with; it’s uncharted territory for a SOAP newbie like me :P . But after a little hard work and some background reading I began to understand the protocol and the sense behind it, and now we have a nice extensible C# proxy for it which handles object synchronisation entirely in the background.

(I’m still confused about the use of XML diffs. Surely if minimisation of network traffic is a priority, XML isn’t the best format to use anyway?)

Once we’d determined which parts of our original design worked and what could replace the bits that didn’t, a new design was drawn up. It mutated slightly over the following week or so, but has stayed broadly the same.

VirtualTest design

  • The TestBuilder generates ISO images containing the necessary programs and data to run the tests. These ISOs will autoplay if attached to an active VM. VIM doesn’t seem to support attaching devices while a VM is running, however, so each VM includes a service that autoplays an attached ISO at system start time.
  • The Resource Manager handles scheduling of jobs. It knows nothing about these jobs except the resources they consume. The Resource Controller (the Managed.VIM namespace, used for talking to VirtualCenter) eventually got split out of this module and became a separate entity.
  • VirtualTest runs as a service and links everything together.

The use of an ISO to inject tasks into a VM is rather similar to the ‘parameter disk’ concept used in our Powershell-based system. It’s also the technique VMware use to install their Tools package on a VM, which incidentally is where I got the idea in the first place…

The design above is actually split into two parts. The testing system consists of the dashboard, the VM, and the contents of the ISO. VirtualTest (the green part) is really just a job control system dealing with VM processing resource; TestBuilder would be better named IsoBuilder, and we’ll have to rename VirtualTest too. Strangely, the potential flexibility of this system only really became clear once the design had been adjusted to make it fully testable. Yet another victory for TDD, I feel.

This split also made dividing the workload simple. I worked on the VirtualTest system, and Ben has been developing the testing framework using Ruby and Selenium. Integrating the two systems consists of implementing a Job object that will build and deploy the appropriate ISO to the VM; apart from that, neither system need know anything about the other, since test results are uploaded to the dashboard by the testing software running from the ISO. We also get for free a way to do on-site testing of client configurations, because the ISOs can be burned to CD and run on any machine.

At this point in time, VirtualTest is not quite ready for deployment. The CLI tools do not yet work across TCP (I may well solve this by dropping Remoting and just using WCF instead) and all they do is add a simple demo job to the queue. The ISO builder for tests has not yet been implemented. Support for on-demand tests is on hold until we actually have something working.

The core of the VirtualTest system demonstrably works, however, and will soon take over from the Powershell scripts that currently schedule our nightly builds. It’s almost complete enough for our needs, but the following improvements could also be made:

  • Fix resource management so resources can be assigned properly. Resources requested by a job and resources viewed as allocated to that job should be the same thing, but presently the resource manager isn’t smart enough to remember how much is allocated, and instead just looks at how much is being used.
  • Fix the whole thing so multiple resource types can be managed. The only resource it understands is ‘VM capacity’. It’d be nice to be able to track registration key usage, so we could make optimal use of Windows product keys and ensure that only one active VM is using a given key at a time; currently every VM has to have a separate key whether it’s running or not, which is somewhat wasteful.
  • It’d also be nice to allocate CPU and memory intelligently instead of abstracting these as ‘VM capacity’, but that’s probably out of our reach.
  • Fix the whole thing so jobs can be scheduled dependent on multiple resource types. This ties in with the registration key thing.
  • Make the Managed.VIM proxy safe for multithreaded use without requiring lots of locking in client code.
  • Ensure that Managed.VIM works seamlessly with VIM version 2, so migration to ESX Server is easy.
  • Guarantee that all Job objects are serialisable, allowing them to be instantiated by a client app and sent across the wire to the VirtualTest service. At present, the Remoting interface demands a Type and an array of constructor arguments, which is horrible.
  • Improve unit test coverage. The Managed.VIM assembly is mostly untested. We need integration tests for this as well.

I’m going to move the codebase over to Sourceforge as soon as possible, at which time I’ll blog again. The ‘testing’ component of this system is Ben’s domain, so I’ll let him explain it.

UPDATE: We now have a Sourceforge project, called VirtualCloud. The codebase has been migrated and the above improvements added to the Feature Request tracker.

→ 1 CommentCategories: Testing · VMware · VirtualCloud · Virtualisation