Wednesday, August 13, 2008 4:43:43 PM (GMT Standard Time, UTC+00:00)

After getting a few silly, random things exorcised from my system, I sat down last night and set to work on the Scrobbles client and libraries, as these have been neglected while I have been working on the core server-side stuff.

Last time I left them, I was dealing with some problems to do with the situation where the user upgrades the Scrobbles client, and that client uses a newer database schema for the local cache, or the web services change so data can no longer be submitted the old way. This is quite a rare occurrence, but it just so happens that in my latest overhaul a couple of months ago I completely changed the web services and now have 500mb of unuploaded data sat on my hard drive needing migrating one way or another. And if I'm going to do it nice, I may as well write a system that can cope if I have to change things again in the future rather than just doing a one-time migration on my computer alone.

I had a few options to choose from, that I could think of.

  • When a new version of Scrobbles is installed, do an in-place migration from the old cache to the new cache
  • Keep the old web services intact and add newer ones seperately, with migration happening server-side per submission
  • Write an adapter for each new schema, mapping old data into new data before passing through any new code

Each of these had its own pros and cons, chiefly to do with the resources that each method would require from either the client computer or the server, but also to do with the maintainability and reliability of each method.

  • Doing an in-place migration would require that it be capable of migration from any previous version to the modern version, and potentially have to migrate across several hundred thousand rows - this is hardly a background operation and would be prone to problems if migration was cancelled by the user.
  • Not breaking older clients wouldn't give users an incentive to upgrade, and the server would have to start having to do quite a bit of work to translate older requests into newer ones, and having resolved to make these web service calls as thin as possible this would go against that.
  • Adapting the data client-side moves the burden of translation from the server to the client, and while translation from any previous version would still be required, not being done in bulk would mean this could be a transparent process.

All the above would require that the client would have to be capable of dealing with there being multiple cache files present, and be able to find out the version of each cache file. Initially this was going to be achieved through naming the cache file by its version, but I've never been one for naming conventions having been completely previously disgusted by the heavy reliance of them in Lionhead's The Movies. I instead added a Metadata table to the cache database and set a version in that. This means this can be checked for with a simple query on opening the cache and the relevant actions chosen.

I decided in the end to go with the final option, of creating adapters around existing code, mapping various methods and classes through a common interface. It involves a bit of work anytime I have to change the data structure between storage and uploading, but it means not having to modify existing code that already works when upgrading. It also means that each client can deal with software that uses older versions of the client library to create old cache databases.

It doesn't strike me as the best solution because it doesn't feel as elegant as I generally like things to be, but it shall suffice as I don't expect to be changing things too often anyway!!

In other news, I have resigned from my post at the University of Reading, and have taken up employment elsewhere. This saddens me slightly, but the new company does look like it's going to provide some interesting times. Because it's a real job with a scary looking contract, I'll refrain from mentioning who they are until I know what their blogging policy is. I don't want to get in trouble by suddenly becoming googleable to those concerned.
Anyway, onwards to a great deal more money, and to a more structured day - it should be interesting (at least, until Scrobbles makes me a millionaire.. ;-))

All comments require the approval of the site owner before being displayed.
Name
E-mail
Home page

Comment (HTML not allowed)  

Enter the code shown (prevents robots):