Thursday, August 14, 2008 12:31:01 AM (GMT Standard Time, UTC+00:00)

Ploughed through some stuff last night, so here is that to-do list with a few things removed, and the most recent items dealt with. The client is so much more stable now I've tweaked the connection string to turn on some options to deal with the ferocity with which the client can push/pull data from the cache. Also added a few new services so data can be submitted in nice little compressed batches, speeding things up to no end. On a train a while ago I wrote the SDK for .NET and I'll be duplicating that for PHP when I get the chance. The Wiki is actually halfway to being populated but I've had some ideas about that elusive graphical snippet designer so perhaps it won't be needed as much as I thought. We'll see.

• Wiki
• SDK for PHP
• Service Documentation
• Snippet Documentation
• Family/Key/Value Documentation

Wednesday, August 13, 2008 4:43:43 PM (GMT Standard Time, UTC+00:00)

After getting a few silly, random things exorcised from my system, I sat down last night and set to work on the Scrobbles client and libraries, as these have been neglected while I have been working on the core server-side stuff.

Last time I left them, I was dealing with some problems to do with the situation where the user upgrades the Scrobbles client, and that client uses a newer database schema for the local cache, or the web services change so data can no longer be submitted the old way. This is quite a rare occurrence, but it just so happens that in my latest overhaul a couple of months ago I completely changed the web services and now have 500mb of unuploaded data sat on my hard drive needing migrating one way or another. And if I'm going to do it nice, I may as well write a system that can cope if I have to change things again in the future rather than just doing a one-time migration on my computer alone.

I had a few options to choose from, that I could think of.

• When a new version of Scrobbles is installed, do an in-place migration from the old cache to the new cache
• Keep the old web services intact and add newer ones seperately, with migration happening server-side per submission
• Write an adapter for each new schema, mapping old data into new data before passing through any new code

Each of these had its own pros and cons, chiefly to do with the resources that each method would require from either the client computer or the server, but also to do with the maintainability and reliability of each method.

• Doing an in-place migration would require that it be capable of migration from any previous version to the modern version, and potentially have to migrate across several hundred thousand rows - this is hardly a background operation and would be prone to problems if migration was cancelled by the user.
  
• Not breaking older clients wouldn't give users an incentive to upgrade, and the server would have to start having to do quite a bit of work to translate older requests into newer ones, and having resolved to make these web service calls as thin as possible this would go against that.
• Adapting the data client-side moves the burden of translation from the server to the client, and while translation from any previous version would still be required, not being done in bulk would mean this could be a transparent process.

All the above would require that the client would have to be capable of dealing with there being multiple cache files present, and be able to find out the version of each cache file. Initially this was going to be achieved through naming the cache file by its version, but I've never been one for naming conventions having been completely previously disgusted by the heavy reliance of them in Lionhead's The Movies. I instead added a Metadata table to the cache database and set a version in that. This means this can be checked for with a simple query on opening the cache and the relevant actions chosen.

I decided in the end to go with the final option, of creating adapters around existing code, mapping various methods and classes through a common interface. It involves a bit of work anytime I have to change the data structure between storage and uploading, but it means not having to modify existing code that already works when upgrading. It also means that each client can deal with software that uses older versions of the client library to create old cache databases.

It doesn't strike me as the best solution because it doesn't feel as elegant as I generally like things to be, but it shall suffice as I don't expect to be changing things too often anyway!!

In other news, I have resigned from my post at the University of Reading, and have taken up employment elsewhere. This saddens me slightly, but the new company does look like it's going to provide some interesting times. Because it's a real job with a scary looking contract, I'll refrain from mentioning who they are until I know what their blogging policy is. I don't want to get in trouble by suddenly becoming googleable to those concerned.
Anyway, onwards to a great deal more money, and to a more structured day - it should be interesting (at least, until Scrobbles makes me a millionaire.. )

Tuesday, August 05, 2008 4:34:02 PM (GMT Standard Time, UTC+00:00)

This weekend I headed up to Kendal and had myself actually quite a good time, despite the gallons of mud caused by a downpour on Friday night. There are many complaints to be had about the organisation and the price considering the actual acts being put on, but I don't care about any of those things cos there was plenty of entertainment to be had nearly all hours of the day if you just loosened up and enjoyed the chilled atmosphere.

Friday

We arrived on Friday evening, and immediately set up the tent because it had stopped raining. No sooner had we finished doing this and got ourselves and stuff inside the tent then the heavens opened up and started soaking people outside still trying to put their tents up. The Friday night lineup was appalling so we just sat in the tent drinking and listening to the ambient sounds coming from the main tent which we'd parked our tent just behind.



Saturday

The previous night's downpour left us with a field full of mud, so after failing to find wellies that fit me, I just went bare foot and enjoyed it.





I ended up stripped down to a pair of shorts and was soon frolicking in the mud for pleasure. Later on while watching the guys photographed below (no idea who they are) I realised I was pissing blood from my foot after cutting it on something, but the mud and crap soon acted as a clotting agent and put a stop to that...



Photos end here as I was getting a bit drunk and I wasn't going to risk falling over in the thick mud and getting the camera dirty. A good evening though, slightly ruined by the Super Furry Animals being a crap band, but only slightly as the local Kendal talent during the day made up for that.

Sunday

Sunday the mud was getting a lot firmer and after grabbing breakfast I settled down in my favourite tent to watch some folk covers of some awesome songs by Johnny Cash and Steve Earle by the Wild Wood Band (known also as the Wyrd Strings when doing their silly stuff). Talented guys. That tent was lovely too, lots of straw to lie down in and a really chilled atmosphere to boot.



The sun was coming out and straw was being provided for people to relax in, and people started making mud sculptures out of the now rather substantial mud.





It was nice to see people enjoying themselves so I spent a couple of hours just wandering about, flitting between the various stages and getting the occasional cup of tea from my other favourite tent (a cafe serving five bean chilli and doing their own little DJ set).





Next up I met up with some of the delightful people from the bsp crowd and partook in smuggled cider and... another other questionable liquid. It was nice to see some familiar faces and I sat outside my favourite tent with them until it was time to go see The Witch and the Robot (one of my favourite bands at the moment). They were, as expected delightfully odd and a pleasure to watch once more.





Eamon Hamilton was on straight after and was another of the very pleasant acts on this day.

After this, we packed up our tent and after getting a photo of the festival flag in action, I put the camera in the car for the rest of the evening.



On arriving back into the festival site, I headed down to the dance tent where it was probably about time for EMF to be performing. For the novelty value they had to be seen and I was lucky enough to turn up just as they started playing "Unbelievable" and left straight after. Timing or what?!

British Sea Power were as always a delight to watch, and I met up with some fellow flag makers who I spent the rest of the evening with along with some other people who I had met during the day. Having people to help wave mine and Jo's massive flag was a real bonus, and by the end of it my arms were absolutely dead.

I had to spend an hour posing with people and the flag whilst they held on to me for support in the thick mud and I sank deeper and deeper... but I'm hoping at least I'll find a good photo to use in the competition.

A good weekend all in all, although slightly marred by the day after where my grandparents made me go to A&E for the aforementioned cut in my foot. They scooped out all the crap, x-rayed it to make sure nothing else was in there, bandaged me up, and put me on antibiotics while getting a telling off from the cutest nurse I have seen in a long time - I was not complaining. (Hey, if you're reading this get in touch yeah? ).

Trains on the way back were cancelled and I ended up getting a taxi from Birmingham to Reading courtesy of the rail services. Say what you like about our public transport but at least they look after you when things go wrong! 

Wednesday, July 30, 2008 12:11:20 PM (GMT Standard Time, UTC+00:00)

With the success of the artwork we presented to The Witch and the Robot a few months ago ( see here ), when British Sea Power announced a festival flag and bunting competition I decided to re-enlist the help of the vaguely talented Jo to add a touch of class to the entry I wanted to put together.

That was over a month ago now.

After an initial three hour design session where we discussed concepts and bored the hell out of Owen with our ramblings, we came up with a vague plan involving a bedsheet and fabric to create a lion and a bear with some beer, and whatever else we came up with during the creation period.

A trip was organised to the rather excecllent Fabric Land to pick up materials and equipment to perform sewing and cutting activities with, and a bed sheet was procured from Woolworths.

Somehow, from this pile of material, Jo managed to craft a brilliant resemblance of the Lion from the third battalion pin I possess, and a really good attempt at the bear from Open Season, BSPS's second album. Owen cut some leaves out, in the style of the leaves from Decline Of [bsp] (BSP's first album) and I... made some stars and the pint of beer. The easy stuff because my fingers don't generally do what I want them to do and the hard stuff would just come out wrong.

All of this material had to be sewn onto the bedsheet by hand, and I've calculated that we've put well over thirty hours into this over the past month. Nevertheless, it has turned out quite well, so thanks to Jo for being a good little creative director, and thanks to Owen for his leaves as I don't think fabric and sewing were really his thing... he looked bored to death, bless.



Now to use the rest of the fabric to make a daft costume, and the rest of the thread to tie Jo up once we're done, she must not be allowed to fall into the hands of the competition.

Wednesday, July 23, 2008 12:13:35 AM (GMT Standard Time, UTC+00:00)

That weekend in Alicante left me interested in using photographs to show people the way I see things in this world.

I can't draw, I can't paint... generally anything I do with my hands is pewp, and when I wielded the camera I felt a bit of power, so I've borrowed Ben's rather nice DSLR for a bit so I can learn how to take photos.

I've been using the manual modes on the camera, doing everything in my power to make each photograph look like what I'm feeling and seeing in a particular scene.

I've spent a week with it now, and I bought a photography magazine and learned all about all this malarkey to do with the setting of aperture size, the exposure times to go with that, and the iso rating of the ... 'film'? And what it all means.

I quickly realise that actually, this photography thing is a very neat fit for somebody like me, because it suddenly all boils down to numbers and knowing what those numbers mean. This is why I get on with programming so well, so I feel at home setting up the camera to capture shots.

Now, I owned a camera once, when I was 7. It was one of those simple film jobbies which I used to do what thing that everybody else does these days with their mobile phones, taking simple shots of whatever it is you are looking at.
I don't want to do that, I want to capture the essence of a scene, to take a bit of the soul home with me and show it to other people so they can understand what I found beautiful about something.

So, without further ado, some pictures from my first weekend of learning with the camera. I am quite pleased with some of the results, especially as this is my first go with a camera since that one I owned when I was 7.

Attempt Two: Switzerland and France
I hadn't read anything yet, so I was mainly winging it from fiddling with settings and working out from hearsasy what they were all about. I wish I could go back and take some more with just what I've learned from the past couple of days!

The full set of Geneva shots can be found here.

A lamp in France



It was quite dark, as the sun was setting, and I noticed the sun was throwing light through the orange of the lamp and onto the tree behind it. I took a few shots of it with different settings, and this is the one that came out best. Perhaps a different angle would have worked better, but it was the first shot that came out that gave me any level of satisfaction.

F-stop: f/5.6
Exposure time: 1/60sec
ISO speed: 400

Had I had the tripod with me, this would have worked a lot better with a lower exposure time, and flipping the ISO speed back to 100 where I didn't get such a grainy picture. Ah well - live and learn. This was before I bought the magazine for education and inspiration.

Olives in the Market


We visited the market and I thought it would be a good chance to try and get some colourful shots of some goods on display. The olives were a no-brainer, and the cloth-stand one came out quite well too.

F-Stop: f/8
Exposure Time: 1/160 sec
ISO Speed: ISO-400

Wtf? ISO 400? That was an accident, and by using the light-meter I compensated by upping the speed of the shot without really thinking about it. Whoops. Had I known about aperture at this point, I'd probably have taken a few shots with different layers of olives in varying amounts of focus. Still would have probably chosen this shot though.

Attempt Three: Campus
Okay, so I read a magazine on the plane, and some articles on the internets. I'm an expert now right? Okay, joking aside, after spending the day at home working, I walked into campus and took a few nice shots and they made me happy.

The full set can be found here

Woodland Trees



Having learned that I can get quite a range of depth of field by using a small aperture size, I took this one by bringing it down, and upping the exposure time to a decent length of time. Sticking the camera on the ground on the tripod, and using a timer so I don't have to touch the camera and blur the photograph. It's quite dark under the trees too, so it's sort of necessary.

F-Stop: f/10
Exposure Time: 2 sec
ISO Speed: ISO-100

As much as I like shooting from the floor, it would have been nice to use a proper tripod in this shot and take it from the vantage point I had personally.

Woodland Floor


The floor is quite a pretty and overlooked place as people stamp all over it, so I got down and dirty and took a shot of the undergrowth. It turned out ok, despite the darkness under the tree canopy.

F-Stop: f/11
Exposure Time: 1 sec
ISO Speed: ISO-400

I upped the ISO earlier because I was taking pictures of some moving animals under the canopy, and forgetting to turn it down again the exposure time wasn't quite as long as other shots. It didn't make the shot too grainy thankfully. I should have used a much smaller aperture size I think, and upped the exposure time, this was quite early on and I hadn't quite worked this out yet.

Woodland Floor


Another shot from the floor, but this time looking at the overall scene. Really narrowing the aperture size so I can get a good focus on the entire thing. This meant an obscene exposure time, but I think it turned out ok.

F-Stop: f/16
Exposure Time: 6 sec
ISO Speed: ISO-100

I think I've got this sussed, and decide to go home. I'm looking forward to learning more, this is only my third outing and I feel as if I'm getting the hang of it already.

I know a few of you actually take photographs as a hobby, have you any hints to give me from looking at the above? Am I making any stupidly rookie mistakes? I wouldn't be surprised if I was...

Monday, July 14, 2008 10:24:53 PM (GMT Standard Time, UTC+00:00)

Fine fine fine, the weekend as it happened.

Friday

Friday was my birthday, although I like to keep this hidden from most people so I don't feel forced into any sort of celebration, take things as they come.

I had organised a meetup of some old friends that night (Dave, Luke and Stears), although the fact that it was my birthday was entirely coincidental, it just being the first Friday night that people were free to go for a few quiet drinks.

This started off as one of those evenings where we sit down over a few slow beers, and talk about geeky things like code, our mobile phone and OS preferences, what we're doing at work, and our recent hijinks, as we rarely see each other there is generally a lot to catch up on.

At about 10pm, we had moved into the territory where bad jokes start getting told, and the conversation becomes normal enough for non-programmers, so I called up Ben and grabbed him, and texted some students to make sure they were in the metal-bar across town before we began the trek.

As we had hardly been drinking enough to really justify three hours dancing wildly in a metal bar, it was decided that we would catch up with ourselves by going through Back of Beyond's more than adequate supply of Sambuca. Rounds were bought with 2-3 sambucas each in them, and at least five of these were purchased.

It was safe to say, that this was the turning point of the evening.

Somehow, we made it to Facebar, were we rendevouzed with a horde of "people that I know". I don't remember much about Facebar, only that it was a lot of fun and that I kept falling over. Somebody suggested doing some strawpedoing and apparently I managed to succeed in this for a change. Probably not good in the long term however. Somewhere between here and home I managed to pick up a facial injury which is a bit embarassing - but oh well these things happen!

At about 3am, it was decided that we would head back to Dave's, and drink Jägerbull as he had a lot of Jägermeister at his house. So we set off, resolving to pick up a large supply of Red Bull from Murco's, our trusted all night petrol station in homeward adventures. We head back to Dave's for Jagermeister and The National, everybody is happy so at 5-30am we retreat to our homes for a good night's sleep.

Or rather, a few hours...

Saturday

We get up at around 10am and resolve to a day of chilling and possibly even a chilli in the evening. Given that I got my face damaged the previous night, I sacrifice the vegetarian stance I've been maintaining (for variety rather than ethical reasons) and set up some bacon and brie sandwiches, buying some really nice and well looked after bacon for this brief lapse. Ben pops around to the house and Stears him and I decide to spend a day down at the cinema, and head down at about 15:30 for this purpose. I get some replacement glasses from specsavers and we go to watch Kung Fu Panda. The woman who sells us the tickets mis-hears us and we don't realise until we're at the screen that actually, we're going to watch Hancock. Ah well, it was entertaining I guess.

We head out for a coffee, and discuss plans, perhaps it would be funny to go fly somewhere on Sunday and grab lunch, possibly Greece, or Italy. This idea is investigated on Stears' phone a bit, and then we head back to the cinema to watch Wanted. Absolutely hilarious movie, recommend it to anyone.

We arrive back at my house after going via Tesco, at around 10:30pm and I decide to look at some flights. Greece is too far away, and Italy is too expensive. At 10:50 we decide "Hey, let's go to Spain and grab some spanish food". AWESOME.

Ben runs to get his passport from his house, and Stears and I feel smug that we keep our in our laptop cases. I book three tickets to spain at 5:40am, and we RUN to the train station to catch the final train to the airport which leaves at 11:34pm.

We spend the morning pissing about at the airport, drinking coffee and setting our Facebook statuses to comical things like "Gone for lunch"

We are relieved on checking in, to find that I have booked our flights correctly and that in a few short hours, we'll be at Alacante drinking beer and eating Paella.

Sunday

We arrive into Alacante and grab a taxi to "the beach". The beach it is and we are amused at how our random adventure has gone so far.

We buy some shorts and towels, and head onto this beach to do some... lounging around and perhaps a bit of sleep. Ben and I go for a swim and are pleasantly surprised by how warm the water is. At 9am, the ambient air temperature is 25 degrees Celcius, so we know we'll be in for a hot day.

I realise I've left the country without bringing a bobble to tie my hair back, and after not showering for 24 hours and in this heat, my hair is going to stick to my face and make me incredibly irritable (I want to do murders where I'm irritable), so we set about looking for a shop that sells bobbles. In this overly-religous country however, the entire town is shut down because it's Sunday morning and everybody is at church.

I improvise by tying my tie around my head like a bandana, and feeling like a prat for the rest of the day. Oh well, I looked like a prat wearing the tie in the first place, so this is probably an improvement.

The rest of the day is spent just pissing about, drinking beer, eating paella and swimming. Ben and I play with his camera and use the long lense to 'snipe' random passers by doing their general activities. We spent the entire day drinking beer, and slowly learning the local lingo. We have an awesome time, but it's one of those awesome times that is hard to put into words because the good time is being generated from just being there and talking nonsense over beer whilst people-watching.








12 hours later, we're a bit worse for wear through tiredness and beer - and in my case almost drowning and we say goodbye to the city and make our way home.

We arrive back in Reading 6am, having left 36 hours beforehand. We're dead on our feet. We've witnessed the sunrise from Gatwick airport twice in a row and are getting dangerously sober.

Monday

A few hours sleep is had, getting up once more at 10am to fix the boiler, and get some work done. Life is back to normal but photos need uploading. What a weekend eh?

The un-edited photo set can be found here.
The flickr set can be found here

Saturday, July 05, 2008 12:59:39 PM (GMT Standard Time, UTC+00:00)

Today, Firefox loses all my bookmarks, it's done this in Fx0.x Fx 1.0 Fx 2.0 and now Fx3.0 - you'd have thought this would have been fixed by now.

Opera takes two hours to upgrade, and then the new version decides it can't render its own UI anymore, and when it finally starts doing that again, I've lost most of my toolbars to the UI eating monster and it's utterly fubarred beyond repair.

Internet Explorer 7... worked. Okay, it wasn't the best with standards, but it worked. The UI is clunky and it's not as pretty as the two aforementioned browsers, but it DID ITS JOB.

I 'upgraded' to the beta of IE8 a while ago, which failed to work with a lot of sites (mainly because the Js model has had an overhaul), and that is what prompted me to start using Opera (and then Firefox when that fell over).

I have decided to go back to IE8. It might be unstable, it might occasionally crash whilst on Facebook, but at least I know it's going to do those things. It so far has never unpredictably just stopped working, or ruined my life, or killed my cat. If I carry on using Opera or Firefox, those things might just happen, and I just can't take that risk.

Wednesday, July 02, 2008 2:15:56 PM (GMT Standard Time, UTC+00:00)

Of course, it goes without saying that the below example has lots of common elements that would be better presented as part of the SDK to save on boiler-plate code.

Click here to see new code for generating that output.

Somewhat better no?

Wednesday, July 02, 2008 12:00:57 AM (GMT Standard Time, UTC+00:00)

Spent most of the evening designing a flag for the upcoming Kendal's Calling and Tan Hill festivals, with my partner in crime Jo, who last worked with me to create the artwork for a present when visiting a band we like called The Witch and the Robot. The flag was a lot harder to come up with an idea for, having a canvas which is so many square feet in diameter. We settled with something we liked in the end though, so we'll have fun sewing and painting that later this week, stand by for photos!

Then the rest of the evening wasted in Diablo II with Owen, but that's all fun and nostalgic so why worry about that eh?

Because of the above, I've barely gotten any Scrobbles done tonight, but I did just sit down and polish off the embedded data from external websites, a very simple fix with a fresh mind and it was done.

So, this is the product of that.

This is a snippet, with nothing in it but an application calling into an external service.

Click here for xml in a new window

Walking through that from top to bottom:

• We have some inputs. These are entered by the user on adding the snippet to their profile. This allows the user to add the snippet multiple times and configure it differently each time.
• We then have the actual application itself, inside a Data section, which simply calls a third party web page and passes it the Character and the Realm originally entered by the user.
• In the Layout section, we simply say "I want this Application's data here". This way, several applications can be utilised within the same snippet, along with all the ordinary stuff you might find in a snippet.

Behind the aspx page being called, the following code can be found:

Click here for code in a new window

All I'm doing here is

• Ensuring the page returns plain old XML
• Creating a context from the sdk which takes care of authentication for us
• Pulling the variables out and pushing them into a query which
• Outputting the results of along with a bunch of text as html. (Well, a limited subset of html, of course).

This results in the following being displayed in the user's page. It's a really trivial example so the amount of overhead required for two lines of text doesn't seem worth it, but I guess that's always the way with trivial examples.

With this, the possibilities for representing data are endless, I can think of hundreds of ways I'd like to use this system so hopefully so will my users. If I get any that is.

It's kinda cool to get this working, as this was one of my original visions when I started last year.

Tomorrow I'll be going all out on Scrobbles again, I think I'm going to work through the documentation and get that completely sorted out, with perhaps a few more API tweaks to allow for direct access to pre-generated queries from external websites.