Friday, October 24, 2008 1:04:51 PM (GMT Standard Time, UTC+00:00)

I picked up JsonFx yesterday as I could really do with a more.. declarative way of creating client-side controls when piddling about with Ajax for a browser-application type thing I am working on. JsonFX seems to provide a nice JBST system, along with data binding and support for compacting scripts/files on the fly, so I'm quite excited about getting into it.

I've had a few difficulties getting it working however, I just kept getting the glorious message "Error loading "~/scripts/sdctls/controls.merge". Either not found or a build error occurred." with no other information supplied. At the time of writing, this error message cannot be found on Google.

The actual JsonFX site is seemingly scant on documentation, and the starter kit was throwing the same wobbly, so I determined that it was either the way I was running the site, or something to do with the environment

Downloading the source for JsonFX and stepping through it, revealed a dependency on the MSScriptControl ActiveX component, presumably for validating scripts before shrinking them and sending them across the wire. Googling for the MSScriptControl dependency came up null though, for some absolutely bizarre reason, perhaps I am just fail at googling today.

Supposedly on not finding the component, an error is supposed to be added to the error handling system in JsonFX and therefore displayed, telling the developer to go download it, but this was not happening.

The ActiveX component can be found here, the Windows Script Control

Problem solved, I've got everything else working by looking at the rather excellent self-documenting starter-kit. If I come across any other gotchas I guess I'll post them, as this library seems rather new and there is very little on the internet about it

Sunday, September 28, 2008 1:22:40 PM (GMT Standard Time, UTC+00:00)

Started the new job about a month ago now, and I'm quite enjoying the actual job itself, the people who work there (and the money they pay).

One thing however, has taken a huge hit. I was often asked by a friend of mine how I managed to get so much stuff done when juggling personal projects with a full time job. The answer has become clear since doing this "real person" thing of commuting and spending standard hours at work. My productivity has absolutely plummeted, I come home too tired to do anything computer related at all. I have ended up either collapsed onto my bed listening to music, or spending time with my newly acquired girlfriend Jo (bad timing, but not complaining). This is not because I do more work at work, but because I'm actually having to travel into work and get up early to prepare, I'm suddenly devoting thirteen hours of my day towards the job instead of the normal seven or eight hours.

I shall resolve to do something about that soon, I can't afford to not be working on my personal projects, Scrobbles or otherwise if I am not putting things out there on the internet, if I am not creating time to learn new technologies then there is little point in being a programmer at all.

Today however, I am working on Scrobbles, my work ethic has not gone anywhere and I have resolved to cross a few things off the list, as pointless at it is in light of the above.

Url re-writing is now done finally, and I'm going to push forwards with getting the rest of the things ticked off as much as possible. And I'll keep you all informed with my coping strategy for developing personal code when faced with the Real World. I think I know what my solution is, but I don't want to talk about it until I know if it is going to work.

Thursday, August 14, 2008 12:31:01 AM (GMT Standard Time, UTC+00:00)

Ploughed through some stuff last night, so here is that to-do list with a few things removed, and the most recent items dealt with. The client is so much more stable now I've tweaked the connection string to turn on some options to deal with the ferocity with which the client can push/pull data from the cache. Also added a few new services so data can be submitted in nice little compressed batches, speeding things up to no end. On a train a while ago I wrote the SDK for .NET and I'll be duplicating that for PHP when I get the chance. The Wiki is actually halfway to being populated but I've had some ideas about that elusive graphical snippet designer so perhaps it won't be needed as much as I thought. We'll see.

• Wiki
• SDK for PHP
• Service Documentation
• Snippet Documentation
• Family/Key/Value Documentation

Wednesday, August 13, 2008 4:43:43 PM (GMT Standard Time, UTC+00:00)

After getting a few silly, random things exorcised from my system, I sat down last night and set to work on the Scrobbles client and libraries, as these have been neglected while I have been working on the core server-side stuff.

Last time I left them, I was dealing with some problems to do with the situation where the user upgrades the Scrobbles client, and that client uses a newer database schema for the local cache, or the web services change so data can no longer be submitted the old way. This is quite a rare occurrence, but it just so happens that in my latest overhaul a couple of months ago I completely changed the web services and now have 500mb of unuploaded data sat on my hard drive needing migrating one way or another. And if I'm going to do it nice, I may as well write a system that can cope if I have to change things again in the future rather than just doing a one-time migration on my computer alone.

I had a few options to choose from, that I could think of.

• When a new version of Scrobbles is installed, do an in-place migration from the old cache to the new cache
• Keep the old web services intact and add newer ones seperately, with migration happening server-side per submission
• Write an adapter for each new schema, mapping old data into new data before passing through any new code

Each of these had its own pros and cons, chiefly to do with the resources that each method would require from either the client computer or the server, but also to do with the maintainability and reliability of each method.

• Doing an in-place migration would require that it be capable of migration from any previous version to the modern version, and potentially have to migrate across several hundred thousand rows - this is hardly a background operation and would be prone to problems if migration was cancelled by the user.
  
• Not breaking older clients wouldn't give users an incentive to upgrade, and the server would have to start having to do quite a bit of work to translate older requests into newer ones, and having resolved to make these web service calls as thin as possible this would go against that.
• Adapting the data client-side moves the burden of translation from the server to the client, and while translation from any previous version would still be required, not being done in bulk would mean this could be a transparent process.

All the above would require that the client would have to be capable of dealing with there being multiple cache files present, and be able to find out the version of each cache file. Initially this was going to be achieved through naming the cache file by its version, but I've never been one for naming conventions having been completely previously disgusted by the heavy reliance of them in Lionhead's The Movies. I instead added a Metadata table to the cache database and set a version in that. This means this can be checked for with a simple query on opening the cache and the relevant actions chosen.

I decided in the end to go with the final option, of creating adapters around existing code, mapping various methods and classes through a common interface. It involves a bit of work anytime I have to change the data structure between storage and uploading, but it means not having to modify existing code that already works when upgrading. It also means that each client can deal with software that uses older versions of the client library to create old cache databases.

It doesn't strike me as the best solution because it doesn't feel as elegant as I generally like things to be, but it shall suffice as I don't expect to be changing things too often anyway!!

In other news, I have resigned from my post at the University of Reading, and have taken up employment elsewhere. This saddens me slightly, but the new company does look like it's going to provide some interesting times. Because it's a real job with a scary looking contract, I'll refrain from mentioning who they are until I know what their blogging policy is. I don't want to get in trouble by suddenly becoming googleable to those concerned.
Anyway, onwards to a great deal more money, and to a more structured day - it should be interesting (at least, until Scrobbles makes me a millionaire.. )

Wednesday, July 02, 2008 2:15:56 PM (GMT Standard Time, UTC+00:00)

Of course, it goes without saying that the below example has lots of common elements that would be better presented as part of the SDK to save on boiler-plate code.

Click here to see new code for generating that output.

Somewhat better no?

Wednesday, July 02, 2008 12:00:57 AM (GMT Standard Time, UTC+00:00)

Spent most of the evening designing a flag for the upcoming Kendal's Calling and Tan Hill festivals, with my partner in crime Jo, who last worked with me to create the artwork for a present when visiting a band we like called The Witch and the Robot. The flag was a lot harder to come up with an idea for, having a canvas which is so many square feet in diameter. We settled with something we liked in the end though, so we'll have fun sewing and painting that later this week, stand by for photos!

Then the rest of the evening wasted in Diablo II with Owen, but that's all fun and nostalgic so why worry about that eh?

Because of the above, I've barely gotten any Scrobbles done tonight, but I did just sit down and polish off the embedded data from external websites, a very simple fix with a fresh mind and it was done.

So, this is the product of that.

This is a snippet, with nothing in it but an application calling into an external service.

Click here for xml in a new window

Walking through that from top to bottom:

• We have some inputs. These are entered by the user on adding the snippet to their profile. This allows the user to add the snippet multiple times and configure it differently each time.
• We then have the actual application itself, inside a Data section, which simply calls a third party web page and passes it the Character and the Realm originally entered by the user.
• In the Layout section, we simply say "I want this Application's data here". This way, several applications can be utilised within the same snippet, along with all the ordinary stuff you might find in a snippet.

Behind the aspx page being called, the following code can be found:

Click here for code in a new window

All I'm doing here is

• Ensuring the page returns plain old XML
• Creating a context from the sdk which takes care of authentication for us
• Pulling the variables out and pushing them into a query which
• Outputting the results of along with a bunch of text as html. (Well, a limited subset of html, of course).

This results in the following being displayed in the user's page. It's a really trivial example so the amount of overhead required for two lines of text doesn't seem worth it, but I guess that's always the way with trivial examples.

With this, the possibilities for representing data are endless, I can think of hundreds of ways I'd like to use this system so hopefully so will my users. If I get any that is.

It's kinda cool to get this working, as this was one of my original visions when I started last year.

Tomorrow I'll be going all out on Scrobbles again, I think I'm going to work through the documentation and get that completely sorted out, with perhaps a few more API tweaks to allow for direct access to pre-generated queries from external websites.

Tuesday, July 01, 2008 1:01:13 AM (GMT Standard Time, UTC+00:00)

I am really tired, as I mentioned previously I used this weekend's train journeys to start on the code that would allow external websites to embed data in a user's page (ala FB). This of course means creating some sort of auth system so those external websites are allowed access to the user's data (done), adding the application node to the snippet schema (done), creating a schema and a transform for the xml that the external website would pass back to Scrobbles for embedding (done), adding the database support for caching the results of these requests (done), writing the code to actually make those requests when pages are being generated (done).
Still left to do is adding support to the layout transform for actually embedding this data into the final page, but that can wait until tomorrow because having done most of the above just now I am truely knackered.

I got bored an hour ago and decided to see how much code I've got under the Scrobbles namespace after just under a year's development in my spare time. I started at the end of August, just after I stopped playing World of Warcraft. So.. close enough. 11 months development if I'm being kind to myself.

In total, I have:

• 723KB of VB across 258 files
• 164KB of C# across 39 files

This is of course filtering out the Visual Studio generated .designer files and etc, as they're rather large, and I didn't write them.

Of these, the following totals were calculated using SlickEdit.

• 19,000 lines of text
• 11,000 lines of code        (58%)
• 2500 lines of comments    (14%)
• 5500 lines of whitespace  (30%)

It's not actually too bad, of course on top of that I have a couple of thousand lines of javascript written, a thousand lines or so of hand-written XML Schemas and another thousand lines of hand-written XML transform file.

Calculating it, it's very roughly 50 lines a day, every single day with no break. Sadly I spend entire weeks not writing code, but it does mean I'm quite pleased with this average.

Drawing on those seemingly pointless software engineering lectures, (This project counts as being organic right?)

• E = 2.4 * (11)^1.05 = 29.762
• D = 2.5 * (E)^0.38 = 9 months
• P = 3

Obviously this is entirely meaningless, as COCOMO is a rather outdated and worthless calculation used against these modern languages which can practically write themselves, but it's still quite nice to see that I've managed to maintain some semblance of productivity this past year. Come on release!

Friday, June 27, 2008 1:21:08 AM (GMT Standard Time, UTC+00:00)

Whew.

Spent this evening firstly trying to get PHP to play nicely with my nice Windows 2003 Server, and then intalling PHPBB on top of that. Once that was done, I wanted to tie together PHPBB with my current authentication system. And that is where the fun really started.

There isn't a lot out there on writing authentication modules for PHPBB, most people seem to write plug-ins for other web software to authenticate against PHPBB rather than the other way around. I decided therefore that the best way would be to take the LDAP plug-in provided with PHPBB and rip the relevant bits out, replacing them with a SOAP web service call to my existing authentication system.

That's right, I'm authenticating across a web service instead of going directly into the database. I couldn't be faffed playing about with the hashing method I use in the ASP.NET application for salting the passwords and storing them in the database, so I decided to use my existing .NET code to do the legwork for me.

It wasn't actually that hard in the end, although I came across a few problems that I should probably document in case I ever have to try this again.

1. Creating a SoapClient around my ASP.NET WSDL endpoint - remember to add ?wsdl to the end of the url for the web service, so that the SoapClient actually gets the wsdl instead of the html placeholder...
2. Calling into the Webservice method.

• $authToken = $Client->Login($username, $password); did not work
• $authToken = $Client->Login( array( 'username' => $username, 'password' => $password, ) ); did work. I don't know why this is the case, as the docs didn't demonstrate this way of calling the method.
  
• My web service method returns a 'string', but in PHP this is represented as a standard StdClass, meant to deal with potentially complex return types from the web service. The actual string, can be found in $authToken->LoginResult - go figure...



3. In PHP, the md5 method returns a lower-case hexadecimal string. In .NET, most examples tend to use the format string "X2", which creates an upper-case hexadecimal string. Wrapping up the password hash with strotupper before passing it to .NET solved this.

The actual process of writing the authentication plug-in couldn't be simpler using the LDAP plug-in as a base. Simply take the username and password, and attempt to authenticate against the web service. If this fails, then try going through the database directly. If it succeeds and the user doesn't exist in PHPBB, then tell PHPBB to create the user. Store the password in a PHPBB hash and retrieve the user's details from the web service. If it exists, then just make sure the password is up to date and carry on.
This way, if the web service goes down due to Scrobbles being updated or whatever, my users can still log in and complain on the forums. Happy days.

Thursday, June 26, 2008 4:58:07 PM (GMT Standard Time, UTC+00:00)

This is the list as it stands for things I need to do before release.

• Core Scrobbles
  • Friendly URLs
  • Referal system for 'earning' queries
  • Embedding Data From Third Party Sites
  • Compressed Raw Data Queries
  • Compressed Batch Data Submission
  • Registration System
  • Arbitrary Views (using Snippet System)
  • Automatic Data Submission (Scrobbles App)
  • Security/Validation of All Existing Forms
• Third Party
  • Online submission of WoW Data
  • Heatmaps (location tracking)
• Community
  • Wiki
    • SDK for PHP and .NET (wrappers around system)
    • Service Documentation
    • Snippet Documentation
    • Family/Key/Value Documentation
    • Sync with Scrobbles DB
      
  • Forums
    • Sync with Scrobbles db
      
    • Requests etc

Seems I mostly just have housekeeping tasks left, so I'm going to run through a few of them tonight.

I have a pile of ideas for lots of 'small' projects once I have this completed and I'm anxious to get started on them. I think longterm, I'll make more money from the small projects per hour put in.

Scrobbles may end up being successful and that will be great, but if it's not, it will make a great technical portfolio item, and that was the reason I started it in the first place.

The small projects are to make money, and by spreading my time over many projects with smaller scope and smaller time input required, I hope to make something of them. Can't wait for the rest of this year to have rolled by. I'm feeling really motivated and can't believe my head can actually generate this many ideas!!

Tuesday, June 24, 2008 10:53:41 AM (GMT Standard Time, UTC+00:00)

Last week on a rather long train journey to Morecambe to see British Sea Power, I decided to write the next layer underneath the top level querying system, to cache 'collections' of individual common queries, which the results of could then also be queried in order to speed things up.

This has been planned since the original xml based querying system, but I had decided that writing it on top of that system would be like slapping a band aid on a broken neck. When something is that unworkable, trying to speed things up just by adding additional caching levels is a lost cause.

Happily, also on the train I wrote yet another version of the core querying system, which used a ridiculous level of nested inner joins to whittle down arbitrary data into a workable form. This is entirely counter intuitive, as you're told academically that joins are expensive, and nesting queries is expensive - and that by combining them you must be in for a world of pain. Imagine my surprise on running the query against SQL Server 2005's rather excellent query visualiser, that the resulting path was not only incredibly simple, but also incredibly fast due to the rapidly diminishing size of the data.

Because of the analytics code I already had written, it was very easy for the initial inner statement to cut down the size of the data by a huge amount in one simple query. Because quite a lot of the core queries tend to revolve around single keys, and single values (for example, "get me all the keys/values where KEY=WoWActionName and VALUE=MINING). Caching the results of this query, and even subsequent queries can result in replacing that part of the overall nested query with ("get me all the keys/values where the query = <>").

Running some trials, with the twenty snippets I have written for testing purposes, resulted in just 10 actual unique cached queries. It is important to note that in the original system I was filtering by date/time in the inner-most query - resulting in a unique query for every single page, and in this version I moved the date/time filter to the outermost query. This meant the results of this query could be re-used across all pages generated for that user.

This cut down the time taken to generate a typical 'all-time' page from 50-60 seconds, to 3-5 seconds. A rather massive increase of performance. (The fifteen seconds listed below wasn't when doing it concurrently)

These cached queries could then be refreshed or culled periodically using the background windows service I have written for Scrobbles, and most importantly rather than these resource intensive queries happening on different threads and really killing the server from over-zealous usage of RAM and therefore over-zealous paging and hard disk thashing, I can keep them on a single pipeline until I have a server that can deal with parallelising the whole shebang.

I foresee this not being enough when I have a huge amount of data in there for each user, the amount of data generated for an eight hour session of World of Warcraft is immense, and I only probably have sixty hours of data in there myself. I'll be able to add date and time to those cached queries later, and combine the daily results across a month, and the monthly results across a year and the yearly results across 'all time'. So I'm not worried about that.

I ran some tests on my laptop on the train, and was able to generate 2000 pages (all the pages possible for my single user over a year) in under ten minutes. This was using a threadpool to generate pages concurrently, as that is how it will happen once on the internet. At any one point during this, thirty pages were being generated simultaneously and taking seconds to complete. SQL Server's ram usage didn't even get above 300mb so I'm fairly confident this will work for my first few users. (Going forward!).

I appreciate the above is probably quite hard to follow, so here is a picture of the whole process - showing how each process has been decoupled to allow parallelisation as required in the future.

In other news, life is really busy at the moment and Scrobbles is having a week or so of hiatus. This weekend saw me at the Natural History Museum watching British Sea Power (again, yes I know), culminating in me lying on the stage drunkely trying to play the cornet which I had 'borrowed' from the band, tonight sees me at Victoria Park for Radiohead, as does tomorrow, Friday sees me at Koko seeing a band I saw a few weeks ago called "Die Die Die" (Awful name), but they were quite cool, and Saturday sees me up in Manchester seeing My Bloody Valentine, the legends that they are.

Yes, I have purchased and am using ear plugs...

Monday, June 16, 2008 1:45:17 AM (GMT Standard Time, UTC+00:00)

There we go, all that effort has paid off.

Time taken to generate a single 'one day' page of even the more complicated Scrobbles info is down to under a second, and down to about fifteen seconds for an 'all time' page view. (I cache these with quite a high longevity - think of the single default view that last.fm gives you and how often that updates..). On the old system with the current amount of data, this was going into the 'several minutes' for even a single day page. (Yeah, XQuery not so good...)

Getting the right balance of normalisation was key to this success, and I have written a lot more code than I would have originally liked to, most of it not being used in the final solution. Some of it quite experimental and rather cool though - and perhaps the key to future attempts in optimising Scrobbles. One of my solutions worked out 'groups' of keys which could isolate the data for most requests down over 90%, to generate permanent tables for crunching data into - and this may end up being a good way to go if my current solution doesn't last the distance. With the data for the entire year, this algorithm only generated ten tables - and I was able to index the entire year's data through this process in under five minutes so it was quite scaleable.

That was a bit hard to integrate with the query system as it stood however, so I'm bypassing it and going straight to the core data store for now.

Should be grand though, the background service can happily be constantly ticking over the 'all time' pages on a low priority (Well of course I have a priority queue based system!), and generating the one-day views as they are requested - and perhaps some sort of balance over the monthly views and yearly views based on what month or year it currently is.

At say, three pages per user, and 5 views to be updated constantly, that's fifteen views taking about 120 seconds in processing time altogether. I could still update every single users page more often than last.fm does (for its users' music pages) and support 500 users on the rather underpowered server this system currently sits on. Not that I would of course, because there is little point in updating pages unless people look at them sometimes.

Lots more work to do yet on making things even faster - but with a firm database design to now stand on, I feel a bit more confident about pushing ahead with the real development of the system.

Saturday, June 14, 2008 1:42:23 PM (GMT Standard Time, UTC+00:00)

My checklist has taken a hit this week, with a busy social calendar (organised by me a couple of months ago in the past apparently).

Not only that but I hit some technical issues after I migrated the server-side install of the Scrobbles system across to a new version, along with the database (a process that took over six hours, there being now over a year's data in there).  A migration that was supposed to make the whole system a lot more scaleable and maneagable in the future. De-coupling page requests from page generation and making a whole load of the resource intensive operations parallelisable.

The new system also removed the data-loss incurred by using discrete blocks of time, and moved the system across to an entirely continuous method of time-based data storage. And most importantly was meant to make it a lot easier to form complicated queries by storing a lot of this data in the original XML format, for querying using the XQuery support built into SQL Server 2005. I thought that there was no way I could do anything better than this with my limited knowledge, and that SQL Server's magic black box would just index this data and keep things fast and nifty for me.

In my trials, this seemed valid, and page generation was no faster or slower than the previous iterations of the querying system. On migrating across a larger data set, SQL Server started eating ridiculous amounts of Ram and CPU, before finally giving up in a big heap. Back to the drawing board again, for probably the fourth time.

It should go without saying that I still think the XQuery support in SQL Server 2005 is fantastic, and I think that the solution was simply not compatible with my needs. It was educational to play with however, and I can think of a few projects which would benefit from having the masses of XML on the hard drive transferred into a database which can then do the hard work of actually querying it for information.

I downloaded a full copy of the working database for testing and set about trying to index this massive amount of data myself. Many conversations were had with different people with varying levels of expertise - my colleagues (Karsten and Pat) were helpful over coffee and a pad of paper, and we came up with a new design which should hopefully have been more efficient. I also had quite a few conversations with Paul Evans who let me know of the oh so many potential pitfalls before I even began work on the new prototype system. Sadly, these pitfalls seemed to pop up all too soon and my query graphs were soon looking just as complicated, if not more complicated than the original XML driven attempt.

I think I've finally come to a proper solution now, which involves creating tables on the fly to fit the needs of the system as it evolves. This is a scary solution for me, as it gives away some of the intelligence of the database design to an indirect process rather than directly from me. It also increases the complexity of the code by quite a bit - and I was hoping by just having the one-size fits all database solution to avoid that. 

Sadly in the world where speed and efficiency counts more than anything else, and as I was originally warned at the start of the week, this seems to be quite a standard compromise in the world of database design.

Sunday, June 08, 2008 3:03:06 PM (GMT Standard Time, UTC+00:00)

The last one was starting to get a bit busy, and I was starting to panic because I'm never going to release if I have to work my way through them all.

I've split it up into things I need to do before I release Scrobbles, and those that can wait until its out the door. For one thing, I should stop assuming that my potential users are going to want things, and should really pass the lead onto them. Unless of course nobody wants to use Scrobbles, in which case I'll either start working on one of my many other ideas, or I'll push Scrobbles development in my own direction until I have what users will actually want off me.

I also realised that developing the snippet editor was going to take me a long time, not through technical difficulty - but actually designing something that wasn't harder to use than simply writing some XML was actually quite challenging. It means less people will be able to create snippets if I don't do it, but I'm not looking at wanting a particularly large amount of users right now anyway, just some - so I can start learning off them.

Focusing on the important list, and developing a good pitch so the World of Warcraft users understand what Scrobbles is all about should be my priorities. With the enormous scope that Scrobbles has, I run the risk of overcomplicating the description and confusing people, and if people don't know what software does, they won't use it.

Before Release
-----------------------------------------------

• Core Scrobbles
  • Friendly URLs
  • Referal system for 'earning' queries
  • Embedding Data From Third Party Sites
  • Compressed Raw Data Queries
  • Compressed Batch Data Submission
  • Registration System
  • Sort the database out [fast fast fast]
  • Arbitrary Views (using Snippet System)
  • Ajax-Driven Page Requests
  • Windows Service for Data Generation
    • Page Generation Queue
    • Pending Data Crunching
  • Automatic Data Submission (Scrobbles App)
  • Security/Validation of All Existing Forms
• Third Party
  • Online submission of WoW Data
  • Heatmaps (location tracking)
• Community
  • Wiki
    • SDK for PHP and .NET (wrappers around system)
    • Service Documentation
    • Snippet Documentation
    • Family/Key/Value Documentation
  • Forums
    • Requests etc

Post Release
-----------------------------------------------

• Core Scrobbles
  • Javascript Snippet Editor
  • Create an intallation manager (DOH, re-invent wheel!?) 
  • XML based 'module' installation/uninstallation 
  • Filterable install list 
  • Rollback on fail
  • Create click-once installer for the installation manager
  • Cache At Query Level (in DB)
    • Add cached query pre-generation to Windows Service
  • Crunch data on commonly used keys, primary, secondary, tertiary (speed up queries)
  • WoWWebStats Emulation
• Logistics
  • Finance of new server
  • Add a 'status' to each account for payment info
  • Provide the means with which to easily pay (Paypal/Google Checkout/Direct CC/Phone??) 
  • Limit the number of queries each user is allowed and make this dependent on account status 
  • Third party limitations (payment system too?), to prevent DoS attacks
• Third Party
  • World of Warcraft Blogging 
    • Develop automatic user creation based on Scrobbles data
    • Choose a decent template for advertising revenue
    • Architect a data-driven pipeline for generation of blog posts based on:
      • Location 
      • Activity 
      • Player Character 
      • Party 
      • ???
    • Develop background process to generate blog posts
    • Use the World of Warcraft Armory to populate character profiles 
    • Add capability for characters to automatically post comments on each other's blogs for purposes of hilarity
  • World of warcraft Avatar/Signature Generation
  • Embeddable Widgets into blogs/etc
  • Generate Vista Gadgets from form
     

So, there we have it. I'm not going to copy and paste this entire list again - I'm going to focus on the things I have to do in order to release the product, and then I'll revisit the second list.

Saturday, June 07, 2008 8:49:17 AM (GMT Standard Time, UTC+00:00)

I did quite well this week to get as much done as I did (In my opinion).

My scrobbles page shows quite an improvement in the amount of code being written anyway.

Here is what I have left - I'm going to work on the Javascript based snippet editor this weekend, as that would be a massive win to acchieve. After I've done that, I'll look at replicating those stats found at WoWWebStats and probably come up with some easily implementable features to bring the user experience more in line with what the users expect.

• Javascript Based Snippet Editing/Creation (MAJOR TASK)
• World of Warcraft Scripts + Research  (WWStats)
  • Replicate their stats using the snippet system (Modifying Lua as needed)
• World of Warcraft Data Submittal
  • Create an online page (using the third party data API) to allow online data submission for World of Warcraft (IE: without using the client)
• Write a background service for all data crunching
  • Asynchronous Page Generation
  • Pending Data Crunching
  • Cached Query Generation
• Snippet format work
  • Add capabiity for third party websites to insert data into snippets (using public services
• Generic Server Work
  • Work out how to finance the purchase of a new server
• Security + Validation
  • Go through all pages and check all user-input for limits/etc [Make sure automatic validation is turned off]
  • Validate postbacks for modifiable data and security concernsLogistics
• Add a 'status' to each account for payment info
  • Provide the means with which to easily pay (Paypal/Google Checkout/Direct CC/Phone??)
  • Limit the number of queries each user is allowed and make this dependent on account status
  • Third party limitations (payment system too?), to prevent DoS attacks
• World of Warcraft Automated Blogging
  • Develop automatic user creation based on Scrobbles data
  • Choose a decent template for advertising revenue
  • Architect a data-driven pipeline for generation of blog posts based on
    • Location
    • Activity
    • Player Character
    • Party
    • ???
  • Develop background process to generate blog posts
  • Use the World of Warcraft Armory to populate character profiles
  • Add capability for characters to automatically post comments on each other's blogs for purposes of hilarity
• Online Community
  • Populate Wiki with 'general' information
  • Populate Wiki with stat family documentation
    • Write a script to do this automatically from the Scrobbles database.
  • Populate Wiki for API documentation
  • Create forums for snippet requests/application request
• Make client application for automatic data submission stable (One is already written, it just needs a lot of work!)
  • Automatic World of Warcraft upload (Make this more atomic)
  • Create an intallation manager (DOH, re-invent wheel!?)
    • Elevated installer process
    • XML based 'module' installation/uninstallation
    • Filterable install list
    • Rollback on fail
  • Create click-once installer for the installation manager

Friday, June 06, 2008 2:19:35 PM (GMT Standard Time, UTC+00:00)

With work on Scrobbles swiftly moving ahead, I find myself thinking about the inevitable and yet seemingly unreachable release date.

"Users are going to want this, so I should add it now.."

How many times have I now said that to myself? "Users are going to want to customize their pages", "Users are going to want to submit custom data", "Users are going to want to embed content in their pages", "Users will need snippets to be configurable so they don't need to write a new one for each 'key' or 'value'"... Each time I do this, it's for a good reason - I don't want to fail as a service, and therefore I need to be the best service around.

As mentioned previously, my main competitor (in the World of Warcraft arena anyway) is probably WWS (WoW Web Stats) - who have a mature, but nowhere near as flexible system as the one I have written. The keyword there however, is "mature". I took a look earlier and the wealth of information available from it is astounding. I can of course do better, and I do aim to write snippets which emulate the statistics that it throws out.

However, I then need to think about the groups of people who will be involved in these events, and think that perhaps they will want to combine their data and compare each other during raids, I need to possibly write a system that allows snippets to link to further in depth data based on a keyword in that snippet, I need to to write a system that allows users to create 'views' of their data between a user-defined period of time, with inputs coming through the existing snippet data. It needs to be really easy to create these views, possibly from templates so that data about a raid can be retrieved within a set period of time.

What about those casual users who are wanting statistics not about raids, but about their day to day activities? There is still a wealth of data that I am still not collecting, and I'm going to have to create a character of each class and profession in order to find out about them. It is absolutely terrifying how much stuff that I might "miss out" in the initial release of the software.

And there is the clincher, if I get it wrong, there is the chance that a future version of the WoW stuff might make previous data invalid. I can't be having that, so it has to be perfect, or at least - forward compatible to begin with - do I need a system for this??

What about those users for whom stats don't mean too much, I need to write that 'third party' website, WoWScrolls.com, so they can see the potential of throwing all their data at Scrobbles. (Public services are *awesome*). How do I achieve that? My colleagues at work have suggested that I use something like AIML to generate the blog posts and I can see their point, but it still leaves me with the daunting task of actually populating the database with "witty" phrases about each location, each task, each type of character and etc - nevermind creating the actual profiles from the data available at the WoW Armory.

My head is full of ideas, and getting that final feature list is not easy - because the moment I allow people to use Scrobbles they're going to start having even more ideas than I can deal with, and being the sole developer it's going to be very hard to keep up with the demand for features - nevermind technical support, complaints and all the normal day to day problems that come with running a website.

There is also that niggling issue, that releasing a service like this feels a bit like throwing down your cards at the end of a poker round, there is always the risk that your opponents might have a full house - and then what do you do?

Where do I call it quits? I could do with Scrobbles being out before the summer holidays so I could just prioritise my list of ideas and just work on them as much as possible before then, throwing it out in whatever form it has at that time. (Limiting the total users so I have time to assess server load and start thinking about monetizing the operation so I can spend more time on it - World of Warcraft is not the be all and end all of this system after all!).

I start to understand why games and software in general can often take such a long time to get out the door, there is always that one little thing that you just know the software will not be complete without. At some point, the users need to start leading the development strategy, and if their ideas conflict with mine - what on earth do I do then?

Monday, June 02, 2008 10:03:41 PM (GMT Standard Time, UTC+00:00)

Yet another evening of a work closer to the end than I was at the beginning.

I really didn't feel like starting, but I forced myself into it, wrote a small list from the main to-do list and got to tick a few things off it.

Seems setting specific tasks is more useful than I'd have thought. I also figured out some stuff which will help me at work tomorrow.

Monday, June 02, 2008 2:39:15 PM (GMT Standard Time, UTC+00:00)

Over the year of my contract with the university, I have been teaching myself how to do web dev - and coming from a background of professional desktop software development, moving to this world was surprisingly difficult.

I don't mean development as in the ability to put together a few pages about myself, or the ability to design a pretty website.I mean development as in putting together a full featured web application wtih the same features as an equivalent desktop version (if one was to be written).

My chosen area of learning revolved around ASP.NET because I'm already very familiar with the .NET framework. This turned out originally to cause me problems, because when developing ASP.NET applications you're given a lot of things you simply do not need, and you're given a structure to work inside of, that may or may not fit your end goals.

I found going to PHP and doing work in that helped, as I was given direct control over the process of form postbacks, and made to do everything myself. This gave me an understanding of the technology underlying ASP.NET and therefore the ability to work within the framework and create syncronous websites.

Obviously, the future is in asyncronous requests - a world without postbacks, and I've spent the past few months getting to grips with javascript, getting it to talk to the server, and architecting my web applications around a combination of syncronous and asyncronous behaviour.

When developing any code, I try to keep everything as organised as possible, to keep functionality in re-usable libraries, to keep presentation and business logic seperate, and this is where my main headache has been - in developing web software that is as organised as my desktop software. Trying to develop and utilise patterns across the entire web application so that once a few concepts have been described, anybody else could find what they were looking for if modifying/re-using any of this code.

Again, surprisingly difficult to do in a web application environment, as you are occasionally forced into mixing your logic and layout with this eery combination of Javascript, XHTML and VB.NET (Pick a fight on my choice of language and I will hurt you).

ASP.NET advocates the use of re-usable web controls, which can be slotted into web forms, just like in the desktop world, and I often find myself putting those in a seperate class library so I can easily have access to them in my visual studio toolbox. But then these controls end up referring to, or requiring certain markup to be available on a page (such as common dialogs inside of div, or web services exposed via the WebMethod system on pages). They therefore end up needing to be heavily commented, "do not use unless these thigns are present". They should probably be entirely private to the website itself - and even then there is the scope for abuse when you have over 50 seperate pages they could end up being used on. Can you say spaghetti code? No wonder PHP tends to be so all over the place - as you're not even forced into using any framework when writing it.

There are dozens of things like this, that the developer ends up just having to make a decision on. If you want to learn how to use individual components of code, learn how to use the framework, learn how to write the code, how to do little things, then there are books, there are websites to learn from. This has never been a problem.

If you want to learn how to put together, how to architect a solution that's elegant and forward-thinking, there is surprisingly little out there. It's left to the developer to work it out. (I'm talking a lower level than just N-Tier diagrams before anybody asks).

If I was working in a company that did web development, then I would no doubt be picking up on these things from my peers, who would have picked it up off their peers, who would have developed and learned from other people too. I am not however, and have ended up with my own style of doing things which may or may not be in keeping with other peoples.

A reflection on where I am now? I think this kind of learning is all very well and good, but it is harmful to productivity if you're doing it for your job. The amount of times I have now written this web client for the MeAggregator to a certain level of functionality before realising that I can't go any further without doing it all an entirely different way. If I was experienced in this field, I would have had it completed a long time ago.

I am lucky to work in a job where this is acceptable, and hope that the final product of this effort reflects the time I've taken to learn how to do things properly through trial and error.

Writing code is easy, writing huge amounts of code is easy - but writing large volumes of code that is understandable and well designed as an overall concept... it takes effort and knowledge.  The former I'm willing to put in, to extreme levels - but the latter can only come with time, and that's something we all wish we had more of.

Sunday, June 01, 2008 8:43:43 PM (GMT Standard Time, UTC+00:00)

Last week I worked quite heavily through the list, and towards core site completion. This leaves me with the tasks of building the third party websites, developing more client software and the creation of more user-friendly systems.

These are the tasks I have left to do, with a few more added. I aim to complete the ones in italic by next Sunday. I am however out all day on Wednesday, and at Microsoft on Thursday/Friday (I think), so time will be tight.

• Generic Server Work
  • Work out how to finance the purchase of a new server
• Snippet format work
  • Add capabiity for third party websites to insert data into snippets (using public services)
• Editing of pages (Ajax stylee, I already had a syncronous version done)
  • Hide/Show editing controls - based on user authentication
  • Renaming of Pages
  • Adding snippets to pages
  • Editing the inputs to those snippets
• Security + Validation
• Go through all pages and check all user-input for limits/etc [Make sure automatic validation is turned off]
• Validate postbacks for modifiable data and security concerns
• Write a background service for all data crunching
• Asynchronous Page Generation
• Pending Data Crunching
• Cached Query Generation
• Logistics
  • Add a 'status' to each account for payment info
  • Provide the means with which to easily pay (Paypal/Google Checkout/Direct CC/Phone??)
  • Limit the number of queries each user is allowed and make this dependent on account status
  • Third party limitations (payment system too?), to prevent DoS attacks.
• World of Warcraft Scripts + Research
  • Check out WowStats and see what stats they present to their users
  • Replicate those stats using the snippet system (Modifying Lua as needed)
• Javascript Based Snippet Editing/Creation (MAJOR TASK)
• World of Warcraft Automated Blogging
  • Choose a technology to build on (Going to build it myself in asp.net, it will actually be easier that way)
  • Develop automatic user creation based on Scrobbles data
  • Choose a decent template for advertising revenue
  • Architect a data-driven pipeline for generation of blog posts based on
    • Location
    • Activity
    • Player Character
    • Party
    • ???
  • Develop background process to generate blog posts
  • Use the World of Warcraft Armory to populate character profiles
  • Add capability for characters to automatically post comments on each other's blogs for purposes of hilarity
• Online Community
  • Populate Wiki with 'general' information
  • Populate Wiki with stat family documentation
    • Write a script to do this automatically from the Scrobbles database.
  • Populate Wiki for API documentation
  • Create forums for snippet requests/application requests
• World of Warcraft Data Submittal
  • Create an online page (using the third party data API) to allow online data submission for World of Warcraft (IE: without using the client)
• Make client application for automatic data submission stable (One is already written, it just needs a lot of work!)
  • Automatic World of Warcraft upload (Make this more atomic)
  • Create an intallation manager (DOH, re-invent wheel!?)
    • Elevated installer process
    • XML based 'module' installation/uninstallation
    • Filterable install list
    • Rollback on fail
  • Create click-once installer for the installation manager

Sunday, May 25, 2008 8:37:06 PM (GMT Standard Time, UTC+00:00)

In the interest of self organisation at work, I'm going to write a blog entry each week over at Redgloo with a list of the things I wish to acchieve at work. My boss can then comment and add things and I can cross them off as I get them done.

This is quite a cool idea, and I'm going to start doing it for Scrobbles as well, although maybe not as regularly. I'm dangerously close to finishing this project (at least to the point where people can start using it), and this week and weekend has seen a forced increase in productivity - the likes of which not seen for quite a while from me. I'm not entirely sure how I managed it, as at the beginning of the week I really didn't want to work on it. I just forced myself to by setting some tasks and got carried away from there. Let's try to continue this eh?

I tend to keep a text file on my desktop containing the current 'to-do' list, and as I have this, I can safely post the list as it stood at the beginning of the week, and cross out the things I got done this week also. I have backdated this entry to a week ago, to reflect this.

The way I'll do this, is each time I post, I'll post a revised version of the list, containing only the things I still need to do, and as I get things completed, I'll cross them out of the most recently written blog entry :) If I think of new things that need adding, I'll either edit the blog entry, or if enough has been achieved to warrant it, I'll make a new entry.

I'll highlight the 'next' planned work actions with italic lettering, and these will be things I aim to get done within a week's time.

• Write the public services for third party websites to retrieve/submit data
• Authentication
• FB Application like system, where users choose who is allowed to pull their data from the site
• Totalling queries
• List Queries
• Interval Queries
• User Info queries
• Generic Server Work
• Update database on server to catch up with development version
• Update server site to take on all this new code I've been writing
• Get the automated backup scripts working again
• Work out how to finance the purchase of a new server
• Snippet format work
• Add capability for user inputs to be given to snippets
• Add capabiity for third party websites to insert data into snippets (using public services)
• Editing of pages (Ajax stylee, I already had a syncronous version done)
• Hide/Show editing controls - based on user authentication
• Adding pages
• Removing pages
• Renaming of Pages
• Adding snippets to pages
• Moving snippets around pages
• Editing the inputs to those snippets
• Javascript Based Snippet Editing/Creation
• World of Warcraft Automated Blogging
• Buy domain (wowscrolls.com)
• Choose a technology to build on
• Develop automatic user creation based on Scrobbles data
• Choose a decent template for advertising revenue
• Architect a data-driven pipeline for generation of blog posts based on
• Location
• Activity
• Player Character
• Party
• ???
• Develop background process to generate blog posts
• Use the World of Warcraft Armory to populate character profiles
• Add capability for characters to automatically post comments on each other's blogs for purposes of hilarity
• Online Community
• Populate Wiki with 'general' information
• Populate Wiki with stat family documentation
• Populate Wiki for API documentation
• Create forums for snippet requests/application requests
• World of Warcraft Data Submittal
• Create an online page (using the third party data API) to allow online data submission for World of Warcraft (IE: without using the client)
• Make client application for automatic data submission stable (One is already written, it just needs a lot of work!)
• Automatic World of Warcraft upload (Make this more atomic)
• Create an intallation manager (DOH, re-invent wheel!?)
• Elevated installer process
• XML based 'module' installation/uninstallation
• Filterable install list
• Rollback on fail
• Create click-once installer for the installation manager
• Logistics
• Add a 'status' to each account for payment info
• Provide the means with which to easily pay (Paypal/Google Checkout/Direct CC/Phone??)
• Limit the number of queries each user is allowed and make this dependent on account status
• Third party limitations (payment system too?), to prevent DoS attacks.

Friday, August 03, 2007 12:28:39 PM (GMT Standard Time, UTC+00:00)

First off, a big thanks to Shirley for hosting the MeAggregator BBQ the other night, I think we can probably claim it as a success, although I do regret making a tit of myself trying to keep with Karsten on the chinese rice wine..

Life is cool at the moment, Redgloo is taking shape on the devlopment machine at work, and I'm doing my best to follow T-rex's expert advice outside of work. Got a party tonight, and tomorrow I'm meeting a girl who doesn't suck for a spot of one on one drinking tutelage .

Anyway, one of the things I aimed to do once I was back in Reading was to buy some of the musical instruments I used to play, and erm.. start playing them again in my free time. First up is the Guitar, as anyone can play guitar, even if you haven't owned one in four years :)

I popped onto Ebay and found a chap who was getting rid of his musical instruments so he could go jetting around the world. I found a nice simple classic guitar which appealed to me and placed some bids and ended up snapping it up for about £150 (Retails new at around £350).

I stayed at home late this morning so I could sign the paper and pick it up from Parcelforce, the seller was amazing and was very honest and forthcoming about the delivery. "I didn't get time to do it today, so I'll make sure to get it sent tomorrow", and he did! It took just under 17 hours for it to get from him to me and I had great fun unwrapping the parcel.

Well, it plays just like he said it plays. It's got a really rich sound, it's amazingly loud and clear compared to other classical guitars I've played in the past and I'm looking forward to learning some new stuff once I've gotten into playing it again.

Starting off with the familiar and over-played, I'll be jamming with The Rolling Stones and moving onto anything which has an interesting finger pick. I'd love to get Buckets of Rain (Couldn't find a video of Dylan doing it himself )Nailed before I visit my dad again, it's such a lovely piece of music and would sound really nice on something which didn't have those orrible steel strings O_o

And here's a picture of me with the lovely thing, Ok I'm posing a bit but I was actually playing something, honest!

Friday, July 06, 2007 8:08:26 PM (GMT Standard Time, UTC+00:00)

Graduation

Yesterday I finally got around to graduating, har har!

Started off as an ordinary work day at my new job at the University, meandering in, having a chat with Karsten and a nice meeting with Shirley before heading down town to meet my Dad. It occurred to me just before leaving the uni that I might need a shirt and tie for this whole graduation thing so I dragged my dad from the train station to John Lewis where in not too much time, and with not too much pain managed to pick a nice black shirt and white tie to go with the shiny black robe they make you wear for these occasions.

We went for a couple of pints at the Hobgoblin as we had some time to kill and I have to say, this no smoking ban thing has a lot going for it. The Hobgoblin is a far more pleasant place without people making you smell horrible while enjoying your pint of Old Rosie.

Skiiip to the London Road campus a couple of hours later and I was merry enough to consider graduating finally. Met up with Stears and his parents and proceeded to get robed in all our finery!

Taking a photo of myself. How clichéd :o

The graduation ceremony itself went smooth enough once I'd worked out where I was meant to be going. I took the holistic method and just followed other people who were considerably more likely than I to know where they were going. I'm not sure how people know what they're meant to do once inside the great hall, but thankfully there were people in front of me who I could copy.

The Deputy Vice Chancellor had a nice blue hat on, so when I went up to shake his hand I decided to show him the appreciation his dress sense so clearly deserved by telling him what a nice hat he was wearing. During his final speech when he was thanking various people for being useful I got a thankyou too, "And thanks goes to Robert for telling me my hat was nice". Ok, I'll not mention that again now, I promise - I was fairly proud about it at the time though :X

So, we're now outside the graduation hall and I'm with my mum and dad who haven't seen each other in like, a gazillion years and I was ready to jump between them to stop any fights that might occur. It turned out unnecessary and we quickly set about taking photos of us to be uploaded onto the internet when they finally make their way to me!

OMG HUNGRY!

Rather than stick around, we all decided to head down town for a beer and an indian, which we duely did. Managing to get through over two hours without resorting to talking about the weather even once! I can't say I recommend the curry house we went to, but the wine was ok and it wasn't too expensive either.

One of the tasks at hand was to get all of my stuff moved from my mother's car into my new house, which I move into tomorrow! It was done and mum gave us a lift to dad's hotel so he could get changed before we commenced more drinkage let mum start driving north again.

I took dad to BoBs and introduced him to the magical wonders of Jagerbull, ho ho ho :D

Dad with Jagerbull!

Aah how I've missed Jagerbull in the long week since I had it last! The Wetherspoons one really wasn't as strong as the DIY ones I was constructing at Glen's 40th birthday but that's probably for the best considering everything else I'd been drinking all day. Back to the Hobgoblin, lots of beer - picked up the passport I'd accidently left there earlier in the day and after a few more pints headed home. Good day, good company and hurrah, I finally have some letters after my name =D

Wednesday, May 16, 2007 2:52:35 PM (GMT Standard Time, UTC+00:00)

I am feeling so much better this week, the project I started on Monday went off well and I had a rudimentary BW2 Model viewer written within a few hours.

I didn't work on it too late and went to bed at a sensible time so I could get up early for a train to Reading for an interview with Shirley, Karsten and Ian Bland.

Obviously spending 8 hours on trains gave me more time to work on the file format, and I did just that. I've got about 12 excel spreadsheets of dumped information and I'm able to fully read every single group of data in every file, even if I don't know exactly what each group does.

I think I'm fairly close to being able to just write a very simple exporter from Blender to generate static single-textured models for use in a BW2 map, but I don't think I'm going to bother unless I feel a surge of enthusiasm - I don't think anybody really wants an exporter enough for it to warrant the effort (I haven't posted anything on the forums and I'm not going to, I don't want the hassle!).

I think the reason I'm feeling all positive, is that I've now got something to look forward to. I'm moving back down to Reading in July to start something new at the University. It will be sad to move on from DriveWorks, but I'm planning on staying in touch and keeping up with things!

I'm quite excited, I'm going to get me another nice little flat like the one I have in Lymm, and buy a guitar so I have something different to do when I'm at home, it's been a while since I owned one of those :)
Once term starts, I think I'll join the climbing society and maybe some others.

I guess I don't want to be all insular and self contained like I have been since the beginning of time. I've gained a lot of confidence after being with Emma for the past couple of years, I guess I stopped worrying what people were thinking quite so much. This breakup now gives me the chance to exercise this confidence and I'm going to do that by meeting new people and making some new friends.

Wednesday, May 02, 2007 7:26:43 PM (GMT Standard Time, UTC+00:00)

My new books arrived today, ShaderX4 and Games Programming Gems 6.

Normally I avoid any book with the word "Game" in the title because they tend to cater for the 14 year old who doesn't understand software dev but thinks it would be cool to make games for a living.

I am pleasantly surprised therefore that the articles in GPG6 are actually very interesting, with topics ranging from how to structure resources for efficient loading, implementing various scripting systems within an engine, component oriented systems (woohoo, no more silly diamonds!) and even a random chapter on NAT hole punching for those wanting to implement a p2p system for dummies :)

ShaderX4 is very technical and I'm beginning to wish I'd grabbed ShaderX2 at the same time to complete my collection. It doesn't dally about like ShaderX1 with masses of introductions and while I can see why, it would have been nice to have a basic HLSL reference on paper. ShaderX2 probably has something like that, being that it came out when HLSL was finally taking hold of the graphics programmers who realised that once you started pushing your pixel shaders past a hundred assembly instructions they were getting a bit unwieldy :p [And don't claim otherwise, you KNOW it's true!).

So I've got a bit of reading to do and then I might be able to pluck up some motivation to carry on working on my project. It reached a bit of a dead end this weekend and since the breakup (yarr, if you're entitled to that info, you can see it, if not, then you can't) I've been feeling too queasy to write any code :)

Now to listen to crazy Icelandic woman and Medulla, what a quirky album.

Sunday, February 25, 2007 2:20:30 PM (GMT Standard Time, UTC+00:00)

It's Sunday and I'm at work.

This in itself is not unusual, I quite often go into the office at the weekend to work on my own projects. It's a quiet location with a nice stereo system and coffee machine which make it 'not so quiet'.

This Sunday however, is different. Tomorrow the product I've been helping get into shape since leaving University last June finally goes out. DriveWorks 6 SP0.0 is about to be released to the masses and boy that is exciting.

We actually finished the product on Friday, but in order for it to go out tomorrow the installers need finalising, start menu shortcuts, executable names and paths, help needs linking in and final logos for the Installshield project need throwing in. (Ok, I admit I have one or two tasks in the actual product that need ploughing through, but I've known about them for a week now and just haven't gotten around to doing anything about them!).

I'm happy to give up my Sunday to do this - as my own projects have hit that brick wall where I can't seem to gather any motivation to push into them. Having spent the entirety of yesterday walking in circles and getting nothing done, sinking into a frustrated depression over this seeming incredibly likely - it's nice to have some goals and targets given to me to work with.

Therapy by work - it's the future.

Tuesday, January 30, 2007 9:46:16 PM (GMT Standard Time, UTC+00:00)

Been thinking about my career lately with regards to what I'm doing and where I'm going.

As I've always done, I've been keeping my CV and portfolio out there - making sure that the relevant people and agents have access to them so when the opportunites arise, and when people consider me to be 'worthy' of entering the games industry I'll join it.

Things have been hotting up lately, what with the recent job offer from that company in London and I think I'd be fairly happy if I were to get a games position by the summer.

I had a conversation with somebody today who's opinion I should probably respect when it comes to me 'being ready side of things. It probably didn't do my ego a lot of good but he said some things which made me happy about the relative state of my portfolio and the chances of my succeeding in the positions that I would like to do.

So, I've decided to let that happen - I'm going to let this person show me where to go, see what opportunities are available and if something comes along that I like, I'll cross that bridge when I come to it and deal with what's left of my lease and DriveWorks when it comes to it :)

It should be interesting what I end up doing over the next few months, it would be nice to have a blog of the months preceeding my joining of the industry I've been aiming for for the pas six years.

Stay tuned for some exciting stories about interviews and rejections ;)

Thursday, January 18, 2007 11:04:49 AM (GMT Standard Time, UTC+00:00)

Or possibly not.

I was called last night from a company based in London asking me if I wanted to come down for a job interview. As I'd already purchased tickets down to Reading this weekend and next weekend is cancelled because DriveWorks needs working on, on the surface it seems ideal that we conduct that interview this weekend.

Ah, not so. I'd unofficially decided that this weekend was going to be cancelled as well, because DriveWorks needs working on (anyone notice a trend here?) so I hadn't made any plans with the powers that be to take the day off work necessary to use these tickets.

So, I don't lie - its not in me, so the idea of taking a day off work under false pretences doesn't hold water. I rang my boss this morning who is currently in Reading and asked him for the day off and told him I was going for the interview.

I consider my current bosses to be friends and I really love the working environment, the amount of leeway given to me and their general attitude towards making DriveWorks a better place us to work. I'm not going to spring surprises on them like this, because it's just plain unfair.

I certainly hope they aren't offended by me at least scoping the place out - games development is somewhere I really want to be, and it's somewhere I've wanted to be for coming up on six years now, and while I'm maybe the happiest employee on the planet, I can't stop a fear of change stop me at least window shopping at whats out there.

It will take a lot to move me away from where I am currently, but it shouldn't stop me at least looking if opportunities come along?

So what do people think? What would do you tell your current employer if you had an impromptu interview like this and needed time off work?

Personally I think it's never worth lying, because when it catches up to you it always bites you in the ass.