Archive for July, 2010

Weeknotes: Talks, Open Correspondence, XMPP

Sunday, July 25th, 2010

I gave a talk at the Oxford Geek Nights about Open Correspondence and letters. At some point I really ought to learn how to give talks. Anyhow Russell Davies was the main speakers and he showed how you could make physical objects from data derived from social networks. (He has a marvellously sane post about the Raoul Moat facebook page.) Anyhow its gathered some people who are interested in contributing. Now I’ve finished the book, I’ve got more time to make changes to the codebase which urgently needs it. Finishing off stuff really. Then making the real changes.

Accounts has been slightly on hold since the wages needed to be run and I didn’t see that accounts or operations would be happy with fugures potentially changing.

The main project has been setting up a notification service to set up the service layer correctly. I’ve finally got the server working so I’m just building a framework. I thought of porting parts of djabberd projects into PHP but I’m just  looking at parts of it but XMPP is certainly a useful tool in getting machines to speak to each other and to develop event driven services.

Tags: , , ,
Posted in projects, weeknotes | No Comments »

Finding a space for NoSQL

Tuesday, July 20th, 2010

ReadWriteWeb have a post on NoSQL (again?) by Audrey Watters which is a brief overview of the area.  The original post points the Heroku blog, where Adam Wiggins outlines the uses of NoSQL. I’m not an expert by any means but use Redis on a daily basis with the Rediska PHP library. I remember having an argument with the IT director when I originally proposed using Redis but I’m glad that the gamble has paid off. The caching system that uses is now far more productive than the earlier version.

Our base is database is MySQL which I like a fair amount for what we do with it but all I needed do was to cache some data. The scripts write a fair amount of data to the cache and then there is one read process to read the entire list before updating the main database. At least I know that the data has some sort of security. It is not a panacea or similar cure all but it does have a place in development for certain jobs.

Best tool and all that?

I can understand why Twitter are not using Cassandra in the main service but are still using it for other projects.  For now. Systems and priorities change and perhaps it will happen in some way.

Despite its meteoric rise, NoSQL is not the answer to everything. It does have a useful place though.

Tags: , ,
Posted in Information Retrieval | No Comments »

BBC’s use of Semantic Web technology in World Cup

Tuesday, July 13th, 2010

Just caught this story on ReadWrite Web about the BBC website’s use of semantic web technology during the World Cup.  Jem Rayfield explains more on the BBC Internet blog about the use of technology.

I’ve still got a fair amount of reading to do but this is the sort of project that makes me rethink the Open Letters project and how it could be used by other sites. It has also given me food for thought for work as well.

Tags: , ,
Posted in Open Knowledge | No Comments »

Weeknotes: documentation, prototyping and cats

Sunday, July 11th, 2010

I’ve spent most of the week either trying to persuade colleagues that rewrites are needed to existing services. I’ve also finally managed to get the initial promise of working from home so hopefully I’ll be able to get the rewrite started on the “quiet” days away from the office. (Although the cat can drive me nuts before she goes to sleep at 10am).

Still working on the accounts project which keeps unravelling a series of underlying problems. Most of them we know about but they appear in all sorts of odd places.

Assuming the world doesn’t fall on my head next time I’m in the office, I’m going to try and spend the day at home on a “Fedex” day. I’m taking the notion from an issue of Wired where they were talking about different ways of working and Atlassian mentioned “Fedex” days where you spend a day building a prototype. What I’d really like to get prototyped is the service bus / queuing system. So fingers crossed.

The impetus came from updating the disaster recovery documentation and writing the first department of the service status documentation (which I wrote after getting the last bit of debugging finished). I know that documentation is not everybody’s favourite thing but I find it useful in rethinking the system and making sure it fits together.

I’ve made time to rewrite the load function for Open Letters. I’ve got the document building the letters in XML and written a rough upload script. Next task is to rewrite the main.py script, test the XML loading and then finished tidying up the initial document.

I’m also looking forward to Textcamp so it’ll be great to get the load finished (as it normalises the function) and get on with doing a presentation for the camp.

I’m also coming to end of writing my book on children’s fantasy. Whilst not technical in an IT sense, I’m thinking of the next project on the New Weird and how to use IT to visualise influences and timelines. The one that worries me is archiving necessary web pages for the research which I need to look towards as I’m not sure whether it is technically illegal.

Tags: ,
Posted in Open Knowledge, weeknotes | No Comments »

Weeknotes: maintenance, and Dickens

Sunday, July 4th, 2010

It seems to be maintenance season again.

Still carry on with the accounts systems and doing some work to those systems for most of the week. It is a slow job but I would rather spend time getting it right rather than rush something ut and spend the next year patching it because we rushed it rather than a need changed.

The rest of the week is spent either developing some new functionality for the admin department or thinking about revamping the existing services. Most of them are fine but a lick of paint and some further optimisation to take care of unanticipated needs wouldn’t go amiss. I suspect that maintenance isn’t high on most developer’s agendas but in a moving and growing company, some systems begin to be left behind when their use either changes or the company outgrows the service. The challenge is trying to minimise user frustration whilst getting the new version out and finished. Mm, time to work on the ‘soft skills’ of people methinks.

In the meanwhile, I’ve created an XML file of letters of Dickens and now am just changing the loading script for Open Correspondence so that there is a normalised way of loading the data into the database. The fact that everything was predicated on one file was annoying me so I made the time to change it. The next thing is to dive back into TEI lite and rework the file so that it fits into an already definted schema. (I don’t see there being any point in this case in trying to create something new as it should be unnecessary.)

Posted in weeknotes | No Comments »