RSSBandit - how would I do it?

Dare posted up a comment on one of my posts about RSSBandit and the problems I was having. He asked what I would suggest to fix the problem.

Before I make my suggestions I want to point out that RSSBandit is free software and Dare devotes serious time to maintaining the software so its not fair for me to criticise his work. The code itself is awesome, it just has a problem when it reaches the number of feeds that I was trying to aggregate.

That said, I do have a few suggestions. The first is to make UI updates high value, there are lots of locations in the code where a call is marshalled across to the UI just to update a single statistic. This is fine for your typical feed load but when you have a thousand feeds the system spends a lot of time synchronising threads for you.

I would suggested having some kind of statistics system where periodically (on idle) the treeview nodes are updated all in one go. To compliment this it may be more efficient to store pre-computed statistics along with the applications data files so that when the program is loaded it shows what it last knew the statistics to be.

The background thread would then just walk the feeds that have been updated since the last UI refresh and update that statistics file. Diagramatically it would look something like this:

By decoupling the UI update from the statistics refresh you give yourself the ability to optimise each algorithm seperately. For example, when refreshing feeds (not necessarily done of this background thread – I know) it may be more efficient to group hosts together.

I subscribe to quite a few blogs at http://blogs.msdn.com but like a lot of hosts it limits you to 2–4 simultaneous connections as defined in one of the HTTP specs. So if you do multithreaded fetches of RSS feeds you could essentially have a bunch of then in a wait state while they are hung up waiting for the web-services to hear their request.

The feasability of doing all of this may be reduced if Dare upgrades RSSBandit to work with Windows Vista because it can exploit the RSS platform that is part of Internet Explorer and it might be more appropriate for Dare to make us greedy feeders wait until then.

Rule and Context Based Artificial Intelligence

I stumbled across this entry by Wesner Moise about intelligence. I was particularly interested in the stuff about Stephen Wolfram who developed Mathematica. I haven’t used Mathematica but I intend to correct that over the next few weeks but some of the theorhetical stuff that Wesner pointed out fits nicely with some things I’ve been pondering on over the past twelve months.

Enterprise applications are all well and good, but they aren’t terribly interesting to write most of the time, so I entertain my brain from time to time with subjects that are quite frankly beyond my mental reach. One of those areas is artificial intelligence.

While those close to the AI community no doubt are proud of the work they have done I can’t help but feel that the majority of the population is unaffected by AI. Sure there are specialised applications in the field such as lending evaluation, entertainment and related areas such as speech recognition and linguistics but we haven’t uncovered some central truth yet, or atleast I don’t think we have – remember, its beyond my mental reach.

To quote Wesner on Jeff Hawkins work:

Jeff found that observation surprising given that sight, hearing and touch seemed very different with fundamentally different qualities. He concludes the that human brain is fundamentally memory-driven machine using pattern recognition techniques—essentially a rules-based machine.

Are we trying to find a complex solution to a simple problem? I subscribe to the theory that our brain is a rules-based engine (probably less sophisticated than the one in BizTalk) but I also think that looking at the rules alone means you miss quite a big piece of the picture.

I believe that humans are essentially a series of devices attached to an organic network. The organic network pushes signals to and from devices and our brain. Every time we get some kind of sensory input we evaluate that input and if the evaluation turns out to be true then we take some kind of action. In the human body that might mean the release of a chemical from a gland for example. Diagramatically, this is how I think of it.

Lets apply this to some software. Lets say we are writing an agent that plays an online game. For the sake of simplicity lets choose a text-based MUD. If I was to write an agent/bot to play this game I could build the sensor that feeds in the stream of text coming over the socket where it could be evaluated by a rule.

In this case that rule could be a regular expression looking for some kind of pattern in the text, if it found it then it could send some data back and get ready for the next round of input. Here we can visually see how we would build those rules up.

Assuming we were handling the text correctly this little algorithm neatly defines how we would handshake with the game. As the system was trained to break up the rules so that there was a reduced possibility of one getting invoked in the wrong context. Our brains do this automatically but with a piece of software we might need to get explicit about what context we are in. I see it like this:

I see a system like this sitting on top of a whole heap of devices which deal with the procedural aspects of getting the job done and leaving this higher-level system to orchestrate them. I see great applications in robotics and semi-autonomous systems. Its certainly interesting to think about - the possibilities are both amazing and scarey.

Dell Customer Service

I was reading Ambrose’s post on his Dell customer service experience and it echos the thousands of very public cries of anguish that are happening out there in customer land.

When I was at code camp I looked up into the audience and noticed that overwhelmingly Dell was the laptop of choice for developers that attended. It will be interesting next year to see how many people are still using them.

The Dell that I have now may well be my last Dell. The machine itself is great (its got a few quirks that I haven’t quite worked out yet), but everytime I ring Dell I feel like I’m talking to a machine that doesn’t really understand or appreciate my needs.

I’m not alone. Dell may find themselves with a wave of customers leaving them unless they start getting localised support staff. First it will be the advanced user base, then they’ll start convincing their family and friends not to buy from Dell and finally the general consensus will be that Dell is no good and they will start loosing corporate clients - which is where it will really hurt them.

It’ll take a while, but once it pronounced enough for them to notice it may be too late. Its one of those frog in the pot things.

TechEd Mediacasting - Spot the Joke!

Frank Arrigo has posted up a link to a page on the Australian TechEd site about how Mick and Cam are going to be producing little vids (and presumably audio) casts of the experience at TechEd.

Mick and Cam are Australias (and the worlds) latest media moguls and run The Podcast Network.

The boys have already posted up some vids including one of Chuck talking about how a lot of the content for the event has been chosen by the local .NET developer community and that there has generally be quite a bit of community involvement.

My favorite video however is the one of Frank in his office talking about what his sessions are. Go and watch it now and think what movie scene it reminds you of.

Thats right! Its like that scene from Spys Like Us with Dan Aykroyd and Chevy Chase when they are in the villiage performing the operation on one of the tribe members. If you have seen the movie you’ll know what I mean!

P.S. Thanks for the spell check Frank!

Cold in Canberra

It is cold in Canberra today. It rained most of last night and it brought the temperature right down today. It actually got cold enough to not only snow up on the mountains around us but also cold enough to snow around the centre of the city – although the snow melted almost as soon as it hit the ground.

This is one of the reasons we moved to Canberra – I love the things that happen in the cold climate. If I go outside tomorrow morning and find snow on the ground I’m declaring a snow day!

Readify: A copper company.

Darren and Graeme have both posted up a few stories about the virtual company lifestyle that seems to be evolving at Readify these days.

I had a bit of a chuckle about Darren’s picture showing how we hooked Bill up into the conference call after he had a few problems with Skype. I knew Bill was having problems before the meeting started but the solution worked perfectly

I was actually late to the conference call because I hadn’t sync’d up my smart phone since the meeting was booked and I had to be politely reminded to attend.

But I didn’t really want to talk about telephony hacks or my lateness to a meeting, rather I really wanted to talk to the concept of a virtual company.

When I joined Readify our internal computing infrastructure was struggling. Wierd stuff would happen to our servers from time to time and they’d just go offline. Most people in the company had the keys to the kingdom (administrator password) and it was pretty much open slather.

A little while later Bill Chesnut joined the team and in some of his idle cycles took responsibility for the infrastructure with a lot of support from both Graeme Armstrong and Dan Green to get it right.

Ever since then there has been precious little unscheduled downtime in the critical systems such as e-mail, VPN and file servinig (well done Bill).

An interesting characteristic of the system we have set up is that no-one has physical access to the system under normal circumstances (Bill does if he is swapping a disk or what have you) as its housed in a data center and everyone gets access to it over the Internet, even from the office in Docklands.

When I am not working on a client-site my preferred virtual office is at Starbucks. While most would question the quality of the coffee I find it good enough for me, and the hot chocolate is really good. There is a power point at several points within the shop and very confortable chairs as well as a few tables here and there.

They actually have an Azure hotspot which I can use an affordable Boingo acount with. We are getting some iBurst cards very shortly which will make us more mobile and help us cope with clients that don’t provide Internet on tap at the desktop.

Its all starting to come together, I probably make two business related Skype calls each day and I’m close to being able to drop my mobile phone plan down a notch.