Monthly Archives: March 2013

Windows Azure Websites Background Worker with Node.js

One of my pet projects at the moment is building a personal productivity application called Smashing Pomodoros. I’m currently accepting registrations for a BETA launch at some point in the future. It is a labor of love because I ultimately want an application I am happy to use myself. However, I didn’t want to talk about Smashing Pomodoros specifically in this blog post, rather I wanted to talk about the fact that I am using Windows Azure Web Sites to host the application and a technique you can use to implement a background worker.

Background

Before I get into the technical details, I wanted to provide some background (no-pun intended) on why this significant. When Windows Azure first went online Microsoft provided Cloud Services where a developer could divide their application up into web and worker roles. Web roles could be used to respond to HTTP requests, and worker roles could be used to perform long running and intensive operations. This web and worker compute model still persists in Windows Azure today and if you are running heavy background processing operations it is definitely worth a look.

That said, many web-sites have very light background processing requirements (like periodically sending out an e-mail to a user) and so that addition of a worker role to a Windows Azure deployment that might add up to as much as an extra $50-$100 in hosting charges wasn’t that appealing to many people. It turns out that in Windows Azure Cloud Services it is possible to provide worker role characteristics (a durable process that doesn’t start when IIS recycles) inside a web-role by injecting code in the WebRole class (derived from RoleEntryPoint) at the root of a Cloud Services web-project. This is a great way to save some costs when using Cloud Services and you only have a light background processing load.

More recently Microsoft has added Windows Azure Web Sites to their offering which increases the speed with which development teams can push out a new version of their application. The problem with Cloud Services (as opposed to Web Sites) is that each time you push out a new deployment to Cloud Services the underlying platform provisions a brand-new virtual machine. This can add 5-10 minutes to deployment times which can be painful for debugging and recovering from issues.

So Windows Azure Web Sites with sub-minute deployment times (for your typical basic web-application) has grown in popularity despite its preview status. The problem is there is no background worker capability built into Windows Azure Websites … or is there?

A Node.js Interlude

Most developers would have heard of Node.js by now. It is a program that allows you to execute JavaScript code and build network enabled applications, such as web-servers. Node executes the JavaScript using Google’s open source V8 JavaScript Engine which is built into Google Chrome and is very fast and stable.

Initially you could only run Node.js on a Linux environment but it has since been extended to work on Mac OS X and more recently Windows! Individuals from Microsoft contributed to this effort in a number of ways, including the development of “iisnode” which is a module for IIS (the web-server built into Windows) which takes some of the pain out of hosting a Node application by wrapping it in some of the reliability features within IIS. Not long after this work was done iisnode got picked up by Windows Azure as well as other hosting providers who wanted to support Node on Windows.

Node.js is used in a number of places on Windows Azure, from Windows Azure Web Sites to Windows Azure Mobile Services and Microsoft have published an SDK specifically to support developers who want to work with Node.js. Getting started with Node.js on Windows Azure is easy, just following Microsoft’s instructions and you’ll have hello world running in no time.

Combining Node.js and ASP.NET

What some people may not realize is that it is possible to combine Node.js and ASP.NET in a single application running on Windows Azure Web Sites. By convention Windows Azure looks for a server.js (or similar) file in the root of the website. If it is present then it assumes that you want to use Node.js and it will do what it can to generate the appropriate configuration on the server side, all your code needs to do is to listen for requests forwarded on from iisnode which fronts the Node application.

However, running Node or ASP.NET are not mutually exclusive. Because all HTTP requests come through the standard HTTP.sys pipeline. It is possible to hive off your Node.js code into a seperate folder and then instruct iisnode to execute in that particular context.

The thing is that your Node application is just a standard node application so you can do all the usual things (provided you stay within your sandbox) that you would expect to be able to do, such as fire events in the background independent of whether there is a HTTP request in play.

Running a Background Worker with Node.js

Having set up your Windows Azure Web Site to support both Node.js and ASP.NET, you are now ready to use your Node application to perform background processing. This is something that Node excels at. All you need to do is set up a timeout that periodically takes some action, the code below is just a starting point that I have inside Smashing Pomodoros which returns some heartbeat information from the Node process.

When you first hit this code inside your application you’ll notice that the runtime value is null. This is because until you hit the application URL the hosting infrastructure doesn’t even bother spinning up the code. This can be a problem, but there are some workarounds.

Avoiding Idle Timeout

Some of the reliability features in IIS will actually make it difficult for you to implement any kind of background processing. Firstly, IIS is configured with an “idle timeout” where if your application does not receive a request with a specified period of time it will shutdown the application pool hosting your code, and in this case that includes the Node.js code. This makes sense in a heavily shared hosting environment you wouldn’t want one application spinning up consuming memory even when it isn’t servicing any requests. Unfortunately that doesn’t take into consideration any background processing that you want to perform.

One way to avoid idle timeout is ensure that your application is getting a steady stream of HTTP requests. You might be able to do this from the Node.js application itself but that doesn’t necessarily solve the problem of when your site is migrated to another server within the Azure Web Sites cluster and is yet to receive a request to kick things off.

To work around this you really need an external solution to hit the web-site and trigger Node.js to start. A recent addition to the Windows Azure Web Sites service offering is URL or endpoint monitoring. If you scale up your Windows Azure Website to a reserved instance you can specify an endpoint that you would like the monitoring infrastructure to hit every five minutes.

Using Windows Azure Mobile Services Scheduler Instead

Another option to replace all of this trickery is Windows Azure Mobile Services (WAMS). Don’t be put off by the title. Whilst WAMS is designed to make building a backend service for mobile device applications easier it has a number of services that you may find useful for web-based applications as well. In this case WAMS includes a scheduler which allows you to execute chunks of JavaScript code which can optionally call out to your Windows Azure Web Site hosted application. I recommend checking out WAMS, and the WAMS scheduler specifically.

The only issue that I see with WAMS is that unless you plan on using it for any kind of mobile device support it could be a bit of a sledge hammer just to get scheduling.

Summary and Hope

In this post I’ve shown a few tricks to try and get a level of background processing going within Windows Azure Web Sites, accepting that idle timeouts and application pool recycling means that you won’t be able to continuously execute code, despite being able to ensure it keeps running either via the new monitoring facilities or Windows Azure Mobile Services.

I really hope that Microsoft comes up with a lightweight scheduling/background worker option for Windows Azure Web Sites so that we don’t need this kind of trickery. Ultimately I think that this facility needs to be built into IIS itself so that developers can specify “jobs” in the web.config file that either run continuously or on a specified schedule. I would fully expect that in a Windows Azure hosting context this execution time would contribute to the CPU time against a Windows Azure Web Site.

Finally I strongly recommend checking out Windows Azure Mobile Services. I think that in the long run I think Windows Azure Mobile Services needs to be broken up into multiple components which can be mixed and matched. I would really love every web-site to have scheduler support without having to take on all of WAMS. For the time being though it is a nicely integrated set of services.

I hope this post helps. Enjoy your adventures with Windows Azure!

Google is shutting down Google Reader.

I saw this come across my Inbox today. It is a list of applications and services that Google is shutting down in the coming months. Among the list is Google Reader. Most of the readers of this blog will be familiar with Google Reader as a web-based RSS/ATOM feed aggregation app with a number of features around feed discovery etc.

I came to use Google Reader after RSS Bandit (my preferred desktop aggregator at the time) failed to handle the load that I was throwing at it. Google Reader filled the void and was probably one of the biggest anchors for me in the Google eco-system. For this reason I am surprised that Google is shutting down the service – citing that it has a dwindling user base.

Is RSS dead?

Perhaps with the evolution of social media (Twitter, Google+, Facebook etc) the need for RSS aggregators has diminished. Basically we now rely on our peers to find and share content via those channels and curate our feed more dynamically.

As a blogger I think that this will have an impact on my readership as I suspect many actually only see my posts because they periodically show up in their Google Reader account. I imagine that the shutdown of Google Reader will likely have a massive impact on professional bloggers.

The real test will be whether all the users of Google Reader bother to re-establish themselves with a new aggregator or whether they’ll just give up on the RSS eco-system as a result.

Impact to Application Developers

My primary means of reading blogs is via Google Reader and a number of applications that interface directly with it, namely Wonder Reader (Windows Phone) and Nextgen Reader (Windows Phone and Windows 8). I’d imagine that the vendors of these applications are scratching their heads right now.

If I were them I would be quickly looking to push out a release that allows me to migrate from Google Reader to another platform of my choice, where the mobile device does the work of copying across the feed and read marks.

Business as Usual for Google

This is not the first time that Google has abandoned what was a fantastic product (albeit with limited adoption). I was a huge fan of Google Wave which the company discontinued. I thought that Google Wave had the potential to really change the way communicate electronically.

It makes sense for a company like Google to continually flush out the products that they think don’t have a future to free up capital for other initiatives. I just wish that they wouldn’t keep doing it to the things that I love using.

Can Google hold onto me as a user?

I’m not convinced that Google is going to be able to hold onto me as a user in the long run. I have got a foot firmly planted in the Microsoft eco-system using a Windows Phone device, SkyDrive, Outlook.com and Windows 8. I have a Google Nexus 7 and use Google Chrome and casually use Google+, but without Google Reader I might forget to login.

I suspect that Google has done the numbers and know that they’ll come out on top in the long run so whether I am a user of their technology is probably not that important to them.

Protests and Invention

Bloggers are a fairly vocal minority group. I expect many of them will protest pretty loudly at the demise of Google Reader before being forced to look for alternatives. You might even see a few community aggregators pop into existence, or perhaps a start-up or two. Existing aggregators will soak up most of the user base that decide to persist with RSS.

Is working from home now considered a right for developers?

WP_20130305_001

This is my work desk at home. When Shelley and I purchased our new home a few years ago we earmarked a space within the house as an office. Originally it was a second living space that we felt would be wasted as a formal dining room or second lounge. About 6 or so months after we moved in we got a cabinet maker to come in and put in built-in shelving, desks and draws.

In the picture above I have access to four screens to do my work. Three 24″ (approx) screens and a smaller screen which is actually a Samsung Slate PC which I use when I am out and about and don’t expect to be doing much coding. The result is a very functional developer workstation with better facilities than you could find in most regular office spaces.

I consider myself to be lucky in this respect. I’m lucky that I have a wife who allows me to commandeer a corner of the house for office space, I’m lucky that I am working with an boss who trusts me enough to work independently from home, and I’m lucky that the infrastructure that the company needs to support remote work.

Alas, I can’t work from home all the time. The reality is that I work within a professional services businesses, site visits and face to face meetings are part of the deal. So even though I can expect to work from home on average 2-3 days per week, there might be weeks where I am in the city every day. You just have to roll with the punches.

In the future, I expect remote work to become even more prevalent. I can foresee a time when even clients don’t want you to come into their offices because they simply don’t want to spend the money on floor space for a transient workforce (we are starting to see this already). Bring it on I say, but then I’m ready for it, and I’ve been living it for a good number of years.

Some companies are ill equipped to handle it. They need to change or they face being out classed by businesses with increased agility when it comes to hiring and retaining talent.

Before I close of this post, I think it is worth making a point about agile development practices and the benefits of co-location. Most agile enthusiasts will tell you that it is better if the team is co-located. This is true to the extent that given an ideal team, you will probably get better output out of them if they are co-located rather than remote. But sometimes to get that ideal team you need to provide conditions that they are happy with. If they aren’t happy travelling 3 hours every day just to be interrupted by pointless meetings then you might just need to live with the ideal team where some or all of the team members are remote. I cannot deny the benefits of regular team bonding sessions, whether it be going out to lunch or other social gatherings.

At all times you need to use common sense and allow some flexibility, and that is why this announcement about Yahoo! work conditions is so surprising.