Scheduling or Dispatching Work in Windows Azure

I have been developing applications for the cloud on Windows Azure since its initial CTP was released last fall at PDC. I just recently upgraded to the latest July CTP of the SDK, released publicly last week. They have made a major improvement in this version: the ability to have multiple web and worker role types per application. This has made it significantly easier for me to schedule/dispatch offline processing in my worker roles. I thought I would share an example of what I call the "Dispatcher Pattern".

Here is a sample of how I can now implement the "Dispatcher Pattern" using multiple worker role types for an application:

Requirements:

• My application needs to perform a background task, such as aggregating different pieces of data, very often.
• I want to be able to scale this work and ensure that I don't have multiple instances doing the same work at the same time.

Solution:

• A "Dispatcher" worker role will quickly determine what work should occur, inserting a queue message into a queue for each piece work. There will only be one instance of the "Dispatcher" running, so you can be confident that the same items are not being inserted into the queue multiple times. The "Dispatcher" role can queue work as frequently as needed.
• A second role type actually does the work. This role type, "Aggregation Role" in this case, pulls a message off the queue, does its work, and repeats. While you only needed one instance of a "Dispatcher", you can scale to as many of this role type as you need to keep up with your queue.

I think this is a very useful pattern when it comes to needing to initiate work to take place in your worker roles in Windows Azure, especially when the work is not triggered from an action taking place in a Web Role. You can now have multiple worker role types in an application, and scale them independently.