Tag Archives: readify

Timing Data Imports into Azure SQL Databases

We often help customers migrate their applications to the cloud. For small applications with small databases this can be pretty straight forward, however as the complexity and volume of data that the application deals with grows you need to do a little bit more planning.

With Azure SQL Database the principal way of importing data is via a *.bacpac file which contains all of the database definition and data. You can upload this file into an Azure Storage Account and then trigger the import via the web-based UI upon creation of the database.

The size and shape of the data you are importing, combined with the service tier that you chose can have a dramatic effect on import time. If you are planning a time critical cut-over then you are going to want to test how long this migration will take in advance, and if necessary step up to a higher performance tier during import and then move down to an appropriate tier for your usual transactional workloads.

As you can see from the following chart, a given database will import in significantly less time if you do it on a higher performance tier.


The exact timings that you achieve will of course depend on your specific database. To help establish those timings you can use a script that I created to collect the data points that went into the above graph – I’ve posted it up on GitHub under the Readify account.

AzureSqlPerformance is a PowerShell module that takes a bunch of arguments and uses them to create a Azure SQL Server, and Database at a specified performance tier, then kicks off an import and times how long (to about 10 seconds accuracy) it takes for the import to complete.

Feel free to check the script out, and if you can make some improvements to the implementation submit a pull request. I kind of stopped working on it as soon as it did what I wanted 🙂

By the way, you can find some of the other pet projects others at Readify have contributed to over at labs.readify.net where we are trying to collect some of these kinds of resources.


Three Windows 8 Application Archetypes

Over the past six months I’ve been working with various Readify customers helping them understand how they can take advantage of Windows 8 within their organisations. In particular I’ve been working with commercial and/or government entities who are looking at deploying the new operating system and want to better understand what benefits the new application model might provide.

From Windows 7 to Windows 8

For those of you just getting familiar with Windows 8, it is important to understand that Windows 8 builds upon the foundation of Windows 7 in that if the application worked with Windows 7, it’ll work with Windows 8. Windows 8 represents a superset of functionality of Windows 7 (as you would expect). One of the significant things that Microsoft added to Windows 8 was an additional model for building applications – these are called “Windows Store Apps” but the title is somewhat misleading because you can actually build these kinds of applications and deploy them within an organisation without publishing them on the public Windows Store. For the remainder of this post I’m going to refer to this new kind of application as a “Modern App” and the old-style of application as a “Legacy App”.

Impact of Mobility and Form Factor

There is no doubt that mobile computing has had a major impact on our society and I think I could argue that the trend hasn’t even hit the corner of the hockey-stick yet. We are seeing increased adoption of touch-based tablet devices and in the consumer space these devices are displacing traditional family computers. The problem is that the legacy approach to building desktop applications just doesn’t scale down well to these kinds of touch screen devices – and so a new approach was needed.

Microsoft has developed the new modern application framework as part of a larger overall rethink on how users want to interact with their PC today and into the future. Microsoft now sees Windows running on high-end engineering PCs used for CAD and software development all the way down to handheld devices used for casual access. Prior to Windows 8 these lower end devices represented a problem for Microsoft because their OS couldn’t be scaled to run efficiently on this kind of hardware. Windows 8 changed all that – the operating system is now significantly more friend on battery consumption and with the modern UI of new Windows 8 applications it works much better on a smaller form factor (whilst maintaining its mouse/keyboard friendly capabilities).

Taking Advantage of the Platform

While it is possible for enterprises to roll out Windows 8 and not redevelop any of their internal business systems I recommend that organisations take a look at their existing business systems and think about the kinds of systems that could benefit from the new platform capabilities. Any sizable business is likely to have dozens of applications that sit within this category.

No doubt organisations will already have Windows 8 application projects underway with more in the pipeline. One of my concerns is that customers are going to start building large monolithic Windows 8 applications which span multiple departments  and have relatively large footprints. One of the benefits of the new Modern App model is that you break applications down into smaller pieces and link them together with techniques like protocol activation and sharing contracts.

The Three Archetypes

What is really needed is a shorthand for talking about the kind of application that is being built to put a fence around the functionality that should be contained within it. After a good period of time working with customers, and thinking about the problem I think I’ve come up with three different application types. They are:

  1. The Activity Hub
  2. The Gateway Application
  3. The Role/Departmental Application

What I’ll do here is expand on each of them a little bit.

The Activity Hub

The Activity Hub is a “one-of” kind of application within any organisation (where an organisation might be an entire business or a significant business unit). It is the one application that every employee/contractor uses several times a day to keep abreast of what is happening across the organisation. In an increasingly mobile and distributed world it is the equivalent of the notice board in the coffee room.

Many organisations that are getting started with Windows 8 have this idea to replace the Windows Store with their own company specific store. Whilst this is a laudable goal I think that it is a case of re-inventing the wheels. I would rather see organisations use technologies such as Windows InTune or System Centre Configuration Manager to deploy applications to user devices and instead re-purpose this hub as a central point for discovery of business intelligence, notifications and other generally available information.

The Gateway Application

The gateway application is a thin layer over an existing line of business application. Rather than expose any really task-like functionality this kind of application is focused on getting data in and out of the existing system via the various shell extension points in Modern Apps (such as sharing contracts, file pickers, search etc).

Gateway applications are important because they represent a relatively cheap way to get data into the hands of people that need it without fully replacing the existing application.

The Role/Departmental Application

The role-focused or departmental application is a fully featured application focused on helping a particular kind of user within the business key their job done quickly. It might include forms for capturing data, or take advantage of capabilities such as NFC (Near Field Communication) for bringing up records for physical items.

Role/Departmental applications represent a bigger investment that gateway applications and would probably be developed initially for those people who need to be mobile (whether that is inside the building, or out of the road). These applications become a rethink of how users work with backend systems on modern devices which might work as a desktop one minute, and as a tablet the next.

Basis for Estimation and Higher Success Rates

One of the interesting side effects of breaking high level application requirements into these three categories is it gives you a practical way to weigh up the likely implementation costs. Most activity hub development projects are going to be very similar in size, as are gateway applications. Departmental applications might be the area of most significant deviation but I believe that organisations should limit the size of these applications to increase the speed with which they can be pushed out but also realise some of the benefits of building smaller applications focused on very specific areas and then linking them together.

For example, you might have a common concept of a customer across two departments, and instead of building screens for customers in two different applications you build a single “customer micro-app” which both departmental applications can launch into when necessary.


I think this is going to be the year of Windows 8 adoption within the enterprise. As part of that I expect many businesses to start embracing a more mobile style of working where all staff take their devices with them wherever they go. To make this work they are going to need new kinds of applications that work well in this style of working. I think that breaking applications down into these three application archetypes will help.

If you are about to kick-off a deployment of Windows 8 in your organisation and you want someone to help come up with an overall approach get in touch. I’ve got some work that I’ve already done in this space which might help.

Is “notgartner” good for my personal brand?

Officially at Readify the Marketing Manager role reports to the Chief Technology Officer. It is a curious perversion of an organisational structure that you won’t find in many companies but it kind of makes sense since the CTO role at Readify has a dual focus around delivery of services internally, but also defining the overall technology strategy for the company that we go to the market with.

I’m not a natural marketer, it isn’t something that I studied before coming into the CTO role and for a good part of my career I tolerated interactions with “marketing people” and never would have dreamt that I would have some level of responsibility for the smooth running of a marketing department.

When I first started working with the marketing team I did what any geek would – I started creating systems and inventing processes not spending a lot of time to really understand what the organisation really expects from the department. Whilst I had collaborated with the marketing team in the CTO role I wasn’t initially responsible for the department but the last 12 months has seen me on a real learning curve.

Of course I’ve been doing lots of self study to get a better grip on the problem, actively participated in the process and learnt lots of lessons (made mistakes) and had some wins. More recently Readify has been starting a planning process which defines the vision for the company over the next three years. That is a pretty tough ask given the rate at which technology changes in this business but at that level its more about the structural supports you need to put into the business to support growth and cope with the pain that comes with it.

These recent events have given me cause to go back to some marketing fundamentals and ask some questions about the Readify brand. What is our brand position, what is our brand promise – things of that nature. Whilst I have a relatively good idea of what those things are for Readify (more work to do), it got me thinking about what my own personal brand is.

I’m assuming that my personal brand and its positioning in other peoples mind is a function of the things that I blog about, the topics that I present on in conferences and the direct interactions that I have with individuals. Another part of my personal brand is my brand name “notgartner”, and I am starting to wonder “what’s in a name?”.

Where is all this coming from? Well, I’m starting to wonder whether it is time to leave “notgartner” behind and start to take some specific steps to reposition myself (and name is important in that).

What do you think?

BPOS to Office 365 Migration Notes

Over the course of the last week Readify has migrated from BPOS to Office 365. For most BPOS customers this should be a non event since Microsoft will work with you to drive the migration process and all you’ll have to do is look after the local client side issues – so normally I would recommend that you wait for Office 365 to come to you courtesy of the Office 365 migration teams that are now moving customers.

A Customer Driven Migration

Our situation was a little bit different however. We adopted BPOS quite early and in our rush to move onto it we didn’t stop to think about which AD domain we were going to associate with the environment. At the time we had two AD domains in use within Readify. We decided to associate with our old one since that is where most of the users were at that point in time. What we didn’t realise was that once associated you were really locked into using that domain unless you wanted to completely remove your user accounts and start the AD synchronisation again (note: I’ve heard that there is a way around this now, but it certainly wasn’t available when we first asked).

Anyway – so our migration was a little bit more complicated because not only did we want to move from BPOS to Office 365, but we also wanted to start synchronisation with a completely new AD domain which was now the home for user accounts, security groups etc.

In the end our migration wasn’t really a migration at all. We simply created a new Office 365 account, and used the MigrationWiz tool to copy mailbox content across from BPOS into freshly minted mailboxes at Office 365. The plan started simple but we hit a stumbling block which resulted in a full day e-mail outage, and no rollback path.

What cooked our goose?

Basically what happened was that there was a resource record in BPOS that was somehow incorrectly located inside the BPOS AD environment. This stopped us removing it, and the associated SMTP proxy address which still tied it to the @readify.net domain name. So even though we had disassociated the @readify.net address from all users, removed the @readify.net domain name from BPOS and requested the clean-up of the FOPE (Forefront) environment in Microsoft Online Services, we were still blocked from verifying the domain in Office 365.

We attempted to re-associate the @readify.net domain name in BPOS, however for some reason the e-mail routing rules in FOPE (presumably) didn’t get re-established properly so we ended up with bouncing e-mails – so our rollback was cut-off. After a conversation with the support team at Microsoft we decided the best course of action was to move forward by removing our BPOS environment completely. This meant that we had to step up the actual mailbox migration which we were hoping to do in the background for users whilst Office 365 started to receive new e-mails.

The MigrationWiz tool is good, but it can’t alter the laws of physics, there was only so much data we could pull out of the BPOS environment. Interestingly transfer into Office 365 was several times faster than transfer out of BPOS so we suspect that there was some throttling going on there. We also had a few users with big mailboxes (mea culpa).

A Devine Hack

Eventually we got all the data across and we gave Microsoft the green-light to decommission the BPOS environment. Unfortunately that item sat in the queue for way too long and we were looking at our second day of an e-mail outage. That was when our system administrator, Nathan, had the bright idea of setting up an external e-mail relay that would translate @readify.net addresses to the @readify.onmicrosoft.com addresses that get associated with an Office 365 instance automatically. This largely worked although a few of our external facing DLs weren’t mapped so we weren’t running at 100%, but at this stage I was happy with 50% Smile Outbound e-mails also went out with @readify.onmicrosoft.com and later @readify.net.au (which we had successfully verified earlier).

The Plea for Help

This was the status quo for most of last week but the job wasn’t finished so I got back onto Microsoft’s case about cleaning up our BPOS environment. After a fairly disappointing call to Microsoft support I put this tweet out to the Internet. That certainly stirred up a bit of a hornets nest. Its not that I have any particular sway within Microsoft but I’m usually pretty pro Microsoft (I like them as a company), it was unlike me which is why I think it got some attention.

Simultaneously we also reached out to some other contacts within Microsoft who were associated with Office 365 and asked for help and this is really what got the ball rolling for us. We got a local contact who managed to find someone in the US who could look directly at the problem for us rather than going through 2-3 layers of support with no real access. Once we got access to these people it took less than 36 hours to nail the problem which had been blocking us for around five days.

Suggestion for Improvement

I think that if there was one improvement that I hope Microsoft would make to Microsoft Online Services, especially BPOS/Office 365 it would be to make it possible to get access to these people without a series of secret handshakes. What took over 20 phone calls to get no where was resolved with 2-3 phone calls to the right individuals. Customers and partners need a panic button of sorts.

On the other hand Windows Azure support has been consistently good, when you make a request they usually know what you are talking about and can help you when you have problems. When you think about the products involved however that makes a lot of sense. BPOS/Office 365 sits on top of Exchange, SharePoint, and Lync – all of which have very specific requirements around DNS/AD – whereas Windows Azure is somewhat independent of all that, at least for the core services.

Credit Where Credit Due

At the height of my disappointment with the experience we were getting I put out that tweet I mentioned earlier. After that the troops within Microsoft did rally to get our problem solved. Specifically I’d like to thank the following people at Microsoft (in no particular order):

  1. Steven Wilks
  2. Andrew Pasco
  3. Lee Hickin
  4. Vajira Weerasekera

They took the time to look into the issue and respond to some e-mails, take phone calls which makes a huge difference.

In Summary

Internally within the Readify support team, last week is already being referred to as “hell week”. I hope I never have to go through anything like that again. But we now know a lot more about the way the internal support organisation at Microsoft works and the importance of finding someone who understands the problem and can show some empathy. It taught me that Readify’s own application support offering has to have as few layers as possible between the engineers and the customers.

I’d like to make a big shout out to a few of the Readify guys, including Andrew Harcourt, Tatham Oddie, Nathan Thomas, Robert McCann, and Tien Phan – all of who spent time after hours working through the problem. Hopefully I haven’t missed anyone.