When you start any human endeavour a big part of the challenge is figuring out the kinds of tools that you are going to need for the job. If all you want to do is change a light-bulb well then a step-ladder is probably going to do the job. However, if you want to go to the moon then you’ll probably want a rocket ship. In theory you could stack ladders on top of each other, but eventually that is going to stop working and you’ll have to face up to the fact that you are going to need to do some serious engineering to reach such a lofty altitude.
Software development as a problem space has huge diversity in the scale of problems that need to be solved. Some days you might be building a simple command-line utility, and others you are developing a multi-tenant, multi-currency, multi-language solution or something in between.
As a software developer you not only have to be able to correctly frame your problem, but you need to figure out what set of tools are going to help you solve it. If you go to complex, you create a monster that no-one wants to work with, if you go to simply you just don’t have the capabilities required to do the job – both mean failure.
A great deal of complex architectures are the result of a poorly framed problem. When you start out your stakeholders will be bullish about the capabilities that require. In an effort to meet all of the requirements you try to figure out how to piece together a solution which meets not only all of the functional requirements, but also the cross cutting concerns. Indeed, it’s the cross cutting concerns that often lead to complexity in application design.
I’m constantly amazed at how, after months of development initially unmovable constraints start to get software as the reality of the cost of meeting all those requirements sinks in. Not only that, but capabilities that the architecture provided but which were never initially considered suddenly become the new “most important feature”.
As developers though, we can’t dodge all the blame. We all want to try shiny new things, and so every project we undertake we push the envelope a little bit. Sometimes these new construction approaches are simply the new common approach taking hold. Other times it is just folly.