I have recently been involved in the overhaul of an established business with poor output into a functioning early/mid stage startup (long story). We are back on track but, honestly, my lessons learned fly in the face of a lot of currently accepted wisdom:
- Choose languages that developers are familiar with, not the best tool for the job
- Avoid microservices where possible, the operational cost considering devops is just immense
- Advanced reliability / redundancy even in critical systems ironically seems to causes more downtime than it prevents due to the introduction of complexity to dev & devops.
- Continuous integration seems to be a plaster on the problem of complex devops introduced by microservices.
- Agile “methodology” when used as anything but a tool to solve specific, discrete, communications issues is really problematic
I think overall we seem to be over-complicating software development. We look to architecture and process for flexibility when in reality its acting as a crutch for lack of communication and proper analysis of how we should be architecting the actual software.
Is it just me?
I was going to reply to the comment on HackerNews, but I quickly realized that I had a lot to say, and it would not have worked well as a comment. Additionally, I felt that others might benefit from reading this question and my response in the future, and I worry that HackerNews comments quickly vanish into the ether never to be re-discovered by anyone at a later date.
To start off, I’d like to address the question of “Is it just me?” It is not. I feel similarly that today we often unnecessarily over-complicate software development. My career spans back to 1999, and I got to see how software development became complicated and the very good reasons why it did. Today, however, there is no balance: you either do exactly what the crowd does or you are wrong. This is symptomatic of being only a Novice as defined by the Dreyfus Model of Skill Acquisition. To put it more plainly, if you are strictly adhering to the rules, you lack experience.
The following is my response to each of the questions posed:
1. Choose languages that developers are familiar with, not the best tool for the job
If we are speaking strictly about languages – and not tools – then yes, I agree wholeheartedly. My primary reason for this is that I need to make my team write better software, and that’s going to be a long, uphill battle depending on the individual. While I am training them, I don’t also need them struggling with unfamiliar syntax because in the hands of experts it may provide the option for an incremental improvement of clarity or performance. If we are talking about tools – which to me covers just about everything outside of a language – then I do not agree. Tools are about objective productivity, language choice is a subjective preference outside of specialized tasks or ecosystem restrictions.
2. Avoid microservices where possible, the operational cost considering devops is just immense
Microservices are a concept, not an implementation. Confusing the two points back to a novice level of experience. My favorite discussion of microservices is the one done by James Lewis and Martin Fowler. To get the most our of this article, however, I feel that the following are minimum prerequisites:
There are many other books that I would in turn consider prerequisites to these, but among those actively engaged in discussions around microservices, I find few who have heard of even these books – much less read them. This is a shame. If you are a fan of microservices, you will be squirming in your seat as you see the little hints predicting that microservices are the way of the future. In fact, when I finally did hear the term “microservices” and inquired as to what they were, I was able to point back to specific patterns or aspects of patterns that demanded what we now refer to as a “microservice”.
DevOps is another topic that is taken to mean a specific implementation than an idea, but this is an area that benefits from more ridged definition than less. Next up on my reading list is The DevOps Handbook, which I am hoping will present in a structured way hundreds of best practices that I can hand to development teams around a large organization so that we can all learn to collaborate more effectively
- Don’t take “microservices” to mean something too specific. Learn the alternatives and when, where, and how to use them.
- Do learn the industry best practices for “DevOps” as it benefits from everyone approaching the same problem the same way.
3. Advanced reliability / redundancy even in critical systems ironically seems to causes more downtime than it prevents due to the introduction of complexity to dev & devops.
I think I understand what you trying to say here, even though on the surface most people would say “You’re doing it wrong”…I admit I would be one of those people. As that is not a very thoughtful answer, let me attempt to expand on my trite rebuke.
“Complexity” is a tricky thing to define. I could argue that the job of software developers is to tame complexity in systems we don’t control, while creating new systems without adding unnecessary complexity. I put complexity into two distinct buckets:
- Complexity we can’t avoid (at least, for now)
- Complexity we can avoid
Our collective goal in the industry (and we do a really good job of this) is remove unnecessary complexity in systems over time such that we trend to simplicity. Our primary technique for doing this is by introducing useful abstractions. I use the word “useful” to mean that there are abstractions that are not useful. One of the canonical abstractions I often use as an example of this is that related to “simplifying” SQL. There are many ORM tools available to “remove the complexity of SQL”, but often the resulting code when observed years after their introduction appear to have increased complexity due to their inability to gracefully handle the inevitable 5% of edge-cases that refuse to play well with the ORM tool. Often, where many teams would adopt a heavy all-in ORM framework, a simple query building API would have done nicely, and resulted in a less complex and more flexible system.
Having said all of this, I am not familiar with your specific situation. I would imagine that the pain you are experiencing is real, but I would maintain that the source of your pain is most likely specific systems, processes, or people – not the general concept of enhancing reliability or redundancy.
In support of your point, there is a bit of heart burn in the industry around the container deployment model when it is used inappropriately, as captured by this brilliant piece of satire by CircleCI. Though the article ostensibly warns about the general use of containers, I take it’s meaning to be “don’t use containers when containers are not appropriate.”
4. Continuous integration seems to be a plaster on the problem of complex devops introduced by microservices.
Generally, not at all – but this is again the difference between understand “continuous integration” as a concept as opposed to a specific set of processes and tools.
Continuous integration means that we don’t all wait until some later date to test all of our work together on production-like systems – we do it continuously, early and often. It generally speaks to frequent check-ins that automatically trigger automated tests. That’s generally though. There are many specific Continuous integration tools and techniques that can introduce so much process and overhead burden as to forget what the original objective was supposed to be: not waiting until the night before deployment to test everything working together. Fun fact: lot’s of organizations still only fully test their systems of the night of a big deployment. They would absolutely benefit from continuous integration but often claim it to be “too hard” to setup.
Now, I am starting to get a feel for your specific development situation, and I can easily imagine that in your context you are right: The inappropriate choice and/or application of specific processes, tools, architectures, and deployment models can put an undue amount of pressure on a CI system to make sense of it all. Sadly, a CI system will reflect the madness that it is supposed to orchestrate, so it’s easy to see it as a band-aid attempting to cover an underlying problem.
5. Agile “methodology” when used as anything but a tool to solve specific, discrete, communications issues is really problematic
Once again: Agile the philosophy, or Agile in practice today? The agile philosophy is (to me at least) best described in Extreme Programming Explained, and well over a decade ago it was my introduction to Agile by my mentor at the time. Around the same time as I was introduced to Kent Beck’s book, I was also reading the equally impressive The Inmates Are Running the Asylum by Alan Cooper. Back then (~2001) I did web user interfaces (called Dynamic HTML back then – pre-AJAX even), and also did website backends in PHP/MySQL. The result is that I had skin in the game both from the standpoint of Beck’s thoughts on software delivery and Cooper’s thoughts on Interaction Design. I read both books, and loved them both. In my mind I automatically blended their seemingly dissenting philosophies into a cohesive whole, and now that I’m an old salty enterprise systems architect and HTML5/iOS user interface designer I still base my day-to-day execution largely on those combined concepts, as they have stood the test of time and served me very well.
Shortly after reading both books, I learned that someone interviewed Beck and Cooper at the same time. It was pretty epic. I could only find “Cooper vs. Beck” on the Wayback Machine, but at least you can read it yourself. At the risk of spoiling it for you, Cooper makes a pretty good case against agility that I have seen hold up time-and-time again.
Now, do I practice Agile? Yes, but I feel Agile must be customized to the organization and is not a one-size-fits-all drop-in turn-key process-in-a-box. My guess would be you are using a non-customized version of Agile someone obtained from a book or website, and therefore you are experiencing more of the pain caused by Agile, and not its benefit. That’s a pity. A team that executes Agile well is a beautiful thing to behold. Unfortunately, those teams are extremely rare.
We look to architecture and process for flexibility when in reality its acting as a crutch for lack of communication and proper analysis of how we should be architecting the actual software.
I completely agree.