The Infancy of the Cloud

The Heterogeneous World of IT Job Titles

When I started working in the IT industry, one of the things that shocked me was the way job titles were used. As an undergrad, I guess I expected some correlation between the training or study plan and the position or role one holds in a company. Yes, that was naive of me, I admit it. Then, the creativity some people exercised when describing their jobs took me by surprise. Anyone who has scrolled through enough LinkedIn profiles will know exactly what I’m talking about.

However, I don’t want to rant about made-up or senseless occupations here. What I want to talk about is a specific position that, on the surface, makes all the sense in the world—just not to me, not yet.

The Cloud Expert

The cloud has become an essential area for IT professionals. If you don’t know anything about the cloud, you’ll be left behind. Most of you will remember how, back in 2006, Amazon released AWS, and the cloud phenomenon started taking off. Some people used to joke that:

It was never the cloud, it was always someone else’s computer.

And yes, that’s more or less what it looked like at the beginning. But then, things started evolving. First, there were continuous additions to the AWS product landscape, followed by Microsoft Azure, Google’s, and IBM’s counterparts. Then came Pivotal’s CloudFoundry and AWS CloudFormation in 2011, Docker containers in 2013, Google’s Kubernetes, AWS Lambda, and HashiCorp’s Terraform in 2014… just to name a few.

It’s 2020, and we need cloud experts. However, in my humble opinion, nobody out there is truly a cloud expert. Not yet. I’m afraid that we haven’t had enough time to digest it all. There’s no standardization, no clear best practices, and, what’s worse, it has not stopped evolving—and it doesn’t look like it will anytime soon.

Yes, you might have attended countless conferences as a speaker talking about cloud technologies, and you might have world-class expertise in the cloud. Or maybe you’ve been using Kubernetes since 2014, and that certainly makes you an expert in that particular technology.

But do you have any certainty about the near future of the cloud? Can you create something complex today that will still adhere to what will be considered best practices in 5 years? I seriously doubt it (just look at how we were doing things five years ago).

Is It a Bad Thing? What Can We Do?

I think this is something quite common in IT, as well as in other fields where things are evolving so rapidly. It has both pros and cons. On the one hand, for us professionals in the field, it’s exciting to experience this evolution firsthand. On the other hand, it generates a lot of noise and fatigue, and you never really know which technologies or techniques will remain relevant a couple of years from now.

There is, however, one thing you can do when starting a new project or making architectural decisions on existing projects in the cloud: forget for a moment how big of an expert you are, try to adopt a beginner’s mindset, and follow the golden rule of software development: the Keep It Simple, Stupid (KISS™) principle.

The cloud came to help us. It’s not there for us to over-engineer our solutions or to take the Resume Driven Development route. Things tend to get complicated on their own, and it’s our responsibility to do our best to keep them as simple as possible.

Some of the clear benefits of the cloud are:

• You don’t have to buy or manage hardware yourself.
• It should be cheaper.
• Software updates are managed or simplified.
• It offers a fast and reliable service (typically 99.9% availability).
• It makes it easier to write software that scales (although if you’re a startup, chances are that you shouldn’t have to worry much about that).

If you misuse the tools that the cloud offers, however, you can easily ruin its associated benefits, even creating brand-new categories of problems. Here are some common problems I’ve experienced when working on legacy projects in the cloud:

• Using too many cloud resources can be very expensive.
• It’s often the case that you cannot easily set up a development environment on your local machine because there are too many dependencies on cloud components. You should try to avoid this early on in the project, or things will quickly get out of hand, making life difficult for you and, even worse, for newer members of your team.
• Sometimes unit tests depend on existing infrastructure (this is a big red flag).
• No appropriate monitoring and tracing mechanisms have been set up to figure out exactly what happened when something went wrong.
• It’s easy to overcomplicate things when trying to apply architectural patterns like microservices or an event-oriented architecture. Sure, using patterns and tools for the sake of using them has always been an anti-pattern, but the cloud can take it to a whole new level. If there are too many components, it gets more difficult than ever to get a clear picture of what’s going on, and documentation tends to stay out of date.
• If there’s no deployment automation from the beginning (Infrastructure as Code), chances are you’re going to regret it.

If You Really Are an Expert, You’ll Be Humble

As Bret Victor pointed out in his talk about the Future of Programming, the IT community is (sadly) still digesting ideas that were first formulated during the ’60s and ’70s.

A good example of this is the renewed interest in Functional Programming and how mainstream Object-Oriented languages like Java, C#, and C++ have recently incorporated FP concepts that were already available in LISP, first specified in 1958.

We simply haven’t had enough time to define best practices for the cloud yet. Since you cannot know the best practices of the future, try to keep things as simple as possible today, because chances are you’ll need to rewrite parts of your code in the coming years. The less code you have today, the easier it will be to modify it tomorrow.

Also, your team will be grateful if you do your best to keep documentation up to date. Using tools like Terraform, CDK, or Pulumi, we can write Infrastructure as Code (IaC). That serves a double purpose: it lets us deploy and manage our cloud resources automatically, while also providing up-to-date documentation of your infrastructure.

What will come next? A tool that generates up-to-date diagrams from our IaC? A WYSIWYG editor for the cloud? At this pace, we’ll see it pretty soon.

Back to the top ↑