This text is a compilation of author’s experience, written to provide new juniour/middle teammates the basis to start developing computational intensive or/and ML based systems. It will take you about 5-10 minutes for reading, all mentioned terms will be referenced at the bottom.
Complexity
It’s a common mistake, when people think software engineer have to only write a code and make features of a product. What makes this role really interesting is a Complexity. Constant fight with entropy of systems engineers are working on, reducing it as much as possible every day in that endless loop of knowledge and pain. Yeah, you get it right. Working on creation of new software especially noncommon one - is using new techs, products, instruments from day to day, and ofc getting some new unusual issues with them.
To understand the concept we should discuss some terms at first.
Abstractions and links between them
We may consider our software as a system of actors/subsystems/submodules with some kind of relationships between them. Links between such elements can be:
hard - when you brake it, entire system will fall down
soft - when you brake it, just a small part won’t work, the system continue to operate, may be with reduced functionality
For example, Core in your product may depends on some external modules/systems, but fallen dawn Billing or Metrics storage won’t affect business critical functionality, your customers just won’t be able to pay for smth for a while and your analytics won’t get a portion of metrics.
And now, let’s just think about how many items and their linkage average human can handle in mind simultaneously - about 6-8 items and about 5-10 links between them. Quite enough to work with small systems: websites, scripts, simple crawlers may be some bots e.t.c.
And what are you gonna do if your system is larger? I mean, large enough to not fit into your mind so you won’t be able to operate abstractions and their links well enough. I guess, you will use such instruments as:
mind technics to remember more
frameworks
diagrams
special software
So, all above - just to handle complex relationships between elements of just one system. And let’s imagine forward - several complex systems which should communicate with each other. But keep in mind - these systems/subsystems are not a static picture. They constantly interact either with users either with developers and engineers.
Usual processes and users interactions in a software system will appear in about 3-10 months of development and growth.
Wave of changes
But why to do it, why you need to keep in mind all this stuff, is it necessary in a daily routines? Short answer - yes. As software is usually just an automation of business processes, it mirrors some nature of business from inside. And the most important - starve for constant growing and adaptation to environment and market changes. To be able to survive in a market, business should constantly evolve and improve itself, so either its software should be updated, fixed, refactored, improved and optimized.
There is no talks here about non working software at all. Your system should be able to operate at least.
And now we approaching to one of a major points: it’s not essential how well your software is and how beautiful you wrote a code, or make it in a few lines instead of several pages. It just need to work as business demands. And it’s only a software engineer’s responsibility to think about how easily it can be changed and improved during timeline. Usually business don’t give a though about it in a short term, and will only start crying about it in a long term. So, the cost of change - what’s important. How adding new feature will affect old ones, how optimization of that will affect costs in maintenance time of another part and so on.
Forgotten realms
How to understand that you have a complex system, not a small one like a project, but a real bitch? There is a very simple answer - just when you need more than one human to work on it.
And there are signs you already have complexity problems:
there is a person, who only keep in mind some information about a subsystem/submodule of a software. No one else know about it enough to operate with.
nobody knows a “whole picture” of your IT landscape, every engineer operates only with own slice of knowledge about an entire system.
engineers leave and at sometimes you heard from a colleague “don’t know what’s happen there”, “it’s a black box for me”
Vortex of doom
What happens if business didn’t allow its engineers to spent enough time to fight a complexity?
There is a very similar phenomenon in a biology, called an Ant Mill - due to some reason ants lost their pheromone tracks and started following nearest fellows ant forming constantly rotating circle. More and more ants join this circle till large group of them involved. Then they will run till their death.
So, the same is for human companies - there is a critical mass of tech debt (different for each company), after reaching it’ll begin a circle of:
01: bug fixing
02: making new bugs while fixing old ones
03: go to 01
And will last till engineers leave such doomed company at all.
Ways to fight it
There is no way to keep complexity at a low level long time. It’s a constant process and require some balance of spending time/money on developing new feature and refactoring an old ones.
It’s more like a hygiene - you just need to do it constantly every period of time. Another analogy - second law of thermodynamics, when you should spent some energy on a system to not let its entropy grow. So what are basic ways to reduce a complexity?
Human in the loop
Rules and conventions
Engineers tend to follow some common rules and conventions made inside a company or outside whatever. Usually it’s:
best practices documents, like this one
code style, which usually should be taken from big tech companies, like Google, Amazon, Netflix and so on
gitflow policies
any other agreements between people to collaborate effectively
Organizational structures
Hierarchy helps to scale people work and do changes in parallel.
It depends which way is optimal for a company in its actual situation. Better to read this book to understand deeper:
in red companies they usually build vertical hierarchy to react faster, tradeoff - lack of creativity among developers/engineers
in green companies there are more horizontal structures with much creativity and agility, tradeoff - lack of speed
Software to the rescue
IDE
Intellesense, autosuggestion, copilot - all of these instruments help engineers to handle more abstractions and links between them at once.
The price of automation
QA checks, CI/CD - all of that makes systems and development processes more inert in a small group of engineers, but let you scale your development up to multiple teams.
Observability
Monitoring, logging, error capturing help a lot.
Here are the helpfullest ones:
Total
Software is a non-static system being affected by constant changes.
Software and development process should be considered as indivisible concept. You mustn’t solve some software problems without thinking about a process and vice versa.
System complexity problem is more noticeable on a scale: scale of people, hardware and system load.
author: Andrey Avdeev, aavdeev.com
editor: Maksim Sidorov
