Don't be a hero

In the wave of news coming from Twitter I wrote a short thread about one presentation I’ve seen one day. Twitter is not the best medium for longer forms so I’ve decided to write more about it here.

Imagine a system or a process with some kind of property. This might be a service for buying books or a process for provisioning access to something. At some point someone has decided that this system must have 99.999% availability or maybe it was decided that every ticket in that queue will be addressed within 24 hours. For a period of time this property is upheld and everything works great. Inevitably however the system starts failing. There might be multiple reasons for that - maybe the complexity increased beyond the initial assumptions. Or maybe we are getting three times more tickets in a given queue than we’ve used to. It doesn’t really matter but the effect is easy to predict - our metrics start to go below the desired level.

Suddenly, a hero appears. It might be an individual who has decided to uphold the property no matter what. Such a person starts working extra hours to meet the goal, sets up extra monitoring, rolls back flaky builds, prunes logs manually, cull the overcommitted thread pools or sits on pager duty all weekends.

Saving a day on a weekly basis becomes addictive. Our culture praises heroism - toil and sacrifice are seen as an ultimate path to success. Medias are full of people telling war stories about 14, 16 or even 18 hours long working days. Eating cold pizza for breakfast is somehow seen as something glorious. One thing that nobody mentions however are associated costs.

First of all - every act of heroism masks the systemic and structural deficiency. By treating the symptoms you are masking the root cause of the problem. Maybe the property you are so desperately trying to uphold is no longer relevant - just nobody told you that. Or maybe there are other causes and the system needs to be re-engineered. One thing is however sure - you are heading for much bigger issues later on. What happens with the system when you decide to move on might become a huge problem for the organization.

Another aspect is that heroism is bad for your mental and physical health. Working a 80 hour week is unsustainable and sooner or later the consequences will catch up with you. Burnout and stomach ulcers are not as fun as you might think. If you are in your twenties and believe you are immortal, think about it from a different angle. Nobody has ever been promoted for doing repetitive L3 type of jobs - regardless of how many hours you put in.

The solution to that problem - let it fail. Best case you will discover that what you were heroically trying to maintain is not that important after all. Worst case the system fails and the management realizes the need for changes - ones that do not require a personal sacrifice.