Marcelo Boeira’s Post

STOP 👏🏻 CREATING 👏🏻 CIRCULAR 👏🏻 DEPENDENCIES 👏🏻 A circular dependency is a relation between two or more systems/components that directly or indirectly depend on each other to function correctly. It might seem obvious, but it often happens in our industry, consciously and unconsciously. I've seen it first-hand multiple times: engineers do not think about the non-functional requirements of their applications or the overall design; they just "write the code"; Soon enough, the circular dependencies present themselves and are often hard to revert - most system-design choices are. That becomes an "accepted" flight risk, increasing the blast radius of failures. They can be as simple as: "creating a deployment infra deployed with your own deployment infra" -> How do you deploy a fix to your deployment if the deployment is down? Or something less obvious but still conceptually problematic:  "hosting your own monitoring stack in the same infra as your applications stack" -> What happens if the part of the infra goes down? How do you debug without your monitoring stack?" Not even big tech is out of that; a known example in the SRE industry was S3's outage, which's side-effects included the unavailability of AWS's status pages hosted… on S3. People had to be communicated over unofficial channels (e.g.: Twitter/Slack/...) that S3 was down because AWS's status page was down because of S3. How to prevent such problems? System Design combined with Non-functional requirements before writing the code. Web applications have changed drastically over the last decade, with the demand for more traffic, more components potentially failing and higher availability times, there is no space for legacy practices that might've worked 10 years ago when we were all back on our own data-centers with maintenance windows scheduled with customers.   Hiring and empowering seasoned engineers with Cloud Experience will facilitate the transition to modern practices by emphasizing good system design and the use of best practices when it comes to reliability. Most people think code is expensive and hard to get, but always remember:  An excellent system design can overcome poor code; an excellent code alone, on the other hand, can't cover a crapy design.  What do you think? Follow-up reading in the comments. #sre #cloud #systemdesign #outage