There's a pattern in many software development blogs, talks, tweets, etc. They often lack context and they're received poorly and discussions end up with people talking clear past each other.
Understanding situational factors makes it far easier to have a meaningful discussion about technology choices. The answers to every one of these questions can affect your designs and technology choices:
- What size team are you talking about? Is the team going to significantly grow in the next year or is there a plan to keep it small?
- What are all the engineers' experience levels and specific technical backgrounds?
- Do you even have a team? Decisions are often wildly different if you're working on your own.
- Did the person that was the main advocate and maintainer of technology X leave the team?
- How profitable is the business (i.e., how much time can you spend on experiments or robustness)?
- Is it OK if infrastructure costs scale linearly (or worse) with activity/customer count?
- What are the data retention and privacy requirements?
- Does this domain really need such a high level of reliability?
- What kind of operations culture is in place already?
- Are you already OK with being paged occasionally or do you want spend an enormous amount of energy and money to avoid this as much as possible?
- What is more important to the customers and business goals, reliability or new features?
- How adaptable do you want the architecture to be?
- How shielded from "lock-in" do we need to be for component or dependency X? (probably going to write a whole post on this one)
- How much do you value high performance testing / developer iteration speed?
- Is this programming language something that most college graduates will know already? (actually matters in some contexts)
- Do you have a platform team that provides capabilities already that you are mandated to use?
- Is your company working towards an internal platform or do you accept the potentially repeated functionality in the future?
- Is this clever but unmaintainable hack going to solve a major problem right now? What is the real long term risk vs. benefit of solving that problem?
- Are you deliberately using more new/exciting technologies to attract candidates?
- Are you simply more interested in the latest exciting technology vs. boring but proven solutions? (no judgment)
- Is the new/exciting thing actually the only thing that solves the problem well? (it happens)
- Do you need to build this thing to be supported by another team or by non-developers?
- Is the culture/management going to accept if this interferes or overlaps with another team's service? (hello 'Big Tech' internal politics)
- Can this feature be a script or a spreadsheet instead?
- Are you or your manager suffering from the sunk cost fallacy? (Often a huge problem. And sometimes it's better to stay the course with the previous solution given how initial replacement estimates can balloon out of control later :-))
- Is this open-sourced or will it be open-sourced in the near future? What's that community like? Are core engineers funded? How well is it maintained? Are you able to contribute features and fixes?
- Is technology X easier to monitor than technology Y? And is that because of the technology or your own monitoring stack?
- Does anyone on the team have experience running X already? Or does a different, trusted team in your company have that experience?
- Are you trying to make sure people on the team develop transferable skills? (recommend that in most cases, both for attraction and retention purposes)
- Is an internal service in your company up to serving this new load or do we need to take over and 'productize' that?
- Does adopting or creating X mean we can re-use it in several places?
- Is that database battle-tested or is it more experimental?
- Is that externally provided service or software from an under-funded or brand new company with just a few customers?
- Are you building this for fun, curiosity, and personal satisfaction? (often the best posts and talks)
None of these questions have a "right or wrong" answer. Some constraints are simply forced on you. Some positions you take are coming from your hard-won experience, world view, or even aesthetics. Some constraints torment you and drive you away from the team eventually. Some things you're working to change. Some constraints are new to you, you resent them a bit, and then they actually become some of the best learning experiences.
And those are just sample questions. I probably could've spent a whole day on the list and made it much longer (and yet still incomplete). Engineers and managers need to put in real hours to think and talk through these types of questions.
And you may have noticed that the questions don't even touch on the specifics of the technologies. There are a whole new series of trade-offs you need to consider after the broader decisions are made and the design begins to get legs. There were no mentions of API contracts, composability, throttling, database partitioning, blast radius minimization, service meshes, infrastructure automation, deployment safety, etc. There were no mentions of monoliths, microservices, NoSQL, event-driven architectures, containers, serverless, etc. (I touched on this topic in a Twitter thread about monoliths vs. microservices.)
There are trade-offs in every decision. From the huge, obvious ones like language or database choice all the way down to the design of an individual function. Sometimes the trade-offs are gnarly and sometimes they're obvious and just need a moment's attention/documentation. Some decisions make other ones harder when considered together; intertwined, multivariate issues abound.
Good engineers and managers know what trade-offs they're making with each choice and deeply question their assumptions. The worst can't see beyond their situation enough to know there was ever a question.
And that's why it's hard to see talks or posts like "clearly you must use X for Y". And why I get suspicious when the term "best practices" is dropped. If you think there's really one perfect solution to a significant problem, it's likely that you haven't been exposed to that many different contexts where that problem rears its head.
I like talks and posts that are more along the lines of "here are my constraints, my aims, and how I approached the problem". I'm not asking anyone to write a book-sized preface, but it's valuable for authors to include some context and not assume the reader is in the same situation (or that the reader can read their mind). And of course it's valuable for the reader to not jump to conclusions either.