Practical Finance for Software Engineering: Part 3 - Understanding Efficiency
(Read Part 2.)
Say, we have a SaaS company with gross margins of 60%. For each dollar of revenue, 40¢ is spent producing the product unit, and we keep 60¢ to reinvest in the product. This 60¢ can be spent on activities that deepen the competitive moat: developing new products, improving existing products, marketing to a new customer segment etc. In order to out-invest and accelerate past our competition, we want to increase this margin to something closer to 80%.
Here is alternative way to frame the same objective: suppose, on average, our competitors are able to reinvest g¢ per dollar of revenue. How can we do better than g?
Remember from Part 1 that Gross Margin is an average. To increase this average, we need to reduce marginal costs (of goods sold).
Economies of Scale
One way to reduce marginal costs is to exploit economies of scale. Some economies of scale happen naturally. For example, suppose we have a new service that demands its own isolated database cluster because it just needs that level of performance predictability. This database cluster is more expensive per query in the beginning (due to fixed costs from just operating the cluster, and minimum sizes of machines), but the per-query cost decreases as the number of queries increase (aka amortization).
Similarly, the smallest size for a feasible Support team is around 3 people, so that no single person has to be on-call 24/7, and we can reasonably handle attrition and PTO. When the number of support tickets is very small (great for you!), the cost per ticket is high, and decreases as the number of support tickets increase.
Hopefully, that increase is correlated with revenue growth, and not because the product’s quality is deteriorating. I would not consider the latter form to be economies of scale.
Other economies of scale require more intention and planning. On AWS, one could purchase Savings Plans and Reserved Instances once there is some predictable floor on daily usage. Beyond $15m/year, one could also negotiate multi-year agreements with AWS that exchange commitments for discounts. All SaaS vendors will offer something of this sort once you are spending more than $6m/year with them: you accept some form of vendor lock-in, and they give you discounts.
Engineering Efficiency
Most of the time, the pursuit of economies of scale is a multi-year effort, and is not actionable on the 6-month (or sometimes even 12-month) roadmap. Two ways to reduce marginal costs which are more actionable are:
- reduce infrastructural waste, and
- reduce application waste.
In performance engineering circles, these are also known as the resource perspective and the workload perspective.
The resource perspective (infrastructural waste) starts with a review of the infrastructure (e.g. CPU, memory, storage, Databricks Units) and looks for waste. We ask the following questions:
- Are reserved (or pre-allocated) infrastructure fully utilized throughout the day?
- Are there opportunities to de-allocate or free infrastructural resources as workload decreases?
- Do we use all of the shared data we keep in memory?
- Do we use all of the shared data we keep in storage?
- Are there bottlenecks we can resolve to increase the overall utilization of the infrastructure?
- What’s over-provisioned, or under-provisioned?
The workload perspective (application waste) starts with a review of the application components that consume the most amount of resources e.g. uses the largest database cluster, keeps the most amount of data in memory cache, or requests the largest containers. We ask the following questions:
- Do we actually have to do all this work?
- Can this work be optimized to use fewer resources?
- What drives this workload? Can it be batched or scheduled?
- Does the application use all of its data in memory/storage?
- Can any of the requests be cached and reused?
Fun fact: spot instances on AWS are cheaper at night. I assume this is true for GCP and Azure as well.
Product Efficiency
We will end Part 3 with one final way to improve efficiency: build a better product. Better products are intuitive and user-friendly, to the extent of predicting common issues and enabling users to resolve them themselves without calling in. Better products reduce cancellations, charge-offs, and returns. Better products have the features that our users want, and none of the bloat they don’t need, and help users discover and learn how to use new features on their own. Better product comes with better marketing, which does not bait customers who are likely to regret their purchase and cancel when then free trial ends.
Stay tuned for Part 4.
