Skip to content
STN New Delhi · Building in the open Filing · 0%

What if we used salt instead of sugar?

Strip out the abstractions that make the system feel runnable. If the parts underneath can't stand on their own, the sweetness was hiding rot.

Raw Belief state Raw Still hot. Written in one pass. Not yet pressure-tested. Mode · proposal Mode proposal A way to think, design, or build differently. architecture Theme architecture Primitives, boundaries, implementation shape, and the consequences of choosing one structure over another.
Filed 9 min read

Conditions · lit fuse

There is a moment in the life of every codebase when it starts to feel real. The framework is humming. The data is flowing. The deploys are deploys. You look at it and think: this works. We have built a thing.

I am here to ruin that moment for you.

What you are looking at is probably not a working system. What you are looking at is a system that has been made runnable by a thick layer of sugar. The sugar is doing more work than you think. Some of it is load-bearing. Some of it is holding up walls that should have been foundations. And the only way to find out which is which is to swap the sugar for salt.

The diagnostic

Here is my proposal. Take your best system. Identify the things that make it feel easy and powerful: the framework, the conventions, the shared mental models, the assumptions about what is “standard,” the happy path, the tooling that quietly handles the mess you never look at. Call all of it sugar.

Now imagine you remove one piece of that sugar. Not because you want to ship without it. Not because I am a deviant. But as a stress test, temporarily. As a way to see whether the rest of the system stands on its own, or whether it collapses into a pile of assumptions that were only ever true because the sugar made them true.

If it collapses, you have found rot. If it stands, you have found the real strength in your system was not its taste but its architecture.

Engineering is not the architecture

This is the distinction I keep coming back to. Engineering is not the same thing as the architecture of the work being done. The architecture is the shape. The engineering is how they were built and whether the ingredients can carry the weight.

You can have a beautiful architecture — layers, patterns, abstractions, all the boxes and arrows you want, and still have a system where no individual part can justify its own existence or the strength of the whole. You can have a system that looks engineered and is actually just a house of cards with excellent interior and exterior design.

The purpose of making you do this is to make you understand that the building blocks have to matter. The ingredients have to carry weight. A framework cannot be the reason your domain logic exists. A convention cannot be the reason your boundary holds. A shared assumption cannot be the reason your system does not fall over. Those things can help. They can make the work easier. But they cannot be the load-bearing structure.

You have to think about the weight of the system that you are envisioning and then provision the ingredients to carry it.

When they are skipped, you do not have engineering. You have an insulated design. It feels like engineering because it produces a running system. You can demo it. You can ship it. You can write tests against it. But it does not make the underlying structure sound and steady. It just makes the temperature comfortable enough that you stop asking whether the walls are load-bearing.

The regrowth test

Think about an amphibian. Cut off its tail or a limb and the body keeps functioning. It is not ideal. It is not the aim. But the organism does not die on the spot. It adjusts. It reallocates. And when the conditions are suitable, it regrows what was lost.

This is not a practical requirement for software. I am not saying your microservice should literally regenerate a failed dependency like some kind of digital salamander. But the underlying principle matters. The design should be self-standing enough that losing one component does not cause the whole system to drop dead on contact.

Too many systems are built like Jenga towers. Pull one block and the thing tips. Not because the block was load-bearing, but because the blocks were never really independent. They were leaning on each other in ways nobody mapped. The architecture looked clean on the diagram because the diagram did not show the hidden dependencies. It was the culmination of the disjointed parts into a cohesive unit but neither of them could support their own weight.

The system did not have a spine from which the whole thing could be supported. It had too many interconnected parts that were not really independent. The regrowth test then surfaced that lack of structure which ultimately caused the system to fail.

The towers stood long enough

Here is a darker, more useful image. When the twin towers were hit by the airplanes, they stood long enough for quite a few people to get evacuated. The question was never whether the towers would fall or not. The question was how long they could stand and how many could be rescued.

That is the system design that should be the aim for a developer too.

It might seem impossible. Software feels fragile by nature. One bad deploy, one corrupted dependency, one cascading failure, and the whole thing goes dark. But there are ways to make sure that when the impact comes, the system does not immediately die.

Error handling with proper error notifications instead of just a 404 and a prayer. Soft close of ongoing, on-the-fly operations in a storage system when power is disrupted, so nothing is left half-written and inconsistent. Redundancy in servers and storage systems so one failure does not become total failure. Deterministic systems taking over for undeterministic ones when the latter go down, so the machine keeps serving the parts of the work that do not require guesswork. There are and can be devised infintely more examples of such deliberate defenses to prevent the system from dying on the spot. The only cost is deliberate engineering.

The argument is not about being strong enough to avoid a nuclear disaster in terms of systems design. That is fantasy. The argument is about being prepared for it. About pressure testing the system down to each component individually, even when that takes valuable dev time. Especially when it takes valuable dev time. I mean we all know how the video-game industry ships these days…

The defensibility test

After you pressure test a component, ask one question: can this part justify its own existence?

Not “does it make the system easier.” Not “does the team like it.” Not “is it idiomatic.” The primary question is whether the part would still deserve to exist if the sugar were gone.

If a layer only exists to make another layer bearable, neither layer is defensible. Both are part of a mutual-aid society of abstraction. They are holding each other up like drunkards leaning against a wall.

If a feature only works because five other features are propping it up, you do not have five features and a sixth. You have a single fragile organism with six heads.

If a design decision cannot survive the question “what does this actually do on its own?” then it is not design. It is decoration. And decoration is fine in a room, but it is dangerous in a system because if it crashes it will reveal the rot that it hid behind it.

And lets not forget the overly complex systems that can be stripped down to their bare bones doing only the core things that they were designed to do. A good system allows you to do that without losing functionality and carefully separates the seams of the system.

Failure is the point

Some systems will fail the salt test. This is not a failure of the method. This is the method working.

The point is to find the cracks on a test bench instead of in production. The point is to know, honestly, which parts of your system are structural and which parts are decorative. The point is to stop conflating it runs with it works.

I know engineers who get angry at this exercise. They say it is artificial. They say nobody deploys without the framework. They say the salt version is not realistic. They are missing the point. The salt version is not a proposal. The salt version is a mirror. It shows you what you look like without your costume.

If you are afraid to look, that is information too.

Where this stops being useful

There is a version of this practice that becomes puritanical. Refuse all sugar on principle. Strip every abstraction. Hand-roll everything. Reject anything that makes development pleasant. That version is not engineering. It is aesthetic asceticism. It is performance art.

The point is not to live without sugar. The point is to know which sugar is taste and which sugar is structural concealment.

Taste sugar is the sugar you chose because it makes the system better: cleaner, faster, safer, more maintainable, more humane to work in. Structural-concealment sugar is the sugar you depend on because without it, the system would fall apart. The first is a choice. The second is a confession.

You want to maximize taste sugar and minimize structural-concealment sugar. The salt substitution is how you tell them apart.

The hard part

The hard part is not the technical work. The hard part is the honesty.

It is embarrassing to discover that a layer you defended in meetings, a pattern you championed, a tool you identified with, is actually sugar hiding rot. It feels like a personal failure. It is not. It is normal. Every system has rot. The question is whether you are willing to smell it and then learn to clean it up properly without masking it.

The engineers I trust the most are the ones who will do the salt substitution in public. They will say,

I thought this layer was load-bearing, but when I removed it, nothing changed. I was wrong. Let us delete it.

That is not weakness. That is the strongest thing an engineer can do. They expose their own vulnerabilities and then come out of it with a better system and a better understanding of the said system.

So what

You do not have to stop using sugar. I am not asking you to live in a cave and write assembly by candlelight. Sugar is good. Sugar is civilization.

But know your sugar. Name it. Test it. Every once in a while, imagine removing it and see what survives. Because the system that runs only because it is sweet is not a system that works. It is a system that has not failed yet.

And failure without warning is the most expensive kind. Better to find the rot in the test kitchen than to serve it at the table with real stakes. Better to know, before the airplanes hit, how long your towers can stand. Are you willing to take a bet on it?

Appears in

More from this theme