Limewater Labs
Scale & reliability

Know it survives real-world volume — measured, not hoped.

We turn vague performance worries into numbers, then move the numbers: load tested against the volume you actually have, bottlenecks fixed at the right layer, and releases that no longer need a maintenance window.

Where this usually starts

It works in the demo. Production has more data than the demo.

It's fine in testing. It falls over on the customers with the most data.

The overnight import crashes, and the next run starts again from zero.

Everyone says the system is slow. Nobody can say how slow, or where.

Every release needs a maintenance window we have to go and ask for.

How we do it

Turn a performance worry into a number, then move it.

Nobody can fix “it feels slow”. Every step here exists to replace an argument about performance with a measurement — and then to make that measurement change in a way a user would notice.

Measure before you optimise

A load-testing harness and dashboards come first, so there is a baseline number to argue with. Performance work that starts from a guess about the bottleneck usually optimises the wrong thing — the profile names somewhere else.

Test at the volume your largest customer has

Systems do not fail on average data. They fail on the biggest tenant, the longest history, the busiest hour. The load profile gets built from your real distribution rather than from a round number someone liked.

Fix the architecture, not only the query

Sometimes the answer is an index. More often it is batching, back-pressure, or splitting a processing path into phases. We say which one it is, and what it would cost, before anybody rewrites anything.

Failures resume; they do not restart

Checkpointed, incremental processing, so a crash nine hours into a historical import costs minutes instead of the whole run. Long jobs that restart from zero are the most expensive kind of unreliable.

Releases and schema changes without a window

Zero-downtime deployments with rollbacks, and database changes that apply in place rather than forcing a full re-import. Uptime is mostly decided by how you deploy, not by how fast the code runs.

Diagnosable at three in the morning

Contextual logging, a snapshot at the moment of failure, and metrics that point at a cause. The next incident should be explainable from the evidence it left behind, not reproduced by hand a week later.

Under the hood: K6 load testing · Grafana + Prometheus · profiling against production-shaped data · checkpointed incremental processing · adaptive batch sizing · parallel processing · zero-downtime deploys + rollbacks · contextual logging · snapshot diagnostics
What the first three months look like

A number in week one, and a better one every month after.

  1. Week 1

    A baseline you can argue with

    A load-testing harness against a production-shaped dataset, dashboards, and the first honest numbers. By the end of the week the conversation is about a named bottleneck instead of a feeling.

  2. ~1 month

    The first bottleneck, gone

    The largest constraint fixed and re-measured, with before and after side by side. We take the one whose removal users can feel, not the one that is most interesting to engineers.

  3. ~3 months

    Reliability, not just speed

    Checkpointing, crash handling, zero-downtime deploys and the diagnostics that go with them — so the system degrades predictably under pressure instead of falling over at the worst hour.

  4. From then on

    The numbers stay yours

    The harness, the dashboards and the load profiles run in your pipeline, owned by your team. Performance stops being an incident and becomes a threshold a build can fail.

Proof, not adjectives

We have kept a system standing at national volume.

Healthcare data platform at national scale

Proven against 150M+ records of real production volume

Constraint
A data pipeline buckling under real hospital volume: out-of-memory crashes, historical imports restarting from zero after a failure, and no way to change the schema without a full re-import.
Approach
Memory problems diagnosed from contextual logging and failure snapshots, then checkpointed incremental imports, adaptive batch sizing and parallel processing — with live and historical load balanced against database pressure.
Result
A pipeline reliable at 150M+ records with 2–3× performance gains, schema changes applied with no downtime, and failures that resume instead of starting over.

And where the job was keeping it up rather than speeding it up:

  • A load-testing framework with dashboards and monitoring for a real-time payments platform — which located the transaction-processing bottleneck and produced the two-phase design that answered it.
  • Zero-downtime deployments and rollbacks across nightly, QA and beta environments for that same platform, with every infrastructure change applied as reviewable version history.
  • A performance-sensitive clinical feature in emergency-care software, made fast enough to be usable at the bedside with server-side caching, subscription-based updates and batched evaluation.

A performance problem nobody has measured is an opinion about one. The first week of this work regularly changes which component everybody was sure was to blame.

Client work is confidential; described here in general terms. See all three engagements — or the two people who would do the work.

Start a conversation

Tell us what is slow, or what breaks under load.

Start with a low-commitment performance assessment: what the system has to survive, what it does today under that load, and a straight answer about what would move the number first — and what would only look like progress.

Email us to get startedhello@limewaterlabs.com