On 17 August 2026 we loaded LEO Chat with 300 requests per second and forcibly killed the engine, the message queue and the cache in the middle of the test. Here is what we checked, why it is not the same as production traffic, and what the result showed.
On 17 August 2026 we did to LEO Chat what usually only happens to a production system by accident: loaded it with 300 requests per second and, in the middle of the test, started forcibly killing processes — the engine, the NATS queue, the Redis cache. Not one by one, gently, but with SIGKILL, which gives a program no chance to shut down cleanly.
This article is about why we did that to our own product and what the result actually showed — including what it does not prove.
Most chat bot tests check whether the bot answers correctly. Less often do they check what happens when part of the system goes down mid-operation — which is exactly what happens in production: a container restart during a deploy, a network glitch, memory exhaustion. The question is not "will a component fail" but "what happens to the customer's message that was being processed at that exact moment."
For LEO Chat this question is particularly sensitive because part of the scenarios are order placement and payment confirmation. A lost message here is not an inconvenience — it is an order nobody saw, or worse, a payment confirmation that never reached the customer.
The tool was k6, a load-testing tool that generates a stream of requests to a defined profile. The target was 300 requests per second, which for a chat bot means simulating several hundred simultaneous conversations with messages arriving one after another.
Alongside the load, the test forcibly killed key components of the stack: the message-processing engine itself, the NATS queue (through which messages pass between services), and Redis (cache and part of the session state). The killing was not a graceful shutdown but SIGKILL: the process disappears instantly, with no chance to finish writing anything unfinished.
The metric measured: how many messages were lost (sent but never acknowledged) and how many were duplicated (the same message processed twice — for example, an order confirmed twice instead of once).
0 lost and 0 duplicated confirmed messages.
This means: despite forcibly killing the engine, the queue and the cache under a load of 300 requests per second, not a single confirmed message disappeared, and none was processed twice. Technically this rests on the NATS queue storing messages persistently (not only in memory) and having a processing acknowledgment mechanism — a service that crashed before acknowledging a message does not remove it from the queue forever; it waits for a retry.
This is a load test on the local full stack — the same service configuration as production, but not production itself, not with real customer traffic and not under real network latency between data centres. Production load can reveal things a local test cannot: specific network delays, resource contention with other services on the same server, behaviour under sustained rather than one-off load.
It is also not a test of 100% availability: while a component was being forcibly killed, some requests experienced a delayed response while the system recovered — the queue did not lose messages, but it did not process them instantly at the moment of failure either. For the customer this means a slower reply for a few seconds, not a lost conversation.
Most chat bot product pages talk about answer accuracy and speed. Behaviour during a failure — rarely, because it is an uncomfortable topic: showing that a system went down, even in a controlled test, means admitting it can go down. We consider it more honest to show this limit with a date and a number than to stay silent about it or promise "100% uptime" backed by nothing.
The same approach — measuring our own limits, not only our strengths — we already applied to LEO Chat's answer accuracy: 89.7% on a reference set of questions, with a direct statement that the rest goes to an operator. The chaos test is the same logic applied to infrastructure instead of answer quality.
If you are choosing a chat bot for a store, message-queue reliability rarely makes it onto the list of questions — yet it is exactly what determines whether a payment confirmation gets lost during a routine server restart at your vendor. Three concrete questions worth asking: are messages stored persistently until processing is acknowledged, is there a retry mechanism after a component failure, and was a real-load test run with a date and a number — not "we tested it," but exactly how many requests and what result.
We publish our own measurements precisely because "trust us" is not an argument here: a verifiable number with a date is an argument, a promise is not.
Your opinion helps us create better content
Help others learn about it - share the article on social networks
Thank you for helping us grow
Writes about what he builds himself: online stores on OpenCart, applications on Next.js, integrations and site speed. The articles carry measurements and checks a reader can repeat on their own project, not general advice. Commercial development since 2015.
Answers to common questions on the topic
A regular load test checks whether a system withstands a stream of requests. A chaos test adds forced killing of components mid-operation on top of that — not a graceful shutdown, but a SIGKILL that gives the process no chance to close correctly. The goal is to check behaviour during a real failure, not speed.
For a chat bot where a conversation is a sequence of several messages, 300 requests per second means simultaneous activity across several hundred conversations. It is not a measure of the system's maximum throughput — the test did not look for a ceiling, it checked behaviour under one fixed load.
No. The result means that in this specific test on 17 August 2026, under this specific load and these specific failures, nothing was lost. It is not a guarantee for every possible condition — production traffic, network latency between data centres, or a prolonged rather than one-off failure could produce a different result.
Because forcibly killing processes in production would mean a real outage for real customers. The local stack is the same service configuration as production, without risking live traffic. This is an honest limitation of the method, and the article states it directly rather than hiding it.
The NATS queue stores messages persistently rather than only in the process's memory, and waits for processing acknowledgment. If the service meant to process a message crashed before acknowledging it, the message stays in the queue and is retried once the service recovers.
Subscribe to our newsletter and receive useful tips, insights and news about web development, marketing and business.
Web developmentOn 21 September 2026, the first day of per-route request tracking, the internal /api/health endpoint received 13,666 requests, 60% of application traffic, and every one of them hit the shared database. Here are the two settings that fixed it, what they cost us, and how to check your own healthcheck.
Web developmentOn 21 September 2026, a type check on our site reported 154 errors, even though the build had been green every time: the check's result was simply thrown away. Inside, we found zeros in link statistics and in the analytics export, sorting that didn't sort, and tests that never ran. Here is how we worked through them, what we changed, and how to check your own project.
Web developmentOn 22 September 2026 we checked the HTTPS certificates on 116 sites we had already measured for market studies, and for 104 of them we compared the state with August. All 116 passed verification, 108 run on 89- and 90-day free certificates, and only one site's certificate has still not renewed, even though with typical settings it should have by now. We walk through the measurements and the limits of the method, and show how to check your own site with a single command.
