Kafka vs Redpanda Performance
published on 2023/05/16
After having spent a fair amount of time benchmarking both I would say that when Redpanda gets the right workload it can really shine - the problem is that there are many workloads where it doesn’t. Its performance is a little more precarious than Apache Kafka. Batch sizes need to be not too small, throughput shouldn’t be too high on high partition workloads and the drives need to be adequately provisioned with enough empty space to allow for the random IO nature of its storage layer. Low throughput is where Redpanda has its most robust end-to-end latency results, then as you increase throughput, the specifics of the workload can have a disproportionate impact on performance. It is also extremely sensitive to drive latency.
This is a wonderful write up. We need more write up about our infrastructure.