Server, Storage Engine, Protocol, Client: Suspects of a MySQL Performance Mystery

<p>While optimizing a new heap storage engine across both MySQL and a PostgreSQL-based database we encountered a puzzling result: while on MySQL the throughput stalled below 500k tpmC, on the other database it achieved over 1 million tpmC. The mystery deepened when three different TPC-C benchmarks each told a conflicting story about MySQL’s speed.</p> <p>This talk details the systematic investigation to resolve these contradictions and reclaim the lost performance. We’ll walk through the methodical process of isolating variables across the entire software stack, dissecting benchmark implementations, profiling execution end-to-end with advanced tools, analyzing client/server protocol behavior, and comparing query optimization plans.</p> <p>The investigation revealed that the performance gap was not caused by a single flaw, but by a cascade of inefficiencies, in multiple areas of the stack. Subtle issues in query planning, protocol handling, and client-side implementation conspired to create overwhelming overhead. By addressing these interconnected problems holistically – through optimizer fixes, protocol enhancements, and client improvements – we transformed MySQL’s performance profile to reveal the engine’s true potential.</p> <p>The outcome was a dramatic turnaround: with additional improvements the performance of the new engine on MySQL reaches almost 2 million tpmC now. </p> <p>This case study underscores a critical lesson: database performance, for OLTP workloads in particular, is determined not by any single component, but by the precise alignment of the entire database stack, from the client down to the storage engine.</p>