Elasticsearch Just Admitted What Columnar Databases Have Known for a Decade
On July 9, Elastic published a blog post announcing Columnar Mode, a new indexing mode shipping in Elasticsearch 9.5 that stores data in columnar form instead of Elasticsearch's traditional document model. Buried in the announcement is a sentence worth sitting with:
"When your job is to read and reason about a lot of data, columnar storage turns nearly every dimension of cost and performance in your favor."
That is not a marketing claim from a columnar database vendor. That is Elasticsearch, sixteen years into being one of the most successful document-oriented search engines ever built, writing on their own engineering blog that the architecture their whole platform was built on is the wrong one for logs, metrics, traces, telemetry, and observability data at scale.
It is a remarkable admission, and it is the right call. It is also worth understanding exactly what it does and does not change, because the gap between "Elasticsearch adds a columnar mode" and "Elasticsearch is now a columnar database" is where the real story is.
Why Elasticsearch was built the way it was
Elastic's post tells this part of the story honestly, so it is worth repeating. Elasticsearch was designed in 2010 for a document-shaped world: JSON payloads, flexible schemas, full-text search. The engine's job was to take a query, find the small number of documents that match, rank them, and return them fast. To do that, it built inverted indexes and stored the original document, the indexed text, and a per-field columnar structure (doc values) all at once, in parallel, for every field, by default.
That was a good trade for the workload it was solving. Rich documents, relatively low volume, every read valuable. You pay a storage and ingestion tax for a large capability surface, and for search-first applications, that tax is worth it.
The problem is that a huge share of what people now put into Elasticsearch is not that workload at all. It is logs. It is metrics. It is traces. It is security telemetry. It is data written once in enormous volume, queried analytically, and rarely read record by record. For that shape of data, storing everything three different ways by default is not a modest tax anymore. It is the entire cost structure of running the cluster.
The columnar bet, twenty years old, converging from everywhere
While document search was maturing, a separate lineage was proving out a different architecture, one built specifically for the "read and reason over a lot of data" problem. Elastic's post traces it well: MonetDB in the early 1990s, C-Store and Michael Stonebraker's work at MIT in the mid-2000s, then Vertica, Redshift, BigQuery, Snowflake, Parquet, Arrow, ClickHouse, DuckDB. Every serious analytics system of the last twenty years has converged on the same answer: store data by column, not by row, and nearly every dimension of query performance improves. I/O, compression, block-skipping, vectorized CPU execution, all of it, at once.
This is not a niche technique. It is the load-bearing architectural decision behind every modern data warehouse and every fast analytical engine that exists today. What is notable is not that Elastic is adopting it. It is that it took this long for the last major document-oriented holdout to concede the point. If you want a deeper primer on how columnar storage actually works and when it matters, our guide to columnar databases walks through the mechanics.
What Columnar Mode actually is, and what it is not
Here is where it is worth reading past the headline. Elastic is explicit, to their credit, about what Columnar Mode does not do:
"This is the difference between us and a pure columnar engine. A pure columnar engine does one of these jobs and asks you to bring another system for the other."
Columnar Mode is an index-level setting inside a document-oriented search engine. It is additive: you opt individual indexes into it, the document mode remains the default and stays fully supported, and the platform's core identity as a search engine does not change. Even in Columnar Mode, Elasticsearch keeps building an inverted index on any field you want searchable, and it can still regenerate the original document from the column store on demand. It is a genuinely well-engineered retrofit. It is not a rebuild.
That distinction matters more than it sounds like it should, because it determines who this actually helps.
Who this is genuinely good news for
If you already run Elasticsearch for logs, security telemetry, or observability, Columnar Mode is a real, no-migration win. Elastic's own framing, "no changes to APIs, dashboards, applications, or downstream integrations," is accurate and it is the correct way to ship an architecture change into a platform hundreds of thousands of organizations already depend on. If your data and your team are already inside the Elastic ecosystem, turning on a columnar index and watching your storage bill drop is close to a free upgrade. That is a genuine, well-executed piece of engineering, and teams in that position should take it.
Where the retrofit runs into its own ceiling
But a retrofit inherits the constraints of what it is retrofitted onto, and Elastic's post is honest about several of them, if you read closely.
The platform still runs on the JVM, still carries the operational model of a distributed search cluster, and still treats the document as a first-class concept the engine has to be able to reconstruct. Columnar Mode has to coexist with all of that, which means every design decision is filtered through "does this break search behavior" before it gets to "does this compete with a purpose-built columnar engine on cost and speed." That is not a criticism. It is the correct priority for a company whose core product is a search engine. But it also means Columnar Mode is optimizing for good enough alongside search, not for winning a benchmark against systems that were never asked to do anything else.
We have seen this in the numbers. On ClickBench, the analytical query benchmark, Arc is 6 to 9x faster than Elasticsearch on the combined score and up to 45x faster on ingestion. Elasticsearch can run analytical queries. It is just not what the engine was built for, and the gap shows.
A system built from the ground up for one job, take timestamped, high-volume data, store it in the cheapest possible open format, and make it fast to query with standard SQL, does not carry any of that weight. No JVM tax. No dual-mode complexity. No document model to preserve compatibility with. Just the columnar path, end to end, because that is the only path that exists.
That is the actual dividing line this announcement draws, not "columnar vs. document" (Elastic just conceded that argument), but purpose-built vs. retrofitted. For a team standing up a new pipeline for sensor data, telemetry, or observability from scratch, with no existing Elasticsearch investment to protect, the calculus is simple: why adopt a columnar mode bolted onto a search engine when you can start on a system that was never anything else?
The bigger signal
Strip away the vendor names and the real story here is bigger than one product update. The company best positioned to defend the document model, the one with the most to lose by conceding the point, just spent an entire engineering blog post explaining, with citations back to Stonebraker and the 2008 SIGMOD literature, why columnar storage is the correct architecture for the workloads defining the next decade of infrastructure: observability, security telemetry, IoT, AI retrieval.
When the last major holdout agrees, the debate is over. What is left is a much narrower and more useful question: for the data you are generating right now, do you want columnar as a mode you turned on inside something else, or columnar as the only thing your database has ever tried to be?
Arc is a columnar analytical database built for exactly this shape of data: high-volume, timestamped, written once and queried often. Open Parquet storage, standard SQL, no document model to carry, no JVM. See how Arc compares to Elasticsearch.