AWS published a case study this month about Comcast's analytics platform for cable modem telemetry. The headline numbers are good: 80x higher concurrent request volume, 38% lower cost, and roughly 350 TiB of data served from Iceberg tables on Amazon S3 Tables.
It is a solid piece of engineering and worth reading in full. But the two sentences that matter most in that post are not the ones in the headline. They are the ones that explain why the team made the choices they made, and they describe a problem that most teams running telemetry at scale have quietly accepted as the cost of doing business.
They picked a storage layer to stop running compaction jobs
Here is AWS describing the decision, verbatim:
they also wanted to alleviate the operational burden of maintaining compaction scripts and snapshot management
Read that again. A team at Comcast's scale, with Comcast's engineering depth, selected a managed storage product substantially because they did not want to keep babysitting table maintenance.
This is the part of the lakehouse story that the marketing never covers. Iceberg gives you open tables, time travel, schema evolution, and engine independence. What it does not give you is a table that stays fast on its own. High-frequency ingest produces a lot of small files. Small files destroy query performance and inflate request costs. So somebody writes a compaction job. Then somebody writes snapshot expiration, because the metadata grows without bound. Then orphan file cleanup. Then a scheduler to run all of it, and monitoring for when the scheduler fails, and a runbook for when compaction collides with a long-running read.
None of that is analytics. It is plumbing that exists because the write path and the table format were never designed together.
Comcast's answer was to buy their way out of it, and S3 Tables is a genuinely good answer if you are already all-in on AWS. But notice what the answer implies: the maintenance burden was large enough to influence a platform decision at a company whose broadband network serves tens of millions of customers.
The savings came from not making copies
The second sentence worth pulling out:
The bulk of the savings comes from avoiding duplicate copies of data
38% cost reduction, and the dominant contributor is not better compression or cheaper compute. It is deleting redundancy.
This is the actual economics of telemetry, and almost nobody prices it honestly. The standard architecture has your data at rest in three or four places at once: in the operational database that received the writes, in the object store the ETL job wrote out, in the warehouse that loaded it for analytics, and often in a separate cold archive for retention compliance. Each copy has storage cost, each hop has egress and compute cost, and each transformation has a failure mode and an on-call rotation attached to it.
The industry talks about this as a data movement problem. It is more useful to think of it as a data duplication problem, because movement is only expensive when it produces another copy that you then have to keep, secure, back up, and reconcile.
Comcast cut copies and saved 38%. That is the finding. The interesting question is how much further that number goes if you cut more of them.
The tier that is still there
Look at the pipeline as described. Telemetry arrives. Apache Airflow runs on AWS Fargate to orchestrate ingest into Iceberg. A GraphQL layer sits on top for query access.
The Airflow tier is doing real work, and given the constraints, it is the right design. But step back and ask what it is fundamentally for. It exists to move telemetry from wherever it lands into a table format that analytics tools can read. It is a translation layer between the system that receives data and the system that queries data.
That layer is only necessary if those are two different systems.
If the database receiving the telemetry writes Parquet natively and registers those same files as an Iceberg table by reference, no copy is made and no orchestration is needed. The operational store and the analytical table are the same bytes. There is no ingest DAG to schedule because there is no ingest step. The table is a view over files that already exist.
This is not a hypothetical architecture. It is what happens when you stop treating the query format as an export target and treat it as the storage format from the start. The moment your database's on-disk layout is open Parquet, the pipeline to get data into open Parquet becomes redundant by definition.
Where this argument stops
Being straight about the boundaries, because the Comcast setup is not a mistake and pretending otherwise would be dishonest.
S3 Tables genuinely solves the maintenance problem. If your telemetry originates in AWS, lands in AWS, and is queried from AWS, then buying managed table maintenance is a rational trade and the operational burden really does go away. The compaction argument does not beat a managed service on its own turf.
What a managed service cannot do is run somewhere else. And a large share of the world's telemetry does not originate in a region. It originates at a cell site, on a factory floor, in a substation, on a vehicle, at a remote gateway, inside an air-gapped enclave. For those workloads the question is not which managed table service to buy. It is whether you can get analytical tables at all without first shipping everything to a cloud you may not be allowed to use, over a link that may not have the bandwidth, at an egress cost that may dwarf the storage.
That is the real split. Not compaction versus managed compaction. It is whether the architecture works when the data cannot leave.
The general lesson
The transferable insight from the Comcast post is not about Iceberg or S3 or GraphQL. It is this: they found 38% by auditing their copies, and they found a platform decision by auditing their maintenance jobs.
Both of those audits are available to anyone, and almost nobody runs them. So run them.
Count the copies. For one telemetry stream, write down every place a byte of it exists at rest. Count storage cost per copy and transfer cost per hop. Most teams find three or four copies and cannot articulate why more than two exist.
Count the maintenance. List every scheduled job whose only purpose is keeping storage healthy. Compaction, snapshot expiry, orphan cleanup, reindexing, tier migration, vacuum. Add up the engineering hours those jobs consume in a year, including incidents. That number is usually larger than anyone expects, and it is pure overhead.
Then ask, for each copy and each job: does this exist because the workload needs it, or because two systems that should have been one system need a bridge between them?
The best cost optimization is not making something cheaper to run. It is stopping running something that did not need to exist.
Source: How Comcast scaled analytics 80x while cutting costs 38% with GraphQL and Amazon S3 Tables, AWS Storage Blog.
Arc is an open source time-series database for telemetry you need to keep. Open Parquet storage, Iceberg tables by reference, standard SQL, one static binary.