Estuary reposted this
Iceberg is quietly becoming the standard for how serious data teams structure their lakehouses. Last week’s Cloudflare announcement was another proof point -- offering managed Iceberg tables and a built-in catalog. When a company like Cloudflare (at the scale and technical depth they operate) bets on Iceberg, it’s worth paying attention. And they’re not alone. 1. Snowflake has Polaris. 2. Databricks backs Unity Catalog. 3. AWS supports it via Glue, Athena and S3 tables. 4. Dremio, Starburst, MotherDuck, the list just keeps growing. At Estuary, we’ve been watching this play out, and more importantly, building around it. We see Iceberg as the table format that finally gives companies flexibility without tradeoffs. Governance without vendor lock-in. Performance without being boxed into a single processing model. But what’s been missing is great real-time integration. We’ve been working hard to fill that gap. Our Iceberg connector now supports: - Glue, Unity, Polaris, and other REST catalogs - S3 Tables - Streaming or batch ingestion from hundreds of sources - Schema evolution that just works - Merge queries for use cases that need precision (think CDC-style upserts) - And soon, built-in table maintenance If your team is starting to treat Iceberg not just as a format, but as an operational data layer—it might be time to talk. This is where the modern data stack is going and we're building for it.
Polaris IS an Iceberg catalog. Unity is a competing technology.
By maintenance, you mean compaction?
David Yaffe I’m curious when OLTP workloads are going to support Iceberg(or a variant) as well. I see so many possibilities in this area. Only if you think about the possibility of reducing the costs of operational databases by using S3 or Cloudflare as operational data layer…
Check out how we support Apache Iceberg workloads: https://estuary.dev/solutions/technology/apache-iceberg/
I think we still need better query engine, dremio is great but building reflections is major overhead.
Director of Product Management at Dremio
4dDavid Yaffe I see that you mention Dremio but you don't support it as a destination https://estuary.dev/integrations/. I'd love to change that and find a way for our companies to work better together.