Data IntegrationPostgresIcebergCDCLakehouseData Engineering

Postgres-to-Iceberg Replication That Analysts Query Through the Postgres Wire

Streambed, built by Cloudflare's former Postgres tech lead, streams WAL changes into Iceberg on S3 and serves analytical queries back through the Postgres protocol via embedded DuckDB, drawing 129 HN points and unusually specific practitioner engagement: TOAST column handling, DDL sync, comparisons against olake, supabase/etl, and Sequin, with one commenter noting the alternatives all have subtle issues. Reliable Postgres CDC to open table formats with zero new query surface is infrastructure teams keep half-building themselves.

Overall

Problem Statement

A platform team fields BI queries that throttle the production database, so they add read replicas that still buckle under analytical scans, then evaluate CDC pipelines: Debezium needs Kafka operations, hosted CDC costs scale punitively, and the new Go-based tools each carry caveats like TOAST columns requiring REPLICA IDENTITY FULL, raised verbatim by an HN commenter from production experience. Meanwhile dashboard teams just want a Postgres endpoint that does not endanger production, the exact interface Streambed's author built after hitting this at Cloudflare.

The Idea

A replication service for data and platform teams that turns a production Postgres into an analytics-ready Iceberg lakehouse queryable through the Postgres wire their BI tools already speak.

Why Now

Iceberg won the open-table-format consolidation in 2025-2026, and every Postgres shop now wants WAL-to-Iceberg without running Debezium and Kafka. The HN thread documents a field of immature alternatives with named subtle issues, while the analytics-on-Postgres pain, read replicas strangled by long queries, predates them all and worsens with data volume.

Target User

Platform and data engineers at Postgres-centric companies with growing analytical load

Target Market

Data replication and lakehouse infrastructure

MVP Scope

Postgres logical replication to Iceberg with snapshot plus streaming.
TOAST handling without REPLICA IDENTITY FULL via object-store backfill.
Automatic DDL synchronization for common change classes.
Postgres-wire query endpoint with DuckDB execution.
No MySQL, no multi-source joins, no BI features at MVP.

Validation Plan

1Run correctness harnesses against the public CDC edge-case suites and publish results versus Sequin and OLake within one quarter.
2Measure replication lag under 10 seconds at 10k writes per second across 5 design partners.
3Convert 8 companies to managed cloud within two quarters.
4Track read-replica retirement: document infrastructure cost reductions per customer.
5Interview 15 teams running Debezium on what would justify a migration within 45 days.

Product Workflow

1Connects to Postgres logical replication with snapshot backfill.
2WAL changes write to Iceberg on S3 with schema evolution handling.
3DDL changes propagate to Iceberg tables automatically.
4Embedded DuckDB serves analytical queries over the Postgres wire protocol.
5Monitoring tracks replication lag and TOAST-handling completeness.

Score Rationale

Market pull

Demand68

Pain68

Pay intent64

Timing74

Friction

Competition32

Build difficulty54

Acquisition difficulty50

Risk56

Conviction

Novelty52

Confidence62

Monetization

Open-source core with a managed cloud priced on change volume and storage under management, the standard infrastructure motion. The buyer comparison is a Debezium-plus-Kafka operations burden or five-figure hosted CDC contracts; pricing at the cost of the read replica it retires makes the procurement conversation short.

Strategic Upside

The Postgres-wire query endpoint is the trojan horse: BI tools connect without migration, and once analytical workloads live on Iceberg the platform extends to multi-source ingestion and warehouse replacement for mid-size companies; the moat question is whether reliability on the long tail of replication edge cases, the thing commenters said everyone gets subtly wrong, can stay ahead of well-funded rivals.

Acquisition Channels

•Postgres community channels, conferences, and the extension ecosystem.
•Data engineering communities where CDC tool comparisons recur weekly.
•Open-source GitHub motion with the HN launch already seeding adoption.
•Iceberg ecosystem partnerships: catalog vendors and query engines.
•Technical content on TOAST, DDL, and edge-case correctness benchmarks.

Competitors

Risks

Databricks, Snowflake, and Confluent all ship managed Postgres-to-Iceberg paths with enterprise distribution.
The HN thread flagged the maker's replies as AI-written and parts of the codebase as vibe-coded, a trust deficit competitors will quote.
CDC correctness failures corrupt analytics silently, and one data-loss incident ends adoption in this buyer segment.
Supabase can bundle its ETL natively for its large hosted-Postgres base.
DuckDB-over-wire query compatibility gaps with real BI tool SQL will generate a long support tail.

Counter Evidence

pg_lake, DuckLake, and Postgres-native extensions keep narrowing the need for a separate replication service.
Commenters noted transformation still happens somewhere, so the no-ETL pitch overstates what the tool removes.
Mid-size teams may simply buy ClickHouse Cloud, which solved Postgres CDC ingestion with low-single-digit-second latency already.

The full brief is free to read

Create a free account to unlock the complete build-ready brief for “Postgres-to-Iceberg Replication That Analysts Query Through the Postgres Wire”, including:

Problem statement & the core idea
Why now & target user / market
MVP scope & feature boundaries
Step-by-step validation plan
Score rationale across 11 dimensions
Monetization model & pricing angle
Competitors with links
Acquisition channels & go-to-market
Risks & counter-evidence

More Data Integration opportunities

Data Integration

Connector Health Monitor and Pipeline Reliability Dashboard for Airbyte

Buyer reviews for Airbyte Cloud consistently highlight reliability concern friction, specifically: Connector quality varies wildly. Salesforce connector is solid; HubSpot connecto; Sync failure debugging is painful. Error messages are often generic Java stack t. This pain is concentrated among Data engineers managing Airbyte connector reliability across production pipelines and creates demand for a focused tool that resolves the gap without requiring a platform switch. The Data Integration category has matured enough that users have committed to Airbyte Cloud as infrastructure, making adjacent tooling more viable than platform replacement.

View opportunity Data Integration

Self-Serve Postgres CDC Replication With TOAST and Schema-Drift Handling

Artie opened self-serve access to its real-time database-to-warehouse replication and the first HN question was whether it handles TOAST columns and schema drift better than Debezium. Mid-market data teams still get paged for broken CDC pipelines weekly. A self-serve CDC product that makes the Debezium failure modes (TOAST, drift, replica identity) invisible is a wedge with proven willingness to pay.

View opportunity