SQLFlow — потоковая обработка данных на SQL

★ 785

SQLFlow is a high-performance stream-processing engine where pipelines are defined simply in SQL ("DuckDB for streaming," a lightweight modern alternative to Flink). Reach for it when you need to process a stream of events on the fly without standing up a heavy stack: read from Kafka, WebSockets, and other sources, transform data with a SQL query, and write the result to PostgreSQL, Kafka topics, or cloud storage (S3) in formats like parquet and iceberg. It is built on DuckDB and Apache Arrow for speed and runs from Docker. Its focus is stream processing in SQL at high speed and low complexity, not batch BI analytics or a data warehouse: the value is describing streaming pipelines declaratively, without Java/Scala code. It is useful to data engineers who need lightweight streaming ETL without deploying and operating a full Flink cluster.