Data processing: Apache Flink Table Store 0.2 offers unified storage
The team behind the framework introduced Flink Table Store, a store for building dynamic tables for streaming and batch processing.
With Flink Table Store, the developers of the stream processing framework want to provide a uniform store that can be used to build dynamic tables for streaming and batch processing in Flink. The software also supports fast data acquisition and data queries.
Versatile method of reading/writing
As described on the Flink website, Flink Table Store is a new type of refreshable data lake that has the following characteristics:
- High data throughput with good query performance.
- Query with primary key filters that should be up to 100ms fast.
- Streaming reads are also available on Lake Storage. Lake Storage also integrates with Apache Kafka to provide second-level streaming reads.
Flink Table Store uses a versatile method to read and write the data and perform OLAP queries. For example, data can be read from historical snapshots in batch mode or from the last offset in streaming mode. As a third possibility, Flink also supports reading incremental snapshots in a hybrid way.
For write operations, the software supports both streaming synchronization from database change log (CDC) and offline data insert/overwrite in batch mode. Furthermore, Table Store not only supports Apache Flink, but also other computing engines such as Apache Hive, Apache Spark and Trino.
Developers who want to use Flink Table Store with these features can find comprehensive instructions on the Apache Flink website to get started quickly.
