Integration
D2 · Resilient
D3 · Performance
~1 phút đọc

Kinesis & streaming data

Xử lý stream real-time. Data Streams, Firehose, Data Analytics, Video Streams.

kinesis
firehose
streaming
real-time

Sơ đồ tổng quan

Đang tải sơ đồ…

Kinesis Data Streams

  • Stream gồm nhiều shard; mỗi shard 1 MB/s in, 2 MB/s out, 1000 records/s in.
  • Retention 24h mặc định, tối đa 365 ngày.
  • Producer: SDK, KPL, Kinesis Agent, CloudWatch Logs subscription.
  • Consumer: SDK, KCL, Lambda, Kinesis Data Analytics, Firehose.
  • On-demand mode: tự scale shard.
  • Enhanced Fan-out: mỗi consumer có throughput riêng 2 MB/s (push, HTTP/2).

Kinesis Data Firehose

  • Fully managed, near real-time (buffer 1 MB / 1 phút).
  • Target: S3, Redshift, OpenSearch, Splunk, HTTP endpoint.
  • Có thể transform data với Lambda, convert JSON → Parquet/ORC.
  • Không giữ data → không replay được; dùng cho ingest → storage.

Kinesis Data Analytics

  • Bây giờ gọi là Amazon Managed Service for Apache Flink.
  • Chạy SQL / Flink job trên stream (Kinesis, Kafka).
  • Output ra Kinesis, Firehose, Lambda.

So sánh Kinesis vs SQS

  • Kinesis: stream, multiple consumers, replay, ordered theo shard, giữ data lâu.
  • SQS: queue, 1 consumer group, delete-after-consume, không replay.
Nội dung liên quan
Quiz liên quan
Integration & Events