Producer
-
KinesisData Engineering 2019. 9. 20. 00:42
Kinesis Data Stream Real-time Data Stream Retention between 1 day to 365 days Ability to reprocess (replay) data Once data is inserted in Kinesis, it can’t be deleted (immutability) Data that share the same partition goes to the same shard (ordering) Producers: AWS SDK, Kinesis Producer Library (KPL), Kinesis Agent Consumers Write your own: Kinesis Client Library (KCL), AWS SDK Managed: AWS Lamb..
-
Apache KafkaData Engineering 2019. 9. 5. 01:33
1. Overview Apache Kafka is a community distributed event streaming platform capable of handling trillions of events a day. Initially conceived as a messaging queue, Kafka is based on an abstraction of a distributed commit log. Since being created and open-sourced by LinkedIn in 2011, Kafka has quickly evolved from messaging queue to a full-fledged event streaming platform. Founded by the origin..