Exploring AWS !!

Day 67

Amazon Kinesis Data Streams

This service is mainly used for processing real time data streams. Data processing applications are known as Kinesis Data Streams applications. This application reads records as streams. These applications use kinesis client library, and run on EC2.

Processed records are sent to dashboards, can be used for generating alerts, can change pricing and advertising strategies, or the data can be sent further to various AWS services.

Features:

For quick and continuous data intake and aggregation. The data can be either log data, application logs, social media, market data feeds, or web clickstream data. The response time is in real time.

Scenarios:

• The data can be pushed directly for processing further. Like let’s say, system and application logs are pushed and are immediately available for processing in seconds. This will prevent the loss of the data log, even if the application fails. Kinesis Data Streams provides accelerated data feed intake because you don’t batch the data on the servers before you submit it for intake.

• The data collected in Kinesis Data Streams can be used for data analysis and reporting in real time.

• Real-time-data analytics, combines the power of parallel processing with the value of real-time data.

• You can create Directed Acyclic Graphs (DAGs) of Kinesis Data Streams applications and data streams.

Benefits:

• Though various data streaming problems can be solved, the common use-case is the real-time aggregation of data, further loading the aggregate data into a data warehouse.

• Durability and elasticity is ensured.

• Kinesis Data Streams application starts to consume the data from the stream immediately once the data gets added.

• Multiple Kinesis Data Streams applications consume data from a stream, so that multiple actions, like archiving and processing, can take place concurrently and independently.

• The Kinesis Client Library enables fault-tolerant consumption of data from streams and provides scaling support for Kinesis Data Streams applications.