Consuming Many Data Streams At Once

Intro

Apache Kafka

  • Data streaming technology.
  • For event driven architecture.
  • Kafka is distributed horizontally.
  • Kafka is redundant backuping itself up.

Kafka for microservices:

  • Breaking a monolithic system of subsequent programs down into independent microservices.
  • Kafka as messaging central bus bonds all independent microservices.
  • Kafka decreases costs of microservices integration.
  • Services providing data of the same nature in Topic are called a producers.
  • Service that takes the data from Topic is called a consumer.

Features

App includes following features:

  • Apache Kafka
  • Pykafka
  • JSON

Demo

Application:

  • Three public transports as producers send their coordinates to Kafka server.
  • Coordinates are bing send all the time as a data stream.
  • One central consumer consumens data streams and tracks them down.

Setup

Following installation required:

  • pip install pykafka
  • Java
  • Kafka
  • Zookeper

Source Code

You can view the source code: HERE