Kafka

From MyInfoRepo
Jump to navigation Jump to search

Kafka manages events, making processing and reading of the events and their data possible. Kafka allows creation of Producers, Consumers and Topics. The producers can publish to brokers on a topic. Consumers subscribe to topics and can request entries from the brokers (logs) from a point onward. There is an at-least-one policy on the reads, meaning the logs do not get deleted on consumption and can be requested again. More on this in the consumer section.

Topics

Topics are like containers of similar events. Data in topics do not have to be unique. Events are written to topics, they are appended and immutable. Since logs are simple, we can get a lot of performance on fetching them.

partitions

brokers

Testing setup

vagrant plugin install vagrant-hostmanager Also, turning hypervisor off: bcdedit /set hypervisorlaunchtype off