Apache Kafka FAQ

These are some common questions I've received in working with Apache Kafka. I'll keep adding more detail What is Apache Kafka ? Apache Kafka is a distributed , partitioned, replicated commit log service What is some common terminology used when discussing Kafka? Event Producer,Partition,Routing,Topic,Broker,Sink,Stream consumers What protocols are supported by Kafka? JSON , Avro Do you have a diagram giving a very high level view of Kafka? This diagram - gives you a flavour of possibilities. What is Zookeper? Zookeper manager distributed processes. It is compulsory part of an Apache Kafka cluster.Kafka Brokers use Zookeper for cluster controller election and... Read more →

Kafka cheatsheet

This cheatsheet covers regularly used commands across different deployments of Kafka. --view error logs on Confluent Kafka confluent log connect tail -f --How to retrieve broker properties from a client step 1 - Enter zookeeper shell zookeeper-shell localhost:2181 step 2 - get /brokers/ids/0 --Check Confluent Current confluent current --Check Confluent version confluent version --List Connectors currently loaded confluent status connectors --List Kafka topics /bin/kafka-topics --zookeeper localhost:2181 --list --Delete a Kafka topic bin/kafka-topics --zookeeper localhost:2181/chroot --delete --topic my_topic_ --Set error logging to DEBUG level edit $(confluent_home)/etc/kafka/connect-log4j.properties log4j.rootLogger=DEBUG,stdout restart Connect --List configured Connectors curl localhost:8083/connectors Read more →