How to Work With Kafka Consumer and Producer in Cloudera

In Section You Will Able to Learn How to Work With Kafka Consumer and Producer
First Configure Kafka Into Cloudera Follow Link Configure Kafka Into Cloudera

Login to GW Node Or Any Broker Node

You Can Use hdfs User Or Your Own User But That User Should Have Kafka Service Access Permission

Create Topic

               
su - hdfs
/usr/bin/kafka-topics --create --zookeeper cmhost.realmprod.local:2181,slave1.realmprod.local:2181,slave2.realmprod.local:2181/kafka --replication-factor 2 --partitions 4 --topic mytopic1 
                
               
how to create kafka topics cloudera
List All Kafka Topics
               
/usr/bin/kafka-topics --zookeeper cmhost.realmprod.local:2181,slave1.realmprod.local:2181,slave2.realmprod.local:2181/kafka --list
                
               
Start Writting Kafka Topics Using Producer
               
/usr/bin/kafka-console-producer --broker-list cmhost.realmprod.local:9092,slave1.realmprod.local:9092,slave2.realmprod.local:9092 --topic mytopic1
                
               
how write data into kafka topic cloudera
Now You Can Read Data Using Consumer

Open New Terminal and Don't Close Producer Terminal

               
/usr/bin/kafka-console-consumer --bootstrap-server cmhost.realmprod.local:9092,slave1.realmprod.local:9092,slave2.realmprod.local:9092 --topic mytopic1 --from-beginning

                
               
how to read data using kafka consumer cloudera
If You Learn to Know More Checkout My Youtube Channel