from my own experience I find that while setting up kafka cluster on AWS we face some issues so just want to highlight them.
a. First setup zookeeper cluster. Let’s say 3 node cluster. Modify each node zoo.conf to publish ip address as internal IP address.
server.id=<internal aws ip1>:2888:3888
server.id=<internal aws ip2>:2888:3888
server.id=<internal aws ip3>:2888:3888
b. Go to kafka server.properties and change broker.id
node1 — broker.id 0
node2 — broker.id 1
node 3 — broker.id2
Change bind address to <internal ip> so that it is not accessible from outside
Change advertised.host.name to <internal ip>
List all zookeeper nodes within your own cluster under setting zookeeper.connect
Start all kafka nodes and you they should be able to create a cluster
Create a topic, publish a message using kafka-console-producer and see if there are not errors.