Kafka on Windows: A Step-by-Step Installation and Setup Guide

Apache Kafka is a powerful and versatile distributed streaming platform that enables real-time data pipelines and messaging systems. While primarily associated with Linux environments, Kafka can also be installed and used on Windows operating systems. Let’s dive into how to get it up and running.

Prerequisites

• Java: Kafka is built on Java, so you’ll need a Java Development Kit (JDK) installed (version 8 or above). You can download it from Oracle’s website or use alternatives like OpenJDK.
• Windows Subsystem for Linux (Optional): While not strictly mandatory, the Windows Subsystem for Linux (WSL) provides a smoother experience when working with tools like Kafka. See Microsoft’s documentation on how to install WSL.

Steps

1. Download Kafka

• • Visit the Apache Kafka website’s download page: 
  • Choose the latest stable release under the “Binary downloads” section.
  • Select the download link suitable for your Scala version (2.12 or 2.13).

1. Extract the Archive

• • Extract the downloaded .tgz file to your desired location (e.g., C:\kafka).

1. Environment Variables (Optional)

• • It is convenient to add Kafka’s bin directory to your system’s PATH environment variable. This allows you to access Kafka commands directly from the command prompt.

1. Start Zookeeper

• • Kafka depends on ZooKeeper for coordination. To do so, change to the Kafka directory (e.g., C: Kafka) and navigate to the config folder.
  • Open the zookeeper.properties file and find the dataDir property. Modify the path to a suitable location for storing Zookeeper data.
  • Run the following command in your command prompt:

1. Bash
2. bin\windows\zookeeper-server-start.bat config\zookeeper.properties
3. Use code 
4. content_copy
5. Start Kafka Broker

• • Open another command prompt window.
  • Open server.properties in the config directory and adjust any settings as needed (though the defaults work for getting started).
• Run the following command:
• Bash
• bin\windows\kafka-server-start.bat config\server.properties
• Use code 
  • content_copy

Basic Testing

1. Create a Topic

• Open a new command prompt window and run:
• Bash
• bin\windows\kafka-topics.bat –create –topic test-topic –bootstrap-server localhost:9092
• Use code 
  • content_copy

1. Produce Messages

• In the same window, type:
• Bash
• bin\windows\kafka-console-producer.bat –broker-list localhost:9092 –topic test-topic
• Use code.
  • content_copy
  • Now type some messages and press Enter after each one.

1. Consume Messages

• Open another command prompt and run:
• Bash
• bin\windows\kafka-console-consumer.bat –bootstrap-server localhost:9092 –topic test-topic –from-beginning
• Use code
  • content_copy
  • You should see the messages you typed earlier.

Congratulations! You’ve successfully installed and configured Kafka on your Windows machine.

Additional Notes:

• Consider using WSL for a more Linux-like experience with Kafka.
• Explore resources like the Apache Kafka Quickstart for further learning.
• Popular GUI tools like Kafka Tool can simplify Kafka management on Windows.