Kafka on Windows: A Step-by-Step Guide

Apache Kafka is a powerful distributed streaming platform for building real-time data pipelines and streaming applications. While primarily associated with Linux-like environments, Kafka can run smoothly on Windows systems. Here’s how to get it up and running on your Windows machine.

Prerequisites

• Java: Kafka is built with Java, so you’ll need a Java Development Kit (JDK) installed. Version 8 or later is required. Download and install from the official Oracle website. 
• Windows Subsystem for Linux (Optional): WSL 2 provides a Linux-like environment within Windows, making the Kafka installation process resemble standard Linux setups. Microsoft’s documentation provides instructions on setting up WSL.

Step 1: Download Apache Kafka

1. Navigate to the official Apache Kafka downloads page: 
2. Under “Binary downloads,” select the latest stable release of Kafka.
3. Choose the download option with your preferred Scala version (2.12 or 2.13 are common choices).

Step 2: Extract the Archive

1. Once downloaded, extract the downloaded .tgz file to your desired location (example: C:\kafka). This will be your Kafka installation directory.
2. You can use tools like 7-Zip or WinRAR to extract the file.

Step 3: Configure Environment Variables (Optional)

• This step is recommended for easier use of Kafka commands.

1. Search for “environment variables” in the Windows Start menu.
2. Click on “Edit the system environment variables.”
3. Under “System variables,” click “New…”
4. Variable Name: KAFKA_HOME Variable Value: (The path to your Kafka installation directory, e.g., C:\kafka)
5. Click “OK”.
6. Find the Path variable under “System variables,” and click “Edit.”
7. Click “New” and add %KAFKA_HOME%\bin\windows
8. Click “OK” on all windows.

Step 4: Start Zookeeper

Apache Kafka utilizes Zookeeper to manage the cluster.

1. Open a command prompt or terminal (or a WSL terminal, if using WSL).
2. Navigate to your Kafka installation directory (e.g., cd C:\kafka)
3. Run the following command to start Zookeeper:
4. bin\windows\zookeeper-server-start.bat config\zookeeper.properties

Step 5: Start Kafka Server

1. Open a new command prompt or terminal.
2. Navigate to your Kafka installation directory.
3. Run the following command to start the Kafka server:
4. bin\windows\kafka-server-start.bat config\server.properties

Step 6: Verification

To ensure Kafka is running:

1. Open a new command prompt or terminal.
2. Run the following command to create a test topic:
3. bin\windows\kafka-topics.bat –create –topic test-topic –bootstrap-server localhost:9092
4. You should see a confirmation message: “Created topic test-topic.”

Congratulations! You’ve successfully installed and started Apache Kafka on your Windows machine.

Additional Notes:

• Firewalls: Ensure your firewall is configured to allow connections to Kafka and Zookeeper ports (the default ports are 2181 for Zookeeper and 9092 for Kafka).
• WSL: If utilizing the Windows Subsystem for Linux, the setup closely resembles a Linux installation.