A minimal configuration for a dockerized kafka project.

Related tags

Miscellaneousdockerkafkaapachezookeeper
Overview

Docker Kafka Quickstart

A minimal configuration for a dockerized kafka project.

Usage:

  • Run this command to build kafka and zookeeper containers, and create the network kafka-network, where your containers can access the endpoint kafka:29092.
# creates the network `kafka-network` and build and
# runs the containers for zookeeper and kafka
docker-compose -f "docker-compose.kafka.yml" up -d --build # remove -d if you want to see logs in the same shell
  • You can then run as many broker/consumer containers on your kafka-network, in this case: generator is a broker, and indicator is a consumer.
# runs the generator and indicator containers
docker-compose up -d --build
  • If you check the logs in your generator container you should be seeing the messages sent to the queuing.sentences.test topic:
### KAFKA_BROKER_URL kafka:29092
09:35:09.135969 {'time': '09:35:08.107419', 'player': 'player2', 'sentence': 'sDZRJk nwENFG I sR VOOZNWll aGDdZV'}
09:35:10.138498 {'time': '09:35:10.138207', 'player': 'player2', 'sentence': 'vgIXxE  sa'}
09:35:11.141023 {'time': '09:35:11.140690', 'player': 'player1', 'sentence': 'E raH jzIeKHqXZF JGGB   tx  M oLusi '}
09:35:12.142630 {'time': '09:35:12.142360', 'player': 'player2', 'sentence': 'WxsPlteC  yxh  ogME dvbtFn l b TxVB  gaMFZe'}
09:35:13.144693 {'time': '09:35:13.144454', 'player': 'player1', 'sentence': ' aIRVm W TeIcKuigy  HPLOvJrQSXSl ZlLHlL FRH'}
09:35:14.146182 {'time': '09:35:14.145966', 'player': 'player1', 'sentence': 'cmMH k phZ LapRdGv C geTDzIT'}
09:35:15.147609 {'time': '09:35:15.147378', 'player': 'player2', 'sentence': 'QihJPzjFM WmkmDzMLl DkyydRVeyEI Rm'}
09:35:16.149716 {'time': '09:35:16.149502', 'player': 'player1', 'sentence': '  C '}
09:35:17.151371 {'time': '09:35:17.151155', 'player': 'player2', 'sentence': ''}
09:35:18.152736 {'time': '09:35:18.152508', 'player': 'player1', 'sentence': 'Oh tgfSl tY iBrxMHf aNX h'}
...
  • And the logs for indicator show the messages recieved on the queuing.sentences.test topic:
### KAFKA_BROKER_URL kafka:29092
Connecting...
Connected to queuing.sentences.test topic!
09:35:13.320414 {'time': '09:35:13.144454', 'player': 'player1', 'sentence': ' aIRVm W TeIcKuigy  HPLOvJrQSXSl ZlLHlL FRH'}
09:35:14.150851 {'time': '09:35:14.145966', 'player': 'player1', 'sentence': 'cmMH k phZ LapRdGv C geTDzIT'}
09:35:15.279800 {'time': '09:35:15.147378', 'player': 'player2', 'sentence': 'QihJPzjFM WmkmDzMLl DkyydRVeyEI Rm'}
09:35:16.153846 {'time': '09:35:16.149502', 'player': 'player1', 'sentence': '  C '}
09:35:17.156841 {'time': '09:35:17.151155', 'player': 'player2', 'sentence': ''}
09:35:18.156372 {'time': '09:35:18.152508', 'player': 'player1', 'sentence': 'Oh tgfSl tY iBrxMHf aNX h'}
09:35:19.160985 {'time': '09:35:19.154674', 'player': 'player2', 'sentence': 'qgSAzkUNm  fBQyruw S U   PvLnharO '}
09:35:20.183376 {'time': '09:35:20.157269', 'player': 'player1', 'sentence': 'OlX SW  a gcpXVGHxW'}

Refs:

For more information about using kafka with Docker: https://hub.docker.com/r/wurstmeister/kafka/

Owner
Nouamane Tazi
Software Engineer | AI Passionate. Seeking end-of-studies 6-month internship.
Nouamane Tazi
GitHub Repository
🎉 🎉 PyComp - Java Code compiler written in python.

🎉 🎉 PyComp Java Code compiler written in python. This is yet another compiler meant for babcock students project which was created using pure python

Alumona Benaiah 5 Nov 30, 2022
Tucan Discord Token Generator - Remastered

TucanGEN-SRC Tucan Discord Token Generator - Remastered Tucan source made better by me. -- idk if it works anymore Includes: hCaptcha Bypass Automatic

Vast 8 Nov 04, 2022
Um pequeno painel de consulta grátis.

[PAINEL-DE-CONSULTA 3.8(BETA)] · Confira meu canal do YouTube. Clique aqui! Nota: Próxima Atualização será a última com coisas novas, o resto será par

276 Jan 05, 2023
This script can be used to get unlimited Gb for WARP.

Warp-Unlimited-GB This script can be used to get unlimited Gb for WARP. How to use Change the value of the 'referrer' to warp id of yours You can down

Anix Sam Saji 1 Feb 14, 2022
XAC HID Gamepad implementation for CircuitPython 7 or above.

CircuitPython_XAC_Gamepad Setup process Install CircuitPython 7 or above in your board. Add the init.py file under \lib\adafruit_hid directory of CIRC

5 Dec 19, 2022
A minimal configuration for a dockerized kafka project.

Docker Kafka Quickstart A minimal configuration for a dockerized kafka project. Usage: Run this command to build kafka and zookeeper containers, and c

Nouamane Tazi 5 Jan 12, 2022
Automator anble you to create automations on your system

WELCOME TO AUTOMATOR BETA This programm is able to create automations on your system. This programm is only an experimantal release; infact it works v

Davide 1 Jan 12, 2022
Automatic and platform-independent unpacker for Windows binaries based on emulation

_ _ __ _ __ _ | | | | / / (_) \ \ | | | | | |_ __ | | _ | | _ __ __ _ ___| | _____ _ __

514 Dec 21, 2022
Beancount: Double-Entry Accounting from Text Files.

beancount: Double-Entry Accounting from Text Files Contents Description Documentation Download & Installation Versions Filing Bugs Copyright and Licen

2.3k Dec 28, 2022
ALSPAC data analysis studying links between screen-usage and mental health issues in children. Provided data has been synthesised.

ADSMH - Mental Health and Screen Time Group coursework for Applied Data Science at the University of Bristol. Overview The data set that you have was

Kai 1 Jan 13, 2022
VHDL to Discrete Logic on PCB Flow

PCBFlow Highly experimental set of scripts to transform a digital circuit described in a hardware description language (VHDL or Verilog) into a discre

Tim 77 Nov 04, 2022
PwnDatas-DB-Project(PDDP)

PwnDatas-DB-Project PwnDatas-DB-Project(PDDP) 安裝依賴： pip3 install pymediawiki 使用： cd /opt git https://github.com/JustYoomoon/PwnDatas-DB-Project.git c

21 Jul 16, 2021
MODeflattener deobfuscates control flow flattened functions obfuscated by OLLVM using Miasm.

MODeflattener deobfuscates control flow flattened functions obfuscated by OLLVM using Miasm.

Suraj Malhotra 138 Jan 07, 2023
Быстрый локальный старт

Быстрый локальный старт

Anton Ogorodnikov 1 Sep 28, 2021
HPomb Is Socail Engineering Tool , Used For Bombing , Spoofing and Anonymity Available For Linux And Android(Termux)

HPomb v2020.02 Coming Soon Created By Secanonm HPomb Is Socail Engineering Tool , Used For Bombing , Spoofing and Anonymity Available For Linux And An

Secanonm 10 Jul 25, 2022
poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions

poetry2nix poetry2nix turns Poetry projects into Nix derivations without the need to actually write Nix expressions. It does so by parsing pyproject.t

Nix community projects 405 Dec 29, 2022
It's a repo for Cramer's rule, which is some math crap or something idk

It's a repo for Cramer's rule, which is some math crap or something idk (just a joke, it's not crap; don't take that seriously, math teachers)

Module64 0 Aug 31, 2022
Watcher for systemdrun user scopes

Systemctl Memory Watcher Animated watcher for systemdrun user scopes. Usage Launch some process in your GNU-Linux or compatible OS with systemd-run co

Antonio Vanegas 2 Jan 20, 2022
PythonCalculator - A simple Calculator made in python using tkinter for GUI

PythonCalculator A simple Calculator made in python using tkinter for GUI. For P

ʀᴇxɪɴᴀᴢᴏʀ 1 Jan 01, 2022
WordlistPasswordGenerator - Shuhfab Basheer

WordlistPasswordGenerator - Shuhfab Basheer Python wordlist generator MAINTAINER

1 Dec 31, 2021