Port binding error when connecting to localhost Kafka broker from localhost Consumer in Docker container

I have a Zookeeper running on port 2181 (default) and a Kafka server listening on port 9090 on my local machine.

When I run kafka CLI locally, or consumer/producer apps locally, I have no problem connecting.

I then try to bundle a Kafka consumer into a Docker container, and run that Docker container locally, e.g.: docker run -p 9092:9092 --rm <DOCKER_IMAGE>

This gives the error: (Error starting userland proxy: Bind for 0.0.0.0:9090 failed: port is already allocated.)

This makes sense since Kafka Server is bound to 9092, as shown in nmap -p 9092 localhost: PORT STATE SERVICE 9092/tcp open XmlIpcRegSvc

I'd have no problem mapping the Docker container to a different port via -p XXX:9090, but how do I get the local Kafka server to listen on that new port without binding to it?

1 Answer

So after some digging I found a few options. (Note: I'm on a mac so #2 may not be applicable to everyone).

1. Include --network=host in the docker run command (as seen here).
2. Don't change the docker run command at all, and instead connect to broker at host.docker.internal:9092 inside the container consumer/publisher code. As seen here.

I wasn't able to get #1 to work out for me (I'm sure it's user error). However #2 worked perfectly and just required a config change inside the container.