Host port conflicts in Docker Compose

Problem

Imaging that you are working on a project that uses Docker Compose, and you or other developers use Docker for Mac or Windows. When you have a service in the project that needs to be accessed from the outside somehow, you can normally simply expose a port on a service and use the container IP:port to access the service from the host system (you can find the container IP from docker container inspect CONTAINER_NAME). This approach does not work when your host system is Windows or Mac because then Docker runs on a separate virtual machine and the networks that the containers are attached to are not visible from the host. If any of the developers are not using Linux for development, you will want to use the ports directive in docker-compose.yml to bind a host port to a container, and the docker software for your operating system will set everything up to send traffic from a specific port to the Docker virtual machine, which will then be forwarded to a specific container on that virtual machine as in the simplified example below.

version: "3"
services:
  web-server:
    image: nginx
    ports:
      - "3000:80"

Developers are likely to assign one of the common port numbers to a service like this (8000, 8080 and 3000 are very common). If for whatever reason you want to run two of these services at the same time for testing purposes (e.g., run different versions), or run two completely different services that happen to bind to the same port number, the hardcoded host port number will pose a problem.

We can run this project and inspect the container to confirm that host port 3000 is assigned to the web-server container:

$ docker-compose up -d
Creating network "test01_default" with the default driver
Creating test01_web-server_1 ... done
$ docker inspect test01_web-server_1 | jq '.[0].NetworkSettings.Ports'
{
    "80/tcp": [
        {
            "HostIp": "0.0.0.0",
            "HostPort": "3000"
        }
    ]
}

Solution

One way to solve the problem is to simply remove or modify the port line in one of the Docker Compose files. This might work short-term, but you will have to remember this constantly as to not commit the change accidentally.

Better Solution

There is a better way. Docker Compose allows using environment variables in docker-compose.yml, and it is even possible to set default values for missing variables. This makes it very easy to set the host port number to one of the commonly used port numbers while allowing the person running the project to change the port to something else via an environment variable without modifying docker-compose.yml. In the example below the port is set top 3000 by default (the part after :-).

version: "3"
services:
  web-server:
    image: nginx
    ports:
      - "${WEB_SERVER_HOST_PORT:-3000}:80"

Applying this change does not affect the container as it is configured to bind to port 3000 on the host as before:

$ docker-compose up -d
test01_web-server_1 is up-to-date

However, if we specify the WEB_SERVER_HOST_PORT variable, the container will be recreated and the host port will be changed to the given value:

$ WEB_SERVER_HOST_PORT=3001 docker-compose up -d
Recreating test01_web-server_1 ... done
$ docker inspect test01_web-server_1 | jq '.[0].NetworkSettings.Ports'
{
    "80/tcp": [
        {
            "HostIp": "0.0.0.0",
            "HostPort": "3001"
        }
    ]
}

The variables that are added to the .env file will be picked up by Docker Compose automatically, so you will not need to specify them every time or export them in your shell unnecessarily:

$ echo WEB_SERVER_HOST_PORT=3002 >> .env
$ docker-compose up -d
Recreating test01_web-server_1 ... done
$ docker inspect test01_web-server_1 | jq '.[0].NetworkSettings.Ports'
{
    "80/tcp": [
        {
            "HostIp": "0.0.0.0",
            "HostPort": "3002"
        }
    ]
}

Conclusion

It is possible to use environment variables Docker Compose files as a workaround for many services binding to the same default host port with minimal effort.