When running `stunnel` as non-root it errors with "setgroups: Operation not permitted"

Question

I need to authenticate to an OIDC server using a client cert (in addition to the normal OIDC client authentication)

To avoid modifying the oauth2-proxy code, I'm setting up a transparent encryption layer using stunnel inside a Docker image.
FWIW - I created a script with setuid bit to launch stunnel and that works, but I'm not sure it's the best solution.

I can start stunnel as root user, but when trying to run stunnel as non-root, it binds to the port, then immediately bails.

Error details -

oauth2-proxy-oauth2-proxy-1  | # # # ACTIVE CONFIG # # #
oauth2-proxy-oauth2-proxy-1  | foreground = yes
oauth2-proxy-oauth2-proxy-1  | setuid = 65532
oauth2-proxy-oauth2-proxy-1  | setgid = 65532
oauth2-proxy-oauth2-proxy-1  | socket = l:TCP_NODELAY=1
oauth2-proxy-oauth2-proxy-1  | socket = r:TCP_NODELAY=1
oauth2-proxy-oauth2-proxy-1  | cert = /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | client = yes
oauth2-proxy-oauth2-proxy-1  | TIMEOUTbusy = 600
oauth2-proxy-oauth2-proxy-1  | TIMEOUTclose = 600
oauth2-proxy-oauth2-proxy-1  | TIMEOUTconnect = 600
oauth2-proxy-oauth2-proxy-1  | TIMEOUTidle = 600
oauth2-proxy-oauth2-proxy-1  | [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | accept = 0.0.0.0:8080
oauth2-proxy-oauth2-proxy-1  | connect = postman-echo.com:443
oauth2-proxy-oauth2-proxy-1  | # # # ACTIVE CONFIG # # #
oauth2-proxy-oauth2-proxy-1  | Starting stunnel...
oauth2-proxy-oauth2-proxy-1  | [ ] Initializing inetd mode configuration
oauth2-proxy-oauth2-proxy-1  | [ ] Clients allowed=512000
oauth2-proxy-oauth2-proxy-1  | [.] stunnel 5.66 on aarch64-alpine-linux-musl platform
oauth2-proxy-oauth2-proxy-1  | [.] Compiled with OpenSSL 3.0.5 5 Jul 2022
oauth2-proxy-oauth2-proxy-1  | [.] Running  with OpenSSL 3.0.7 1 Nov 2022
oauth2-proxy-oauth2-proxy-1  | [.] Threading:PTHREAD Sockets:POLL,IPv6 TLS:ENGINE,OCSP,PSK,SNI
oauth2-proxy-oauth2-proxy-1  | [ ] errno: (*__errno_location())
oauth2-proxy-oauth2-proxy-1  | [ ] Initializing inetd mode configuration
oauth2-proxy-oauth2-proxy-1  | [.] Reading configuration from file /etc/stunnel/stunnel.conf
oauth2-proxy-oauth2-proxy-1  | [.] UTF-8 byte order mark not detected
oauth2-proxy-oauth2-proxy-1  | [ ] No PRNG seeding was required
oauth2-proxy-oauth2-proxy-1  | [ ] Initializing service [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | [ ] stunnel default security level set: 2
oauth2-proxy-oauth2-proxy-1  | [ ] Ciphers: HIGH:!aNULL:!SSLv2:!DH:!kDHEPSK
oauth2-proxy-oauth2-proxy-1  | [ ] TLSv1.3 ciphersuites: TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256
oauth2-proxy-oauth2-proxy-1  | [ ] TLS options: 0x2100000 (+0x0, -0x0)
oauth2-proxy-oauth2-proxy-1  | [ ] Session resumption enabled
oauth2-proxy-oauth2-proxy-1  | [ ] Loading certificate from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Certificate loaded from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Loading private key from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Private key loaded from file: /etc/stunnel/stunnel.pem
oauth2-proxy-oauth2-proxy-1  | [ ] Private key check succeeded
oauth2-proxy-oauth2-proxy-1  | [:] Service [httpsconnect] needs authentication to prevent MITM attacks
oauth2-proxy-oauth2-proxy-1  | [ ] DH initialization skipped: client section
oauth2-proxy-oauth2-proxy-1  | [ ] ECDH initialization
oauth2-proxy-oauth2-proxy-1  | [ ] ECDH initialized with curves X25519:P-256:X448:P-521:P-384
oauth2-proxy-oauth2-proxy-1  | [.] Configuration successful
oauth2-proxy-oauth2-proxy-1  | [ ] Deallocating deployed section defaults
oauth2-proxy-oauth2-proxy-1  | [ ] Binding service [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | [ ] Listening file descriptor created (FD=9)
oauth2-proxy-oauth2-proxy-1  | [ ] Setting accept socket options (FD=9)
oauth2-proxy-oauth2-proxy-1  | [ ] Option SO_REUSEADDR set on accept socket
oauth2-proxy-oauth2-proxy-1  | [ ] Service [httpsconnect] (FD=9) bound to 0.0.0.0:8080
oauth2-proxy-oauth2-proxy-1  | [!] setgroups: Operation not permitted (1)
oauth2-proxy-oauth2-proxy-1  | [ ] Unbinding service [httpsconnect]
oauth2-proxy-oauth2-proxy-1  | [ ] Service [httpsconnect] closed (FD=9)
oauth2-proxy-oauth2-proxy-1  | [ ] Service [httpsconnect] closed

Dockerfile

FROM quay.io/oauth2-proxy/oauth2-proxy:v7.3.0 as builder

FROM alpine:3
COPY --from=builder /etc/nsswitch.conf /etc/nsswitch.conf
COPY --from=builder /bin/oauth2-proxy /bin/oauth2-proxy
COPY --from=builder /etc/ssl/private/jwt_signing_key.pem /etc/ssl/private/jwt_signing_key.pem


ARG         STUNNEL_VERSION=${STUNNEL_VERSION:-5.66-r0}
ARG         LIBRESSL_VERSION=${LIBRESSL_VERSION:-3.6.1-r0}
ARG         APP_USER=${APP_USER:-65532}

RUN         apk update && apk add --no-cache openssl stunnel=${STUNNEL_VERSION} libressl==${LIBRESSL_VERSION}

ENV         ACCEPT_IP=0.0.0.0 \
            ACCEPT_PORT=8080 \
            SERVICE=httpsconnect \
            DESTINATION_PORT=443 \
            DESTINATION_HOST=0.0.0.0 \
            CLIENT=yes \
            STUNNEL_VERSION=${STUNNEL_VERSION} \
            APP_USER=${APP_USER}


COPY --chown=${APP_USER}:${APP_USER} docker-entrypoint.sh stunnel.sh /

RUN rm /etc/stunnel/stunnel.conf && \
    chown root:root /stunnel.sh && \
    chmod ug+s /stunnel.sh && \
    chmod +x /stunnel.sh && \
    chmod +x /docker-entrypoint.sh && \
    mkdir -p /var/log/stunnel && \
    touch /var/log/stunnel/stunnel.log && \
    chown -R ${APP_USER}:${APP_USER} /var/log/stunnel && \
    chown ${APP_USER}:${APP_USER} /etc/stunnel 

# UID/GID 65532 is also known as nonroot user in distroless image
USER ${APP_USER}:${APP_USER}

ENTRYPOINT [ "./docker-entrypoint.sh" ]

Answer 1

It's possible to run stunnel as non-root.
I've modified the Dockerfile and entrypoint script to start the process as a defined user, update file/folder ownership to the provided user, and skipped the setid/setgid parameters for stunnel so that it doesn't try to change the process owner.
Also, I needed to update the PID location in the stunnel config.

Dockerfile -

FROM quay.io/oauth2-proxy/oauth2-proxy:v7.3.0 as builder

FROM alpine:3
COPY --from=builder /etc/nsswitch.conf /etc/nsswitch.conf
COPY --from=builder /bin/oauth2-proxy /bin/oauth2-proxy
COPY --from=builder /etc/ssl/private/jwt_signing_key.pem /etc/ssl/private/jwt_signing_key.pem


ARG         STUNNEL_VERSION=${STUNNEL_VERSION:-5.66-r0}
ARG         LIBRESSL_VERSION=${LIBRESSL_VERSION:-3.6.1-r0}
# UID/GID 65532 is also known as nonroot user in distroless image
ARG         APP_USER=${APP_USER:-65532}

RUN         apk update && apk add --no-cache openssl stunnel=${STUNNEL_VERSION} libressl==${LIBRESSL_VERSION}

ENV         ACCEPT_IP=0.0.0.0 \
            ACCEPT_PORT=8080 \
            SERVICE=httpsconnect \
            DESTINATION_PORT=443 \
            DESTINATION_HOST=0.0.0.0 \
            CLIENT=yes \
            STUNNEL_VERSION=${STUNNEL_VERSION} \
            APP_USER=${APP_USER}

COPY --chown=${APP_USER}:${APP_USER} docker-entrypoint.sh /

RUN rm /etc/stunnel/stunnel.conf && \
    chmod +x /docker-entrypoint.sh && \
    mkdir -p /var/log/stunnel && \
    chown ${APP_USER}:${APP_USER} /etc/stunnel 

USER ${APP_USER}:${APP_USER}

ENTRYPOINT [ "./docker-entrypoint.sh" ]

Entrypoint (adapted from another example)

#!/bin/sh

to_file() {

    TEXT="${1}"
    FILE="${2}"
    ECHO="$(command -v echo)"

    ${ECHO} "${TEXT}" >> "${FILE}"
}

cd /etc/stunnel

if [ -f stunnel.conf ]
then
    rm -f stunnel.conf
fi

to_file "
foreground = yes
pid = /etc/stunnel/stunnel.pid
debug = info
output = /etc/stunnel/stunnel.log
socket = l:TCP_NODELAY=1
socket = r:TCP_NODELAY=1
cert = /etc/stunnel/stunnel.pem
client = ${CLIENT:-no}" "stunnel.conf"

for DOM in $(echo $SNI | sed "s/,/ /g")
do
    to_file "SNI = ${DOM:-}" "stunnel.conf"
done

to_file "TIMEOUTbusy = 600
TIMEOUTclose = 600
TIMEOUTconnect = 600
TIMEOUTidle = 600
[${SERVICE}]
accept = ${ACCEPT_IP}:${ACCEPT_PORT}
connect = ${DESTINATION_HOST}:${DESTINATION_PORT}" "stunnel.conf"

if ! [ -f stunnel.pem ]
then
    openssl req -x509 -nodes -newkey rsa:2048 -days 3650 -subj '/CN=stunnel' \
                -keyout stunnel.pem -out stunnel.pem
    chmod 600 stunnel.pem
fi

echo "# # # ACTIVE CONFIG # # #"
cat "stunnel.conf"
echo "# # # ACTIVE CONFIG # # #"

echo "Starting stunnel..."
exec stunnel &
echo "Starting oauth2-proxy..."
exec /bin/oauth2-proxy "$@"