I am facing issues with building the binary file for my inventory microservice when using Kafka consumer from github.com/confluentinc/confluent-kafka-go/kafka on my Mac M1. Initially, I set CGO_ENABLED=0 to disable the use of C but now it is set to CGO_ENABLED=1 because that Kafka Go module works with C code. I am also using docker compose to spin up containers on Docker Desktop. The following is the error message I am getting when building the binary for my microservice:
cd ../inventory-service && env CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -o inventoryApp ./cmd/api
# runtime/cgo
linux_syscall.c:67:13: error: implicit declaration of function 'setresgid' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
linux_syscall.c:67:13: note: did you mean 'setregid'?
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/unistd.h:593:6: note: 'setregid' declared here
linux_syscall.c:73:13: error: implicit declaration of function 'setresuid' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
linux_syscall.c:73:13: note: did you mean 'setreuid'?
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/unistd.h:595:6: note: 'setreuid' declared here
make: *** [build_inventory] Error 2
When CGO_ENABLED=0, I get this error message:
cd ../inventory-service && env CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o inventoryApp ./cmd/api
internal/kafka/consumer.go:4:2: imported and not used: "log"
internal/kafka/consumer.go:9:78: undefined: kafka.Consumer
internal/kafka/consumer.go:11:25: undefined: kafka.NewConsumer
internal/kafka/consumer.go:11:44: undefined: kafka.ConfigMap
internal/kafka/consumer.go:28:57: undefined: kafka.Consumer
make: *** [build_inventory] Error 2
Makefile
build_inventory:
@echo "Build inventory binary..."
cd ../inventory-service && env CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -o ${INVENTORY_BINARY} ./cmd/api
docker-compose.yml
inventory-service:
build:
context: ./../inventory-service
dockerfile: ./../inventory-service/Dockerfile
depends_on:
inventory-mysql:
condition: service_healthy
restart: always
ports:
- '8002:8002'
deploy:
mode: replicated
replicas: 1
environment:
MYSQL_USER: inventory-mysql
MYSQL_PASSWORD: password
MYSQL_HOST: inventory-mysql
MYSQL_PORT: '3306'
MYSQL_DATABASE: imsdb
SERVICE_PORT: '8002'
zookeeper:
image: confluentinc/cp-zookeeper:7.3.0
container_name: zookeeper
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
broker:
image: confluentinc/cp-kafka:7.3.0
ports:
# To learn about configuring Kafka for access across networks, see
# https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/
- '9092:9092'
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_INTERNAL:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092,PLAINTEXT_INTERNAL://broker:29092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
Dockerfile
FROM alpine:latest
RUN mkdir /app
COPY inventoryApp /app
CMD ["/app/inventoryApp"]
Update
This is my new Dockerfile but I am facing other issues.
FROM golang:1.20.4-alpine3.17 AS builder
RUN set -ex &&\
apk add --no-progress --no-cache \
gcc \
musl-dev
ENV GO111MODULE=on \
GOOS=linux \
GOARCH=arm64
RUN apk add --no-progress --no-cache gcc musl-dev
WORKDIR /app
COPY go.mod go.sum ./
# install dependencies
RUN go mod download
COPY . .
RUN go build -tags musl -ldflags '-extldflags "-static"' -o inventoryApp ./cmd/api
FROM scratch
WORKDIR /app
COPY --from=builder /app/inventoryApp .
CMD ["./inventoryApp"]
Errors:
16.94 /usr/local/go/pkg/tool/linux_arm64/link: running gcc failed: exit status 1
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/confluentinc/confluent-kafka-go/[email protected]/kafka/librdkafka_vendor/librdkafka_musl_linux_arm64.a(rdkafka_sasl_cyrus.o): in function `rd_kafka_sasl_cyrus_close':
16.94 (.text+0xb4): undefined reference to `sasl_dispose'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/confluentinc/confluent-kafka-go/[email protected]/kafka/librdkafka_vendor/librdkafka_musl_linux_arm64.a(rdkafka_sasl_cyrus.o): in function `rd_kafka_sasl_cyrus_recv':
16.94 (.text+0x198): undefined reference to `sasl_client_step'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0x1c0): undefined reference to `sasl_errdetail'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0x354): undefined reference to `sasl_getprop'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0x384): undefined reference to `sasl_getprop'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0x3a4): undefined reference to `sasl_getprop'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/confluentinc/confluent-kafka-go/[email protected]/kafka/librdkafka_vendor/librdkafka_musl_linux_arm64.a(rdkafka_sasl_cyrus.o): in function `rd_kafka_sasl_cyrus_client_new':
16.94 (.text+0xf64): undefined reference to `sasl_client_new'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0xfc4): undefined reference to `sasl_client_start'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0xfe4): undefined reference to `sasl_errdetail'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0x10fc): undefined reference to `sasl_listmech'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0x1170): undefined reference to `sasl_errstring'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: /go/pkg/mod/github.com/confluentinc/confluent-kafka-go/[email protected]/kafka/librdkafka_vendor/librdkafka_musl_linux_arm64.a(rdkafka_sasl_cyrus.o): in function `rd_kafka_sasl_cyrus_global_init':
16.94 (.text+0x16cc): undefined reference to `sasl_client_init'
16.94 /usr/lib/gcc/aarch64-alpine-linux-musl/12.2.1/../../../../aarch64-alpine-linux-musl/bin/ld: (.text+0x16fc): undefined reference to `sasl_errstring'
16.94 collect2: error: ld returned 1 exit status
16.94