Memory usage of Micro Docker containers

74 views Asked by At

I have several Payara Micro Docker services in a stack, with some services having more than one replica. Their containers are of images that extent Payara Micro's image. It's pretty basic:

FROM payara/micro:5.2022.5-jdk11
COPY target/*.war $DEPLOY_DIR
EXPOSE 8080 8181
CMD ["--deploymentDir", "/opt/payara/deployments", "--contextroot", "a"]

These applications do nothing more than expose a couple of REST endpoints and marshal data to/from their databases (SQL Server and/or MongoDB) - with very little data manipulation/processing/shaping:

JAX-RS Endpoint -> Local EJB -> Database

Without any resource restrictions in place, the containers allocate 1GB of memory!

I thought that I could drop the container memory allocation to less than 100MB, but the containers just fail due to memory requirements. I've settle on max 700MB with an initial of 500MB, and the containers seem to initially use a whopping 460MB of memory just idling. Keep in mind these are every small apps that do little more than basic CRUD (some are even just read only!).

  a-service:
    image: repo/services-a:1.2.3
    depends_on:
      - mongo
    ports:
      - "8081:8080"
    deploy:
      replicas: 2
    env_file: .env
    networks:
      - net
    environment:
      - MEM_MAX_RAM_PERCENTAGE=80
    deploy:
      resources:
        limits:
          cpus: '1.00'
          memory: 700M
        reservations:
          cpus: '0.50'
          memory: 500M

Here is a screenshot from Live Charts in Docker Desktop of one of the services starting up:

Memory usage upon start-up

Here is another screenshot after running a simple stress test:

for run in {1..2000}; do curl -s http://127.0.0.1:8081/a/api/v1/document/details/62fbd94e2b276e0f3d6fcc30 > /dev/null; done

Memory usage during stress test

As you can see, there is an insignificant bump in memory usage (~10MB) for the entire 2000 requests.

Even at 500MB/service, this is excessive. Couple that with several services/containers, it's prohibitive.

What can I do to reduce the memory footprint of these Payara Micro containers? Ideally, I'd like to get them down to about 50MB (or even 100MB), but any reduction at this point is welcomed.

Metrics:

base_classloader_loadedClasses_count 22590
base_classloader_loadedClasses_total_total 22617
base_classloader_unloadedClasses_total_total 27
base_cpu_availableProcessors 1
base_cpu_systemLoadAverage 0.09912109375
base_gc_time_total{name="MarkSweepCompact"} 451
base_gc_time_total{name="Copy"} 639
base_gc_total_total{name="MarkSweepCompact"} 6
base_gc_total_total{name="Copy"} 136
base_jvm_uptime_seconds 1665.407
base_memory_committedHeap_bytes 1.73539328E8
base_memory_committedNonHeap_bytes 2.04734464E8
base_memory_maxHeap_bytes 5.67672832E8
base_memory_maxNonHeap_bytes -1
base_memory_usedHeap_bytes 1.38958704E8
base_memory_usedNonHeap_bytes 1.982758E8
base_thread_count 86
base_thread_daemon_count 72
base_thread_max_count 97
vendor_system_cpu_load 0.00928

jstat:

$ jstat -gc 1
 S0C    S1C    S0U    S1U      EC       EU        OC         OU       MC     MU    CCSC   CCSU   YGC     YGCT    FGC    FGCT    CGC    CGCT     GCT   
5824.0 5824.0 970.5   0.0   46848.0  41339.2   116800.0   93516.7   136320.0 132541.5 17024.0 15556.6    142    0.659   6      0.451   -          -    1.111

0

There are 0 answers