mercure database growing

168 views Asked by At

My mercure.db looks

-rw------- 1 root root 49G Oct 24 11:32 mercure.db

this is my docker-compose

version: "3.1"
services:
    mercure:
      image: dunglas/mercure
      restart: unless-stopped
      volumes:
        - mercure_data:/data
        - mercure_config:/config
      ports:
        - '3001:80'
      logging:
        driver: "json-file"
        options:
          max-size: "100m"
          max-file: "1"
      deploy:
        resources:
          limits:
            cpus: '2.00'
            memory: 8G
      environment:
        - LOG_LEVEL= error
        - SERVER_NAME= :80
        - CORS_ALLOWED_ORIGINS=*
      
    
    volumes:
      mercure_data:
      mercure_config:

Is there a way to configure Mercure to automatically purge or delete messages after a certain period, or to set a limit on the number of messages stored? I am looking for a solution where I can maintain the performance without regularly interrupting the service for cleanup.

Essentially, I have no need to retain undelivered messages. Once a message has been sent, I'd prefer for it to be deleted immediately to prevent unnecessary storage usage. Currently, to mitigate the database's growth, I find myself manually stopping the server, deleting the database, and creating it anew. This approach is not sustainable nor efficient.

I read - https://mercure.rocks/docs/hub/config#bolt-adapter

cleanup_frequency chances to trigger history cleanup when an update occurs, must be a number between 0 (never cleanup) and 1 (cleanup after every publication), default to 0.3

So, if the default is 0.3, I think in my environment not working.

mercure:
    image: dunglas/mercure
    deploy:
      resources:
        limits:
          cpus: '2.00'
          memory: 8G
    environment:
        - MERCURE_TRANSPORT_URL=bolt:///data/mercure.db?size=1000&cleanup_frequency=1

When I add this connection, it stops working:

mercure?topic=%2Fcounter%2F125      fetch   615.595600f6.js:2   0 B Pending

All time "pending".

But when I remove parameters, and restart, it works again:

  environment:
    #        - MERCURE_TRANSPORT_URL=bolt:///data/mercure.db?size=1000&cleanup_frequency=0.5 
    #        - MERCURE_TRANSPORT_URL=bolt:///data/mercure.db?size=1000
    - MERCURE_TRANSPORT_URL=bolt:///data/mercure.db

With options 1 and 2 it does not work, but option 3 works.

I see in Caddyfile , so guess that $MERCURE_TRANSPORT_URL is env var.

route {
    root * /srv/app/public
    mercure {
        # Transport to use (default to Bolt)
        transport_url {$MERCURE_TRANSPORT_URL:bolt:///data/mercure.db}

about "Where did you read about that environment variable? Since you have a volume for your configuration, I assume you have a configuration file, right?":

No, I'm utilizing docker-compose and haven't employed any additional configuration files.

"Might be because the URL encoding. Have you tried adding the transport URL directly to the configuration file?" I don`t use configuration file - i use config from here: https://mercure.rocks/docs/hub/install section Docker Compose

here is example config:

# docker-compose.yml
version: '3'

services:
  mercure:
    image: dunglas/mercure
    restart: unless-stopped
    environment:
      # Uncomment the following line to disable HTTPS
      #SERVER_NAME: ':80'
      MERCURE_PUBLISHER_JWT_KEY: '!ChangeThisMercureHubJWTSecretKey!'
      MERCURE_SUBSCRIBER_JWT_KEY: '!ChangeThisMercureHubJWTSecretKey!'
    # Uncomment the following line to enable the development mode
    #command: /usr/bin/caddy run --config /etc/caddy/Caddyfile.dev
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - mercure_data:/data
      - mercure_config:/config

volumes:
  mercure_data:
  mercure_config:

Like you see, config is only for dev mode.

Where is example of production configuration file / possible to run production version with some limits / working cleanup without extra configs ?

0

There are 0 answers