Configure Varnish backend in kubernetes deployment

Question

I'm experimenting with spinning up a Varnish instance to speed up a slow but static endpoint of a service. The service is already running in a Kubernetes cluster so I'm basing the setup on the official docker image and following the advice of baking the VCL file into the image:

FROM varnish:6.2

COPY default.vcl /etc/varnish/

I'm starting with a small amount of configuration in default.vcl:

vcl 4.0;

backend default {
        .host = "172.17.0.1:8018"; # IP for local experimentation
}

# only cache '/v1/xyz/...'
sub vcl_recv {
    if (req.url ~ "^/v1/xyz/") {
        return (hash);
    }
    return (pass);
}

I would like to be able to declare the target backend in the deployment file, either using env vars or cli args.

The -b flag seemed perfect for the job, but fails with Error: Only one of -b or -f can be specified.

And using std.getenv in the backend block doesn't work either:

import std;

backend default {
        .host = std.getenv("VARNISH_TAGET_HOST");
}

results in

Expected CSTR got 'std'
(program line 369), at
('/etc/varnish/default.vcl' Line 6 Pos 17)
        .host = std.getenv("VARNISH_TAGET_HOST");
----------------###------------------------------

Is there some way (not including sed-like hacks) by which I can configure the backend without hardcoding it into the VCL?

Answer 1

Varnish Enterprise has dynamic backends

Varnish Cache, the open source version of Varnish, only allows static backend definitions.

The only way you can define backends on-the-fly, is by using Varnish Enterprise, the commercial version of the software.

See https://docs.varnish-software.com/varnish-cache-plus/vmods/goto/ for more information about the dynamic backends feature.

Why -b & -f cannot be combined

Apparently the -b parameter is a shorthand for the following command:

varnishadm vcl.inline boot << EOF
vcl 4.1;

backend default {
    .host = "<addr>";
}
EOF

So in fact -b already creates and loads VCL in the background, which makes this option mutually exclusive with -f