I'm assuming that I must have made a mistake in my Varnish configuration. (I'm running version 6.0.7.) Here is the relevant section:
####----SECTION THREE: PURGE RULES----####
# Access Control List to define which IPs can purge content
acl purge {
"localhost";
"127.0.0.1";
"<my home IP address>";
"::1";
}
####----SECTION FOUR: PROCESSING RULES----####
sub vcl_recv {
# Do not allow purging from non-approved IPs
if (req.method == "PURGE") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send PURGE requests."));
}
return (purge);
}
# Allow banning regexes
if (req.method == "BAN") {
if (!client.ip ~ purge) {
return (synth(405, "This IP is not allowed to send BAN requests."));
}
ban("req.http.host == " + req.http.host + " && req.url ~ ^" + req.url);
return (synth(200, "Ban added"));
}
# Allow purging regexes
if (req.method == "PURGE") {
if (req.http.X-Purge-Method == "regex") {
ban("req.url ~ " + req.url + " && req.http.host ~ " + req.http.host);
return (synth(200, "Banned"));
} else {
return (purge);
}
(N.B.: In my setup, Varnish is running on port 6081, with HAProxy in front of it.)
Using this setup, from either my home IP address or the server Varnish is running on, I am able to clear individual pages from the Varnish cache by running, e.g.:
curl -i -X PURGE https://example.com/page/
I can also clear the entire domain from the Varnish cache by running, e.g.:
curl -i -X BAN https://example.com
Likewise, I can clear individual pages from the Varnish cache by running, e.g.:
curl -i -X PURGE http://<IP of Varnish server>:6081/page/ -H "Host: example.com”
I can also clear the entire domain from the Varnish cache by running, e.g.:
curl -i -X BAN http://<IP of Varnish server>:6081 -H "Host: example.com”
But I have noticed that I cannot clear the entire domain by running either:
curl -i -X PURGE https://example.com
or:
curl -i -X PURGE http://<IP of Varnish server>:6081 -H "Host: example.com”
To clear the entire domain, I have to use BAN instead. I do not know if this is a problem or a misconfiguration, but it seems to be, because although I have read that PURGE is not used to clear everything, the Wordpress plugin I am using seems to be trying to, by sending a PURGE request to domain.com/.*
when I hit the “Clear ALL Varnish Cache” button.
This, frankly, confuses me. Because it doesn’t work either in the plug-in itself or if I try it from the command line. What am I missing? Is the plugin just out of date? Did Varnish used to be able to clear the entire cache this way? I'd appreciate any pointers. Thanks!
Here's a cleaned up version of the purging logic in your VCL:
Purging
The
PURGE
HTTP method is what triggers the cache invalidation. Without any headersreturn(purge);
will be called, which will invalidate the exact URL that was used.If the
X-Purge-method
request header is added, and is set toregex
, a regular expression match will occur. Invalidating multiple objects using a regular expression cannot be done withreturn(purge);
, and requires banning.Bans
The
ban()
function adds a ban expression to the ban list. Expressions on this list are matched to all the objects that are stored in cache. The matching ones are removed.You can consult the ban list using the following command on your Varnish server:
Lurker-friendly bans
The ban lurker, a special thread that processes the ban list, doesn't have access to the request context. If you want this thread to remove the objects from cache, based on request parameters like the URL or the hostname, you need to apply som trickery.
As you can see, the pattern I suggest, matches on
obj.http.x-url
&obj.http.x-host
. These are set invcl_backend_response
via 2 custom response headers.If you don't do this, the ban lurker cannot match the objects, and the responsibility will shift to the next request. Even without these so-called lurker-friendly bans, banning will still work, but it is not as efficient.
Testing it
The first example will invalidate
http://example.com/page
from the cacheThe next example will invalidate all objects whose URL starts with
http://example.com/page
:The last example will invalidate all objects from cache for the
example.com
domain:Here's the ban list item for that last example:
As you can see, it matches all URLs for the
example.com
host. When you perform an HTTP request after that, you'll see theAge: 0
response header. This indicates that this response has been stored in cache for zero seconds. This means the ban was successful.