With the new softpurge vmod you can do cache invalidation in Varnish that only affects objects if your backend is up. This means that you can purge all you want, and in the normal case everything works as expected, but if your backend/origin server is unhappy you can serve stale content instead of a 503.
It introduces softpurge.softpurge(); where you would normally use purge; (in vcl_hit and vcl_miss).
The trick is of course that we only reduce the TTL of cached objects (and their Vary-ants), but keep the grace period. This gives us the additional win of increased concurrency while your backend recreates the content after a purge, all but the first client requesting a page withing the page creation time is served the stale one, per usual varnish grace behaviour.
You can find the vmod on my github account. Please remember that this is an early version which has not seen heavy production use yet.
PS: We’re looking into doing softer bans (the other way of doing cache invalidation in Varnish) as well. This looks to be a bit more involved, so we (Varnish Software) are looking for sponsors for this task. Ping me (or Ruben) if this sounds interesting.
Yay! Great stuff, looking forward to checking out how this works.