Search
Varnish Enterprise

Changelog for 6.0.x

Varnish Enterprise 6.0.13r9 (2024-11-28)

New features:

  • vmod_udo director constructor now has an optional argument to specify director type when creating the director.

Bugs fixed:

  • Fixes a race condition in the ESI error cleanup code that would in certain cases lead to a failed assertion. This issue was first introduced in the previous release, and only impacts users of ESI. (VS issue #2391)

Varnish Enterprise 6.0.13r8 (2024-11-13)

New features:

  • vmod_jwt reader now has get methods for custom claim fields. See the vmod_jwt manual page for more information.

  • New feature flag esi_include_onerror that enables the support of the onerror="continue" attribute of <esi:include/> tags.

    When this feature flag is set, failure of an included fragment would abort the parent request, unless the onerror="continue" attribute was set for the failing fragment. Fragments are considered to be failed if an error occurs during the backend fetch, or the response status for the fragment is different from 200 and 204. When the feature flag is not set, the default behaviour remains the same, ie: all the fragments are included regardless of the outcome of their backend fetches.

    Note that enabling this feature when you have cached esi objects on a persistent storage from a previous release is not supported and will result in undefined behavior.

  • Parallel ESI processing is now subject to a new parameter esi_limit. This parameter determines the maximum number of includes in flight at each ESI level for a single delivery. The default value of 10 provides a theoretical maximum of 50 simultaneous subrequests with the default max_esi_depth limit of 5.

  • vmod_s3 now has a signer object that can be used to sign backend requests without using a director. See the vmod_s3 manual page for more information.

  • The MSE4 data format version has changed with this release. This in order to optimize Ykey handling.

    • After upgrading to this version, existing file devices from the previous format will no longer be usable. The system will report a "Invalid file device version" error if one is attempting to start the Varnish daemon using existing data files.
    • To reinitialize the file devices with a clean cache, run the mkfs.mse4 utility with the -f (force) flag and the configure command. Note that this will delete and recreate the file devices, clearing the cache of all content.
  • MSE4 Ykey handling of persisted objects has been significanly improved.

    • The varnishstat counters related to Ykey handling in MSE4 have been reworked.
  • An MSE4 configuration parameter has been deprecated and removed. In the Book sections, the key ykey_buckets is no longer recognized as a valid key.

  • The default value of the parameter startup_timeout has been increased from 60 seconds to 10 minutes. This is to ensure that the startup sequence is not affected in some scenarios.

    • The default varnish.service systemd service file has been changed to include a TimeoutStartSec=720 to facilitate this change.

Bugs fixed:

  • Fixed a bug in vmod_kvstore that allowed the creation of VSC counters containing illegal characters in their names/descriptions that would cause a panic in varnishstat. The key and comment arguments of the .counter() and .gauge() methods must now only contain valid JSON string characters when varnishstat is set to true, otherwise the varnishstat argument will be reset to false. (VS issue #2329)
  • Print a warning if VSM cannot be locked in memory. (4121)
  • Increase the amount of memory permitted to be locked during manager startup as much as the system will allow. (4193)
  • Fix an issue in MSE4 where the MAIN.n_object counter would not be decremented when objects were evicted to make space, either in the book or the store. (VS issue #2362)
  • Fix an issue in MSE4 where the MAIN.n_object_{hitmiss|hitpass} aren't decremented when objects are evicted. (VS issue #2363)
  • Fixed an issue in MSE4 where a zero body length ephemeral object fetched using chunked encoding would cause Varnish to hang up on the client. (VS issue #2348)
  • Fix an issue in MSE4 where the system would cause runtime assertions when the store segment size configured at exactly 4GB. (VS issue #2330)
  • Fixed IPv6 support for .set_ip_tos() in vmod_std. (d0a23b2)

Varnish Enterprise 6.0.13r7 (2024-10-15)

New features:

  • Experimental support for HTTP CONNECT (RFC9110, 9.3.6) has been added. This feature is disabled by default, and the default VCL will not honor a CONNECT request, even when the feature is enabled. When enabled, e.g. through param.set experimental +vcl_connect, the VCL program will need to return (connect) in sub vcl_recv, typically after checking that the request method is CONNECT.
  • varnishncsa recognizes CONNECT requests and the handling formatter can now produce connect in addition to the pre-existing hit, miss, pass, synth and pipe.
  • There is a new ConnectAcct log record, which shows accounting for a request where Varnish acts as a CONNECT proxy.
  • There are new counters for CONNECT, for example MAIN.s_connect and MAIN.s_connect_hdrbytes_in, but there are no counters in VMOD accounting tracking this.
  • cluster.vcl has been updated to version 2.1 with the following improvements:
    • ESI resolution is now disabled when delivering documents with ESI tags to other nodes in the cluster. This avoids accidental caching of personalized content on non-primary nodes.
    • Self-routing is no longer attempted during the self-identification phase when all nodes in the cluster are marked unhealthy. This avoids unnecessary retries.
    • Setting the X-Cluster-Skip header no longer skips accounting of the request. To skip accounting, a new X-Cluster-Skip-Accounting header can be used.
    • Accounting is no longer immediately performed when a node fails to fetch from another cluster node. Instead, accounting is performed if the fallback limit is reached or a successful response is obtained.
    • The X-Cluster-Trace header has been improved to show ERROR on a cache HIT on a shortlived synthetic object inserted by a fetch error with requests on the waitinglist. The trace header now also shows HITSTREAM on streaming cache HITs.
    • A trace header growth issue on 304 revalidations from origin has been fixed. The trace header will now indicate that a 304 revalidation has taken place.
  • vmod_s3 can optionally sign backend requests with the AWSv4 algorithm. See the vmod_s3 manual page for more information.
  • The port of a listen endpoint given with the -a argument to varnishd can now also be a numerical port range like 80-89. (4136)
  • There are two new parameters controlling behavior and limits for when there are too many undiscarded VCL instances (max_vcl and max_vcl_handling). (2713)
  • vmod_mse4 will no longer fail the VCL compilation when the Varnish server is not configured to use MSE4 as its storage stevedore.
  • vmod_mse4 has a new boolean function call is_mse4(), which returns true when the Varnish instance is configured to use MSE4 as its storage stevedore.
  • vmod_mse (MSE version 3) has a new boolean function call is_mse3(), which returns true when the Varnish instance is configured to use MSE3 as its storage stevedore.
  • varnishlog, varnishncsa and varnishtop now support the -E option to show ESI transactions. Note that this is a breaking change for varnishlog that was previously showing ESI transactions by default and will now only show them when -E is specified. (3468)
  • varnishhist now support an E filter for its -P argument to show ESI transactions. (3468)
  • varnishlog now supports the -u switch for unbuffered output. (3468)
  • The Begin and Link vsl tags have a new optional field Child task sub-level that shows the ESI level for ESI transactions. (3828)
  • A new reset_req() function in vmod_headerplus will reset req or bereq depending on the calling context to its initial state without the need for a complete std.rollback(). (VS issue #2148)
  • Two new counters, MAIN.transit_stored and MAIN.transit_buffered, provide insight into the transit buffer's operation.
  • Acceptor threads in varnishscoreboard now shows more information on their listen endpoints.
  • Two new VCL variables, bereq.is_hitpass and bereq.is_hitmiss, have been added. These flags are copies of the corresponding flags on the client side.

Bugs fixed:

  • Fixed a race condition in vmod_headerplus that would cause an assertion due to NULL-pointer dereference.
  • Fixed a bug in vmod_http that would lead to a segmentation fault when supplying a NULL value to .req_set_sparam().
  • Fixed urlplus.query_set() triggering a wrong "Out of workspace" VCL failure when call on a non-existing header.
  • Fix an issue in mkfs.mse4 when attempting to resize the book, where an assertion is thrown in the function resize_book_write_qslottable(). (VS issue #2285)
  • Fixed varnishlog/varnishncsa wrongly applying the -k limit before filtering transactions, resulting in fewer results than expected.
  • VSL performance and stability improvements that should reduce overruns likelihood. (3468)
  • The address of a client is no longer lost for varnishscoreboard when its session enters a waiter. (VS issue #2309)
  • The JPEG operations in vmod_image have been reimplemented in a way that reduces internal memory copying and stack consumption. vmod_image now depends on the libturbojpeg library. (VS issues #1406 and #1407)
  • Fixed a bug in vmod_image that would in certain cases prevent it from decoding images with embedded thumbnails. (VS issue #1510)
  • Fixed a bug where some background threads would show up without its custom name in utilities like top. (VS issue #2302)
  • Fixed a regression in vmod_brotli that could result in a failed decompression on valid brotli input. (VS issue #2312)

Varnish Enterprise 6.0.13r6 (2024-08-26)

New features:

  • cluster.vcl has been updated to version 2.0. The VCL has been improved in several ways, but does not introduce changes that would prevent existing VCLs from compiling. Existing cluster deployments will continue to function in mostly the same way as before, with one significant exception: the replicas cluster option has been removed and setting this to true now has no effect.
  • vmod_utils has a new http_range_support() function. This allows configuring HTTP range support on a per-request basis.
  • Optimized the behavior of the MSE4 ban journalling code. This includes renaming and removal of some of the MSE4 ban journal counters.
  • A new feature flag, vcl_ban, default on (enabled), controls the usage of ban() in VCL. When this flag is turned off, bans can only be issued through the CLI.
  • vmod_tcp.set_socket_pace() has gained the new optional argument scope to set the pace either for the whole session or just for the current request.
  • Logging for H2 connection level errors has been improved. (4095 and 4150)

Bugs fixed:

  • Fixed an issue in MSE4 and its ban journalling code. For certain patterns of ban journal entries, it could end up creating an invalid journal, which would assert and crash the system. Subsequent restart attempts would continue to find an invalid journal and fail the restart. (VS issue #2246)
  • Fixed an issue in MSE4 when doing range requests on ephemeral objects and the object failed the fetch from the backend inside a critical time window. This would lead to the Varnish daemon executing an assertion panic in function mse4_obj_m_iterator_noboc(). (VS issue #2261)
  • Prevent vmod_deviceatlas3 from using a database after failing to load it. (VS issue #1963)
  • Correct vmod_deviceatlas3.lookup() documentation regarding the possible error string return values.
  • vmod_aclplus will only log matches now, instead of providing a log line for all tested ACL.

Varnish Enterprise 6.0.13r5 (2024-07-12)

New features:

  • The vmod_headerplus regex functions have been optimized to use little or no workspace. This also provides a small performance improvement.
  • The MSE4 checksum algorithms have been tuned to make use of AVX2 and AVX512F SIMD instructions when available on the host CPU.
  • MSE4 now has configurable store checksum algorithms. The algorithms available are XXH3, CRC32, CRC32C and CRC64. The algorithm is fixed at the time the store file device is created.
  • The block size configuration option on MSE4 books and stores has been removed. The block size in use is now fixed and always 4096, which is the same as the previous default value for block size.
  • Added vmod_deviceatlas3. This vmod supports version 3.x of the DeviceAtlas API and database.

Bugs fixed:

  • A bug allowed us to end up with duplicate TLS certificate identifiers in configurations that used a combination of the the TLS configuration file and the TLS CLI commands. (VS issue #2212)
  • Fixed an internal race in MSE4 when freeing up memory space for objects removed from memory. This issue would typically result in an assert error. (VS issue #2213)
  • vmod_udo now performs a dynamic backend refresh if DNS changes during a backend fetch, giving subsequent retries within the same task a fresh set of backends. (VS issue #2209)
  • Fixed an issue in MSE4 which would lead to an assert when trimming an object would entirely remove the last storage chunk. (VS issue #2223)
  • Fixed an issue in MSE4 that would assert during bootstrap when the number of Ykeys was high. (VS issue #2222)
  • Return values from vmod_ykey functions now more accurately reflect the number of objects affected by the call. (VS issue #2225)

Varnish Enterprise 6.0.13r4 (2024-06-19)

Bugs fixed:

  • Fixed an issue in MSE4 which could lead to asserts during daemon startup due to Vary instruction reference counting problems. (VS issue #2209)
  • Fixed an issue in MSE4 where the Varnish daemon would not have the right privileges to read the file device statelog during startup.

Varnish Enterprise 6.0.13r3 (2024-06-14)

New features:

  • In vmod_headerplus, the .from_json() method gained a new bolean parameter: skip. Setting it will prevent adding headers that are already present in the target object (no matter their value). Setting both skip and overwrite will trigger a VCL failure.
  • The man page documentation of several VMODs has been updated.

Bugs fixed:

  • Cached DNS entries in ActiveDNS now get the same TTL as the resolution interval dictated by the ttl_rule of the DNS group. This makes it possible to lower the DNS cache TTL with ttl_rule set to force.
  • Fixed an issue where the management process could get stuck and not answer CLI commands when MSE4 is enabled and the child process crashes. (VS issue #2195)
  • Fixed an issue with MSE4 that would result in crashes when handling range requests on ephemeral objects. (VS issue #2198)
  • Fixed an issue with MSE4 that would result in crashes when handling range requests on persisted objects. (VS issue #2197)
  • Fixed an issue with MSE4 related to autodetection of io_uring capability on Ubuntu 22.04. Specifically, io_uring is broken in Ubuntu kernel image version 5.15.0-112. The autodetection feature will now detect the issue in that kernel version, and fall back to classic AIO when the bug is present.

Varnish Enterprise 6.0.13r2 (2024-06-10)

New features:

  • Massive Storage Engine version 4. Please see the release announcement for more information.
  • In vmod_http it is now possible to skip the response body with the new resp_skip_body optional argument for http.req_send(). This avoids accumulating response body bytes when they are not needed and a HEAD request is not appropriate.
  • In vmod_mmdb it is now possible to import mmdb databases in Varnish's working directory, to prevent undefined behavior when a database is modified while being used, and avoid opening it multiple times. Importing large mmdb databases may be ill-advised, especially when the working directory size is limited.
  • Ban evaluation for persisted objects will no longer block cache lookups for object variants while paging in the object headers from disk.
  • A new parameter shutdown_delay determines how long we wait before terminating the varnishd process upon reception of SIGTERM. Another parameter shutdown_close controls whether listen sockets should be closed during that time or not.
  • A new manual page called varnish-numa(7) has been added, covering NUMA (Non Uniform Memory Access) topics and how to use NUMA awareness with Varnish Enterprise.
  • New CLI command tls.cert.reload. This command reloads all TLS certificates from disk based on the existing configuration.

Bugs fixed:

  • The varnishscoreboard output would sometimes display two tasks on the same line. (VS issue #2124)
  • It was not possible to set up a VDP filter (delivery processor) from VCL in conjunction with vmod_slicer without breaking certain expectations. This is now allowed. (VS issue #2126)
  • Fix a bug in the TLS configuration code that allowed us to end up with ciphers that were incompatible with the loaded certificates. (VS issue #2132)
  • When vmod_brotli decompressed backend responses with chunked encoding involved, it could leave the session in an inconsistent state. (VS issue #2127)
  • The unbrotli command in varnishtest would fail with a compression ratio higher than 90%.
  • A code path in the panic handling had an explicit assertion that could result in a recursive panic. (VS issue #2144)
  • A recursive panic could hang a thread forever. Depending on circumstances this thread could block other threads, including subsequent panics. (VS issue #2145)
  • A bug was fixed in vmod_vha where the forbid_replication() function was incorrectly restricted to client context only. It is now allowed in backend context as well. (VS issue #2150)
  • Fixed a crash in libvarnishapi that would occur while detaching from the shared memory. This could result in a panic with vmod_stat. (VS issues #2157 and #2052)
  • Fixed an issue in vmod_stat where a crash could be caused by running out of workspace. (VS issue #2161)
  • Fixed a bug in ActiveDNS that lead to the default update rule being always instead of ignore_error as documented. (VS issue #2183)

Varnish Enterprise 6.0.13r1 (2024-05-14)

New features:

  • A new VMOD feature has been introduced, allowing VMOD developers to limit the set of VCL subroutines from where a VMOD function or method can be called. Trying to use a restricted VMOD function/method outside its limited set of allowed VCL subroutines will be detected by the VCL compiler and will fail the compilation of that VCL. This means that if, for example, a VMOD function is restricted to vcl_recv and a VCL calls it in vcl_backend_fetch, that VCL will fail to compile and will not be loaded. (3915)

    This feature has been applied to all existing VMODs, meaning that starting from this release, VCL configurations that used to compile with no errors and where VMOD functions/methods were misused will no longer compile, even if the call appears in a part of the VCL that is never executed. All restrictions can be found in the respective VMOD documentation or man page.

    A temporary debug flag vcc_lenient_restrict is added in this release to turn restriction violations into warnings and allow the loading of the VCL despite the fact that a function/method is used outside its allowed set of subroutines. The debug flag can be enabled from the CLI like follows:

    param.set debug +vcc_lenient_restrict

    or by adding the following to the varnishd command line:

    -p debug=+vcc_lenient_restrict

    Please keep in mind that this debug flag is just meant to be used as a workaround and will be removed in a future release. If your VCL is no longer compiling with this release, you should review it and fix all the reported violations.

Varnish Enterprise 6.0.12r9 (2024-04-30)

New features:

  • New parameter slicer_excess_ratio, which pertains to how large we allow the final segment in a response to be when doing Slicer processing.

  • vmod_udo director health can now be checked from client context with std.healthy(). The director will be considered healthy if any of the backends in the director report as healthy.

  • IP address lookups can be skipped by vmod_mmdb when several keys are looked up consecutively for the same IP address.

  • Better error messages for thread pool tweaks. For example if increasing thread_pool_min failed because it would exceed thread_pool_max, this is mentioned in the error message. (3099)

  • The ruleset objects in vmod_rewrite have a new .field() method to extract a specific field after matching a rule. This can be used as a more readable alternative to "only-matching" rewrites.

  • New -Q option to read VSL queries from a file in varnishlog, varnishncsa and other logging utilities. A VSL query file can have one query per line, blank lines and comments starting with a #.

    The classic -q option also accepts multi-line queries. When more than one query is present, this is equivalent to having a single query that is an or expression of all the queries.

    A VSL query file can look like this:

    $ cat common-errors.vslq
    # All error tags
    *Error
    
    # HTTP server errors
    *Status >= 500
    
    # Custom backend errors
    BerespHeader:X-Custom-Error
    

    There can be multiple -Q and -q options. They are again treated like an or expression of all queries:

    varnishlog -Q common-errors.vslq -q 'RespStatus == 403'
    

    For very long queries that should span multiple lines for legibility, or existing multi-line queries, use a back-slash at the end of a line to continue the query on the next line. (3001)

Bugs fixed:

  • http/2: A bug in the Upgrade: h2c handling led to an inconsistent state that could in certain cases lead to an erroneous PROTOCOL_ERROR connection error. (4096)
  • Fix a bug where an assert could trigger if memory_target was set to 0. Also, enforce a minimum memory_target of 1M for the positive range.
  • vmod_slicer: Fix a bug where a conditional request with a Range header could be processed as a 304 response, but mistakenly get a 206 response status code applied.
  • vmod_slicer: Fix a bug where Slicer would mistakenly process Slicer subrequests on a HEAD if the request also contained conditional headers (If-None-Match/If-Modified-Since).
  • vmod_slicer: Fix a regression introduced in 6.0.12r7 that would make an invalidation of a slicer segment also invalidate all other Vary objects under the same hash key. (VS issue #2106)
  • Failure to read MSE3 store statistics is no longer a fatal error in degradable mode. (VS issue #2100)
  • Some sections were missing from the vtc(7) manual page. (VS issue #2004)

Varnish Enterprise 6.0.12r8 (2024-04-18)

Bugs fixed:

  • Fix a bug where an assert could trigger after having processed 1 billion transactions and the VXID wrapped back to zero. (VS issue #2094)

Varnish Enterprise 6.0.12r7 (2024-04-04)

New features:

  • varnishncsa has a new %{Varnish:default_format}x log format, which is the same used when neither -f nor -F command line options are specified. It is useful for appending/prepending with other formatters.
  • Introduce vmod_prng: vmod_prng lets you generate synthetic backend responses with "null" bytes or pseudo random bytes. This functionality is intended for testing purposes, and should not be used in any security related applications.
  • Introduce time variable to req, resp, bereq, beresp, and obj objects. (3562)
  • vmod_udo now supports more consistent hashing for DNS groups based on SRV records. With the new dns_group.set_hash_rule() method in vmod_activedns, the hash of each backend can be based on the service name described by each SRV record instead of the combination of IP and port. This can be useful for Kubernetes-like environments where IPs are dynamic but service names stay the same.
  • vmod_udo dynamic backends now support backend connection queueing configured through the dns_group backend template.
  • It is no longer mandatory to load a database with vmod_mmdb. It results in systematic lookup misses.
  • In-Core TLS now supports performing discard and subsequent load of the same ID in one commit.
  • ActiveDNS now provides detailed DNS resolution logs under the ADNS tag.

Bugs fixed:

  • Fix a bug where timeout_idle could be bypassed for HTTPS traffic. (VS issue #2075)
  • Add missing Start and Process VSL timestamps to pipe transactions. (3759)
  • The logged HTTP response code for an HTTP/2 stream being shut down after detection of a rapid reset event has been changed from a 503 to a 408. (VS issue #1938)
  • The parameter h2_max_header_list_size will now be enforced by interrupting the HTTP/2 connection if the size of the incoming header set exceeds this value. By default a value of 150% of http_req_size will be used. (3709)
  • vmod_udo directors now route slicer subrequests consistently when using the hash type. All slicer subrequests for a given object are now routed to the same backend. Previously, a random backend was chosen. (VS issue #2056)
  • Improved error logging in vmod_tls on tls.client_verified() failure. (VS issue #1995)
  • Soft-purging a stale object with vmod_ykey or vmod_purge no longer resets the objects expiry timer to the beginning of its grace period. (4085)
  • Lookup misses no longer log an error for keys in vmod_mmdb, aligning with the lack of error logs for misses on IP addresses.
  • Prevent vmod_deviceatlas from using a database after failing to load it. (VS issue #1965)

Varnish Enterprise 6.0.12r6 (2024-01-31)

New features:

  • vmod_udo now supports backend priority for both static and dynamic backends. The priority selection algorithm is implemented according to the SRV record priority specification in RFC2782. In short, all backends of a higher priority are tried before a backend of a lower priority is selected.
  • vmod_accounting has a new get_namespace() function to get the currently set namespace.
  • vmod_activedns has a new optional host parameter when creating a new dns_group, equivalent to calling .set_host() after creating the group.
  • Improved and more consistent logging in vmod_jwt. Logging has been added to more error paths, and now uses the Error log tag instead of VCL_Log.
  • vmod_jwt now supports HMAC keys in JWK. See the vmod_jwt manual page for more information.
  • varnishtest has a new include command, which can be used to build more complex test cases and reuse VTC fragments across tests.
  • vmod_utils has a new dyn_probe.probe_custom() method that can generate a custom backend health probe at request time.
  • Add parameter tls_ja3 that enables JA3 fingerprint calculation.
  • vmod_tls has a new ja3() function that returns the JA3 fingerprint for the TLS connection when parameter tls_ja3 is enabled.

Bugs fixed:

  • Fix a bug in vmod_slicer relating to a flag being persisted across transactions which could under certain circumstances lead to a crash. (VS issue #1952)
  • Fix a bug where an http/2 session thread could be left holding a VCL reference, resulting in discarded VCLs not being released. (VS issue #1946)
  • Fix a bug in vmod_accounting where a log message would falsely indicate counters were not updated.
  • Fix a bug in vmod_accounting which would prevent the adding of a key if it uses one of the reserved key names as a prefix.
  • Fix a bug in vmod_slicer where slicer.failed() returned false when the failure was due to the response being fetch processed. (VS issue #1959)
  • Fix a bug that could lead to a panic during startup if duplicate listen endpoints were specified at the command line. (VS issue #1956)
  • Fix a bug in vmod_s3 that could lead to a panic when a director tried to create a backend in a cold VCL. (VS issue #1892)
  • Plug a leak in vmod_stat where a prometheus backend could run out of workspace and fail to render a response. (VS issue #2003)

Varnish Enterprise 6.0.12r5 (2023-12-07)

New features:

  • Introduce CLI commands for handling certificates. You can now load, discard and update certificates without restarting Varnish.
  • Introduce TLS listening endpoint as a -a option.
  • Introduce optional_no_ca as an option for client-verify.
  • vmod_tls has a new function .client_verified() that can be used to check the certificate verification.

Varnish Enterprise 6.0.12r4 (2023-11-29)

Bugs fixed:

  • Fix a bug in vmod_sqlite3 where sqlite3.open could trigger an assert if it failed when called from vcl_init.
  • Fix a bug in vmod_stat that would trigger an assert if called from a client-side VCL subroutine.
  • Fix a bug in vmod_bodyaccess where certain invocations would trigger an assert if used outside of a client-side VCL subroutine.
  • Improve the resilience against HTTP/2 clients behaving in a suboptimal way. (VS issue #1904)
  • HTTP/2 panics could result in too much output per stream making the panic string truncated. We now limit the output to the first three streams.

Varnish Enterprise 6.0.12r3 (2023-11-17)

Bugs fixed:

  • A mistake in the previous release introduced a race condition that would under certain workloads lead to a panic.

Varnish Enterprise 6.0.12r2 (2023-11-16)

New features:

  • vmod_utils has a new function .lookup_file() that searches for a file in the vcl_path used at VCL load time and returns its absolute path.

Bugs fixed:

  • Prevent pipe transactions from sending two Transfer-Encoding headers. (VS issue #1879)
  • vmod_probe_proxy global override does not apply to tcponly probes anymore.
  • Assert in mse_signal() could be wrong if the child dies before MSE has fully initialized the environment. (VS issue #1884)
  • The child could hang during exit if some atexit() callback function kept running. This problem was observed if a VMOD loaded a shared object. The fix was to not call atexit() callback functions in the child.
  • Fix an issue where purging of sliced objects could lead to a race with the purging of slice dependencies, causing a workspace panic. (VS issue #1883)
  • Fix an issue that caused TLS session resumption to fail when a client certificate was used. (VS issue #1878)
  • Fix an issue where closing file descriptors would take a long time. (VS issue #1885)

Varnish Enterprise 6.0.12r1 (2023-10-23)

New features:

  • Added mitigation options and visibility for HTTP/2 "rapid reset" attacks (CVE-2023-44487).

    Global rate limit controls have been added as parameters, which can be overridden per HTTP/2 session from VCL using the new vmod h2:

    • The h2_rapid_reset parameter and h2.rapid_reset() function define a threshold duration for an RST_STREAM to be classified as "rapid": If an RST_STREAM frame is parsed sooner than this duration after a HEADERS frame, it is accounted against the rate limit described below.

      The default is one second.

    • The h2_rapid_reset_limit parameter and h2.rapid_reset_limit() function define how many "rapid" resets may be received during the time span defined by the h2_rapid_reset_period parameter / h2.rapid_reset_period() function before the HTTP/2 connection is forcibly closed with a GOAWAY and all ongoing VCL client tasks of the connection are aborted.

      The defaults are 100 and 60 seconds, corresponding to an allowance of 100 "rapid" resets per minute.

    • The h2.rapid_reset_budget() function can be used to query the number of currently allowed "rapid" resets.

    • Sessions closed due to rapid reset rate limiting are reported as SessClose RAPID_RESET in vsl(7) and accounted to main.sc_rapid_reset in vsc as visible through varnishstat(1).

  • The vcl_req_reset feature (controllable through the feature parameter, see varnishd(1)) has been added and enabled by default to terminate client side VCL processing early when the client is gone.

    req_reset events trigger a VCL failure and are reported to vsl(7) as Timestamp: Reset and accounted to main.req_reset in vsc as visible through varnishstat(1).

    In particular, this feature is used to reduce resource consumption of HTTP/2 "rapid reset" attacks.

    Note that req_reset events may lead to client tasks for which no VCL is called ever. Presumably, this is thus the first time that valid vcl(7) client transactions may not contain any VCL_call records.

Bugs fixed:

  • The socket.close command could trigger a panic under certain circumstances. (VS issue #1856)
  • The socket.open command could fail to reacquire privileged ports. (VS issue #1860)

Varnish Enterprise 6.0.11r7 (2023-09-27)

Bugs fixed:

  • The parameter timeout_reqbody was accidentally enabled by default. (VS issue #1845)

Varnish Enterprise 6.0.11r6 (2023-09-21)

New features:

  • MSE 3 has a new fault tolerance facility which allows it to continue running with a subset of its configuration in the event of hardware failure.

    See the varnish-mse(7) manual page for more information.

  • New commands socket.open, socket.close and socket.list for the CLI. They can be used to decide whether to accept or refuse new client traffic. When sockets are closed, ongoing requests are processed until they complete, but new requests are refused and connections are closed. For now, it is only possible to open or close all listen sockets.

    See the varnish-cli(7) manual page for more information.

  • The old parameters of timeout_req and timeout_reqbody from Varnish Plus 4.0 are back due to popular demand. timeout_reqbody is disabled by default.

  • Refined statistics for the reasons why HTTP/2 sessions are closed. (3507)

  • New VTC tunnel command that acts as a proxy between two peers. A tunnel can pause and control how much data goes in each direction. It can also be used to trigger socket timeouts, possibly in the middle of protocol frames, without having to change how the peers are implemented.

  • The counter MAIN.http1_iovs_flush has been added to track the number of premature writev() calls due to an insufficient number of IO vectors.

  • vmod_jwt now supports the JWS header parameter kid, which can be used to select a JWK when verifying and is only used for this purpose. An optional parameter check_kid has been added to the .verify() method, which determines whether the kid is used in verification with JWKs.

    See the vmod_jwt(3) manual page for more information.

  • vmod_jwt method .verify() can now skip the timestamp validation checks of the exp and nbf claims. New boolean parameters check_exp and check_nbf control whether to skip the checks. The default behavior is to validate the exp and nbf claims if present.

  • vmod_jwt can now generate and verify signatures of arbitrary data with new methods .generate_raw() and .verify_raw().

  • New vmod_utils function .hex2integer() can convert hexadecimal numbers to decimal numbers.

  • Better column alignment in the varnishscoreboard output.

  • The .length() and .is_<type>() functions in vmod_json now have an optional element argument to inspect the JSON context more deeply than the root node.

  • vmod-s3 now has support for creating backends with arbitrary port numbers, enabling TLS for backends created with port 443.

  • vmod_jwt methods .set_key(), .generate(), .generate_raw(), and .to_string() now support base64url encoded HMAC keys (secrets). These methods now have an optional encoding parameter to indicate if the secret is encoded.

  • A warning message will be logged to standard output and syslog if the working directory is not mounted on tmpfs. (VS issue #560)

  • The error message received when the working directory is on a filesystem mounted with noexec has been improved. (3943)

Bugs fixed:

  • The Process timestamp for vcl_synth {} was wrongly issued before the VCL callback, now it gets emitted after VCL returns for consistency with vcl_deliver {}
  • Two bugs in the ban expression parser have been fixed where one of them could lead to a panic if a ban expression with an empty header name was issued. (3962)
  • Fixed a potential race condition in the backend probe code. (3957)
  • Added missing header checks specific to HTTP/2. (3952)
  • Fixed a bug in vmod_slicer that could lead to a panic when attempting to slice an empty response. (VS issue #1821)
  • It is no longer possible for a backend to exceed last_byte_timeout under certain conditions.
  • Maintain HPACK consistency when HTTP/2 streams are refused.
  • Fixed a crash in vmod_headerplus that could happen when used from sub vcl_pipe. (VS issue #1804)

Varnish Enterprise 6.0.11r5 (2023-08-14)

New features:

  • Introduce headerplus.attr_exists() which returns true if a given attribute exists within a specified header.
  • new vmod_utils function apply_cache_headers can parse beresp to align beresp.ttl, beresp.grace and beresp.keep on them.

Bugs fixed:

  • Fixes a bug in vmod_digest. (VS issue #1789)

Varnish Enterprise 6.0.11r4 (2023-07-12)

With this release, the VRT version has been bumped from 6.8.0 to 6.9.0, which means that

  • VMODs need to be recompiled for 6.0.11r4 to work with this version. No substantial code changes will be necessary. Only VMODs you have compiled yourself are subject to the statement above. Packaged VMODs will always follow the VRT version of varnishd.
  • If the Varnish packages are upgraded without restarting, it is very likely that a VCL load will fail. In general, a restart is advised whenever the packages have been upgraded.

New features:

  • Backend tasks can now queue if the backend has reached its max_connections. This allows the task to wait for a connection to become available rather than immediately failing. This feature is enabled by setting both of the new parameters added.

    New parameters: backend_wait_timeout sets the amount of time a task will wait. backend_wait_limit sets the maximum number of tasks that can wait.

    These parameters can also be set as backend attributes .wait_timeout and .wait_limit.

    New counters: backend_wait count of tasks that waited in queue for a connection. backend_wait_fail count of tasks that waited in queue but did not get a connection within the wait_timeout.

  • VCL backend probe gained an .expect_close boolean attribute. By setting to to false, backends which fail to honor Connection: close can be probed.

    Notice that the probe .timeout needs to be reached for a probe with .expect_close = false to return. (3886)

  • Backend health is checked once and cached per transaction in vmod_udo, and a new .reset() can clear the transaction's cache for either health check or exhaustion status.

  • A new .set_identity() method for vmod_udo directors enable the manual identification of a Varnish node as being one of its own backends. This is an alternative to the dynamic .self_identify() approach. Another new method .is_identified() confirms whether a Varnish server is identified as one of its backends, through either means.

  • The .self_identify() method in vmod_udo takes a comma-separated list of identifiers to detect loops in a self-routing cluster.

  • varnishncsa: Add support for the -T option

  • The Slicer has a change in behavior in its handling of serving a 304 Not Modified response. Slicer will now ensure it visits all of the relevant segments also on a 304, to ensure they stay relevant in terms of LRU reordering.

  • varnishtest: Add new tls_config arguments sess_out and sess_in for persisting and resuming a TLS session.

  • varnishtest: Add a new tls.sess_reused command for use with expect, to query if a reused session was negotiated.

  • Add a .find() method to vmod_aclplus. This method returns the rule that was matched from the supplied ACL list.

  • Add a gauge function to vmod_kvstore. This function creates a key and sets its value.

  • A new .set_quick_ack() function for vmod-tcp can now be used to send of an immediate acknowledgement to clients sending small messages without using TCP_NODELAY. A new .get_quick_ack() function can also be used to retrive the current setting. See vmod_tcp(3) manual page for more information.

  • The rendering of counters made by varnishstat moved to libvarnishapi. As a result JSON rendering is now shared between varnishstat and vmod_stat.

    This move brings a change to the libvarnishapi.so soname, and might require a rebuild of third-party applications linking to it. Rebuilding should not be needed, as libvarnishapi should remain binary-compatible, but care should be taken to ensure this before upgrading.

  • vmod_jwt can now sign and verify signatures with elliptic curve (ECDSA) algorithms ES256, ES384, ES512. The method .set_jwk() allows JWKs with those algorithms and supports curves P-256, P-384, P-521.

Bugs fixed:

  • Fix bug in vmod_aclplus where .match() would return an error when matching against ::/0
  • Prevent the vmod_file backend from storing fetch_chunksize body bytes when fetching an empty file.

Varnish Enterprise 6.0.11r3 (2023-06-14)

Bugs fixed:

  • Fix a bug relating to scoreboard task tracking that could in certain cases cause a panic when Varnish was out of worker threads. (VS issue #1698)
  • Let tasks linger in the varnishscoreboard output until completion when task tracking is disabled. This is needed to verify consistency checks related to queuing. (VS issue #1718)
  • Fix an issue in vmod-utils where fast_304() would result in a too high TTL if the 304 from the backend contained an Age header. (VS issue #1737)
  • Fix a race condition in vmod-goto that could lead to a panic. (VS issue #1741)
  • Fix an unlikely deadlock situation in vmod-activedns relating to discarding a VCL. (VS issue #1715)
  • Fix a race condition that could lead to a deadlock in vmod-activedns on VCL temperature change. (VS issue #1725)

Varnish Enterprise 6.0.11r2 (2023-06-01)

New features:

  • Introduce vmod-s3. This VMOD provides functionality for using Varnish as a caching proxy in front of S3, including acting as a dynamic backend director for S3 endpoints.
  • Add headerplus.attr_count() to return the number of attributes of a given header.
  • Add a contains() method to vmod-kvstore. This method returns true if the key exists in the kvstore and is not expired. Returns false if the key does not exist or is expired. Note that if the key is expired, it is deleted from the kvstore.
  • Add a decode() function to vmod-uri. This provides a simple utility function for URI decoding of unreserved characters.
  • Log storage name on allocation failures. (VS issue #1702)
  • Reduced pressure on storage when large request bodies are streamed to the backend. (VS issue #1704)
  • Feature settings validate_client_responses and validate_backend_requests are now enabled by default.
  • Allow any negative expression in VCL. (3124)
  • Better panic backtraces for SIGQUIT in the cache process. (3917)

Bugs fixed:

  • Fix a bug where we did not respect the max_restarts limit if a VCL transaction restart was triggered from sub vcl_synth. (VS issue #1694)
  • Fix a bug in vmod-slicer that would lead to a panic if a slicer subrequest performed a VCL return (fail). (VS issue #1674)
  • Fix a bug in backend probe where timeout would display a Poll error 0 (Success) error message. (VS issue #1724)
  • Fix unset bereq.body not removing cached request bodies. (VS issue #1665)
  • Fix a performance issue in http/2 upload processing. (VS issue #1722)
  • The http/2 request parser now allows URLs starting out with // as per the updated http/2 specification (rfc9113).
  • Fix a bug that would lead to a panic if std.healthy() was invoked on a vmod-goto backend in sub vcl_pipe. (VS issue #1709)
  • Fix a bug in vmod-aclplus that could lead to a double free. (VS issue #1720)
  • Fix a deadlock situation that would occur when beresp.transit_buffer was used in combination with a hit-for-miss or a hit-for-pass object. (VS issue #1673)
  • Avoid panic in vmod-goto when the client certificate string isn't guaranteed to be constant, leading to duplicate backend entries. (VS issue 1679)
  • Avoid panic in vmod-goto when the TTL rule allows the TTL configuration to change, leading to duplicate backend entries. (VS issue 1679)

Varnish Cache Plus 6.0.11r1 (2023-03-10)

New features:

  • Add counters n_object_hitmiss and n_object_hitpass for the number of hit-for-miss and hit-for-pass objects in the cache.
  • In varnishtest, the tls.cert objects have gained a new subject_alt_names operation for validating subject alt names.
  • vmod_rewrite can now be configured to use braces ({ and }) as an input field separator.
  • New vmod_http_pool_timeout parameter to avoid connections remaining in the CLOSE-WAIT state when there is no vmod_http activity. (VS issue #1656)

Bugs fixed:

  • Plug a leak in the scoreboard task tracking involving sub-requests, waiting lists and tracking slots starvation. (VS issue #1655)
  • Fix an issue where for certain configurations the TLS 1.3 ciphersuite configuration was not properly applied. (VS issue #1654)
  • Better reporting of acceptor tasks in the scoreboard for unix-domain socket endpoints. (VS issue #1661)
  • Fix a vmod_accounting bug that would lead to an assert if add_keys() was called from sub vcl_init. (VS issue #1675)
  • Fix a bad example in the documentation of vmod_image. (VS issue #1648)

Varnish Cache Plus 6.0.10r5 (2023-01-24)

New features:

  • Improved varnishtest DNS and file. Files can now be mounted on any path, even if parts of the path do not exist. There is a new feature priv_network, which brings the test into a new network namespace. DNS servers are now declared without a name (dns { <config> }). File -match and -match-not have been removed.
  • Add parameter query_keep_equal_sign to vmod_urlplus. When enabled this will keep the equal sign in the query string if the value is empty.
  • ESI will log an error when a sub-request's response is not delivered because the client disconnected before it was its turn to be transmitted.
  • New counters ACCG_DIAG.[be]req_dropped for the rare case where a client or backend task would start with vmod_accounting loaded, but unloaded before its completion. In this case keys are no longer accounted for.
  • Task tracking for the scoreboard is now performed per thread pool. The vst_space parameter is expressed per thread_pools. This allows task tracking to respect the experimental numa_aware parameter. As a result, probes can no longer be tracked in their waiting state.
  • IP conversions could fail with a cryptic getnameinfo = -12 Unknown error when running out of workspace. This conversion is now performed on the stack to better report the underlying workspace overflow. (3765)
  • New counter shm_bytes for the number of bytes written to the shared memory log.
  • Added optional position parameter to urlplus.query_add and urlplus.query_get.
  • TLS: Support for verification of client certificates. This is configurable via configuration settings client-verify and client-verify-ca. More information here.
  • Added tls.client_cert(), which can be used to query whether the client presented a certificate and provide the subject name of that certificate.
  • TLS: The tls-protos setting will now be applied at the connection-level, decoupled from any certificates in the same configuration scope.
  • TLS: The ciphers and ciphersuites options can now be specified inside of a pem-file configuration block. This setting will take precedence over any other configured ciphers configuration for that particular certificate context.

Bugs fixed:

  • tls.cfg: Avoid logging a warning for missing DH params when provided in a separate file. (VS issue #1603)
  • Only IP addresses can be used with vmod_aclplus, this is now enforced.
  • slicer: Fix a bug that could lead to a panic if a previously uncacheable object turned into a cacheable object. (VS issue #1628)
  • An ESI sub-request's keys for vmod_accounting could be leaked if it turned out to not be delivered. (VS issue #1608)
  • Under rare circumstances a vmod_accounting leak could be caught and turned into a panic. (VS issue #1608)
  • Fix a vmod_http assert when calling http.resp_copy_headers from vcl_recv. This will now terminate the transaction instead.
  • Fix two cases of incorrect references to function names in vmod_http log messages. (VS issue #1577)
  • Fixed race condition when using vmod_stat filters. (VS issue #1598)
  • Fix an issue in MSE waterlevel handling that could when the system is under high load cause an assert from attempts to purge an object that was still being fetched. (VS issue #1607)

Varnish Cache Plus 6.0.10r4 (2022-11-30)

New features:

  • Add remove_all_duplicates() function to vmod_cookieplus.
  • Add bitmap resolution parameter to backend_prometheus() in vmod_stat.
  • Add ability to filter vmod_stat Prometheus output by Accounting namespace and VCL name.
  • Add webp input support to vmod_image.
  • Add crop support to vmod_image.
  • Add ability to lock aspect ratio when resizing using vmod_image.
  • Add set_pipe_timeout() function to vmod_utils.
  • Add Slicer and Range request support to vmod_file.
  • Add a VHA6 setting to change the linger time of dynamic origin backends.
  • ActiveDNS now uses the group candidate port when resolving an SRV record with port number 0, instead of failing the resolution.
  • Add get_hits() function to vmod_stale.
  • The Slicer now allows a slicer subrequest fetch to do a return (retry) from vcl_backend_error.
  • New varnishstat counters SLICER.meta_req, SLICER.meta_cache_hit, SLICER.meta_cache_miss, SLICER.slice_req, SLICER.slice_cache_hit, SLICER.slice_cache_miss, SLICER.slice_beresp_200 and SLICER.slice_beresp_206.
  • Add a reset() function to vmod_json.
  • The description of acceptor tasks in varnishscoreboard includes the listen address it works with and the thread pool it operates on.
  • Processing logs at rest from a regular file should be faster by a 6 to 9 fold depending on grouping. (3429)
  • Add parameter backend_cooloff. This controls the delay in between cleanup and deletion of a backend.
  • Add set_subtype() method to vmod_udo directors. This is especially useful for implementing partial replication in a two-tier sharding architecture.
  • Two new feature flags, validate_client_responses and validate_backend_requests for turning on validation of client and backend requests, respectively. At some CPU cost, this will reduce the chance of invalid requests and responses when VCL changes headers and pseudo headers.
  • Request and response validation in vmod_http following the new validation of requests described above.
  • In vmod_uri, add an encode parameter to several functions for setting values. With encode set to true, the new values will be percent encoded as they are set.

Bugs fixed:

  • Fix an issue where subsequent calls to reset() in vmod_urlplus would use up workspace.
  • Fix an issue where subsequent calls to reset() in vmod_headerplus would fail a request.
  • Bodyaccess: len_req_body now returns 0 when the POST body is empty.
  • Bodyaccess: hash_req_body no longer logs an error when the POST body is empty.
  • Slicer: Fix a panic on invalid response body status triggered via vmod_file.
  • Slicer: Fix inadvertent cache miss on certain final segment ranges.
  • Allow varnishncsa to see the client IP address for bad requests. (VS issue #1515)
  • Attempt at structuring and clarifying YKEY logs.
  • Reduced stack usage when unbuffered log records are added. (VS issue #1553)
  • Slicer: Fix an issue where we would needlessly close the connection after delivery of a superseded object.
  • Fix an issue where calling std.healthy() on a UDO director from a client context would lead to an assert.
  • Fix a rare issue where a worker threads in vmod_goto would race against a VCL cold event during backend creation, leading to a CLI deadlock.
  • Varnish no longer includes an If-Modified-Since header in a backend fetch if the stale object has a Date header equal to the Last-Modified header. (3852)
  • Fix a crash when Slicer segments are purged by the MSE waterlevel thread. (VS issue #1595)

Varnish Cache Plus 6.0.10r3 (2022-10-04)

Security Update

Bugs fixed:

  • Fix an issue in vmod_crypto where the functions crypto.urandom() and crypto.uuid_v4() could cause a crash if the system has run out of file descriptors. (VS issue #1521)
  • Fix a denial of service vulnerability when using the proxy protocol version 2. (VSV00011)

Varnish Cache Plus 6.0.10r2 (2022-08-16)

New features:

  • varnishscoreboard: Introduce new step esi_dispatch. This indicates that a task is currently dispatching a set of ESI subrequest tasks.
  • Improved parallel ESI request scheduling, yielding better performance for certain workloads.
  • Add prometheus format for vmod_kvstore counters in vmod_stat

Bugs fixed:

  • Background fetch tasks are abandoned instead of being queued if there is no thread available immediately. A grace hit could otherwise block until the background fetch released its reference to the request. If there is no thread available, it also means that an opportunistic background fetch can be skipped. See the MAIN.bgfetch_no_thread counter. (3827)
  • When creating the revalidated object during backend conditional fetches, correctly account for the number of auxiliary object attributes when these are present on the stale object. (VS issue #1494)
  • Fix a bug in vmod_slicer that would cause a panic when revalidating a sliced object if the slicer was not enabled during revalidation. (VS issue #1496)
  • varnishscoreboard: Ensure that all ESI tasks report their parent request VXID and not the session's VXID as parent.
  • varnishscoreboard: More precise task step reporting. (VS issue #1287)
  • Fix a bug in vmod_stat where get_value would incorrectly clamp the value (< INT_MAX).
  • Varnish WAF: Fail transactions gracefully instead of crashing if call to setup_transaction() is missing.
  • Systemd service config: Start varnishd after nss-lookup.target to avoid start up failures after boot.

Varnish Cache Plus 6.0.10r1 (2022-07-25)

New features:

  • New version of vmod_akamai, adding support for HTTP-proxy and HTTP-proxy-tunneling using CONNECT method.
  • Add init_req() and init_resp() functions to vmod_headerplus. These functions serve as context aware versions of headerplus.init().
  • Varnish Cache Plus packages for Ubuntu 22.04 LTS (Jammy Jellyfish).

Bugs fixed:

  • Fixes an issue where gzip compressed ESI content is stored as uncompressed if brotli compression is enabled. (VS issue #1442)
  • Fixes an issue where vmod-image would fail if the content used gzip compression. (VS issue #1435)
  • Dynamic udo backend VSCs are now hidden while the backends are lingering. (VS issue #1467)
  • kvstore VSCs are now hidden when the VCL goes cold. (VS issue #1459)
  • Increasing the lifetime of an object in the cache through stale.rearm(), did not fully work. The TTL and grace was changed, but the time of death of the object was not updated, with the result that the object could be removed earlier than intended. Now all relevant properties of a stale object is updated on stale.rearm(). (VS issue #1488)
  • Fix an issue where removal of object slices during MSE database waterlevel handling would lead to an assert. (VS issue #1444)
  • Fix an issue with pooled backend connections getting stuck in FIN_WAIT_2 state for a prolonged period of time. (VS issue #1445)
  • Fix an issue in vmod_rewrite where an assert was triggered if the given rule set violated the min_fields setting. (VS issue #1454)
  • Fix a problem with workspace exhaustion leading to an assert during LRU handling of sliced objects. (VS issue #1482)
  • Language improvements and spelling corrections in the package included man-pages.
  • Fix an issue in AWS VCL where the equal sign would be removed from a query string if the value was empty.

Varnish Cache Plus 6.0.9r7 (2022-05-20)

New features:

  • The manpage documentation for all the package included VMODs has been overhauled. Information previously only available at https://docs.varnish-software.com is now available in the manpages as well.
  • In varnishtest, process objects have gained a new match-text operation for validating repeated text patterns.

Bugs fixed:

  • A problem has been found in the change introduced in version 6.0.9r5 labelled "Faster recycling of reusable backend connections during a fetch.". The problem would result in a segmentation fault error while handling certain types of failed fetches. This change has been reverted. (VS issue #1436)

Varnish Cache Plus 6.0.9r6 (2022-05-04)

New features:

  • New version of vmod_http. The new version allows for better connection reuse, caching of hostname lookups, and gives better performance when many simultaneous requests are active.

    Note that this version will limit the protocols enabled in the underlying CURL execution engine to HTTP and HTTPS, and that HTTP/1.1 will be the preferred HTTP version. This can be changed by adjusting the corresponding Varnish runtime parameters (vmod_http_require_http and vmod_http_prefer_http_11).

    The new version is VCL compatible with the previous version, and no adjustments to VCL programs is necessary.

  • The documentation of ykey.stat_ functions has improved, and some unnecessary flags have been removed because they give no data. The feature is marked as experimental for now.

Varnish Cache Plus 6.0.9r5 (2022-04-28)

New features:

  • Introduce YKEY stat functions to query the cache for statistics. This feature is experimental and some minor changes to the API in the coming releases are to be expected.

  • Introduce libadns, an asynchronous dns resolution library. VMODs can use this library to configure a domain to be be actively resolved by the ActiveDNS service, and set up callbacks to receive dns updates each time the domain is resolved.

  • Introduce vmod_activedns, a VMOD to create dns_groups. A dns_group contains a set of rules for dns resolution, and a template for creating dynamic backends. Compatible VMODs can subscribe to updates from a dns_group in order to create dynamic backends.

  • Introduce support for subscribing to dns_group updates in vmod_udo, adding full support for dynamic backends.

  • Add functions .get_identifier(), .self_identify(), and .self_is_next() to vmod_udo. These functions are useful for implementing self-identification and self-routing. Works with both static and dynamic clusters.

  • Add experimental cluster.vcl. This VCL can be included to enable self-routing within a cluster, with optional partial replication across nodes. This VCL is subject to change in future releases.

  • Add prometheus format for dynamic udo backend names in vmod_stat

  • Add -delay argument to the varnishtest dns server

  • Add experimental headerplus.write_req0() function which can be used to make label VCLs start from a modified set of headers and URL.

  • Added utils.force_fresh(), which can be used to force a backend request to happen on a fresh connection.

  • Adjusted counters in varnishstat to mitigate negative gauges being represented with very large values. This can happen when a gauge is decremented and its value is flushed before its prior increment was. A negative gauge is represented as zero, since gauges must have been strictly positive before being decremented. This happens mostly in benchmarking conditions.

    A new -r option and r key binding for varnishstat toggle between raw and adjusted gauges in the output. (VS issue #967)

  • Faster recycling of reusable backend connections during a fetch. (VS issue #1383)

  • Add call_backend_response option to probe-proxy.vcl.

  • Add remove_duplicate function to vmod_cookieplus.

  • Add first or last occurrence selection to cookieplus.get().

  • Add support for alternative Akamai Sureroute testobject in the Akamai connector by also intercepting the /akamai/sureroute-test-object.html endpoint.

  • The Slicer now asks for only the bytes needed from the stevedore to fulfill a range request rather than asking for the whole slice. This reduces the load on the disk when using the Slicer with the persisted MSE.

  • There is now TLS support in varnishtest. Read more about this by running man vtc and search for TLS.

Bugs fixed:

  • Fix workspace allocation handling in vmod_rewrite. (VS issue #1333)
  • Fix workspace allocation handling in vmod_var. (VS issue #1361)
  • Added missing logs when a client transaction ends after leaving the waiting list because the client is going away, or already gone.
  • Fix vmod_deviceatlas to correctly maintain the database(s) for the lifetime of the loaded VCL. Important Note: deviceatlas.loadfile() will in the future only be permitted from the vcl_init subroutine.
  • Fix a memory leak in vmod_image during error handling. (VS issue #1368)
  • Improve error reporting in the resulting panic message on IO errors originating in the MSE AIO code paths. (VS issue #1425)

Varnish Cache Plus 6.0.9r4 (2021-12-21)

Security Update

Bugs fixed:

  • In vmod_accounting, attempts to create namespaces using the reserved word std (referring to the always present and always charged namespace) will now be ignored. This also means that VCLs that explicitly attempt to create a namespace called std, will no longer have their statistics charged twice on that namespace. (VS issue #1351)
  • The documentation of the vmod_accounting namespacing rules, and its total and sub keys has been improved to ease in understanding and configuration of the feature.
  • Fix a request body timing issue. (VS issue #1353, VSV00008)

Varnish Cache Plus 6.0.9r3 (2021-12-09)

Bugs fixed:

  • Always provide some form of fallback in std.ip() (3746)
  • Fix problems related to handling slicer segment invalidation when freeing objects due to LRU activity. (VS issues #1341 & #1343)
  • Fix an out of workspace condition in vmod_stat.
  • Correctly mark the workspace overflowed when the vmod_ykey function ykey.get_hashed_keys() runs out of workspace.
  • Fix an out of workspace condition when using Parallel ESI. (VS issue #1345)
  • The Via header changes introduced in Varnish Cache Plus version 6.0.9r1 have been reverted, making the Via header sent by default be the string Via: 1.1 varnish (Varnish/6.0). (VS issue #1344)
  • Improvements to our SELinux policy.

Varnish Cache Plus 6.0.9r2 (2021-11-30)

Bugs fixed:

  • Fix a race condition between the object expiry thread and some newly added object flags controlling object lifetimes. This could lead to flag updates being lost, leading to asserts. (VS issue #1337)

Varnish Cache Plus 6.0.9r1 (2021-11-24)

New features:

  • When using probe-proxy.vcl adding multiple backends to the gateway director will fall back to the next backend on a retry.
  • A new req.hash_ignore_vary flag is available, deprecating the hash_ignore_vary() function from vmod_utils.
  • The Via header automatically added by Varnish now has a new version string identifying the version as 6.0-plus. The Via header can be removed in sub vcl_backend_deliver.
  • The Slicer has been reintroduced, and should be fully functional. This makes it possible to transparently break down very large objects into smaller parts that are individually cached. For more information see the Slicer VMOD documentation page at https://docs.varnish-software.com/varnish-cache-plus/vmods/slicer/

Bugs fixed:

  • Fix a race condition between purges and certain object flags, which could cause issues with Ykey house keeping and subsequent asserts. (VS issue #993)
  • Improve error messages given when failing to authenticate CLI connections due to not being able to generate random data. (VS issue #1306)
  • Correctly reset the received SIGHUP flag in libvarnishapi when no handling function for this signal is registered. (3437)
  • Fixed an issue in VMOD Image where JPEG warnings were treated as errors. (VS issue #1291)
  • Add an alias for the VHA6 option allow_locahost with allow_localhost.
  • Fix a race condition occurring when importing VMOD Accounting during high traffic load. (VS issue #1313)
  • Account for empty header values in headerplus.attr_delete() and headerplus.attr_set(). (VS issue #1294)
  • Fix an issue when calling purge frequently on a single hash key, and there is a very large number of objects on that hash key. This could lead to the purge operations taking a very long time. (VS issue #1315)
  • Added check for NULL in the .global_get function in VMOD var. (VS issue #1325)

Varnish Cache Plus 6.0.8r7 (2021-10-20)

Bugs fixed:

  • The Slicer VMOD and core changes have been temporarily removed. Once some critical issues have been corrected, it will be released again. (VS issue #1289)

Varnish Cache Plus 6.0.8r6 (2021-10-19)

Note: This release has been removed from the repositories due to a critical bug in the new Slicer feature.

New features:

  • Introduce VMOD Image including a function to convert JPEG and PNG images to WebP images.
  • Introduce the new Slicer VMOD and accompanying core features. This makes it possible to transparently break down very large objects into smaller parts that are individually cached. For more information see the Slicer VMOD documentation page at https://docs.varnish-software.com/varnish-cache-plus/vmods/slicer/ .
  • New parameter h2_rxbuf_storage to select which storage backend temporary HTTP/2 receive buffers should be allocated from. This defaults to Transient.
  • Varnish internal zlib library is now namespaced, to avoid naming conflicts with any externally linked in zlib library.

Bugs fixed:

  • Fix a race condition in vmod_goto that was introduced in 6.0.7r2. This could lead asserts in rare circumstances. (VS issue #1280)
  • Fix error cause reporting on MSE store AIO errors.
  • Fix a memory leak in vmod_tls when using the cert_sign() and cert_key() functions. (VS issue #1284)
  • Fix accumulation of vmod_http idle connections. These should now be closed in a timely manner when not being reused immediately. (VS issue #1209)
  • vmod_tcp function get_estimated_rtt() now returns floating point values instead of rounded integers. (VS issue #1248)
  • Headerplus's attr_get(), attr_set(), attr_delete() can now handle quoted values of a key value pair. headerplus.attr_delete() adds a parameter for the attribute delimiter attr_delim defaulting to =. headerplus.attr_set() requires a matching attribute delimiter. (VS issue #1198)

Varnish Cache Plus 6.0.8r5 (2021-10-08)

Bugs fixed:

  • Fixed an off-by-one bug in xbody.log_body that would in rare cases lead to a panic (VS issue #1247).
  • Fix a race in varnishtest cyclic barriers where the barrier could loose count of the number of waiters and deadlock the test. (11)
  • Experimental VMOD vmod_stat has received several format fixes to both its prometheus and JSON output backends.
  • Experimental VMOD vmod_stat now has the ability to pull a named statistics value from VCL.
  • The memory pool varnishtstat values for ssl_buf and busyobj will now have the pool number appended correctly, showing as MEMPOOL.ssl_buf<pool-no> and MEMPOOL.busyobj<pool-no>. (VS issue #1275)
  • Fix a problem with vmod_mmdb when passing NULL pointer as the input IP address. (VS issue #1244)
  • Set the TRUSTED_FIRST OpenSSL certificate verification flag on all platforms. This is the default behavior since OpenSSL version 1.1.0, but this applies the flag also on the platform using an older OpenSSL (CentOS 7). This helps with verifying certificates when there are multiple crossigned verification paths, and only some that are valid. (VS issue #1277)
  • The socket option inheritance checks now correctly identifies situations where UDS and TCP listening sockets behave differently, and are no longer subject to the order the inheritance checks happens to be executed (3732).

Varnish Cache Plus 6.0.8r4 (2021-08-26)

New features:

  • Add urlplus.query_set() to update query values.
  • Implement backend.list -j for varnishadm.
  • Add a counter for the number of ESI sub requests.
  • New timestamps Fetch, Connected and Process for backend transactions.
  • Add grace_hit counter to vmod_accounting counter set for keys.

Bugs fixed:

  • Backend_health log records no longer display (null) as the name the first time it is logged. (VS issue #878)
  • Fixed VCL temperature race with dynamic backend probes that would cause the child process to fail an assert on startup. (VS issue #1194)
  • varnishscoreboard: Fix various missing timestamp value issues.
  • varnishscoreboard: Fix a missing Address field issue.
  • varnishlog: Empty JSON arrays will not be printed when logs are filtered by either client or backend transaction types.
  • Handle large synthetic bodies exceeding the maximum chunk size in effect (either parameter fetch_maxchunksize for regular storages, or memcache_chunksize for MSE). (VS issue #1228)
  • vsl: Give up when it's too late to buffer records (3451)
  • Plugged some non-essential one-time memory leaks.

Other changes:

  • The vcl.show -v command now prints the built-in VCL last. It used to be the loaded VCL, then the built-in, then the loaded VCL includes. From now on it's the loaded VCL, then its includes, and finally the built-in code common to all loaded VCLs.

Varnish Cache Plus 6.0.8r3 (2021-06-15)

New features:

  • Increased the MAX_HDR limit in varnishtest from 50 to 256.

Bugs fixed:

  • Fix a vmod_goto bug introduced in 6.0.8r2, where creating two identical dns_director objects would trigger an assert. (VS issue #1176)
  • Fix a NULL pointer dereference in an error handling path in the JWT VMOD.
  • Fix a NULL pointer dereference when setting namespaces in the Accounting VMOD.
  • Fail correctly when the Stat VMOD is attempted called from Client context.
  • Better error handling for workspace errors when allocating fetch and delivery processors. (VS issue #1177)
  • Correct an H/2 buffer handling issue introduced in 6.0.8r2. (VS issue #1182)

Varnish Cache Plus 6.0.8r2 (2021-06-01)

New features:

  • Introduce vmod_accounting, create customizable summarizing statistics with with namespace separation as stat counters (see man vmod_accounting).
  • goto directors now accept the option extra_string argument which adds a suffix to the generated backend names.
  • goto DNS message buffer size is increased from 4096 to 65535 bytes.
  • goto DNS cache compaction interval decreased to run at most once every ten seconds.
  • goto.dns_backend() can now be used from vcl_pipe.
  • Introduce vmod_uri to make it easier to parse a URI into the five generic components: scheme, authority, path, query and fragment (see man vmod_uri).
  • Introduce vmod_udo, a new director unifying most of the existing ones (shard, weighted random, fallback) under a common interface with extra features (see man vmod_udo).
  • vmod_saintmode blacklist methods/functions have been deprecated and replaced by corresponding denylist callables.
  • vmod_saintmode objects now have an extra .denylist() method that can be used anywhere in a backend context.
  • Add weighted backends to the shard director.
  • Introduce vmod_probe_proxy to allow users to redirect probes to a new location or enables probes to first pass through VCL.
  • Introduce vmod_stat to expose varnish-counters via a backend, in both JSON and prometheus formats. It is released as a tech preview meaning the output formats are subjects to change in the near future.
  • Introduce transit buffering using the beresp.transit_buffer VCL variable, which allows controlling how far ahead the receiver is allowed to be compared to the sender.
  • Add a limit to the number of operations done in xbody.capture() and xbody.regsub().
  • Base64 encode a response body with utils.base64_encode().
  • Lower the memory footprint of kvstore keys.
  • Use the new experimental parameter -p reuseport=on to apply SO_REUSEPORT on all listen sockets. This allows better load distribution across pools. See param.show reuseport for more details.
  • Use the new experimental parameter -p numa_aware=on to make varnishd more efficient on systems with more than one NUMA node. See param.show numa_aware for more details.
  • Add a new VTC command, -bodyfrom, to send a file as the request or response body. Usable from client and server sections.

Bugs fixed:

  • Fix a vmod_goto bug where some parameters were not taken into account when deciding whether dns_backend() should reuse an existing backend. Also enabled dns_backend() to reuse backends from dns_director(). (VS issue #1107)
  • Fix a vmod_goto bug where a dns backend/director with ttl = 0 resolving a resource record with ttl = 0 would cause the dns response to be cached indefinitely. (VS issue #1127)
  • Fix a vmod_goto bug in the /etc/nsswitch.conf parser where newlines were not considered delimiting characters. (VS issue #1125)
  • Fix vmod_goto memory leaks triggered by invalid VCL inputs
  • Fix an issue where varnishd will sleep for 1s when attempting to shutdown. (3569)
  • Fix an issue where directors.round_robin() can sometimes be empty when a single backend is sick. (3474)
  • Fix an issue where a backend fetch can stall after a client has disconnected. (3556)
  • Better support for binary data when using xbody.capture(). (VS issue #1105)
  • Fix a panic situation in vbf_stp_condfetch(). (3558)
  • Fix an issue in directors.shard() where calling reconfigure() on an empty director breaks subsequent shard directors. Also changed an internal canon_point calculation. (3593)
  • Fix a segfault in cookieplus when using regex. (VS issue #1164)
  • HTTP/2 request body handling now buffers data as necessary to avoid head of line blocking on the session socket and long delays in serving streams as a result. (VS issue #1129)
  • Fix a null pointer dereference in utils.time_format().
  • Bring back vmod_akamai CCU API logging. (VS issue #1106).
  • Fix a problem with VMOD shard and vcl_pipe. (3361)

Varnish Cache Plus 6.0.8r1 (2021-03-10)

New features:

  • Add 304 support to VMOD File's file.backend().
  • Add auto decrement mode to kvstore.counter() and kvstore.limit() to check the value of an auto decrementing counter.
  • Add std.bytes() for string to bytes conversion.
  • Add utils.hash_ignore_vary().
  • Improved logging of ykey events.

Bugs fixed:

  • Reset the status and response on a retry. (3525)
  • synthbackend.mirror() will now fail transactions where the body couldn't be fully ingested.
  • Fix a synthbackend.mirror() race condition were concurrent synthetic backend fetches could interfere with each other, leading to a double free and a crash. (VS issue #1097)
  • Fix a potential data race when referencing resp.reason in sub vcl_synth. (VS issue #1083)
  • Fix ESI panic when an error is encountered during transmission. (VS issue #1080)
  • Fix an erroneous short response when a backend stream fails early. (VS issue #1091)
  • Improve watchdog handling. Most notably, only trigger when there is no movement for the highest priority.
  • Fix a problem where the threads_limited counter was incremented when it should not be. (3531)
  • Fix panic when using headerplus in an ESI or Edgestash sub requests.

Varnish Cache Plus 6.0.7r3 (2021-02-09)

New features:

  • Introduce resp.send_timeout to set the send_timeout on a per-request basis from vcl.
  • varnishncsa: add support for Epoch timestamps. (VS issue #1050)
  • MSE variable memory_target can now be negative for non % tagged value. This allow to specify the amount of memory you want to keep for your system. e.g. : If you have X G of RAM and set memory_target to -2G, MSE will be told to use only X - 2 G .
  • Object misses can now be counted via utils.backend_misses() to allow objects to be inserted on the second miss.
  • Introduce utils.waitinglist() which returns true if there are other clients waiting for this object.

Bugs fixed:

  • Fix a bug where VMOD cleanup routines were not executed for ESI subrequests that exited abnormally. (VS issue #1051)
  • Fix a vmod_goto crash when discarding a VCL with a director for a zero ttl domain. (VS issue #1069)
  • Fix a vmod_goto bug where a the last non-zero weight backend in a priority block would always be picked. (VS issue #1061)
  • Fix a bug where the destruction of a dynamic backend races against a backend probe, resulting in a crash. (VS issue #1060)
  • Fix a persisted MSE race condition that could in exceptional circumstances assert on range requests that failed outside the successful fetch range. (VS issue #1062)
  • varnishadm: Properly handle short writes. (VS issue #1033)
  • Close a race between VCP_Control() and vbp_thread that could in rare circumstances cause an assert related to probe status upon VCL temperature change. (3362)

Varnish Cache Plus 6.0.7r2 (2021-01-14)

New features:

  • Introduce new vmod_stale counter, MAIN.fetch_stale_rearm for rearm() and revive()
  • A new function, utils.resolve_backend() is now available.
  • Introduce new VHA6 setting to disable VCS logging.
  • varnishncsa: Log request time in milliseconds (VS issue #1019).
  • varnishtest has a new file resource: Create a temporary file that can be bind mounted over any file in the file system. Bind mounts are test-local with no side effects outside the test process. (VS issue #984)
  • varnishtest has a new dns resource: Configure a test-local dns server that answers all DNS queries during the test. The dns server is test-local with no side effects outside the test process. (VS issue #994)
  • Introduce new port_rule optional parameter to vmod_goto. Default abide will use port from SRV record, while force will use the port parameter. (VS issue #1026)
  • Introduce a new VCL long string format via """...""".

Bugs fixed:

  • Fix a vmod_goto crash with SRV due to a race condition in health reporting. (VS issue #1043)
  • Fix the vmod_goto algorithm for distribution over a set of backends based on priority and weight from SRV records. (VS issue #1043)
  • Fix a vmod_xbody bug where a single end anchor match fails on a chunked response. (VS issue #1035)
  • Fix a bug where beresp.uncacheable cannot be unset after a retry. (VS issue #1038)
  • Invalid assert walking away from the waiting list. (VS issue #1042)

Varnish Cache Plus 6.0.7r1 (2020-12-21)

New features:

  • The tls VMOD can now also be used in vcl_backend_response, to inspect the state of a TLS backend connection. See documentation at https://docs.varnish-software.com/varnish-cache-plus/ for more details.
  • SSL/TLS backends now support transmitting a client certificate as part of the TLS handshake. See https://docs.varnish-software.com/varnish-cache-plus/features/backend-ssl/ for more details.
  • A brand new varnishscoreboard can now track tasks in addition to worker threads. It can also track more than VCL tasks. The varnishd parameters scoreboard_active and scoreboard_enable were renamed respectively to vst_space and thread_pool_track, but were kept as aliases for compatibility.
  • New kvstore STATIC scope, this allows a kvstore to stay intact across VCL reloads.
  • The newly added VHA6 broadcaster_ssl_verify_peer and broadcaster_ssl_verify_host settings allow you to use self-signed certificates on the broadcaster.
  • Introduce a new VMOD, VMOD Format for easier string formatting.
  • VMOD YKey has a new .namespace_reset function.
  • A new function, server_fingerprint, has been added to the mse VMOD.
  • A new function, bytes2string, has been added to the utils VMOD.

Bugs Fixed:

  • Fix an issue with MSE book database waterlevel drift. This can lead to excessive waterlevel activity due to the wrong space usage data being used to control the algorithm. (VS issue 996)
  • Fixed an issue with vmod_http where libcurl would use signals, degrading performance. This fix also affects VHA6.
  • YKey .namespace would panic if supplied with a NULL value (VS issue 995)
  • ban_cutoff would sometimes incorrectly count the total number of bans.
  • VMOD goto would panic when revalidating a transient backend while a cached backend for the same domain exists in cache. (VS issue 1007)

Varnish Cache Plus 6.0.6r10 (2020-10-16)

New features:

  • Varnish now comes with built-in support for Brotli compressed objects. To make use of this feature, it is required to import the brotli VMOD in your VCL. When the VMOD is in effect in the currently running VCL, backend requests will have brotli added to the Accept-Encoding header. The brotli VMOD exposes functions to control the behavior. See the documentation at https://docs.varnish-software.com/varnish-cache-plus/vmods/brotli/ for more information.
  • The file VMOD now has an append function.
  • More detailed listing of storage data in the panic output when using MSE.
  • VCL now allows forcing no connection reuse of backend connections by setting a 'Connection: close' header on either the bereq in vcl_backend_fetch or beresp in vcl_backend_response. Doing this overrides what the backend hinted to for reusable connections. (3400)
  • Introduce an API to register and control custom fetch processors through VMODs.
  • The utils VMOD now has functions for doing bit operations on integers.

Bugs fixed:

  • Fix an out-of-workspace related panic in the http/2 delivery code. (VS issue #916)
  • Fix a bug where we ended up using the wrong timeout if last_byte_timeout was set to a value smaller than first_byte_timeout. (VS issue #947)
  • VHA6 will now account inflight requests against the rate limit.
  • Improved thread creation handling on queued tasks. (2942)
  • Allow VMOD headerplus function attr_set to append only a name, without requiring a value.
  • Fix an out of workspace assertion error in HTTP/2 delivery. (VS issue #916)
  • Fix garbled minimal response protocol error responses sent to clients on TLS connections. (VS issue #952)
  • Fix a parsing error in the urlplus VMOD get_dirname function, where it would for input strings two segments long return the default value instead of the first segment. (VS issue #961)
  • Fix a refcounting issue related to request bodies and restarts. (3433)
  • Use the request workspace consistently for all VCL processing running in vcl_pipe. (3329)
  • The goto VMOD now honors the default search domain from /etc/resolv.conf, or an override set in a LOCALDOMAIN environment variable, when resolving domain names. (VS issue #959)

Varnish Cache Plus 6.0.6r9 (2020-09-30)

Bugs fixed:

  • Fix an issue with client side TLS idle timeout handling, that would cause early hangup of otherwise healthy client TLS connections when the system was under load. (VS issue #888)
  • Fix an assert problem related to timeouts when sending a minimal response protocol error messages in HTTP/1. (VS issue #951)

Varnish Cache Plus 6.0.6r8 (2020-08-11)

New features:

  • New VMOD called resolver. This enables making reverse IP DNS resolution and verification from VCL.
  • New veribot VCL library to enable domain-based access control using the resolver VMOD.
  • New is_healthy field in backend sections in varnishstat. The is_healthy field will be either a 1 if the backend is currently healthy, or a 0 if it is unhealthy. In interactive varnishstat it will show as either true or false.
  • The Akamai connector now comes with VCL 4.1 compatible VCL files. (VS issue #892)

Bugs fixed:

  • Fix a streaming delivery problem when having fetches of unknown length and using the MSE stevedore. (VS issue #904)
  • Fix a bug where timeouts were not properly used during certain kinds of backend TLS handshake failures. (VS issue #880)
  • Fix an issue in vmod_goto that fail to reorder backends correctly on priority changes in SRV records. (VS issue #873)
  • Fix a data set consistency problem in vmod_goto that could surface when backends are removed through DNS updates. (VS issue #874)
  • Fix a storage allocation error handling issue. (3273)
  • Changed the default value of the vsl_reclen runtime parameter from 255 to 4084 bytes.
  • Fix parsing errors in vmod_str. (VS issue #897)
  • Fix a bug in the shard director that could cause an assert if a lazy resolve is used together with a return (pipe). (VS issue #910)
  • On RPM based systems, Varnish will no longer automatically be restarted as part of the package upgrade process. This change is to be consistent with RPM packaging guidelines. Note that even though the varnishd process continues to run, the system may experience problems loading new VCLs after an upgrade until the varnishd process has been restarted. (VS pkg issue #209)
  • Fix a condition where running out of client workspace will assert on H2 delivery. (3382)

Varnish Cache Plus 6.0.6r7 (2020-06-30)

Bugs fixed:

  • Do not implicitly import the xkey VMOD when enabling VHA6. (VS issue #867)
  • Fix an issue that would cause asserts when attempting to restart a persisted MSE cache, when both the xkey VMOD and the memory governor are enabled. (VS issue #868)
  • Fix a bug where would attempt to reuse a backend connection after it was closed. (3266)
  • Fix a bug in str.split() where we in certain cases would drop the initial character of the result string. (VS issue 865)
  • Revert a packaging change that caused errors on upgrade of deb packages.

Varnish Cache Plus 6.0.6r6 (2020-06-23)

New features:

  • Added uuid_v4() to VMOD crypto.
  • Added log_body() to VMOD xbody.
  • VHA6 will attempt to skip broadcasting to itself. Requires Broadcaster 1.3.0 or later.
  • Added VHA6 state hooks for advanced custom use cases.
  • Increased number of available VMOD http instances from 10 to 100.
  • Introduce VMOD headerplus for advanced header access and manipulation.
  • Allow resizing MSE journals

Bugs fixed:

  • Fixed an assert that obscured the proper error message when varnishd refused to start because of an error reported during the execution of vcl_init in the initial command line specified VCL file. This would happen only when MSE was in use. (VS issue 651)
  • Fixed two issues that would lead to an asserts if invalid status codes were attempted set by the VCL configuration. (3301)
  • Make sure that the global published statistics are kept updated even in the case where all worker threads become idle at the same time, and no new work items come in. This could previously show up as an issue when a server was taken out of rotation abruptly, where the statistics appeared skewed due to unflushed buffers. (VS issue 828)
  • Fix error handling of unparseable memory_target parameter strings. (VS issue 849)
  • Fix error where a use after free is possible in GOTO SRV DNS resolution. (VS issue 851)
  • Fixed process leak when using file.exec(). (VS issue 841)
  • Fix a worker pool reserve prioritization issue. (2418)

Varnish Cache Plus 6.0.6r5 (2020-04-27)

New features:

  • GotoResolv has been added as a new VSL Timestamp to log when a task needs to wait for a DNS resolution (when using vmod_goto).
  • New VMOD http parameter vmod_http_max_conn to control the maximum number of connections to leave open.
  • Introduce VMOD tls. This VMOD lets you query for attributes relating to a TLS connection when using Varnish's builtin TLS support. See vmod_tls(3) for details.

Bugs fixed:

  • Fix bug introduced in 6.0.6r3 where local /etc/hosts resolution in VMOD goto did not work.
  • Fix a bug in the MSE object iteration code that could assert when handling a range request, and the fetch thread aborted due to a short read (VS issue 809).

Varnish Cache Plus 6.0.6r4 (2020-04-07)

Bugs fixed:

  • Fix an issue where the Ban Lurker and the Memory Governor would step on each other's toes. The ban lurker now patiently waits until the Memory Governor has set itself up.
  • Fix a bug in the stale vmod which would incorrectly treat a hit-for-miss object as a stale object for a transaction. (VS issue 791).

Varnish Cache Plus 6.0.6r3 (2020-03-30)

New features:

  • Introduce VMOD stale. This VMOD allows for proper stale if error functionality.
  • VMOD goto SRV support.
  • New VHA6 options for stale transfer, streaming transfer, and fast_304().
  • Import VMOD sqlite3.
  • New JWT VMOD function: writer.parse()

Bugs fixed:

  • Fix scoreboard column alignment.
  • Fix kvstore out of workspace panic.
  • Fix TTL logging for bereq.uncacheable / Keep Age information on passes. (3226)
  • Fix another clean up race on libssl resources on process exit.
  • Correct rushing behavior when releasing objects that will not be used due to restart in vcl_hit. (VS issue 776).

Varnish Cache Plus 6.0.6r2 (2020-03-18)

New features:

  • Builtin TLS support is introduced. See varnish-tls-conf(7) for configuration details.
  • MSE Memory Governor. This is an advanced memory management system for MSE that enables setting memory limits for the whole Varnish process, instead of the traditional accumulated cache object payload size. Varnish will then handle the cache size automatically, scaling it up or down in accordance with traffic patterns and memory availability. Note that it requires a configuration change to enable the feature. See varnish-mse(7) manual page for more information.
  • The JWT VMOD was introduced.
  • Export cookies as JSON with cookieplus.as_json().

Bugs fixed:

  • Skip utils.fast_304() TTL re-arm if response is uncacheable.
  • Fixed a bug in the ACLplus VMOD that caused the match() function to fail certain ACL checks.
  • Fixed a crash when a NULL string is given to rewrite.match().

Varnish Cache Plus 6.0.6r1 (2020-02-14)

New features:

  • New VRT function VRT_FlushThreadCache(). This can be used from VMODs to flush the thread memory allocation cache.
  • New VHA fetch timeouts, origin override, localhost detection, and error counters
  • Add utils.mod() for modulo operations.
  • Add utils.fast_random() and utils.fast_random_int() functions for faster pseudo random number generation. These functions are not fit for cryptography applications.
  • vmod_accept now follows the specification better, instead of strict string comparison, allowing a generic tag to match a more specific one.
  • Improved persistent MSE startup performance through the use of a new pseudo random number generator. This is the same random generator as in utils.fast_random() and utils.fast_random_int().
  • Add multiple levels of logging to VMOD HTTP.

Bugs fixed:

  • Fixed bug where VHA would omit new Edgestash delimiters.
  • Disallow VHA stale object transfers.
  • Fix VHA origin header where identity is appended multiple times.
  • Fixed self counting issue in the VHA broadcast_candidates counter.
  • Fixed bug where VHA would omit the broadcaster_group.
  • Fixed special character escaping in varnishlog -S (json output).
  • Fixed a formatting error on invalid VCL_TIME to string conversions. (VS issue #741)
  • Fix a bug on retries of backend requests and uncached request body. (3093)
  • Fix a clean up race on libssl resources on process exit that would give spurious error messages. (VS issue #720)
  • Various H2 bug fixes and improvements have been back ported from the master branch. (2395, 2572, 2905, 2923, 2930, 2931, 2933, 2934, 2937, 2938, 2967, 3086, 3082)

Varnish Cache Plus 6.0.5r3 (2020-01-07)

Security Update

New features:

  • Make VHA6 aware of fast_304 usage, propagating its use on replication events.

Bugs fixed:

  • Correct the minimum heap sorting of next deadline DNS update in vmod_goto. (VS issue #707)
  • Fix a minor memory leak in vmod_goto (VS issue #702)
  • Improve handling of cooling backends. This fixes a rare crash.
  • Fix a denial of service vulnerability when using the proxy protocol version 2. (VSV00005)

Varnish Cache Plus 6.0.5r2 (2019-11-27)

New features:

  • Counters showing ban journal usage in persisted MSE caches.
  • Add json.contains() to search for a key.
  • Add if-range support in conditional fetches (3111).

Bugs fixed:

  • Fix a crash when the user labels a VCL twice (2834).
  • Fix a backend probe scheduling issue that would calculate the wrong sleep time until the next probe. (3115)
  • Fix ban handling in persisted MSE when out of journal space and using the database for additional storage.
  • Fix an issue where the Content-Encoding response header was permitted to be overridden in a conditional fetch.
  • Fix a crash when json.parse_resp_index() and MSE are used together (VS issue 689).
  • Fix a parsing bug in cookieplus where a double delimiter would end up in the cookie name (VS issue 678).

Varnish Cache Plus 6.0.5r1 (2019-10-24)

New features:

  • The MSE VMOD is introduced. Use it to select specific stores or set an object to memory only.
  • Extend VHA6 option force_update to request scope.
  • Add granular VHA6 SSL options for peer to origin communication.
  • Add the option to have a none backend in VCL. In effect, this allows you to start with no (real) backends.
  • A namespace feature has been added to ykey.
  • Added last_byte_timeout to backend fetches.
  • Error handling in VMOD crypto has been improved.
  • Error handling when the VSM volume runs out of space improved.
  • Add a MSE persistent object book insertion timeout feature, which will fall back to non-persisted memory only objects when the timeout is exceeded.
  • Make the number of simultaneous MSE book readers configurable, and increase the default number of simultaneous book readers from 126 to 4096.
  • Optimize VMOD http in cases where large amounts of data was fetched in parallel using asynchronous requests.

Bugs fixed:

  • Fix VHA6 bug where req.* ban evaluation gets skipped.
  • Fix VHA6 bug where Accept header gets removed when used in a Vary response.
  • Fix bug where goto did not respect the user specified TTL (VS issue 598).
  • Allow multiple IPs for the same hostname in /etc/hosts when using VMOD goto (VS issue 647).
  • In VMOD http, req_add_header() will also copy empty VCL headers.
  • Fix a crash with MSE where an uninitialized read would cause a panic in mse_populator_thread (VS issue 660).
  • Better SELinux integration.
  • The idle timeout can now be set per session, by setting the sess.timeout_idle variable in VCL.
  • Fix a problem where the ban lurker would skip objects. (3007)

Varnish Cache Plus 6.0.4r3 (2019-09-27)

Security Update

New features:

  • The str VMOD has been added, with several useful string functions.
  • The experimental scoreboard tool is introduced. It is disabled by default, and you need to turn it on in varnishd to use it.
  • VMOD utils: Add the vcl_name function.
  • VMOD aclplus: IPv6 support added.

Bugs fixed:

  • VHA: Work around a bug in curl where an error number was silently set.
  • varnishncsa: Make empty default ncsa strings empty when in JSON mode.
  • Fixed clearing of a state variable that could case an information leak (VSV00004)

Varnish Cache Plus 6.0.4r2 (2019-09-11)

New features:

  • varnishncsa has a new -j option that will cause its output lines to be formatted in a JSON compatible way.

Bugs fixed:

  • Fixed an MSE assert that could trigger during high LRU activity, constrained memcache size and long object writeout queues.
  • Fixed a VSM bug that would cause varnishlog like utilities to not produce log data. This could trigger when the varnish management process is running root, the cache worker as a non-privileged user, and the log utility run as the same user as the cache worker.

Varnish Cache Plus 6.0.4r1 (2019-09-05)

New features:

  • New return(error()) syntax to sub vcl_backend_fetch and sub vcl_backend_response.
  • Added utils.fast_304() which does an optimized 304 response update.
  • New VHA6 HMAC signatures on origin responses.
  • Added VHA6 Ykey support.
  • Incremental VSM updates. With this change, added or removed VSM segments (ie. varnishstat counters) will be done incrementally instead of complete republishments of the entire set of VSM segments. This reduces the load in the utilities (varnishncsa, varnishstat etc.) when there are frequent changes to the set.
  • Optimize the VSM and VSC subsystems to handle large sets of counters more gracefully.

Bugs fixed:

  • Fixed VHA6 bug where return(pass) responses are candidates for replication.
  • Fixed scenarios where improper vmod_http usage would assert.
  • Fix several resource leaks in libvarnishapi that would cause the utilities to incrementally go slower and use CPU cycles after many changes to the set of VSM segments.

Varnish Cache Plus 6.0.3r9 (2019-08-19)

Security Update

Bugs fixed:

  • Fix a bug from the previous release related to the patch to allow varnishlog to ignore unrelated VSM changes. The optimization would not take effect when varnishd and the log utility were running as different users.
  • Fix a request and response parsing bug. This is the fix of VSV00003, announced on 2019-09-03. (This change log was updated after the announcement).

Varnish Cache Plus 6.0.3r8 (2019-08-06)

This release focuses heavily on bug fixes and performance improvements.

Varnish Shared Memory improvements

  • Batch publishing of VSM segments. With this change, VSM segment removals and additions are published in batches at the given interval (interval is configurable, defaults to 1 second). This limits the frequency at which the utilities needs to refresh the list of available VSMs. Note that this change only affects varnishstat in interactive mode, where there may ie be up to 1 second delay before the counters for a new backend becomes visible.
  • Fix a counter iteration bug that could cause varnishstat to crash.
  • Allow varnishlog and its cousins (varnishncsa, VHA and the VSC agent) to ignore VSM changes that does not affect its functionality. This keeps them from having to reindex the Varnish Shared Memory (VSM) data every time a counter segment is added or removed.

Other bugs:

  • VMOD crypto: Fail gracefully when HMAC key is empty, or has not been set
  • Edgestash: Fail gracefully when out of workspace.
  • Edgestash: Fail gracefully when edgestash functions are called from the wrong sub.
  • VMOD http: Fix bug where headers was not copied in the correct way.
  • VMOD urlplus: Add function urlplus.get_dirname()
  • VMOD kvstore: Do not copy VSC counters when cloning
  • Prevent VSC counters from being used in a request context
  • Fix vcl.show for labels. Where we would previously crash, we now simply print the vcl that the label is pointing at.
  • VHA 6: Correctly handle the X-Broadcast-Group header
  • MSE: Fix a bug where transient hit-for-pass objects stored in MSE would crash Varnish. (VS issue 561)
  • VMOD urlplus: Add null check in the .query_add() function.
  • Eliminate crash in extremely rare conditions during lookup, so far only observed during ykey purge.

Varnish Cache Plus 6.0.3r7 (2019-07-23)

  • Improve persisted MSE startup performance. At restarts of the cache, data structures for the persisted objects needs to be set up. This can be CPU intensive, and with this rewrite of the subsystem all of the available CPU resources will be utilized to get it done quickly.
  • Ensure consistent view of gzipdata in ESI's stripgzip code.
  • Remove 64 limit for VHA names.
  • New VMOD mmdb - Varnish Enterprise GeoIP2. Supersedes the now deprecated VMOD geoip.
  • New VMOD utils, for misc functions.
  • Handle Home and End keys in varnishstat.
  • Permit subsequent conditional requests on 304 (2871).
  • Use VSBs to generate code for VMOD objects (2880).
  • STRANDS improvements (2733, 2873)
  • Wake up vbp_thread after a probe is done (2976).
  • Fix an issue in goto with regard to responses containing both CNAME and A/AAAA records. (VS issue 559)
  • Fix an issue with ykey purges when the purge operation caused mutex contention in the main hash critbit tree. (VS issue 558)

Varnish Cache Plus 6.0.3r6 (2019-06-13)

  • Tolerate a null address string in std.ip() (2993)
  • Correctly account the cost of aux attributes in persisted MSE. (VS issue 394)
  • Add Varnish High Availability 6.
  • Rename kvstore key count to __keys

Varnish Cache Plus 6.0.3r5 (2019-05-22)

  • Add param.reset command to varnishadm
  • The xbody VMOD now has a hash_body function.
  • VMOD xbody fixes: Reset priv data when retrying and better workspace failure handling.
  • LDJSON (Line Delimited JSON) support for varnishlog added.
  • Akamai connector: Add support for purge_cachetag, purge_cpcode, api and delete.
  • Fix a rare crash when gunzip streaming to a client.
  • The accept VMOD now does case insensitive matching.
  • Update ban code improvements to match Varnish Cache
  • VMODs digest and vsthrottle updated to latest upstream versions.
  • Waiting list improvements (Varnish Cache issues 2977 and 2978)
  • Now std.ip() can optionally take a port number or service name argument. This is documented in the vmod_std(3) manual. (See Varnish Cache pull request 2993)

Varnish Cache Plus 6.0.3r4 (2019-04-10)

  • Fix a problem where a Varnish Fetch Processor was used with MSE (VS issue 471).
  • Fix a bug where the goto VMOD would crash when called from the wrong VCL sub routine (VS issue 459).
  • Fix a bug in the rtstatus.html file
  • Add option allow_exec to control if the file VMOD's exec function can be called.
  • Add regsub to cookieplus
  • VMOD goto now filters out more unwanted DNS record types.
  • ACL support added in VMOD goto.

Varnish Cache Plus 6.0.3r3 (2019-03-18)

  • Optimize the ban lurker's handling of dying objects.
  • For some ESI and Edgestash requests, the request structure and was not properly cleaned up. This has been fixed (VS issues 444 and 456).
  • Fix a rare crash when MSE is used with ESI/ES (VS issue 454).
  • Fix an incorrect panic when using a backreference in xbody.regsub().
  • Fix double accounting and logging of hitmiss.

Varnish Cache Plus 6.0.3r2 (2019-03-05)

  • MSE3: Upgrade the embedded LMDB database to version 0.9.23
  • MSE3: Reduce the time span of open read transactions that would otherwise prevent database page reuse (VS issue 415)
  • MSE3: Fix the reclaim logic for open database transactions after crashes (VS issue 439)
  • MSE3: Add some debugging aid CLI functions to get runtime information.
  • MSE3: Fix pruning of persisted bans when the bans were marked complete early. (VS issue 443)
  • Fix an SELinux issue related to missing port rights.
  • Open the SELinux policy for Unix Domain Sockets
  • Change the default varnish.service file to listen to "localhost" instead of "127.0.0.1" for IPv6 only hosts.
  • Handle NULL arguments gracefully in vmod-aclplus. (VS issue 365)
  • Prevent some potential buffer overflows in string handling in vmod-goto and vmod-crypto.
  • Fix some edge cases for trailing slash handling in vmod-urlplus (VS issue 445)

Varnish Cache Plus 6.0.3r1 (2019-02-21)

  • Included vtree.h in the distribution for vmods and renamed the red/black tree macros from VRB_* to VRBT_* to disambiguate from the acronym for Varnish Request Body.

  • Added req.is_hitmiss and req.is_hitpass (2743)

  • Fix assigning <bool> == <bool> (2809)

  • Add error handling for STV_NewObject() (2831)

  • Fix VRT_fail for 'if'/'elseif' conditional expressions (2840)

  • Add VSL rate limiting (2837)

    This adds rate limiting to varnishncsa and varnishlog.

  • For varnishtest -L, also keep VCL C source files.

  • Make it possible to change varnishncsa update rate. (2741)

  • Tolerate null IP addresses for ACL matches.

  • Many cache lookup optimizations.

  • Display the VCL syntax during a panic.

  • Update to the VCL diagrams to include hit-for-miss.

  • Fix a gzip data race

  • Akamai-connector can assume session timeouts

  • Kvstore locks

  • vmod-rewrite .add_rules() method can take a type (including "any")

  • New rule type "glob" in vmod-rewrite

  • Fix the startup waterlevel purge code in MSE3

  • Add dynamic VSC counters to KVStore

  • Add vmod-http body functions

  • vmod-http can copy req headers from CSV

  • Add function to create URL from a backend/director in vmod-http

Varnish Cache Plus 6.0.2r1 (2019-01-28)

New features

  • Ykey - secondary hashes like Xkey, but with better scalability, more flexibility and persisted MSE support. See the vmod_ykey man page for more information.

Bugs fixed

  • Fix a memory leak when using BackendSSL with hostname validation, where we would leak the X509 data structure of the peer's certificate (VS issue 390).
  • Fix a memory panic issue with Edgestash partials where JSON context is inherited from the parent request.
  • Honor the ip_version parameter in all cases in vmod-goto.
  • Fix and test objhead refcount for hit-for-pass (2654, 2754, 2760)
  • Allow a string argument to return(fail("Because!")); (2694)
  • Improve VCC error messages (2696)
  • Fix obj.hits in vcl_hit (2746)
  • Improvements to how PRIV_TASK and PRIV_TOP are initialized (2708, 2749)
  • fixed varnishhist display error (2780)
  • In Error: out of workspace log entries, the workspace name is now reported in lowercase
  • Adjust code generator python tools to python 3 and prefer python 3 over python 2 where available
  • Clear the IMS object attribute when copying from a stale object (2763)
  • Implement and test ECMA-48 "REP" sequence to fix test case u00008.vtc on some newer platforms. (2668)
  • Don't mess with C-L when responding to HEAD (2744)
  • Align handling of STRINGS derived types (2745)
  • Fix some stats metrics (VSC) which were wrongly marked as _gauge_
  • Varnishhist: Ignore non-positive values when accumulating (2773)
  • Fix production of VTC documentation (2777)
  • Fix varnishd -I (2782)
  • Fix varnishstat -f in curses mode (interactively, without -1, 2787)
  • Changed the default of the thread_pool_watchdog parameter to 60 seconds to match the cli_timeout default
  • Fix warmup/rampup of the shard director (2823)
  • Fix VRT_priv_task for calls from vcl_pipe {} (2820)
  • Fix vmod object constructor documentation in the vmodtool.py - generated RST files
  • Vmod developers are advised that anything returned by a vmod function/method is assumed to be immutable. In other words, a vmod must not modify any data which was previously returned.
  • Content-Length header is not rewritten in response to a HEAD request, allows responses to HEAD requests to be cached independently from GET responses.
  • return(fail("mumble")) can have a string argument that is emitted by VCC as an error message if the VCL load fails due to the return. (2694)
  • Handle an out-of-workspace condition in HTTP/2 delivery more gracefully (2589)
  • Added a thread pool watchdog which will restart the worker process if scheduling tasks onto worker threads appears stuck. The new parameter thread_pool_watchdog configures it. (2418, 2794)
  • Clarify and test object slimming for hfp+hfm. (2768)
  • Allow PRIORITY frames on closed streams (2775)
  • Hardening of the h2_frame_f callbacks (2781)
  • Added a JSON section to varnish-cli(7) (2783)
  • Improved varnish log client performance (2788)
  • Change nanosecond precision timestamps into microseconds (2792)
  • Only dlclose() Vmods after all "fini" processing (2800)
  • Fix VRT_priv_task for calls from vcl_pipe {} and test for it (2820)
  • Shard director: For warmup/rampup, only consider healthy backends (2823)

Varnish Cache Plus 6.0.1r5 (2018-12-18)

New features

  • XBody array support and expanded backreference support.
  • Reduce locking in some critical areas.

Bugs fixed

  • Fix kvstore issue where only a single request scope kvstore is visible.
  • Fix a buffer overflow in MSE when having a large number of stores on a single book. (VS issue 380)
  • Fix bug in VMOD urlplus where leading slash is ignored for 1 char long URLs
  • Stop the ban lurker from evaluating dying objects. (VS issue 383)
  • Fix obj.hits in vcl_hit. (2746)

Varnish Cache Plus 6.0.1r4 (2018-12-05)

Bugs fixed

  • Varnish now won't rewrite the content-length header when responding to any HEAD request, making it possible to cache HEAD requests independently from the GET ones (previously a HEAD request had to be a pass to avoid this rewriting) (2705)
  • Align handling of STRINGS derived types. This removes some compile time errors that were unnecessary. (2745)
  • Fix a race condition with regard rescheduling of requests on waitinglists, causing a workspace release assertion. (VS issue 372)
  • Fix a bug related to streaming delivery of chunked responses when using the malloc or file stevedores. (VS issue 375)

Varnish Cache Plus 6.0.1r3 (2018-11-20)

New features

  • VMOD_cookieplus: various fixes

  • New VMOD; synthbackend

  • Massive Storage Engine for Varnish Cache Plus 6.0

    This is the first release of Varnish Cache Plus 6.0 that includes the Massive Storage Engine (MSE). See the varnish-mse manpage for configuration and usage details.

    Please note the following:

    • This is the next generation of MSE, and the configuration and usage has changed compared to the previous version in Varnish Cache Plus 4.1.
    • The use of X-key in a persisted MSE configuration is not recommended because the X-key data structures are not persisted and needs to be rebuilt on each restart. A new implementation of X-key with persisted data structures is in the making, and will be included in a future release.
    • If you have been using a tech preview release of Varnish Cache Plus 6.0 with MSE, you will, due to a change in the on disk data format, have to reset the storage files using the mkfs.mse utility and its -f flag before starting the Varnish daemon.

Varnish Cache Plus 6.0.1r2 (2018-10-01)

Fixed bugs

  • In VMOD urlplus it is now possible to handle query parameters where a key appears multiple times, and even to clean up repeated equal key/value pairs.
  • Fixed a backend response issue
  • Improvements to backend SSL to allow for better performance when using OpenSSL prior to version 1.1.

Varnish Cache Plus 6.0.1r1 (2018-09-19)

New features

  • vmod_rewrite: New method - ruleset.add_rules()
  • vmod_rewrite: Matcher types - different ways to match in .replace() and friends
  • vmod_xbody: Play better with ESI
  • vmod_xbody: Increased performance
  • vmod_xbody: JSON capture output in vcl_synth
  • vmod_xbody: Edgestash can now directly use Xbody JSON:
  • Added std.fnmatch() (2737)
  • The variable req.grace is back. (2705)
  • Importing the same VMOD multiple times is now allowed, if the file_id is identical.

varnishstat

  • The counters

    • sess_fail_econnaborted
    • sess_fail_eintr
    • sess_fail_emfile
    • sess_fail_ebadf
    • sess_fail_enomem
    • sess_fail_other

    now break down the detailed reason for session accept failures, the sum of which continues to be counted in sess_fail.

VCL and bundled VMODs

  • VMOD unix now supports the getpeerucred(3) case.

bundled tools

  • varnishhist: The format of the -P argument has been changed for custom profile definitions to also contain a prefix to match the tag against.
  • varnishtest: syslog instances now have to start with a capital S.

Fixed bugs which may influence VCL behavior

  • When an object is out of grace but in keep, the client context goes straight to sub vcl_miss instead of sub vcl_hit. The documentation has been updated accordingly. (2705)

Fixed bugs

  • Several H2 bugs (2285, 2572, 2623, 2624, 2679, 2690, 2693)
  • Make large integers work in VCL. (2603)
  • Print usage on unknown or missing arguments (2608)
  • Assert error in VPX_Send_Proxy() with proxy backends in pipe mode (2613)
  • Holddown times for certain backend connection errors (2622)
  • Enforce Host requirement for HTTP/1.1 requests (2631)
  • Introduction of '-' CLI prefix allowed empty commands to sneak through. (2647)
  • VUT apps can be stopped cleanly via vtc process -stop (2649, 2650)
  • VUT apps fail gracefully when removing a PID file fails
  • varnishd startup log should mention version (2661)
  • In curses mode, always filter in the counters necessary for the header lines. (2678)
  • Assert error in ban_lurker_getfirst() (2681)
  • Missing command entries in varnishadm help menu (2682)
  • Handle string literal concatenation correctly (2685)
  • varnishtop -1 does not work as documented (2686)
  • Handle sigbus like sigsegv (2693)
  • Panic on return (retry) of a conditional fetch (2700)
  • Wrong turn at cache/cache_backend_probe.c:255: Unknown family (2702, 2726)
  • VCL failure causes TASK_PRIV reference on reset workspace (2706)
  • Accurate ban statistics except for a few remaining corner cases (2716)
  • Assert error in vca_make_session() (2719)
  • Assert error in vca_tcp_opt_set() (2722)
  • VCL compiling error on parenthesis (2727)
  • Assert error in HTC_RxPipeline() (2731)
  • vmod_http; headers are now case insensitive

Varnish Cache Plus 6.0.0r2 (unreleased)

Bugs fixed

Varnish Cache Plus 6.0.0r1 (unreleased)

This version is up-to-date with Varnish Cache 6.0.0

New features

  • New VMOD; urlplus

Bugs fixed

  • Several fixes that improve the stability and resilience of Varnish when H2 (HTTP version 2) is enabled.

Varnish Cache Plus 6.0.0r0 (2018-03-08)

Varnish Cache Plus is an enhanced version of Varnish Cache. The 6.0 series was forked from Varnish Cache 6.0.0, and then features were ported from Varnish Cache Plus 4.1.

There are also several new features that are only available in 6.0.

All Plus only features are described on our docs web site.

Note that this version (6.0.0r0) was released in Limited Availability. It does not contain all the changes in Varnish Cache 6.0.0, but there are no major from the user's perspective. The next version will be up to date with Varnish Cache 6.0.0 and also contain additional fixes and improvements.

Features ported from Varnish Cache Plus 4.1

Parallel ESI

Varnish Cache Plus will fetch ESI (Edge Side Includes) in parallel. For pages with many ESI includes this can speed up page loading greatly.

Edgestash

All the features of Edgestash is available in Varnish Cache Plus 6.0.

Backend SSL/TLS

Varnish Cache Plus supports backend SSL/TLS through the OpenSSL library. This is enabled in the same way as in previous versions.

Varnish Cache VMODs

Many Plus only VMODs have been brought to 6.0:

  • VMOD aclplus
  • VMOD cookieplus
  • VMOD directors
  • VMOD edgestash
  • VMOD goto
  • VMOD http
  • VMOD kvstore
  • VMOD leastconn
  • VMOD paywall (not available in 4.1)
  • VMOD rewrite
  • VMOD rtstatus
  • VMOD session
  • VMOD vha (Part of Varnish High Availability)

Included: Open source Varnish Modules

Also bundled are the following VMODs, collectively known as Varnish Modules:

  • VMOD bodyaccess
  • VMOD cookie
  • VMOD header
  • VMOD saintmode
  • VMOD tcp
  • VMOD var
  • VMOD vsthrottle
  • VMOD xkey

New features in Varnish Cache Plus 6.0

The following features are exclusive to Varnish Cache Plus 6.0

Varnish Total Encryption

This new feature is the last piece in getting end-to-end encryption in Varnish Cache Plus, and, as far as we know, any HTTP cache. Enabling Varnish Total Encryption will make even in-memory data encrypted, and this will protect you against "data leak" bugs like Meltdown and Spectre.

Additional VMODs

  • VMOD crypto (Part of Varnish Total Encryption)

Updates from the Varnish Cache project

The changelog below is identical with the Varnish Cache project. The list is not exhaustive, but should contain all major changes from the user's point of view.

Varnish Cache 6.0.0 (2018-03-15)

Usage

  • Fixed implementation of the max_restarts limit: It used to be one less than the number of allowed restarts, it now is the number of return(restart) calls per request.

  • The cli_buffer parameter has been removed

  • Added back umem storage for Solaris descendants

  • The new storage backend type (stevedore) default now resolves to either umem (where available) or malloc.

  • Since varnish 4.1, the thread workspace as configured by workspace_thread was not used as documented, delivery also used the client workspace.

    We are now taking delivery IO vectors from the thread workspace, so the parameter documentation is in sync with reality again.

    Users who need to minimize memory footprint might consider decreasing workspace_client by workspace_thread.

  • The new parameter esi_iovs configures the amount of IO vectors used during ESI delivery. It should not be tuned unless advised by a developer.

  • Support Unix domain sockets for the -a and -b command-line arguments, and for backend declarations. This requires VCL >= 4.1.

VCL and bundled VMODs

  • return (fetch) is no longer allowed in vcl_hit {}, use return (miss) instead. Note that return (fetch) has been deprecated since 4.0.

  • Fix behaviour of restarts to how it was originally intended: Restarts now leave all the request properties in place except for req.restarts and req.xid, which need to change by design.

  • req.storage, req.hash_ignore_busy and req.hash_always_miss are now accessible from all of the client side subs, not just vcl_recv{}

  • obj.storage is now available in vcl_hit{} and vcl_deliver{}.

  • Removed beresp.storage_hint for VCL 4.1 (was deprecated since Varnish 5.1)

    For VCL 4.0, compatibility is preserved, but the implementation is changed slightly: beresp.storage_hint is now referring to the same internal data structure as beresp.storage.

    In particular, it was previously possible to set beresp.storage_hint to an invalid storage name and later retrieve it back. Doing so will now yield the last successfully set stevedore or the undefined (NULL) string.

  • IP-valued elements of VCL are equivalent to 0.0.0.0:0 when the connection in question was addressed as a UDS. This is implemented with the bogo_ip in vsa.c.

  • beresp.backend.ip is retired as of VCL 4.1.

  • workspace overflows in std.log() now trigger a VCL failure.

  • workspace overflows in std.syslog() are ignored.

  • added return(restart) from vcl_recv{}.

  • The alg argument of the shard director .reconfigure() method has been removed - the consistent hashing ring is now always generated using the last 32 bits of a SHA256 hash of "ident%d" as with alg=SHA256 or the default.

    We believe that the other algorithms did not yield sufficiently dispersed placement of backends on the consistent hashing ring and thus retire this option without replacement.

    Users of .reconfigure(alg=CRC32) or .reconfigure(alg=RS) be advised that when upgrading and removing the alg argument, consistent hashing values for all backends will change once and only once.

  • The alg argument of the shard director .key() method has been removed - it now always hashes its arguments using SHA256 and returns the last 32 bits for use as a shard key.

    Backwards compatibility is provided through vmod blobdigest with the key_blob argument of the shard director .backend() method:

    • for alg=CRC32, replace:

      <dir>.backend(by=KEY, key=<dir>.key(<string>, CRC32))
      

      with:

      <dir>.backend(by=BLOB, key_blob=blobdigest.hash(ICRC32,
        blob.decode(encoded=<string>)))
      

      Note: The vmod blobdigest hash method corresponding to the shard director CRC32 method is called ICRC32

  • for alg=RS, replace:

    <dir>.backend(by=KEY, key=<dir>.key(<string>, RS))
    

    with:

    <dir>.backend(by=BLOB, key_blob=blobdigest.hash(RS,
      blob.decode(encoded=<string>)))
    
  • The shard director now offers resolution at the time the actual backend connection is made, which is how all other bundled directors work as well: With the resolve=LAZY argument, other shard parameters are saved for later reference and a director object is returned.

    This enables layering the shard director below other directors.

  • The shard director now also supports getting other parameters from a parameter set object: Rather than passing the required parameters with each .backend() call, an object can be associated with a shard director defining the parameters. The association can be changed in vcl_backend_fetch() and individual parameters can be overridden in each .backend() call.

    The main use case is to segregate shard parameters from director selection: By associating a parameter object with many directors, the same load balancing decision can easily be applied independent of which set of backends is to be used.

  • To support parameter overriding, support for positional arguments of the shard director .backend() method had to be removed. In other words, all parameters to the shard director .backend() method now need to be named.

  • Integers in VCL are now 64 bits wide across all platforms (implemented as int64_t C type), but due to implementation specifics of the VCL compiler (VCC), integer literals' precision is limited to that of a VCL real (double C type, roughly 53 bits).

    In effect, larger integers are not represented accurately (they get rounded) and may even have their sign changed or trigger a C compiler warning / error.

  • Add VMOD unix.

  • Add VMOD proxy.

Logging / statistics

  • Turned off PROXY protocol debugging by default, can be enabled with the protocol debug flag.
  • added cache_hit_grace statistics counter.
  • added n_lru_limited counter.
  • The byte counters in ReqAcct now show the numbers reported from the operating system rather than what we anticipated to send. This will give more accurate numbers when e.g. the client hung up early without receiving the entire response. Also these counters now show how many bytes was attributed to the body, including any protocol overhead (ie chunked encoding).

bundled tools

  • varnishncsa refuses output formats (as defined with the -F command line argument) for tags which could contain control or binary characters. At the time of writing, these are: %{H2RxHdr}x, %{H2RxBody}x, %{H2TxHdr}x, %{H2TxBody}x, %{Debug}x, %{HttpGarbage}x and %{Hash}x
  • The vtc server -listen command supports UDS addresses, as does the client -connect command. vtc remote.path and remote.port have the values 0.0.0.0 and 0 when the peer address is UDS. Added remote.path to vtc, whose value is the path when the address is UDS, and NULL (matching <undef>) for IP addresses.

C APIs (for vmod and utility authors)

  • We have now defined three API Stability levels: VRT, PACKAGE, SOURCE.
  • New API namespace rules, see phk_api_spaces_
  • Rules for including API headers have been changed: * many headers can now only be included once * some headers require specific include ordering * only cache.h _or_ vrt.h can be included
  • Signatures of functions in the VLU API for bytestream into text serialization have been changed
  • vcl.h now contains convenience macros VCL_MET_TASK_B, VCL_MET_TASK_C and VCL_MET_TASK_H for checking ctx->method for backend, client and housekeeping (vcl_init/vcl_fini) task context
  • vcc files can now contain a $Prefix stanza to define the prefix for vmod function names (which was fixed to vmod before)
  • vcc files can contain a $Synopsis stanza with one of the values auto or manual, default auto. With auto, a more comprehensive SYNOPSIS is generated in the doc output with an overview of objects, methods, functions and their signatures. With manual, the auto-SYNOPSIS is left out, for VMOD authors who prefer to write their own.
  • All Varnish internal SHA256* symbols have been renamed to VSHA256*
  • libvarnish now has VNUM_duration() to convert from a VCL duration like 4h or 5s
  • director health state queries have been merged to VRT_Healthy()
  • Renamed macros: * __match_proto__() -> v_matchproto_() * __v_printflike() -> v_printflike_() * __state_variable__() -> v_statevariable_() * __unused -> v_unused_ * __attribute__((__noreturn__) -> v_noreturn_
  • ENUMs are now fixed pointers per vcl.
  • Added VRT_blob() utility function to create a blob as a copy of some chunk of data on the workspace.
  • Directors now have their own admin health information and always need to have the (struct director).admin_health initialized to VDI_AH_* (usually VDI_AH_HEALTHY).

Other changes relevant for VMODs

  • PRIV_* function/method arguments are not excluded from auto-generated vmod documentation.

Fixed bugs which may influence VCL behaviour

  • After reusing a backend connection fails once, a fresh connection will be opened (2135).

Fixed bugs

  • Honor first_byte_timeout for recycled backend connections. (1772)
  • Limit backend connection retries to a single retry (2135)
  • H2: Move the req-specific PRIV pointers to struct req. (2268)
  • H2: Don't panic if we reembark with a request body (2305)
  • Clear the objcore attributes flags when (re)initializing an stv object. (2319)
  • H2: Fail streams with missing :method or :path. (2351)
  • H2: Enforce sequence requirement of header block frames. (2387)
  • H2: Hold the sess mutex when evaluating r2->cond. (2434)
  • Use the idle read timeout only on empty requests. (2492)
  • OH leak in http1_reembark. (2495)
  • Fix objcore reference count leak. (2502)
  • Close a race between backend probe and vcl.state=Cold by removing the be->vsc under backend mtx. (2505)
  • Fail gracefully if shard.backend() is called in housekeeping subs (2506)
  • Fix issue #1799 for keep. (2519)
  • oc->last_lru as float gives too little precision. (2527)
  • H2: Don't HTC_RxStuff with a non-reserved workspace. (2539)
  • Various optimizations of VSM. (2430, 2470, 2518, 2535, 2541, 2545, 2546)
  • Problems during late socket initialization performed by the Varnish child process can now be reported back to the management process with an error message. (2551)
  • Fail if ESI is attempted on partial (206) objects.
  • Assert error in ban_mark_completed() - ban lurker edge case. (2556)
  • Accurate byte counters (2558). See Logging / statistics above.
  • H2: Fix reembark failure handling. (2563 and 2592)
  • Working directory permissions insufficient when starting with umask 027. (2570)
  • Always use HTTP/1.1 on backend connections for pass & fetch. (2574)
  • EPIPE is a documented errno in tcp(7) on linux. (2582)
  • H2: Handle failed write(2) in h2_ou_session. (2607)

Varnish Cache 5.2.1 (2017-11-14)

Bugs fixed

  • 2429 - Avoid buffer read overflow on vcl_backend_error and -sfile
  • 2492 - Use the idle read timeout only on empty requests.

Varnish Cache 5.2.0 (2017-09-15)

  • The cli_buffer parameter has been deprecated (2382)

Varnish Cache 5.2-RC1 (2017-09-04)

Usage

  • The default for the the -i argument is now the hostname as returned by gethostname(3)
  • Where possible (on platforms with setproctitle(3)), the -i argument rather than the -n argument is used for process names
  • varnishd -f honors vcl_path (#2342)
  • The MAIN.s_req statistic has been removed, as it was identical to MAIN.client_req. VSM consumers should be changed to use the latter if necessary.
  • A listen address can take a name in the -a argument. This name is used in the logs and later will possibly be available in VCL.

VCL

  • VRT_purge fails a transaction if used outside of vcl_hit and vcl_miss (#2339)
  • Added bereq.is_bgfetch which is true for background fetches.
  • Added VMOD purge (#2404)
  • Added VMOD blob (#2407)

C APIs (for vmod and utility authors)

  • The VSM API for accessing the shared memory segment has been totally rewritten. Things should be simpler and more general.
  • VSC shared memory layout has changed and the VSC API updated to match it. This paves the way for user defined VSC counters in VMODS and later possibly also in VCL.
  • New vmod vtc for advanced varnishtest usage (#2276)

Varnish Cache 5.1.3 (2017-08-02)

Security Update

Bugs fixed

  • 2379 - Correctly handle bogusly large chunk sizes (VSV00001)

Varnish Cache 5.1.2 (2017-04-07)

  • Fix an endless loop in Backend Polling (#2295)
  • Fix a Chunked bug in tight workspaces (#2207, #2275)
  • Fix a bug relating to req.body when on waitinglist (#2266)
  • Handle EPIPE on broken TCP connections (#2267)
  • Work around the x86 arch's turbo-double FP format in parameter setup code. (#1875)
  • Fix race related to backend probe with proxy header (#2278)
  • Keep VCL temperature consistent between mgt/worker also when worker protests.
  • A lot of HTTP/2 fixes.

Varnish Cache 5.1.1 (2017-03-16)

  • Fix bug introduced by stubborn old bugger right before release 5.1.0 was cut.

Varnish Cache 5.1.0 (2017-03-15)

  • Added varnishd command-line options -I, -x and -?, and tightened restrictions on permitted combinations of options.
  • More progress on support for HTTP/2.
  • Add return(fail) to almost all VCL subroutines.
  • Restored the old hit-for-pass, invoked with return(pass(DURATION)) from vcl_backend_response. hit-for-miss remains the default. Added the cache_hitmiss stat, and cache_hitpass only counts the new/old hit-for-pass cases. Restored HitPass to the Varnish log, and added HitMiss. Added the HFP prefix to TTL log entries to log a hit-for-pass duration.
  • Rolled back the fix for #1206. Client delivery decides solely whether to send a 304 client response, based on client request and response headers.
  • Added vtest.sh.
  • Added vxid as a lefthand side for VSL queries.
  • Added the setenv and write_body commands for Varnish test cases (VTCs). err_shell is deprecated. Also added the operators -cliexpect, -match and -hdrlen, and -reason replaces -msg. Added the ${bad_backend} macro.
  • varnishtest can be stopped with the TERM, INT and KILL signals, but not with HUP.
  • The fallback director has now an extra, optional parameter to keep using the current backend until it falls sick.
  • VMOD shared libraries are now copied to the workdir, to avoid problems when VMODs are updated via packaging systems.
  • Bump the VRT version to 6.0.
  • Export more symbols from libvarnishapi.so.
  • The size of the VSL log is limited to 4G-1b, placing upper bounds on the -l option and the vsl_space and vsm_space parameters.
  • Added parameters clock_step, thread_pool_reserve and ban_cutoff.
  • Parameters vcl_dir and vmod_dir are deprecated, use vcl_path and vmod_path instead.
  • All parameters are defined, even on platforms that don't support them. An unsupported parameter is documented as such in param.show. Setting such a parameter is not an error, but has no effect.
  • Clarified the interpretations of the + and - operators in VCL with operands of the various data types.
  • DURATION types may be used in boolean contexts.
  • INT, DURATION and REAL values can now be negative.
  • Response codes 1000 or greater may now be set in VCL internally. resp.status is delivered modulo 1000 in client responses.
  • IP addresses can be compared for equality in VCL.
  • Introduce the STEVEDORE data type, and the objects storage.SNAME in VCL. Added req.storage and beresp.storage; beresp.storage_hint is deprecated.
  • Retired the umem stevedore.
  • req.ttl is deprecated.
  • Added std.getenv() and std.late_100_continue().
  • The fetch_failed stat is incremented for any kind of fetch failure.
  • Added the stats n_test_gunzip and bans_lurker_obj_killed_cutoff.
  • Clarified the meanings of the %r, %{X}i and %{X}o formatters in varnishncsa.

Bugs fixed

  • 2251 - varnishapi.pc and varnishconfdir
  • 2250 - vrt.h now depends on vdef.h making current vmod fail.
  • 2249 - "logexpect -wait" doesn't fail
  • 2245 - Varnish doesn't start, if use vmod (vmod_cache dir was permission denied)
  • 2241 - VSL fails to get hold of SHM
  • 2233 - Crash on "Assert error in WS_Assert(), cache/cache_ws.c line 59"
  • 2227 - -C flag broken in HEAD
  • 2217 - fix argument processing -C regression
  • 2207 - Assert error in V1L_Write()
  • 2205 - Strange bug when I set client.ip with another string
  • 2203 - unhandled SIGPIPE
  • 2200 - Assert error in vev_compact_pfd(), vev.c line 394
  • 2197 - ESI parser panic on malformed src URL
  • 2190 - varnishncsa: The %r formatter is NOT equivalent to "%m http://%{Host}i%U%q %H"
  • 2186 - Assert error in sml_iterator(), storage/storage_simple.c line 263
  • 2184 - Cannot subtract a negative number
  • 2177 - Clarify interactions between restarts and labels
  • 2175 - Backend leak between a top VCL and a label
  • 2174 - Cflags overhaul
  • 2167 - VCC will not parse a literal negative number where INT is expected
  • 2155 - vmodtool removes text following $Event from RST docs
  • 2151 - Health probes do not honor a backend's PROXY protocol setting
  • 2142 - ip comparison fails
  • 2148 - varnishncsa cannot decode Authorization header if the format is incorrect.
  • 2143 - Assert error in exp_inbox(), cache/cache_expire.c line 195
  • 2134 - Disable Nagle's
  • 2129 - stack overflow with >4 level esi
  • 2128 - SIGSEGV NULL Pointer in STV__iter()
  • 2118 - "varnishstat -f MAIN.sess_conn -1" produces empty output
  • 2117 - SES_Close() EBADF / Wait_Enter() wp->fd <= 0
  • 2115 - VSM temporary files are not always deleted
  • 2110 - [CLI] vcl.inline failures
  • 2104 - Assert error in VFP_Open(), cache/cache_fetch_proc.c line 139: Condition((vc->wrk->vsl) != 0) not true
  • 2099 - VCC BACKEND/HDR comparison produces duplicate gethdr_s definition
  • 2096 - H2 t2002 fail on arm64/arm32
  • 2094 - H2 t2000 fail on arm64/arm32
  • 2078 - VCL comparison doesn't fold STRING_LIST
  • 2052 - d12.vtc flaky when compiling with suncc
  • 2042 - Send a 304 response for a just-gone-stale hitpass object when appropriate
  • 2041 - Parent process should exit if it fails to start child
  • 2035 - varnishd stalls with two consecutive Range requests using HTTP persistent connections
  • 2026 - Add restart of poll in read_tmo
  • 2021 - vcc "used before defined" check
  • 2017 - "%r" field is wrong
  • 2016 - confusing vcc error when acl referenced before definition
  • 2014 - req.ttl: retire or document+vtc
  • 2010 - varnishadm CLI behaving weirdly
  • 1991 - Starting varnish on Linux with boot param ipv6.disable=1 fails
  • 1988 - Lost req.url gives misleading error
  • 1914 - set a custom storage for cache_req_body
  • 1899 - varnishadm vcl.inline is overly obscure
  • 1874 - clock-step related crash
  • 1865 - Panic accessing beresp.backend.ip in vcl_backend_error{}
  • 1856 - LostHeader setting req.url to an empty string
  • 1834 - WS_Assert(), cache/cache_ws.c line 59
  • 1830 - VSL API: "duplicate link" errors in request grouping when vsl_buffer is increased
  • 1764 - nuke_limit is not honored
  • 1750 - Fail more gracefully on -l >= 4GB
  • 1704 - fetch_failed not incremented

Varnish Cache 5.0.0 (2016-09-15)

  • Documentation updates, especially the what's new and upgrade sections.
  • Via: header made by Varnish now says 5.0.
  • VMOD VRT ABI level increased.
  • [vcl] obj.(ttl|age|grace|keep) is now readable in vcl_deliver.
  • Latest devicedetect.vcl imported from upstream.
  • New system wide VCL directory: /usr/share/varnish/vcl/
  • std.integer() can now convert from REAL.

Bugs fixed

  • 2086 - Ignore H2 upgrades if the feature is not enabled.
  • 2054 - Introduce new macros for out-of-tree VMODs
  • 2022 - varnishstat -1 -f field inclusion glob doesn't allow VBE backend fields
  • 2008 - Panic: Assert error in VBE_Delete()
  • 1800 - PRIV_TASK in vcl_init/fini

Varnish Cache 5.0.0-beta1 (2016-09-09)

This is the first beta release of the upcoming 5.0 release.

The list of changes are numerous and will not be expanded on in detail.

The release notes contain more background information and are highly recommended reading before using any of the new features.

Major items:

  • VCL labels, allowing for per-vhost (or per-anything) separate VCL files.
  • (Very!) experimental support for HTTP/2.
  • Always send the request body to the backend, making possible to cache responses of POST, PATCH requests etc with appropriate custom VCL and/or VMODs.
  • hit-for-pass is now actually hit-for-miss.
  • new shard director for loadbalancing by consistent hashing
  • ban lurker performance improvements
  • access to obj.ttl, obj.age, obj.grace and obj.keep in vcl_deliver

News for Vmod Authors

  • workspace and PRIV_TASK for vcl cli events (init/fini methods)
  • PRIV_* now also work for object methods with unchanged scope.

Varnish Cache 4.1.10 (2018-04-25)

Changes since 4.1.9:

  • New counter added: cache_hit_grace (see 2455)
  • Doc: Specify that time_firstbyte is in seconds (see 2456)
  • Fix issue #1799 for keep (2519)
  • New way of calculating TTLs in VCL (2555)
  • Added n_lru_limited counter (see 2569)

Bugs fixed

  • 1772 - Honor first_byte_timeout for recycled backend connections
  • 2135 - Limit Backend Connection retries to a single retry
  • 2450 - Pass delivery abandoned does not qualify as an error
  • 2495 - Avoid leaking an OH ref on reembark failure
  • 2502 - objcore reference count leak
  • 2530 - Varnish shouldn't test gunzip for range responses
  • 2582 - Assert error in http1_minimal_response()
  • 2609 - Fix memory leak of vary string on stevedore alloc fail

Varnish Cache 4.1.9 (2017-11-14)

Changes since 4.1.8:

  • Added bereq.is_bgfetch which is true for background fetches.
  • Add the vtc feature ignore_unknown_macro.
  • Expose to VCL whether or not a fetch is a background fetch (bgfetch)
  • Ignore req.ttl when keeping track of expired objects (see 2422)
  • Move a cli buffer to VSB (from stack).
  • Use a separate stack for signals.

Bugs fixed

  • 2337 and 2366 - Both Upgrade and Connection headers are needed for WebSocket now
  • 2372 - Fix problem with purging and the n_obj_purged counter
  • 2373 - VSC n_vcl, n_vcl_avail, n_vcl_discard are gauge
  • 2380 - Correct regexp in examples.
  • 2390 - Straighten locking wrt vcl_active
  • 2429 - Avoid buffer read overflow on vcl_backend_error and -sfile
  • 2492 - Use the idle read timeout only on empty requests

Varnish Cache 4.1.8 (2017-08-02)

Security Update

Changes since 4.1.7:

  • Update in the documentation of timestamps

Bugs fixed

  • 2379 - Correctly handle bogusly large chunk sizes (VSV00001)

Varnish Cache 4.1.7 (2017-06-28)

Changes since 4.1.7-beta1:

  • Add extra locking to protect the pools list and refcounts
  • Don't panic on a null ban

Bugs fixed

  • 2321 - Prevent storage backends name collisions

Varnish Cache 4.1.7-beta1 (2017-06-15)

Changes since 4.1.6:

  • Add -vsl_catchup to varnishtest
  • Add record-prefix support to varnishncsa

Bugs fixed

  • 1764 - Correctly honor nuke_limit parameter
  • 2022 - varnishstat -1 -f field inclusion glob doesn't allow VBE backend fields
  • 2069 - Health probes fail when HTTP response does not contain reason phrase
  • 2118 - "varnishstat -f MAIN.sess_conn -1" produces empty output
  • 2219 - Remember to reset workspace
  • 2320 - Rework and fix varnishstat counter filtering
  • 2329 - Docfix: Only root can jail

Varnish Cache 4.1.6 (2017-04-26)

  • Introduce a vxid left hand side for VSL queries. This allows matching on records matching a known vxid.
  • Environment variables are now available in the stdandard VMOD; std.getenv()
  • Add setenv command to varnishtest

Bugs fixed

  • 2200 - Dramatically simplify VEV, fix assert in vev.c
  • 2216 - Make sure Age is always less than max-age
  • 2233 - Correct check when parsing the query string
  • 2241 - VSL fails to get hold of SHM
  • 2270 - Newly loaded auto VCLs don't get their go_cold timer set
  • 2273 - Master cooling problem
  • 2275 - If the client workspace is almost, but not quite exhaused, we may not be able to get enough iovec's to do Chunked transmission.
  • 2295 - Spinning loop in VBE_Poll causes master to kill child on CLI timeout
  • 2301 - Don't attempt to check if varnishd is still running if we have already failed.
  • 2313 - Cannot link to varnishapi, symbols missing

Varnish Cache 4.1.5 (2017-02-09)

  • No code changes since 4.1.5-beta2.

Varnish Cache 4.1.5-beta2 (2017-02-08)

  • Update devicedetect.vcl

Bugs fixed

  • 1704 - Reverted the docfix and made the fech_failed counter do what the documentation says it should do
  • 1865 - Panic accessing beresp.backend.ip in vcl_backend_error
  • 2167 - VCC will not parse a literal negative number where INT is expected
  • 2184 - Cannot subtract a negative number

Varnish Cache 4.1.5-beta1 (2017-02-02)

Bugs fixed

  • 1704 - (docfix) Clarify description of fetch_failed counter
  • 1834 - Panic in workspace exhaustion conditions
  • 2106 - 4.1.3: Varnish crashes with "Assert error in CNT_Request(), cache/cache_req_fsm.c line 820"
  • 2134 - Disable Nagle's
  • 2148 - varnishncsa cannot decode Authorization header if the format is incorrect.
  • 2168 - Compare 'bereq.backend' / 'req.backend_hint' myDirector.backend() does not work
  • 2178 - 4.1 branch does not compile on FreeBSD
  • 2188 - Fix vsm_free (never incremented)
  • 2190 - (docfix)varnishncsa: The %r formatter is NOT equivalent to...
  • 2197 - ESI parser panic on malformed src URL

Varnish Cache 4.1.4 (2016-12-01)

Bugs fixed

  • 2035 - varnishd stalls with two consecutive Range requests using HTTP persistent connections

Varnish Cache 4.1.4-beta3 (2016-11-24)

  • Include the current time of the panic in the panic output
  • Keep a reserve of idle threads for vital tasks

Bugs fixed

  • 1874 - clock-step related crash
  • 1889 - (docfix) What does -p flag for backend.list command means
  • 2115 - VSM temporary files are not always deleted
  • 2129 - (docfix) stack overflow with >4 level esi

Varnish Cache 4.1.4-beta2 (2016-10-13)

Bugs fixed

  • 1830 - VSL API: "duplicate link" errors in request grouping when vsl_buffer is increased
  • 2010 - varnishadm CLI behaving weirdly
  • 2017 - varnishncsa docfix: "%r" field is wrong
  • 2107 - (docfix) HEAD requestes changed to GET

Varnish Cache 4.1.4-beta1 (2016-09-14)

  • [varnishhist] Various improvements
  • [varnishtest] A cmd feature for custom shell-based checks
  • Documentation improvements (do_stream, sess_herd, timeout_linger, thread_pools)
  • [varnishtop] Documented behavior when both -p and -1 are specified

Bugs fixed

  • 2027 - Racy backend selection
  • 2024 - panic vmod_rr_resolve() round_robin.c line 75 (be) != NULL
  • 2011 - VBE.*.conn (concurrent connections to backend) not working as expected
  • 2008 - Assert error in VBE_Delete()
  • 2007 - Update documentation part about CLI/management port authentication parameter
  • 1881 - std.cache_req_body() w/ return(pipe) is broken

Varnish Cache 4.1.3 (2016-07-06)

  • Be stricter when parsing request headers to harden against smuggling attacks.

Varnish Cache 4.1.3-beta2 (2016-06-28)

  • New parameter vsm_free_cooldown. Specifies how long freed VSM memory (shared log) will be kept around before actually being freed.
  • varnishncsa now accepts -L argument to configure the limit on incomplete transactions kept. (Issue 1994)

Bugs fixed

  • 1984 - Make the counter vsm_cooling act according to spec
  • 1963 - Avoid abort when changing to a VCL name which is a path
  • 1933 - Don't trust dlopen refcounting

Varnish Cache 4.1.3-beta1 (2016-06-15)

  • varnishncsa can now access and log backend requests. (PR #1905)
  • [varnishncsa] New output formatters %{Varnish:vxid}x and %{VSL:Tag}x.
  • [varnishlog] Added log tag BackendStart on backend transactions.
  • On SmartOS, use ports instead of epoll by default.
  • Add support for TCP Fast Open where available. Disabled by default.
  • [varnishtest] New syncronization primitive barriers added, improving coordination when test cases call external programs.

Bugs fixed

  • 1971 - Add missing Wait_HeapDelete
  • 1967 - [ncsa] Remove implicit line feed when using formatfile
  • 1955 - 4.1.x sometimes duplicates Age and Accept-Ranges headers
  • 1954 - Correctly handle HTTP/1.1 EOF response
  • 1953 - Deal with fetch failures in ved_stripgzip
  • 1931 - Allow VCL set Last-Modified to be used for I-M-S processing
  • 1928 - req->task members must be set in case we get onto the waitinglist
  • 1924 - Make std.log() and std.syslog() work from vcl_{init,fini}
  • 1919 - Avoid ban lurker panic with empty olist
  • 1918 - Correctly handle EOF responses with HTTP/1.1
  • 1912 - Fix (insignificant) memory leak with mal-formed ESI directives.
  • 1904 - Release memory instead of crashing on malformed ESI
  • 1885 - [vmodtool] Method names should start with a period
  • 1879 - Correct handling of duplicate headers on IMS header merge
  • 1878 - Fix a ESI+gzip corner case which had escaped notice until now
  • 1873 - Check for overrun before looking at the next vsm record
  • 1871 - Missing error handling code in V1F_Setup_Fetch
  • 1869 - Remove temporary directory iff called with -C
  • 1883 - Only accept C identifiers as acls
  • 1855 - Truncate output if it's wider than 12 chars
  • 1806 - One minute delay on return (pipe) and a POST-Request
  • 1725 - Revive the backend_conn counter

Varnish Cache 4.1.2 (2016-03-04)

  • [vmods] vmodtool improvements for multiple VMODs in a single directory.

Bugs fixed

  • 1860 - ESI-related memory leaks
  • 1863 - Don't reset the oc->ban pointer from BAN_CheckObject
  • 1864 - Avoid panic if the lurker is working on a ban to be checked.

Varnish Cache 4.1.2-beta2 (2016-02-25)

  • [vmods] Passing VCL ACL to a VMOD is now possible.
  • [vmods] VRT_MINOR_VERSION increase due to new function: VRT_acl_match()
  • Some test case stabilization fixes and minor documentation updates.
  • Improved handling of workspace exhaustion when fetching objects.

Bugs fixed

  • 1858 - Hit-for-pass objects are not IMS candidates

Varnish Cache 4.1.2-beta1 (2016-02-17)

  • Be stricter when parsing a HTTP request to avoid potential HTTP smuggling attacks against vulnerable backends.
  • Some fixes to minor/trivial issues found with clang AddressSanitizer.
  • Arithmetric on REAL data type in VCL is now possible.
  • vmodtool.py improvements to allow VMODs for 4.0 and 4.1 to share a source tree.
  • Off-by-one in WS_Reset() fixed.
  • "https_scheme" parameter added. Enables graceful handling of compound request URLs with HTTPS scheme. (Bug 1847)

Bugs fixed

  • 1739 - Workspace overflow handling in VFP_Push()
  • 1837 - Error compiling VCL if probe is referenced before it is defined
  • 1841 - Replace alien FD's with /dev/null rather than just closing them
  • 1843 - Fail HTTP/1.0 POST and PUT requests without Content-Length
  • 1844 - Correct ENUM handling in object constructors
  • 1851 - Varnish 4.1.1 fails to build on i386
  • 1852 - Add a missing VDP flush operation after ESI:includes.
  • 1857 - Fix timeout calculation for session herding.

Varnish Cache 4.1.1 (2016-01-28)

  • No code changes since 4.1.1-beta2.

Varnish Cache 4.1.1-beta2 (2016-01-22)

  • Improvements to VCL temperature handling added. This opens for reliably deny warming a cooling VCL from a VMOD.

Bugs fixed

  • 1802 - Segfault after VCL change
  • 1825 - Cannot Start Varnish After Just Restarting The Service
  • 1842 - Handle missing waiting list gracefully.
  • 1845 - Handle whitespace after floats in test fields

Varnish Cache 4.1.1-beta1 (2016-01-15)

  • Format of "ban.list" has changed slightly.
  • [varnishncsa] -w is now required when running deamonized.
  • [varnishncsa] Log format can now be read from file.
  • Port fields extracted from PROXY1 header now work as expected.
  • New VCL state "busy" introduced (mostly for VMOD writers).
  • Last traces of varnishreplay removed.
  • If-Modified-Since is now ignored if we have If-None-Match.
  • Zero Content-Length is no longer sent on 304 responses.
  • vcl_dir and vmod_dir now accept a colon separated list of directories.
  • Nested includes starting with "./" are relative to the including VCL file now.

Bugs fixed

  • 1796 - Don't attempt to allocate a V1L from the workspace if it is overflowed.
  • 1794 - Fail if multiple -a arguments return the same suckaddr.
  • 1763 - Restart epoll_wait on EINTR error
  • 1788 - ObjIter has terrible performance profile when busyobj != NULL
  • 1798 - Varnish requests painfully slow with large files
  • 1816 - Use a weak comparison function for If-None-Match
  • 1818 - Allow grace-hits on hit-for-pass objects, [..]
  • 1821 - Always slim private & pass objects after delivery.
  • 1823 - Rush the objheader if there is a waiting list when it is deref'ed.
  • 1826 - Ignore 0 Content-Lengths in 204 responses
  • 1813 - Fail if multiple -a arguments return the same suckaddr.
  • 1810 - Improve handling of HTTP/1.0 clients
  • 1807 - Return 500 if we cannot decode the stored object into the resp.*
  • 1804 - Log proxy related messages on the session, not on the request.
  • 1801 - Relax IP constant parsing

Varnish Cache 4.1.0 (2015-09-30)

  • Documentation updates.
  • Stabilization fixes on testcase p00005.vtc.
  • Avoid compiler warning in zlib.
  • Bug 1792: Avoid using fallocate() with -sfile on non-EXT4.

Varnish Cache 4.1.0-beta1 (2015-09-11)

  • Redhat packaging files are now separate from the normal tree.
  • Client workspace overflow should now result in a 500 response instead of panic.
  • [varnishstat] -w option has been retired.
  • libvarnishapi release number is increased.
  • Body bytes sent on ESI subrequests with gzip are now counted correctly.
  • [vmod-std] Data type conversion functions now take additional fallback argument.

Bugs fixed

  • 1777 - Disable speculative Range handling on streaming transactions.
  • 1778 - [varnishstat] Cast to integer to prevent negative values messing the statistics
  • 1781 - Propagate gzip CRC upwards from nested ESI includes.
  • 1783 - Align code with RFC7230 section 3.3.3 which allows POST without a body.

Varnish Cache 4.1.0-tp1 (2015-07-08)

Changes between 4.0 and 4.1 are numerous. Please read the upgrade section in the documentation for a general overview.

Changes from 4.0.3-rc3 to 4.0.3 (2015-02-17)

  • No changes.

Changes from 4.0.3-rc2 to 4.0.3-rc3 (2015-02-11)

  • Superseded objects are now expired immediately.

Bugs fixed

  • 1462 - Use first/last log entry in varnishncsa.
  • 1539 - Avoid panic when expiry thread modifies a candidate object.
  • 1637 - Fail the fetch processing if the vep callback failed.
  • 1665 - Be more accurate when computing client RX_TIMEOUT.
  • 1672 - Do not panic on unsolicited 304 response to non-200 bereq.

Changes from 4.0.3-rc1 to 4.0.3-rc2 (2015-01-28)

  • Assorted documentation updates.

Bugs fixed

  • 1479 - Fix out-of-tree builds.
  • 1566 - Escape VCL string question marks.
  • 1616 - Correct header file placement.
  • 1620 - Fail miss properly if out of backend threads. (Also 1621)
  • 1628 - Avoid dereferencing null in VBO_DerefBusyObj().
  • 1629 - Ditch rest of waiting list on failure to reschedule.
  • 1660 - Don't attempt range delivery on a synth response

Changes from 4.0.2 to 4.0.3-rc1 (2015-01-15)

  • Support older autoconf (< 2.63b) (el5)
  • A lot of minor documentation fixes.
  • bereq.uncacheable is now read-only.
  • obj.uncacheable is now readable in vcl_deliver.
  • [varnishadm] Prefer exact matches for backend.set_healthy. Bug 1349.
  • Hard-coded -sfile default size is removed.
  • [packaging] EL6 packages are once again built with -O2.
  • [parameter] fetch_chunksize default is reduced to 16KB. (from 128KB)
  • Added std.time() which converts strings to VCL_TIME.
  • [packaging] packages now Provide strictABI (gitref) and ABI (VRT major/minor) for VMOD use.

Bugs fixed

  • 1378 - Properly escape non-printable characters in varnishncsa.
  • 1596 - Delay HSH_Complete() until the storage sanity functions has finished.
  • 1506 - Keep Content-Length from backend if we can.
  • 1602 - Fix a cornercase related to empty pass objects.
  • 1607 - Don't leak reqs on failure to revive from waitinglist.
  • 1610 - Update forgotten varnishlog example to 4.0 syntax.
  • 1612 - Fix a cornercase related to empty pass objects.
  • 1623 - Fix varnishhist -d segfault.
  • 1636 - Outdated paragraph in Vary: documentation
  • 1638 - Fix panic when retrying a failed backend fetch.
  • 1639 - Restore the default SIGSEGV handler during pan_ic
  • 1647 - Relax an assertion for the IMS update candidate object.
  • 1648 - Avoid partial IMS updates to replace old object.
  • 1650 - Collapse multiple X-Forwarded-For headers

Changes from 4.0.2-rc1 to 4.0.2 (2014-10-08)

New since 4.0.2-rc1:

  • [varnishlog] -k argument is back. (exit after n records)
  • [varnishadm] vcl.show is now listed in help.

Changes from 4.0.1 to 4.0.2-rc1 (2014-09-23)

New since 4.0.1:

  • [libvmod-std] New function strstr() for matching substrings.
  • server.(hostname|identity) is now available in all VCL functions.
  • VCL variable type BYTES was added.
  • workspace_client default is now 9k.
  • [varnishstat] Update interval can now be subsecond.
  • Document that reloading VCL does not reload a VMOD.
  • Guru meditation page is now valid HTML5.
  • [varnishstat] hitrate calculation is back.
  • New parameter group_cc adds a GID to the grouplist of VCL compiler sandbox.
  • Parameter shm_reclen is now an alias for vsl_reclen.
  • Workspace overflows are now handled with a 500 client response.
  • VCL variable type added: HTTP, representing a HTTP header set.
  • It is now possible to return(synth) from vcl_deliver.
  • [varnishadm] vcl.show now has a -v option that output the complete set of VCL and included VCL files.
  • RHEL7 packaging (systemd) was added.
  • [libvmod-std] querysort() fixed parameter limit has been lifted.
  • Fix small memory leak in ESI parser.
  • Fix unreported race/assert in V1D_Deliver().

Bugs fixed

  • 1553 - Fully reset workspace (incl. Vary state) before reusing it.
  • 1551 - Handle workspace exhaustion during purge.
  • 1591 - Group entries correctly in varnishtop.
  • 1592 - Bail out on workspace exhaustion in VRT_IP_string.
  • 1538 - Relax VMOD ABI check for release branches.
  • 1584 - Don't log garbage/non-HTTP requests. [varnishncsa]
  • 1407 - Don't rename VSM file until child has started.
  • 1466 - Don't leak request structs on restart after waitinglist.
  • 1580 - Output warning if started without -b and -f. [varnishd]
  • 1583 - Abort on fatal sandbox errors on Solaris. (Related: 1572)
  • 1585 - Handle fatal sandbox errors.
  • 1572 - Exit codes have been cleaned up.
  • 1569 - Order of symbols should not influence compilation result.
  • 1579 - Clean up type inference in VCL.
  • 1578 - Don't count Age twice when computing new object TTL.
  • 1574 - std.syslog() logged empty strings.
  • 1555 - autoconf editline/readline build issue.
  • 1568 - Skip NULL arguments when hashing.
  • 1567 - Compile on systems without SO_SNDTIMEO/SO_RCVTIMEO.
  • 1512 - Changes to bereq are lost between v_b_r and v_b_f.
  • 1563 - Increase varnishtest read timeout.
  • 1561 - Never call a VDP with zero length unless done.
  • 1562 - Fail correctly when rereading a failed client request body.
  • 1521 - VCL compilation fails on OSX x86_64.
  • 1547 - Panic when increasing shm_reclen.
  • 1503 - Document return(retry).
  • 1581 - Don't log duplicate Begin records to shmlog.
  • 1588 - Correct timestamps on pipelined requests.
  • 1575 - Use all director backends when looking for a healthy one.
  • 1577 - Read the full request body if shunted to synth.
  • 1532 - Use correct VCL representation of reals.
  • 1531 - Work around libedit bug in varnishadm.

Changes from 4.0.0 to 4.0.1 (2014-06-24)

New since 4.0.0:

  • New functions in vmod_std: real2time, time2integer, time2real, real.
  • Chunked requests are now supported. (pass)
  • Add std.querysort() that sorts GET query arguments. (from libvmod-boltsort)
  • Varnish will no longer reply with "200 Not Modified".
  • Backend IMS is now only attempted when last status was 200.
  • Packaging now uses find-provides instead of find-requires. [redhat]
  • Two new counters: n_purges and n_obj_purged.
  • Core size can now be set from /etc/sysconfig/varnish [redhat]
  • Via header set is now RFC compliant.
  • Removed "purge" keyword in VCL. Use return(purge) instead.
  • fallback director is now documented.
  • %D format flag in varnishncsa is now truncated to an integer value.
  • persistent storage backend is now deprecated. https://www.varnish-cache.org/docs/trunk/phk/persistent.html
  • Added format flags %I (total bytes received) and %O (total bytes sent) for varnishncsa.
  • python-docutils >= 0.6 is now required.
  • Support year (y) as a duration in VCL.
  • VMOD ABI requirements are relaxed, a VMOD no longer have to be run on the same git revision as it was compiled for. Replaced by a major/minor ABI counter.

Bugs fixed

  • 1269 - Use correct byte counters in varnishncsa when piping a request.
  • 1524 - Chunked requests should be pipe-able.
  • 1530 - Expire old object on successful IMS fetch.
  • 1475 - time-to-first-byte in varnishncsa was potentially dishonest.
  • 1480 - Porting guide for 4.0 is incomplete.
  • 1482 - Inherit group memberships of -u specified user.
  • 1473 - Fail correctly in configure when rst2man is not found.
  • 1486 - Truncate negative Age values to zero.
  • 1488 - Don't panic on high request rates.
  • 1489 - req.esi should only be available in client threads.
  • 1490 - Fix thread leak when reducing number of threads.
  • 1491 - Reorder backend connection close procedure to help test cases.
  • 1498 - Prefix translated VCL names to avoid name clashes.
  • 1499 - Don't leak an objcore when HSH_Lookup returns expired object.
  • 1493 - vcl_purge can return synth or restart.
  • 1476 - Cope with systems having sys/endian.h and endian.h.
  • 1496 - varnishadm should be consistent in argv ordering.
  • 1494 - Don't panic on VCL-initiated retry after a backend 500 error.
  • 1139 - Also reset keep (for IMS) time when purging.
  • 1478 - Avoid panic when delivering an object that expires during delivery.
  • 1504 - ACLs can be unreferenced with vcc_err_unref=off set.
  • 1501 - Handle that a director couldn't pick a backend.
  • 1495 - Reduce WRK_SumStat contention.
  • 1510 - Complain on symbol reuse in VCL.
  • 1514 - Document storage.NAME.free_space and .used_space [docs]
  • 1518 - Suppress body on 304 response when using ESI.
  • 1519 - Round-robin director does not support weight. [docs]

Changes from 4.0.0 beta1 to 4.0.0 (2014-04-10)

New since 4.0.0-beta1:

  • improved varnishstat documentation.
  • In VCL, req.backend_hint is available in vcl_hit
  • ncurses is now a dependency.

Bugs fixed

  • 1469 - Fix build error on PPC
  • 1468 - Set ttl=0 on failed objects
  • 1462 - Handle duplicate ReqURL in varnishncsa.
  • 1467 - Fix missing clearing of oc->busyobj on HSH_Fail.

Changes from 4.0.0 TP2 to 4.0.0 beta1 (2014-03-27)

New since TP2:

  • Previous always-appended code called default.vcl is now called builtin.vcl. The new example.vcl is recommended as a starting point for new users.
  • vcl_error is now called vcl_synth, and does not any more mandate closing the client connection.
  • New VCL function vcl_backend_error, where you can change the 503 prepared if all your backends are failing. This can then be cached as a regular object.
  • Keyword "remove" in VCL is replaced by "unset".
  • new timestamp and accounting records in varnishlog.
  • std.timestamp() is introduced.
  • stored objects are now read only, meaning obj.hits now counts per objecthead instead. obj.lastuse saw little use and has been removed.
  • builtin VCL now does return(pipe) for chunked POST and PUT requests.
  • python-docutils and rst2man are now build requirements.
  • cli_timeout is now 60 seconds to avoid slaughtering the child process in times of high IO load/scheduling latency.
  • return(purge) from vcl_recv is now valid.
  • return(hash) is now the default return action from vcl_recv.
  • req.backend is now req.backend_hint. beresp.storage is beresp.storage_hint.

Bugs fixed

  • 1460 - tools now use the new timestamp format.
  • 1450 - varnishstat -l segmentation fault.
  • 1320 - Work around Content-Length: 0 and Content-Encoding: gzip gracefully.
  • 1458 - Panic on busy object.
  • 1417 - Handle return(abandon) in vcl_backend_response.
  • 1455 - vcl_pipe now sets Connection: close by default on backend requests.
  • 1454 - X-Forwarded-For is now done in C, before vcl_recv is run.
  • 1436 - Better explanation when missing an import in VCL.
  • 1440 - Serve ESI-includes from a different backend.
  • 1441 - Incorrect grouping when logging ESI subrequests.
  • 1434 - std.duration can now do ms/milliseconds.
  • 1419 - Don't put objcores on the ban list until they go non-BUSY.
  • 1405 - Ban lurker does not always evict all objects.

Changes from 4.0.0 TP1 to 4.0.0 TP2 (2014-01-23)

New since from 4.0.0 TP1

  • New VCL_BLOB type to pass binary data between VMODs.
  • New format for VMOD description files. (.vcc)

Bugs fixed

  • 1404 - Don't send Content-Length on 304 Not Modified responses.
  • 1401 - Varnish would crash when retrying a backend fetch too many times.
  • 1399 - Memory get freed while in use by another thread/object
  • 1398 - Fix NULL deref related to a backend we don't know any more.
  • 1397 - Crash on backend fetch while LRUing.
  • 1395 - End up in vcl_error also if fetch fails vcl_backend_response.
  • 1391 - Client abort and retry during a streaming fetch would make Varnish assert.
  • 1390 - Fix assert if the ban lurker is overtaken by new duplicate bans.
  • 1385 - ban lurker doesn't remove (G)one bans
  • 1383 - varnishncsa logs requests for localhost regardless of host header.
  • 1382 - varnishncsa prints nulls as part of request string.
  • 1381 - Ensure vmod_director is installed
  • 1323 - Add a missing boundary check for Range requests
  • 1268 - shortlived parameter now uses TTL+grace+keep instead of just TTL.
  • Fix build error on OpenBSD (TCP_KEEP)
  • n_object wasn't being decremented correctly on object expire.
  • Example default.vcl in distribution is now 4.0-ready.

Open issues

  • 1405 - Ban lurker does not always evict all objects.

Changes from 3.0.7-rc1 to 3.0.7 (2015-03-23)

  • No changes.

Changes from 3.0.6 to 3.0.7-rc1 (2015-03-18)

  • Requests with multiple Content-Length headers will now fail.
  • Stop recognizing a single CR (r) as a HTTP line separator. This opened up a possible cache poisoning attack in stacked installations where sslterminator/varnish/backend had different CR handling.
  • Improved error detection on master-child process communication, leading to faster recovery (child restart) if communication loses sync.
  • Fix a corner-case where Content-Length was wrong for HTTP 1.0 clients, when using gzip and streaming. Bug 1627.
  • More robust handling of hop-by-hop headers.
  • [packaging] Coherent Redhat pidfile in init script. Bug 1690.
  • Avoid memory leak when adding bans.

Changes from 3.0.6rc1 to 3.0.6 (2014-10-16)

  • Minor changes to documentation.
  • [varnishadm] Add termcap workaround for libedit. Bug 1531.

Changes from 3.0.5 to 3.0.6rc1 (2014-06-24)

  • Document storage.<name>.* VCL variables. Bug 1514.
  • Fix memory alignment panic when http_max_hdr is not a multiple of 4. Bug 1327.
  • Avoid negative ReqEnd timestamps with ESI. Bug 1297.
  • %D format for varnishncsa is now an integer (as documented)
  • Fix compile errors with clang.
  • Clear objectcore flags earlier in ban lurker to avoid spinning thread. Bug 1470.
  • Patch embedded jemalloc to avoid segfault. Bug 1448.
  • Allow backend names to start with if, include or else. Bug 1439.
  • Stop handling gzip after gzip body end. Bug 1086.
  • Document %D and %T for varnishncsa.

Changes from 3.0.5 rc 1 to 3.0.5 (2013-12-02)

varnishd

  • Always check the local address of a socket. This avoids a crash if server.ip is accessed after a client has closed the connection. Bug #1376

Changes from 3.0.4 to 3.0.5 rc 1

varnishd

  • Stop printing error messages on ESI parse errors
  • Fix a problem where Varnish would segfault if the first part of a synthetic page was NULL. Bug #1287
  • If streaming was used, you could in some cases end up with duplicate content headers being sent to clients. Bug #1272
  • If we receive a completely garbled request, don't pass through vcl_error, since we could then end up in vcl_recv through a restart and things would go downhill from there. Bug #1367
  • Prettify backtraces on panic slightly.

varnishlog

  • Correct an error where -m, -c and -b would interact badly, leading to lack of matches. Also, emit BackendXID to signify the start of a transaction. Bug #1325

varnishadm

  • Handle input from stdin properly. Bug #1314

Changes from 3.0.4 rc 1 to 3.0.4 (2013-06-14)

varnishd

  • Set the waiter pipe as non-blocking and record overflows. Bug #1285
  • Fix up a bug in the ACL compile code that could lead to false negatives. CVE-2013-4090. Bug #1312
  • Return an error if the client sends multiple Host headers.

Changes from 3.0.3 to 3.0.4 rc 1

varnishd

  • Fix error handling when uncompressing fetched objects for ESI processing. Bug #1184
  • Be clearer about which timeout was reached in logs.
  • Correctly decrement n_waitinglist counter. Bug #1261
  • Turn off Nagle/set TCP_NODELAY.
  • Avoid panic on malformed Vary headers. Bug #1275
  • Increase the maximum length of backend names. Bug #1224
  • Add support for banning on http.status. Bug #1076
  • Make hit-for-pass correctly prefer the transient storage.

varnishlog

  • If -m, but neither -b or -c is given, assume both. This filters out a lot of noise when -m is used to filter. Bug #1071

varnishadm

  • Improve tab completion and require libedit/readline to build.

varnishtop

  • Reopen log file if Varnish is restarted.

varnishncsa

  • Handle file descriptors above 64k (by ignoring them). Prevents a crash in some cases with corrupted shared memory logs.
  • Add %D and %T support for more timing information.

Other

  • Documentation updates.
  • Fixes for OSX
  • Disable PCRE JIT-er, since it's broken in some PCRE versions, at least on i386.
  • Make libvarnish prefer exact hits when looking for VSL tags.

Changes from 3.0.2 to 3.0.3 (2012-08-20)

varnishd

  • Fix a race on the n_sess counter. This race made varnish do excessive session workspace allocations. Bug #897.
  • Fix some crashes in the gzip code when it runs out of memory. Bug #1037. Bug #1043. Bug #1044.
  • Fix a bug where the regular expression parser could end up in an infinite loop. Bug #1047.
  • Fix a memory leak in the regex code.
  • DNS director now uses port 80 by default if not specified.
  • Introduce idle_send_timeout and increase default value for send_timeout to 600s. This allows a long send timeout for slow clients while still being able to disconnect idle clients.
  • Fix an issue where <esi:remove> did not remove HTML comments. Bug #1092.
  • Fix a crash when passing with streaming on.
  • Fix a crash in the idle session timeout code.
  • Fix an issue where the poll waiter did not timeout clients if all clients were idle. Bug #1023.
  • Log regex errors instead of crashing.
  • Introduce pcre_match_limit, and pcre_match_limit_recursion parameters.
  • Add CLI commands to manually control health state of a backend.
  • Fix an issue where the s_bodybytes counter is not updated correctly on gunzipped delivery.
  • Fix a crash when we couldn't allocate memory for a fetched object. Bug #1100.
  • Fix an issue where objects could end up in the transient store with a long TTL, when memory could not be allocated for them in the requested store. Bug #1140.
  • Activate req.hash_ignore_busy when req.hash_always_miss is activated. Bug #1073.
  • Reject invalid tcp port numbers for listen address. Bug #1035.
  • Enable JIT for better performing regular expressions. Bug #1080.
  • Return VCL errors in exit code when using -C. Bug #1069.
  • Stricter validation of acl syntax, to avoid a crash with 5-octet IPv4 addresses. Bug #1126.
  • Fix a crash when first argument to regsub was null. Bug #1125.
  • Fix a case where varnish delivered corrupt gzip content when using ESI. Bug #1109.
  • Fix a case where varnish didn't remove the old Date header and served it alongside the varnish-generated Date header. Bug #1104.
  • Make saint mode work, for the case where we have no object with that hash. Bug #1091.
  • Don't save the object body on hit-for-pass objects.
  • n_ban_gone counter added to count the number of "gone" bans.
  • Ban lurker rewritten to properly sleep when no bans are present, and otherwise to process the list at the configured speed.
  • Fix a case where varnish delivered wrong content for an uncompressed page with compressed ESI child. Bug #1029.
  • Fix an issue where varnish runs out of thread workspace when processing many ESI includes on an object. Bug #1038.
  • Fix a crash when streaming was enabled for an empty body.
  • Better error reporting for some fetch errors.
  • Small performance optimizations.

varnishncsa

  • Support for tn in varnishncsa format strings.
  • Add new format: %{VCL_Log:foo}x which output key:value from std.log() in VCL.
  • Add user-defined date formatting, using %{format}t.

varnishtest

  • resp.body is now available for inspection.
  • Make it possible to test for the absence of an HTTP header. Bug #1062.
  • Log the full panic message instead of shortening it to 512 characters.

varnishstat

  • Add json output (-j).

Other

  • Documentation updates.
  • Bump minimum number of threads to 50 in RPM packages.
  • RPM packaging updates.
  • Fix some compilation warnings on Solaris.
  • Fix some build issues on Open/Net/DragonFly-BSD.
  • Fix build on FreeBSD 10-current.
  • Fix libedit detection on *BSD OSes. Bug #1003.

Changes from 3.0.2 rc 1 to 3.0.2 (2011-10-26)

varnishd

  • Make the size of the synthetic object workspace equal to http_resp_size and add workaround to avoid a crash when setting too long response strings for synthetic objects.
  • Ensure the ban lurker always sleeps the advertised 1 second when it does not have anything to do.
  • Remove error from vcl_deliver. Previously this would assert while it will now give a syntax error.

varnishncsa

  • Add default values for some fields when logging incomplete records and document the default values.

Other

  • Documentation updates
  • Some Solaris portability updates.

Changes from 3.0.1 to 3.0.2 rc 1 (2011-10-06)

varnishd

  • Only log the first 20 bytes of extra headers to prevent overflows.
  • Fix crasher bug which sometimes happened if responses are queued and the backend sends us Vary. Bug #994.
  • Log correct size of compressed when uncompressing them for clients that do not support compression. Bug #996.
  • Only send Range responses if we are going to send a body. Bug #1007.
  • When varnishd creates a storage file, also unlink it to avoid leaking disk space over time. Bug #1008.
  • The default size of the -s file parameter has been changed to 100MB instead of 50% of the available disk space.
  • The limit on the number of objects we remove from the cache to make room for a new one was mistakenly lowered to 10 in 3.0.1. This has been raised back to 50. Bug #1012.
  • http_req_size and http_resp_size have been increased to 8192 bytes. This better matches what other HTTPds have. Bug #1016.

VCL

  • Allow relational comparisons of floating point types.
  • Make it possible for VMODs to fail loading and so cause the VCL loading to fail.

varnishncsa

  • Fixed crash when client was sending illegal HTTP headers.
  • %{Varnish:handling} in format strings was broken, this has been fixed.

Other

  • Documentation updates
  • Some Solaris portability updates.

Changes from 3.0.1 rc 1 to 3.0.1 (2011-08-30)

varnishd

  • Fix crash in streaming code.
  • Add fallback director, as a variant of the round-robin director.
  • The parameter http_req_size has been reduced on 32 bit machines.

VCL

  • Disallow error in the vcl_init and vcl_fini VCL functions.

varnishncsa

  • Fixed crash when using -X.
  • Fix error when the time to first byte was in the format string.

Other

  • Documentation updates

Changes from 3.0.0 to 3.0.1 rc 1 (2011-08-24)

varnishd

  • Avoid sending an empty end-chunk when sending bodyless responses.
  • http_resp_hdr_len and http_req_hdr_len were set to too low values leading to clients receiving HTTP 400 Bad Request errors. The limit has been increased and the error code is now HTTP 413 Request entity too large.
  • Objects with grace or keep set were mistakenly considered as candidates for the transient storage. They now have their grace and keep limited to limit the memory usage of the transient stevedore.
  • If a request was restarted from vcl_miss or vcl_pass it would crash. This has been fixed. Bug #965.
  • Only the first few clients waiting for an object from the backend would be woken up when object arrived and this lead to some clients getting stuck for a long time. This has now been fixed. Bug #963.
  • The hash and client directors would mistakenly retry fetching an object from the same backend unless health probes were enabled. This has been fixed and it will now retry a different backend.

VCL

  • Request specific variables such as client.* and server.* are now correctly marked as not available in vcl_init and vcl_fini.
  • The VCL compiler would fault if two IP comparisons were done on the same line. This now works correctly. Bug #948.

varnishncsa

  • Add support for logging arbitrary request and response headers.
  • Fix crashes if hitmiss and handling have not yet been set.
  • Avoid printing partial log lines if there is an error in a format string.
  • Report user specified format string errors better.

varnishlog

  • varnishlog -r now works correctly again and no longer opens the shared log file of the running Varnish.

Other

  • Various documentation updates.
  • Minor compilation fixes for newer compilers.
  • A bug in the ESI entity replacement parser has been fixed. Bug #961.
  • The ABI of VMODs are now checked. This will require a rebuild of all VMODs against the new version of Varnish.

Changes from 3.0 beta 2 to 3.0.0 (2011-06-16)

varnishd

  • Avoid sending an empty end-chunk when sending bodyless responses.

VCL

  • The synthetic keyword has now been properly marked as only available in vcl_deliver. Bug #936.

varnishadm

  • Fix crash if the secret file was unreadable. Bug #935.
  • Always exit if varnishadm can't connect to the backend for any reason.

Changes from 3.0 beta 1 to 3.0 beta 2

varnishd

  • thread_pool_min and thread_pool_max now each refer to the number of threads per pool, rather than being inconsistent as they were before.
  • 307 Temporary redirect is now considered cacheable. Bug #908.
  • The stats command has been removed from the CLI interface. With the new counters, it would mean implementing more and more of varnishstat in the master CLI process and the CLI is single-threaded so we do not want to do this work there in the first place. Use varnishstat instead.

VCL

  • VCL now treats null arguments (unset headers for instance) as empty strings. Bug #913.
  • VCL now has vcl_init and vcl_fini functions that are called when a given VCL has been loaded and unloaded.
  • There is no longer any interpolation of the right hand side in bans where the ban is a single string. This was confusing and you now have to make sure bits are inside or outside string context as appropriate.
  • Varnish is now stricter in enforcing no duplication of probes, backends and ACLs.

varnishncsa

  • varnishncsa now ignores piped requests, since we have no way of knowing their return status.

VMODs

  • The std module now has proper documentation, including a manual page

Changes from 2.1.5 to 3.0 beta 1

Upcoming changes

  • The interpretation of bans will change slightly between 3.0 beta 1 and 3.0 release. Currently, doing ban("req.url == req.url") will cause the right hand req.url to be interpreted in the context of the request creating the ban. This will change so you will have to do ban("req.url == " + req.url) instead. This syntax already works and is recommended.

varnishd

  • Add streaming on pass and miss. This is controlled by the beresp.do_stream boolean. This includes support for compression/uncompression.
  • Add support for ESI and gzip.
  • Handle objects larger than 2G.
  • HTTP Range support is now enabled by default
  • The ban lurker is enabled by default
  • if there is a backend or director with the name default, use that as the default backend, otherwise use the first one listed.
  • Add many more stats counters. Amongst those, add per storage backend stats and per-backend statistics.
  • Syslog the platform we are running on
  • The -l (shared memory log file) argument has been changed, please see the varnishd manual for the new syntax.
  • The -S and -T arguments are now stored in the shmlog
  • Fix off-by-one error when exactly filling up the workspace. Bug #693.
  • Make it possible to name storage backends. The names have to be unique.
  • Update usage output to match the code. Bug #683
  • Add per-backend health information to shared memory log.
  • Always recreate the shared memory log on startup.
  • Add a vcl_dir parameter. This is used to resolve relative path names for vcl.load and include in .vcl files.
  • Make it possible to specify -T :0. This causes varnishd to look for a free port automatically. The port is written in the shared memory log so varnishadm can find it.
  • Classify locks into kinds and collect stats for each kind, recording the data in the shared memory log.
  • Auto-detect necessary flags for pthread support and VCC_CC flags. This should make Varnish somewhat happier on Solaris. Bug #663
  • The overflow_max parameter has been renamed to queue_max.
  • If setting a parameter fails, report which parameter failed as this is not obvious during startup.
  • Add a parameter named shortlived. Objects whose TTL is less than the parameter go into transient (malloc) storage.
  • Reduce the default thread_add_delay to 2ms.
  • The max_esi_includes parameter has been renamed to max_esi_depth.
  • Hash string components are now logged by default.
  • The default connect timeout parameter has been increased to 0.7 seconds.
  • The err_ttl parameter has been removed and is replaced by a setting in default.vcl.
  • The default send_timeout parameter has been reduced to 1 minute.
  • The default ban_lurker sleep has been set to 10ms.
  • When an object is banned, make sure to set its grace to 0 as well.
  • Add panic.show and panic.clear CLI commands.
  • The default http_resp_hdr_len and http_req_hdr_len has been increased to 2048 bytes.
  • If vcl_fetch results in restart or error, close the backend connection rather than fetching the object.
  • If allocating storage for an object, try reducing the chunk size before evicting objects to make room. Bug #880
  • Add restart from vcl_deliver. Bug #411
  • Fix an off-by-up-to-one-minus-epsilon bug where if an object from the backend did not have a last-modified header we would send out a 304 response which did include a Last-Modified header set to when we received the object. However, we would compare the timestamp to the fractional second we got the object, meaning any request with the exact timestamp would get a 200 response rather than the correct 304.
  • Fix a race condition in the ban lurker where a serving thread and the lurker would both look at an object at the same time, leading to Varnish crashing.
  • If a backend sends a Content-Length header and we are streaming and we are not uncompressing it, send the Content-Length header on, allowing browsers to diplay a progress bar.
  • All storage must be at least 1M large. This is to prevent administrator errors when specifying the size of storage where the admin might have forgotten to specify units.

Tools

common

  • Add an -m $tag:$regex parameter, used for selecting some transactions. The parameter can be repeated, in which case it is logically and-ed together.

varnishadm

  • varnishadm will now pick up the -S and -T arguments from the shared memory log, meaning just running it without any arguments will connect to the running varnish. Bug #875
  • varnishadm now accepts an -n argument to specify the location of the shared memory log file
  • add libedit support

varnishstat

  • reopen shared memory log if the varnishd process is restarted.
  • Improve support for selecting some, but not all fields using the -f argument. Please see the documentation for further details on the use of -f.
  • display per-backend health information

varnishncsa

  • Report error if called with -i and -I as they do not make any sense for varnishncsa.
  • Add custom log formats, specified with -F. Most of the Apache log formats are supported, as well as some Varnish-specific ones. See the documentation for further information. Bug #712 and bug #485

varnishtest

  • add -l and -L switches which leave /tmp/vtc.* behind on error and unconditionally respectively.
  • add -j parameter to run tests in parallel and use this by default.

varnishtop

  • add -p $period parameter. The units in varnishtop were previously undefined, they are now in requests/period. The default period is 60 seconds.

varnishlog

  • group requests by default. This can be turned off by using -O
  • the -o parameter is now a no-op and is ignored.

VMODs

  • Add a std VMOD which includes a random function, log, syslog, fileread, collect,

VCL

  • Change string concatenation to be done using + rather than implicitly.
  • Stop using %xx escapes in VCL strings.
  • Change req.hash += value to hash_data(value)
  • Variables in VCL now have distinct read/write access
  • bereq.connect_timeout is now available in vcl_pipe.
  • Make it possible to declare probes outside of a director. Please see the documentation on how to do this.
  • The VCL compiler has been reworked greatly, expanding its abilities with regards to what kinds of expressions it understands.
  • Add beresp.backend.name, beresp.backend.ip and beresp.backend.port variables. They are only available from vcl_fetch and are read only. Bug #481
  • The default VCL now calls pass for any objects where beresp.http.Vary == "*". Bug #787
  • The log keyword has been moved to the std VMOD.
  • It is now possible to choose which storage backend to be used
  • Add variables storage.$name.free_space, storage.$name.used_space and storage.$name.happy
  • The variable req.can_gzip tells us whether the client accepts gzipped objects or not.
  • purge is now called ban, since that is what it really is and has always been.
  • req.esi_level is now available. Bug #782
  • esi handling is now controlled by the beresp.do_esi boolean rather than the esi function.
  • beresp.do_gzip and beresp.do_gunzip now control whether an uncompressed object should be compressed and a compressed object should be uncompressed in the cache.
  • make it possible to control compression level using the gzip_level parameter.
  • obj.cacheable and beresp.cacheable have been removed. Cacheability is now solely through the beresp.ttl and beresp.grace variables.
  • setting the obj.ttl or beresp.ttl to zero now also sets the corresponding grace to zero. If you want a non-zero grace, set grace after setting the TTL.
  • return(pass) in vcl_fetch has been renamed to return(hit_for_pass) to make it clear that pass in vcl_fetch and vcl_recv are different beasts.
  • Add actual purge support. Doing purge will remove an object and all its variants.

Libraries

  • libvarnishapi has been overhauled and the API has been broken. Please see git commit logs and the support tools to understand what's been changed.
  • Add functions to walk over all the available counters. This is needed because some of the counter names might only be available at runtime.
  • Limit the amount of time varnishapi waits for a shared memory log to appear before returning an error.
  • All libraries but libvarnishapi have been moved to a private directory as they are not for public consumption and have no ABI/API guarantees.

Other

  • Python is now required to build
  • Varnish Cache is now consistently named Varnish Cache.
  • The compilation process now looks for kqueue on NetBSD
  • Make it possible to use a system jemalloc rather than the bundled version.
  • The documentation has been improved all over and should now be in much better shape than before

Changes from 2.1.4 to 2.1.5 (2011-01-25)

varnishd

  • On pass from vcl_recv, we did not remove the backends Content-Length header before adding our own. This could cause confusion for browsers and has been fixed.
  • Make pass with content-length work again. An issue with regards to 304, Content-Length and pass has been resolved.
  • An issue relating to passed requests with If-Modified-Since headers has been fixed. Varnish did not recognize that the 304-response did not have a body.
  • A potential lock-inversion with the ban lurker thread has been resolved.
  • Several build-dependency issues relating to rst2man have been fixed. Varnish should now build from source without rst2man if you are using tar-balls.
  • Ensure Varnish reads the expected last CRLF after chunked data from the backend. This allows re-use of the connection.
  • Remove a GNU Make-ism during make dist to make BSD happier.
  • Document the log, set, unset, return and restart statements in the VCL documentation.
  • Fix an embarrassingly old bug where Varnish would run out of workspace when requests come in fast over a single connection, typically during synthetic benchmarks.
  • Varnish will now allow If-Modified-Since requests to objects without a Last-Modified-header, and instead use the time the object was cached instead.
  • Do not filter out Content-Range headers in pass.
  • Require -d, -b, -f, -S or -T when starting varnishd. In human terms, this means that it is legal to start varnishd without a Vcl or backend, but only if you have a CLI channel of some kind.
  • Don't suppress Cache-Control headers in pass responses.
  • Merge multi-line Cache-Control and Vary header fields. Until now, no browsers have needed this, but Chromium seems to find it necessary to spread its Cache-Control across two lines, and we get to deal with it.
  • Make new-purge not touch busy objects. This fixes a potential crash when calling VRT_purge.
  • If there are everal grace-able objects, pick the least expired one.
  • Fix an issue with varnishadm -T :6082 shorthand.
  • Add bourn-shell like "here" documents on the CLI. Typical usage: vcl.inline vcl_new << 42 backend foo {...} sub vcl_recv {...} 42
  • Add CLI version to the CLI-banner, starting with version 1.0 to mark here-documents.
  • Fix a problem with the expiry thread slacking off during high load.

varnishtest

  • Remove no longer existing -L option.

Changes from 2.1.3 to 2.1.4

varnishd

  • An embarrasing typo in the new binary heap layout caused inflated obj/objcore/objhdr counts and could cause odd problems when the LRU expunge mechanism was invoked. This has been fixed.
  • We now have updated documentation in the reStructuredText format. Manual pages and reference documentation are both built from this.
  • We now include a DNS director which uses DNS for choosing which backend to route requests to. Please see the documentation for more details.
  • If you restarted a request, the HTTP header X-Forwarded-For would be updated multiple times. This has been fixed.
  • If a VCL contained a % sign, and the vcl.show CLI command was used, varnishd would crash. This has been fixed.
  • When doing a pass operation, we would remove the Content-Length, Age and Proxy-Auth headers. We are no longer doing this.
  • now has a string representation, making it easier to construct Expires headers in VCL.
  • In a high traffic environment, we would sometimes reuse a file descriptor before flushing the logs from a worker thread to the shared log buffer. This would cause confusion in some of the tools. This has been fixed by explicitly flushing the log when a backend connection is closed.
  • If the communication between the management and the child process gets out of sync, we have no way to recover. Previously, varnishd would be confused, but we now just kill the child and restart it.
  • If the backend closes the connection on us just as we sent a request to it, we retry the request. This should solve some interoperability problems with Apache and the mpm-itk multi processing module.
  • varnishd now only provides help output the current CLI session is authenticated for.
  • If the backend does not tell us which length indication it is using, we now assume the resource ends EOF at.
  • The client director now has a variable client.identity which is used to choose which backend should receive a given request.
  • The Solaris port waiter has been updated, and other portability fixes for Solaris.
  • There was a corner case in the close-down processing of pipes, this has now been fixed.
  • Previously, if we stopped polling a backend which was sick, it never got marked as healthy. This has now been changed.
  • It is now possible to specify ports as part of the .host field in VCL.
  • The synthetic counters were not locked properly, and so the sms_ counters could underflow. This has now been fixed.
  • The value of obj.status as a string in vcl_error would not be correct in all cases. This has been fixed.
  • Varnish would try to trim storage segments completely filled when using the malloc stevedore and the object was received chunked encoding. This has been fixed.
  • If a buggy backend sends us a Vary header with two colons, we would previously abort. We now rather fix this up and ignore the extra colon.
  • req.hash_always_miss and req.hash_ignore_busy has been added, to make preloading or periodically refreshing content work better.

varnishncsa

  • varnishncsa would in some cases be confused by ESI requests and output invalid lines. This has now been fixed.

varnishlog

  • varnishlog now allows -o and -u together.

varnishtop

  • varnishtop would crash on 32 bit architectures. This has been fixed.

libvarnishapi

  • Regex inclusion and exclusion had problems with matching particular parts of the string being matched. This has been fixed.

Changes from 2.1.2 to 2.1.3

varnishd

  • Improve scalability of critbit.
  • The critbit hash algorithm has now been tightened to make sure the tree is in a consistent state at all points, and the time we wait for an object to cool off after it is eligible for garbage collection has been tweaked.
  • Add log command to VCL. This emits a VCL_log entry into the shared memory log.
  • Only emit Length and ReqEnd log entries if we actually have an XID. This should get rid of some empty log lines in varnishncsa.
  • Destroy directors in a predictable fashion, namely reverse of creation order.
  • Fix bug when ESI elements spanned storage elements causing a panic.
  • In some cases, the VCL compiler would panic instead of giving sensible messages. This has now been fixed.
  • Correct an off-by-one error when the requested range exceeds the size of an object.
  • Handle requests for the end of an object correctly.
  • Allow tabulator characters in the third field of the first line of HTTP requests
  • On Solaris, if the remote end sends us an RST, all system calls related to that socket will return EINVAL. We now handle this better.

libvarnishapi

  • The -X parameter didn't work correctly. This has been fixed.

Changes from 2.1.1 to 2.1.2

varnishd

  • When adding Range support for 2.1.1, we accidentally introduced a bug which would append garbage to objects larger than the chunk size, by default 128k. Browsers would do the right thing due to Content-Length, but some load balancers would get very confused.

Changes from 2.1.1 to 2.1.1

varnishd

  • The changelog in 2.1.0 included syntax errors, causing the generated changelog to be empty.
  • The help text for default_grace was wrongly formatted and included a syntax error. This has now been fixed.
  • varnishd now closes the file descriptor used to read the management secret file (from the -S parameter).
  • The child would previously try to close every valid file descriptor, something which could cause problems if the file descriptor ulimit was set too high. We now keep track of all the file descriptors we open and only close up to that number.
  • ESI was partially broken in 2.1.0 due to a bug in the rollback of session workspace. This has been fixed.
  • Reject the authcommand rather than crash if there is no -S parameter given.
  • Align pointers in allocated objects. This will in theory make Varnish a tiny bit faster at the expense of slightly more memory usage.
  • Ensure the master process process id is updated in the shared memory log file after we go into the background.
  • HEAD requests would be converted to GET requests too early, which affected pass and pipe. This has been fixed.
  • Update the documentation to point out that the TTL is no longer taken into account to decide whether an object is cacheable or not.
  • Add support for completely obliterating an object and all variants of it. Currently, this has to be done using inline C.
  • Add experimental support for the Range header. This has to be enabled using the parameter http_range_support.
  • The critbit hasher could get into a deadlock and had a race condition. Both those have now been fixed.

varnishsizes -----------~

  • varnishsizes, which is like varnishhist, but for the length of objects, has been added..

Changes from 2.0.6 to 2.1.0

varnishd

  • Persistent storage is now experimentally supported using the persistent stevedore. It has the same command line arguments as the file stevedore.

  • obj.* is now called beresp.* in vcl_fetch, and obj.* is now read-only.

  • The regular expression engine is now PCRE instead of POSIX regular expressions.

  • req.* is now available in vcl_deliver.

  • Add saint mode where we can attempt to grace an object if we don't like the backend response for some reason.

    Related, add saintmode_threshold which is the threshold for the number of objects to be added to the trouble list before the backend is considered sick.

  • Add a new hashing method called critbit. This autoscales and should work better on large object workloads than the classic hash. Critbit has been made the default hash algorithm.

  • When closing connections, we experimented with sending RST to free up load balancers and free up threads more quickly. This caused some problems with NAT routers and so has been reverted for now.

  • Add thread that checks objects against ban list in order to prevent ban list from growing forever. Note that this needs purges to be written so they don't depend on req.*. Enabled by setting ban_lurker_sleep to a nonzero value.

  • The shared memory log file format was limited to maximum 64k simultaneous connections. This is now a 32 bit field which removes this limitation.

  • Remove obj_workspace, this is now sized automatically.

  • Rename acceptors to waiters

  • vcl_prefetch has been removed. It was never fully implemented.

  • Add support for authenticating CLI connections.

  • Add hash director that chooses which backend to use depending on req.hash.

  • Add client director that chooses which backend to use depending on the client's IP address. Note that this ignores the X-Forwarded-For header.

  • varnishd now displays a banner by default when you connect to the CLI.

  • Increase performance somewhat by moving statistics gathering into a per-worker structure that is regularly flushed to the global stats.

  • Make sure we store the header and body of object together. This may in some cases improve performance and is needed for persistence.

  • Remove client-side address accounting. It was never used for anything and presented a performance problem.

  • Add a timestamp to bans, so you can know how old they are.

  • Quite a few people got confused over the warning about not being able to lock the shared memory log into RAM, so stop warning about that.

  • Change the default CLI timeout to 10 seconds.

  • We previously forced all inserts into the cache to be GET requests. This has been changed to allow POST as well in order to be able to implement purge-on-POST semantics.

  • The CLI command stats now only lists non-zero values.

  • The CLI command stats now only lists non-zero values.

  • Use daemon(3) from libcompat on Darwin.

  • Remove vcl_discard as it causes too much complexity and never actually worked particularly well.

  • Remove vcl_timeout as it causes too much complexity and never actually worked particularly well.

  • Update the documentation so it refers to sess_workspace, not http_workspace.

  • Document the -i switch to varnishd as well as the server.identity and server.hostname VCL variables.

  • purge.hash is now deprecated and no longer shown in help listings.

  • When processing ESI, replace the five mandatory XML entities when we encounter them.

  • Add string representations of time and relative time.

  • Add locking for n_vbe_conn to make it stop underflowing.

  • When ESI-processing content, check for illegal XML character entities.

  • Varnish can now connect its CLI to a remote instance when starting up, rather than just being connected to.

  • It is no longer needed to specify the maximum number of HTTP headers to allow from backends. This is now a run-time parameter.

  • The X-Forwarded-For header is now generated by vcl_recv rather than the C code.

  • It is now possible to not send all CLI traffic to syslog.

  • It is now possible to not send all CLI traffic to syslog.

  • In the case of varnish crashing, it now outputs a identifying string with the OS, OS revision, architecture and storage parameters together with the backtrace.

  • Use exponential backoff when we run out of file descriptors or sessions.

  • Allow setting backend timeouts to zero.

  • Count uptime in the shared memory log.

  • Try to detect the case of two running varnishes with the same shmlog and storage by writing the master and child process ids to the shmlog and refusing to start if they are still running.

  • Make sure to use EOF mode when serving ESI content to HTTP/1.0 clients.

  • Make sure we close the connection if it either sends Connection: close or it is a HTTP/1.0 backend that does not send Connection: keep-alive.

  • Increase the default session workspace to 64k on 64-bit systems.

  • Make the epoll waiter use level triggering, not edge triggering as edge triggering caused problems on very busy servers.

  • Handle unforeseen client disconnections better on Solaris.

  • Make session lingering apply to new sessions, not just reused sessions.

varnishstat

  • Make use of the new uptime field in the shared memory log rather than synthesizing it from the start time.

varnishlog

  • Exit at the end of the file when started with -d.

varnishadm

  • varnishadm can now have a timeout when trying to connect to the running varnishd.
  • varnishadm now knows how to respond to the secret from a secured varnishd

Changes from 2.0.5 to 2.0.6

varnishd

  • 2.0.5 had an off-by-one error in the ESI handling causing includes to fail a large part of the time. This has now been fixed.
  • Try harder to not confuse backends when sending them backend probes. We half-closed the connection, something some backends thought meant we had dropped the connection. Stop doing so, and add the capability for specifying the expected response code.
  • In 2.0.5, session lingering was turned on. This caused statistics to not be counted often enough in some cases. This has now been fixed.
  • Avoid triggering an assert if the other end closes the connection while we are lingering and waiting for another request from them.
  • When generating backtraces, prefer the built-in backtrace function if such exists. This fixes a problem compiling 2.0.5 on Solaris.
  • Make it possible to specify the per-thread stack size. This might be useful on 32 bit systems with their limited address space.
  • Document the -C option to varnishd.

Changes from 2.0.4 to 2.0.5

varnishd

  • Handle object workspace overruns better.
  • Allow turning off ESI processing per request by using set req.esi = off.
  • Tell the kernel that we expect to use the mmap-ed file in a random fashion. On Linux, this turns off/down readahead and increases performance.
  • Make it possible to change the maximum number of HTTP headers we allow by passing --with-max-header-fields=NUM rather than changing the code.
  • Implement support for HTTP continuation lines.
  • Change how connections are closed and only use SO_LINGER for orderly connection closure. This should hopefully make worker threads less prone to hangups on network problems.
  • Handle multi-element purges correctly. Previously we ended up with parse errors when this was done from VCL.
  • Handle illegal responses from the backend better by serving a 503 page rather than panic-ing.
  • When we run into an assertion that is not true, Varnish would previously dump a little bit of information about itself. Extend that information with a backtrace. Note that this relies on the varnish binary being unstripped.
  • Add a session_max parameter that limits the maximum number of sessions we keep open before we start dropping new connections summarily.
  • Try to consume less memory when doing ESI processing by properly rolling back used workspace after processing an object. This should make it possible to turn sess_workspace quite a bit for users with ESI-heavy pages.
  • Turn on session_linger by default. Tests have shown that session_linger helps a fair bit with performance.
  • Rewrite the epoll acceptor for better performance. This should lead to both higher processing rates and maximum number of connections on Linux.
  • Add If-None-Match support, this gives significant bandwidth savings for users with compliant browsers.
  • RFC2616 specifies that ETag, Content-Location, Expires, Cache-Control and Vary should be emitted when delivering a response with the 304 response code.
  • Various fixes which makes Varnish compile and work on AIX.
  • Turn on TCP_DEFER_ACCEPT on Linux. This should make us less suspecible to denial of service attacks as well as give us slightly better performance.
  • Add an .initial property to the backend probe specification. This is the number of good probes we pretend to have seen. The default is one less than .threshold, which means the first probe will decide if we consider the backend healthy.
  • Make it possible to compare strings against other string-like objects, not just plain strings. This allows you to compare two headers, for instance.
  • When support for restart in vcl_error was added, there was no check to prevent infinte recursion. This has now been fixed.
  • Turn on purge_dups by default. This should make us consume less memory when there are many bans for the same pattern added.
  • Add a new log tag called FetchError which tries to explain why we could not fetch an object from the backend.
  • Change the default srcaddr_ttl to 0. It is not used by anything and has been removed in the development version. This will increase performance somewhat.

varnishtop

  • varnishtop did not handle variable-length log fields correctly. This is now fixed.
  • varnishtop previously did not print the name of the tag, which made it very hard to understand. We now print out the tag name.

Changes from 2.0.3 to 2.0.4

varnishd

  • Make Varnish more portable by pulling in fixes for Solaris and NetBSD.
  • Correct description of -a in the manual page.
  • Ensure we are compiling in C99 mode.
  • If error was called with a null reason, we would crash on Solaris. Make sure this no longer happens.
  • Varnish used to crash if you asked it to use a non-existent waiter. This has now been fixed.
  • Add documentation to the default VCL explaining that using Connection: close in vcl_pipe is generally a good idea.
  • Add minimal facility for dealing with TELNET option negotiation by returning WONT to DO and DONT requests.
  • If the backend is unhealthy, use a graced object if one is available.
  • Make server.hostname and server.identity available to VCL. The latter can be set with the -i parameter to varnishd.
  • Make restart available from vcl_error.
  • Previously, only the TTL of an object was considered in whether it would be marked as cacheable. This has been changed to take the grace into consideration as well.
  • Previously, if an included ESI fragment had a zero size, we would send out a zero-sized chunk which signifies end-of-transmission. We now ignore zero-sized chunks.
  • We accidentally slept for far too long when we reached the maximum number of open file descriptors. This has been corrected and accept_fd_holdoff now works correctly.
  • Previously, when ESI processing, we did not look at the full length, but stopped at the first NULL byte. We no longer do that, enabling ESI processing of binary data.

varnishtest

  • Make sure system "..." returns successfully to ensure test failures do not go unnoticed.
  • Make it possible to send NULL bytes through the testing framework.

Changes from 2.0.2 to 2.0.3

varnishd

  • Handle If-Modified-Since and ESI sub-objects better, fixing a problem where we sometimes neglected to insert included objects.
  • restart in vcl_hit is now supported.
  • Setting the TTL of an object to 0 seconds would sometimes cause it to be delivered for up to one second - epsilon. This has been corrected and we should now never deliver those objects to other clients.
  • The malloc storage backend now prints the maximum storage size, just like the file backend.
  • Various small documentation bugs have been fixed.
  • Varnish did not set a default interval for backend probes, causing it to poll the backend continuously. This has been corrected.
  • Allow "true" and "false" when setting boolean parameters, in addition to on/off, enable/disable and yes/no.
  • Default to always talking HTTP 1.1 with the backend.
  • Varnish did not make sure the file it was loading was a regular file. This could cause Varnish to crash if it was asked to load a directory or other non-regular file. We now check that the file is a regular file before loading it.
  • The binary heap used for expiry processing had scalability problems. Work around this by using stripes of a fixed size, which should make this scale better, particularly when starting up and having lots of objects.
  • When we imported the jemalloc library into the Varnish tree, it did not compile without warnings. This has now been fixed.
  • Varnish took a very long time to detect that the backend did not respond. To remedy this, we now have read timeouts in addition to the connect timeout. Both the first_byte_timeout and the between_bytes_timeout defaults to 60 seconds. The connect timeout is no longer in milliseconds, but rather in seconds.
  • Previously, the VCL to C conversion as well as the invocation of the C compiler was done in the management process. This is now done in a separate sub-process. This prevents any bugs in the VCL compiler from affecting the management process.
  • Chunked encoding headers were counted in the statistics for header bytes. They no longer are.
  • ESI processed objects were not counted in the statistics for body bytes. They now are.
  • It is now possible to adjust the maximum record length of log entries in the shmlog by tuning the shm_reclen parameter.
  • The management parameters listed in the CLI were not sorted, which made it hard to find the parameter you were looking for. They are now sorted, which should make this easier.
  • Add a new hashing type, "critbit", which uses a lock-less tree based lookup algorithm. This is experimental and should not be enabled in production environments without proper testing.
  • The session workspace had a default size of 8k. It is now 16k, which should make VCLs where many headers are processed less prone to panics.
  • We have seen that people seem to be confused as to which actions in the different VCL functions return and which ones don't. Add a new syntax return(action) to make this more explicit. The old syntax is still supported.
  • Varnish would return an error if any of the management IPs listed in the -T parameter could not be listened to. We now only return an error if none of them can be listened to.
  • In the case of the backend or client giving us too many parameters, we used to just ignore the overflowing headers. This is problematic if you end up ignoreing Content-Length, Transfer-Encoding and similar headers. We now give out a 400 error to the client if it sends us too many and 503 if we get too many from the backend.
  • We used panic if we got a too large chunked header. This behaviour has been changed into just failing the transaction.
  • Varnish now supports an extended purge method where it is possible to do purge req.http.host ~ "web1.com" && req.url ~ "\.png" and similar. See the documentation for details.
  • Under heavy load, Varnish would sometimes crash when trying to update the per-request statistics. This has now been fixed.
  • It is now possible to not save the hash string in the session and object workspace. This will save a lot of memory on sites with many small objects. Disabling the purge_hash parameter also disables the purge.hash facility.
  • Varnish now supports !~ as a "no match" regular expression matcher.
  • In some cases, you could get serialised access to "pass" objects. We now make it default to the default_ttl value; this can be overridden in vcl_fetch.
  • Varnish did not check the syntax of regsub calls properly. More checking has been added.
  • If the client closed the connection while Varnish was processing ESI elements, Varnish would crash while trying to write the object to the client. We now check if the client has closed the connection.
  • The ESI parser had a bug where it would crash if an XML comment would span storage segments. This has been fixed.

VCL Manual page --------------~

  • The documentation on how capturing parentheses work was wrong. This has been corrected.
  • Grace has now been documented.

varnishreplay

  • varnishreplay did not work correctly on Linux, due to a too small stack. This has now been fixed.

Changes from 2.0.1 to 2.0.2

varnishd

  • In high-load situations, when using ESI, varnishd would sometimes mishandle objects and crash. This has been worked around.

varnishreplay

  • varnishreplay did not work correctly on Linux, due to a too small stack. This has now been fixed.

Changes from 2.0 to 2.0.1

varnishd

  • When receiving a garbled HTTP request, varnishd would sometimes crash. This has been fixed.
  • There was an off-by-one error in the ACL compilation. Now fixed.

Red Hat spec file ----------------~

  • A typo in the spec file made the .rpm file names wrong.

Changes from 1.1.2 to 2.0

varnishd

  • Only look for sendfile on platforms where we know how to use it, which is FreeBSD for now.
  • Make it possible to adjust the shared memory log size and bump the size from 8MB to 80MB.
  • Fix up the handling of request bodies to better match what RFC2616 mandates. This makes PUT, DELETE, OPTIONS and TRACE work in addition to POST.
  • Change how backends are defined, to a constant structural defintion style. See https://www.varnish-cache.org/wiki/VclSyntaxChanges for the details.
  • Add directors, which wrap backends. Currently, there's a random director and a round-robin director.
  • Add "grace", which is for how long and object will be served, even after it has expired. To use this, both the object's and the request's grace parameter need to be set.
  • Manual pages have been updated for new VCL syntax and varnishd options.
  • Man pages and other docs have been updated.
  • The shared memory log file is now locked in memory, so it should not be paged out to disk.
  • We now handle Vary correctly, as well as Expect.
  • ESI include support is implemented.
  • Make it possible to limit how much memory the malloc uses.
  • Solaris is now supported.
  • There is now a regsuball function, which works like regsub except it replaces all occurrences of the regex, not just the first.
  • Backend and director declarations can have a .connect_timeout parameter, which tells us how long to wait for a successful connection.
  • It is now possible to select the acceptor to use by changing the acceptor parameter.
  • Backends can have probes associated with them, which can be checked with req.backend.health in VCL as well as being handled by directors which do load-balancing.
  • Support larger-than-2GB files also on 32 bit hosts. Please note that this does not mean we can support caches bigger than 2GB, it just means logfiles and similar can be bigger.
  • In some cases, we would remove the wrong header when we were stripping Content-Transfer-Encoding headers from a request. This has been fixed.
  • Backends can have a .max_connections associated with them.
  • On Linux, we need to set the dumpable bit on the child if we want core dumps. Make sure it's set.
  • Doing purge.hash() with an empty string would cause us to dump core. Fixed so we don't do that any more.
  • We ran into a problem with glibc's malloc on Linux where it seemed like it failed to ever give memory back to the OS, causing the system to swap. We have now switched to jemalloc which appears not to have this problem.
  • max_restarts was never checked, so we always ended up running out of workspace. Now, vcl_error is called when we reach max_restarts.

varnishtest

  • varnishtest is a tool to do correctness tests of varnishd. The test suite is run by using make check.

varnishtop

  • We now set the field widths dynamically based on the size of the terminal and the name of the longest field.

varnishstat

  • varnishstat -1 now displays the uptime too.

varnishncsa

  • varnishncsa now does fflush after each write. This makes tail -f work correctly, as well as avoiding broken lines in the log file.
  • It is possible to get varnishncsa to output the X-Forwarded-For instead of the client IP by passing -f to it.

Build system -----------~

  • Various sanity checks have been added to configure, it now complains about no ncurses or if SO_RCVTIMEO or SO_SNDTIMEO are non-functional. It also aborts if there's no working acceptor mechanism
  • The C compiler invocation is decided by the configure script and can now be overridden by passing VCC_CC when running configure.

Changes from 1.1.1 to 1.1.2

varnishd

  • When switching to a new VCL configuration, a race condition exists which may cause Varnish to reference a backend which no longer exists (see ticket #144). This race condition has not been entirely eliminated, but it should occur less frequently.
  • When dropping a TCP session before any requests were processed, an assertion would be triggered due to an uninitialized timestamp (see ticket #132). The timestamp is now correctly initialized.
  • Varnish will now correctly generate a Date: header for every response instead of copying the one it got from the backend (see ticket #157).
  • Comparisons in VCL which involve a non-existent string (usually a header which is not present in the request or object being processed) would cause a NULL pointer dereference; now the comparison will simply fail.
  • A bug in the VCL compiler which would cause a double-free when processing include directives has been fixed.
  • A resource leak in the worker thread management code has been fixed.
  • When connecting to a backend, Varnish will usually get the address from a cache. When the cache is refreshed, existing connections may end up with a reference to an address structure which no longer exists, resulting in a crash. This race condition has been somewhat mitigated, but not entirely eliminated (see ticket #144.)
  • Varnish will now pass the correct protocol version in pipe mode: the backend will get what the client sent, and vice versa.
  • The core of the pipe mode code has been rewritten to increase robustness and eliminate spurious error messages when either end closes the connection in a manner Varnish did not anticipate.
  • A memory leak in the backend code has been plugged.
  • When using the kqueue acceptor, if a client shuts down the request side of the connection (as many clients do after sending their final request), it was possible for the acceptor code to receive the EOF event and recycle the session while the last request was still being serviced, resulting in a assertion failure and a crash when the worker thread later tried to delete the session. This should no longer happen (see ticket #162.)
  • A mismatch between the recorded length of a cached object and the amount of data actually present in cache for that object can occasionally occur (see ticket #167.) This has been partially fixed, but may still occur for error pages generated by Varnish when a problem arises while retrieving an object from the backend.
  • Some socket-related system calls may return unexpected error codes when operating on a TCP connection that has been shut down at the other end. These error codes would previously cause assertion failures, but are now recognized as harmless conditions.

varnishhist

  • Pressing 0 though 9 while varnishhist is running will change the refresh interval to the corresponding power of two, in seconds.

varnishncsa

  • The varnishncsa tool can now daemonize and write a PID file like varnishlog, using the same command-line options. It will also reopen its output upon receipt of a SIGHUP if invoked with -w.

varnishstat

  • Pressing 0 though 9 while varnishstat is running will change the refresh interval to the corresponding power of two, in seconds.

Build system -----------~

  • Varnish's <queue.h> has been modified to avoid conflicts with <sys/queue.h> on platforms where the latter is included indirectly through system headers.
  • Several steps have been taken towards Solaris support, but this is not yet complete.
  • When configure was run without an explicit prefix, Varnish's idea of the default state directory would be garbage and a state directory would have to be specified manually with -n. This has been corrected.

Changes from 1.1 to 1.1.1

varnishd

  • The code required to allow VCL to read obj.status, which had accidentally been left out, has now been added.
  • Varnish will now always include a Connection: header in its reply to the client, to avoid possible misunderstandings.
  • A bug that triggered an assertion failure when generating synthetic error documents has been corrected.
  • A new VCL function, purge_url, provides the same functionality as the url.purge management command.
  • Previously, Varnish assumed that the response body should be sent only if the request method was GET. This was a problem for custom request methods (such as PURGE), so the logic has been changed to always send the response body except in the specific case of a HEAD request.
  • Changes to run-time parameters are now correctly propagated to the child process.
  • Due to the way run-time parameters are initialized at startup, varnishd previously required the nobody user and the nogroup group to exist even if a different user and group were specified on the command line. This has been corrected.
  • Under certain conditions, the VCL compiler would carry on after a syntax error instead of exiting after reporting the error. This has been corrected.
  • The manner in which the hash string is assembled has been modified to reduce memory usage and memory-to-memory copying.
  • Before calling vcl_miss, Varnish assembles a tentative request object for the backend request which will usually follow. This object would be leaked if vcl_miss returned anything else than fetch. This has been corrected.
  • The code necessary to handle an error return from vcl_fetch and vcl_deliver had inadvertantly been left out. This has been corrected.
  • Varnish no longer prints a spurious "child died" message (the result of reaping the compiler process) after compiling a new VCL configuration.
  • Under some circumstances, due to an error in the workspace management code, Varnish would lose the "tail" of a request, i.e. the part of the request that has been received from the client but not yet processed. The most obvious symptom of this was that POST requests would work with some browsers but not others, depending on details of the browser's HTTP implementation. This has been corrected.
  • On some platforms, due to incorrect assumptions in the CLI code, the management process would crash while processing commands received over the management port. This has been corrected.

Build system -----------~

  • The top-level Makefile will now honor $DESTDIR when creating the state directory.
  • The Debian and RedHat packages are now split into three (main / lib / devel) as is customary.
  • A number of compile-time and run-time portability issues have been addressed.
  • The autogen.sh script had workarounds for problems with the GNU autotools on FreeBSD; these are no longer needed and have been removed.
  • The libcompat library has been renamed to libvarnishcompat and is now dynamic rather than static. This simplifies the build process and resolves an issue with the Mac OS X linker.

Changes from 1.0.4 to 1.1

varnishd

  • Readability of the C source code generated from VCL code has been improved.
  • Equality (==) and inequality (!=) operators have been implemented for IP addresses (which previously could only be compared using ACLs).
  • The address of the listening socket on which the client connection was received is now available to VCL as the server.ip variable.
  • Each object's hash key is now computed based on a string which is available to VCL as req.hash. A VCL hook named vcl_hash has been added to allow VCL scripts to control hash generation (for instance, whether or not to include the value of the Host: header in the hash).
  • The setup code for listening sockets has been modified to detect and handle situations where a host name resolves to multiple IP addresses. It will now attempt to bind to each IP address separately, and report a failure only if none of them worked.
  • Network or protocol errors that occur while retrieving an object from a backend server now result in a synthetic error page being inserted into the cache with a 30-second TTL. This should help avoid driving an overburdened backend server into the ground by repeatedly requesting the same object.
  • The child process will now drop root privileges immediately upon startup. The user and group to use are specified with the user and group run-time parameters, which default to nobody and nogroup, respectively. Other changes have been made in an effort to increase the isolation between parent and child, and reduce the impact of a compromise of the child process.
  • Objects which are received from the backend with a Vary: header are now stored separately according to the values of the headers specified in Vary:. This allows Varnish to correctly cache e.g. compressed and uncompressed versions of the same object.
  • Each Varnish instance now has a name, which by default is the host name of the machine it runs on, but can be any string that would be valid as a relative or absolute directory name. It is used to construct the name of a directory in which the server state as well as all temporary files are stored. This makes it possible to run multiple Varnish instances on the same machine without conflict.
  • When invoked with the -C option, varnishd will now not just translate the VCL code to C, but also compile the C code and attempt to load the resulting shared object.
  • Attempts by VCL code to reference a variable outside its scope or to assign a value to a read-only variable will now result in compile-time rather than run-time errors.
  • The new command-line option -F will make varnishd run in the foreground, without enabling debugging.
  • New VCL variables have been introduced to allow inspection and manipulation of the request sent to the backend (bereq.request, bereq.url, bereq.proto and bereq.http) and the response to the client (resp.proto, resp.status, resp.response and resp.http).
  • Statistics from the storage code (including the amount of data and free space in the cache) are now available to varnishstat and other statistics-gathering tools.
  • Objects are now kept on an LRU list which is kept loosely up-to-date (to within a few seconds). When cache runs out, the objects at the tail end of the LRU list are discarded one by one until there is enough space for the freshly requested object(s). A VCL hook, vcl_discard, is allowed to inspect each object and determine its fate by returning either keep or discard.
  • A new VCL hook, vcl_deliver, provides a chance to adjust the response before it is sent to the client.
  • A new management command, vcl.show, displays the VCL source code of any loaded configuration.
  • A new VCL variable, now, provides VCL scripts with the current time in seconds since the epoch.
  • A new VCL variable, obj.lastuse, reflects the time in seconds since the object in question was last used.
  • VCL scripts can now add an HTTP header (or modify the value of an existing one) by assigning a value to the corresponding variable, and strip an HTTP header by using the remove keyword.
  • VCL scripts can now modify the HTTP status code of cached objects (obj.status) and responses (resp.status)
  • Numeric and other non-textual variables in VCL can now be assigned to textual variables; they will be converted as needed.
  • VCL scripts can now apply regular expression substitutions to textual variables using the regsub function.
  • A new management command, status, returns the state of the child.
  • Varnish will now build and run on Mac OS X.

varnishadm

  • This is a new utility which sends a single command to a Varnish server's management port and prints the result to stdout, greatly simplifying the use of the management port from scripts.

varnishhist

  • The user interface has been greatly improved; the histogram will be automatically rescaled and redrawn when the window size changes, and it is updated regularly rather than at a rate dependent on the amount of log data gathered. In addition, the name of the Varnish instance being watched is displayed in the upper right corner.

varnishncsa

  • In addition to client traffic, varnishncsa can now also process log data from backend traffic.
  • A bug that would cause varnishncsa to segfault when it encountered an empty HTTP header in the log file has been fixed.

varnishreplay

  • This new utility will attempt to recreate the HTTP traffic which resulted in the raw Varnish log data which it is fed.

varnishstat

  • Don't print lifetime averages when it doesn't make any sense, for instance, there is no point in dividing the amount in bytes of free cache space by the lifetime in seconds of the varnishd process.
  • The user interface has been greatly improved; varnishstat will no longer print more than fits in the terminal, and will respond correctly to window resize events. The output produced in one-shot mode has been modified to include symbolic names for each entry. In addition, the name of the Varnish instance being watched is displayed in the upper right corner in curses mode.

varnishtop

  • The user interface has been greatly improved; varnishtop will now respond correctly to window resize events, and one-shot mode (-1) actually works. In addition, the name of the Varnish instance being watched is displayed in the upper right corner in curses mode.

Changes from 1.0.3 to 1.0.4

varnishd

  • The request workflow has been redesigned to simplify request processing and eliminate code duplication. All codepaths which need to speak HTTP now share a single implementation of the protocol. Some new VCL hooks have been added, though they aren't much use yet. The only real user-visible change should be that Varnish now handles persistent backend connections correctly (see ticket #56).

  • Support for multiple listen addresses has been added.

  • An "include" facility has been added to VCL, allowing VCL code to pull in code fragments from multiple files.

  • Multiple definitions of the same VCL function are now concatenated into one in the order in which they appear in the source. This simplifies the mechanism for falling back to the built-in default for cases which aren't handled in custom code, and facilitates modularization.

  • The code used to format management command arguments before passing them on to the child process would underestimate the amount of space needed to hold each argument once quotes and special characters were properly escaped, resulting in a buffer overflow. This has been corrected.

  • The VCL compiler has been overhauled. Several memory leaks have been plugged, and error detection and reporting has been improved throughout. Parts of the compiler have been refactored to simplify future extension of the language.

  • A bug in the VCL compiler which resulted in incorrect parsing of the decrement (-=) operator has been fixed.

  • A new -C command-line option has been added which causes varnishd to compile the VCL code (either from a file specified with -f or the built-in default), print the resulting C code and exit.

  • When processing a backend response using chunked encoding, if a chunk header crosses a read buffer boundary, read additional bytes from the backend connection until the chunk header is complete.

  • A new ping_interval run-time parameter controls how often the management process checks that the worker process is alive.

  • A bug which would cause the worker process to dereference a NULL pointer and crash if the backend did not respond has been fixed.

  • In some cases, such as when they are used by AJAX applications to circumvent Internet Explorer's over-eager disk cache, it may be desirable to cache POST requests. However, the code path responsible for delivering objects from cache would only transmit the response body when replying to a GET request. This has been extended to also apply to POST.

    This should be revisited at a later date to allow VCL code to control whether the body is delivered.

  • Varnish now respects Cache-control: s-maxage, and prefers it to Cache-control: max-age if both are present.

    This should be revisited at a later date to allow VCL code to control which headers are used and how they are interpreted.

  • When loading a new VCL script, the management process will now load the compiled object to verify that it links correctly before instructing the worker process to load it.

  • A new -P command-line options has been added which causes varnishd to create a PID file.

  • The sendfile_threshold run-time parameter's default value has been set to infinity after a variety of sendfile()-related bugs were discovered on several platforms.

varnishlog

  • When grouping log entries by request, varnishlog attempts to collapse the log entry for a call to a VCL function with the log entry for the corresponding return from VCL. When two VCL calls were made in succession, varnishlog would incorrectly omit the newline between the two calls (see ticket #95).
  • New -D and -P command-line options have been added to daemonize and create a pidfile, respectively.
  • The flag that is raised upon reception of a SIGHUP has been marked volatile so it will not be optimized away by the compiler.

varnishncsa

  • The formatting callback has been largely rewritten for clarity, robustness and efficiency.

    If a request included a Host: header, construct and output an absolute URL. This makes varnishncsa output from servers which handle multiple virtual hosts far more useful.

  • The flag that is raised upon reception of a SIGHUP has been marked volatile so it will not be optimized away by the compiler.

Documentation

  • The documentation, especially the VCL documentation, has been greatly extended and improved.

Build system

  • The name and location of the curses or ncurses library is now correctly detected by the configure script instead of being hardcoded into affected Makefiles. This allows Varnish to build correctly on a wider range of platforms.
  • Compatibility shims for clock_gettime() are now correctly applied where needed, allowing Varnish to build on MacOS X.
  • The autogen.sh script will now correctly detect and warn about automake versions which are known not to work correctly.

®Varnish Software, Wallingatan 12, 111 60 Stockholm, Organization nr. 556805-6203