SearchVarnish Enterprise
The utils vmod is a VMOD for useful functions that don’t require their own VMOD.
STRING newline()
Return a string consisting of a newline escape sequence, \n.
Arguments: None
Type: Function
Returns: String
STRING time_format(STRING format, BOOL local_time = 0, [TIME time])
Format the time according to format. This is an interface for strftime.
Arguments:
format accepts type STRING
local_time accepts type BOOL with a default value of 0 optional
time accepts type TIME
Type: Function
Returns: String
VOID fast_304()
Perform a fast 304 insert. This revalidates the stale objects instead of performing the HTTP standard’s complicated rules (see https://httpwg.org/specs/rfc7232.html#status.304 and https://httpwg.org/specs/rfc7234.html#freshening.responses) for calculating new headers.
When fast_304() is not used, the standard requires Varnish to create a new object with new headers, where most use cases are better off with simply revalidating the existing objects.
A fast_304 will conduct a simple book update, whereas a normal 304 will perform a full copy of the object payload. For big objects, this represents a potentially very significant reduction in IO cost and it is strongly recommended to use fast_304 when using MSE.
Arguments: None
Type: Function
Returns: None
Restricted to: vcl_backend_response
BOOL is_fast_304()
Was a fast 304 performed.
Arguments: None
Type: Function
Returns: Bool
Restricted to: vcl_backend_response
BOOL was_max_conn()
This function can only be called in sub vcl_backend_error. Returns true if the reason that we are in sub vcl_backend_error is, in fact, that the selected backend’s maximum connections was reached when a new connection was requested.
Arguments: None
Type: Function
Returns: Bool
Restricted to: vcl_backend_error
STRING vcl_name()
The name of the current VCL.
Arguments: None
Type: Function
Returns: String
OBJECT dyn_probe([STRING url], [STRING request], [BOOL tcponly], [INT expected_response], [DURATION timeout], [DURATION interval], [INT initial], [INT window], [INT threshold], [BOOL expect_close])
Create a dynamic probe in vcl_init. The advantage of creating a probe this
way is that any of the probe attributes can be determined at VCL load time, for
example from environment variables. If not set, the attributes have the same
default values as regular probes, and are subject to the same restrictions.
The configured probe can be retrieved with .probe(), and can optionally be
further customized at request time with .probe_custom.
Arguments:
url accepts type STRING
request accepts type STRING
tcponly accepts type BOOL
expected_response accepts type INT
timeout accepts type DURATION
interval accepts type DURATION
initial accepts type INT
window accepts type INT
threshold accepts type INT
expect_close accepts type BOOL
Type: Object
Returns: Object.
PROBE .probe()
Get the probe specified during the creation of the object.
Example
import utils;
import goto;
sub vcl_init {
# Get the probe URL from an environment variable
new dyn_probe = utils.dyn_probe(std.getenv("PROBE_URL"));
# Create a dynamic backend with the above probe
new goto_dir = goto.dns_director("foo.example.com", probe = dyn_probe.probe());
}
Arguments: None
Type: Method
Returns: Probe
PROBE .probe_custom(STRING url, [STRING header1], [STRING header2], [STRING header3], [STRING header4], [STRING header5], [STRING header6], [STRING header7], [STRING header8], [STRING header9], [STRING header10])
Create or reuse a custom probe. The probe will inherit the attributes set at the
creation of this object, with the exception of the url, request, and
tcponly. The probe request will instead be generated based on the arguments
given to this method.
The memory allocated to custom probes will be held for the lifetime of the VCL they are created in, and is only released once the VCL is discarded. The dyn_probe object has an internal cache to reuse custom probe definitions, so calling this method multiple times with the same arguments will not consume extra memory. Each unique custom probe definition will consume about 120 bytes plus the length of the generated request string.
Example
import utils;
import goto;
sub vcl_init {
# Create a dynamic probe
new dyn_probe = utils.dyn_probe("/health");
}
sub vcl_backend_fetch {
# Create a custom probe with the same Host header as the backend fetch
set bereq.backend = goto.dns_backend(bereq.http.Host, probe = dyn_probe.probe_custom("/health", "Host: " + bereq.http.Host, "Connection: close"));
}
Arguments:
url accepts type STRING
header1 accepts type STRING
header2 accepts type STRING
header3 accepts type STRING
header4 accepts type STRING
header5 accepts type STRING
header6 accepts type STRING
header7 accepts type STRING
header8 accepts type STRING
header9 accepts type STRING
header10 accepts type STRING
Type: Method
Returns: Probe
INT mod(INT a, INT b)
Return the result of a mod b, or triggers an error if b is 0. Be
aware that if a or b is negative, the output may be negative.
Arguments:
a accepts type INT
b accepts type INT
Type: Function
Returns: Int
REAL fast_random(REAL lo, REAL hi)
Returns a pseudo random real number between lo and hi. This is a drop-in replacement for std.random(), but with a much higher focus on speed. Like std.random(), this should not be used for cryptographical applications.
set beresp.http.random-number = utils.fast_random(1, 100);
Arguments:
lo accepts type REAL
hi accepts type REAL
Type: Function
Returns: Real
INT fast_random_int(INT m)
When m is a positive number, returns a quasi random integer between 0 and m-1 (inclusive). If m is zero or negative, a random INT is returned, which can be either positive or negative.
set beresp.http.random-int = 1 + utils.fast_random_int(3);
After the above, the header random-int will be “1”, “2” or “3”.
Arguments:
m accepts type INTType: Function
Returns: Int
VOID vcp_vfp_reset()
Clear out any configured VDP VFPs
Arguments: None
Type: Function
Returns: None
Restricted to: backend
INT bitwise_and(INT a, INT b)
Performs bitwise and operation.
Arguments:
a accepts type INT
b accepts type INT
Type: Function
Returns: Int
INT bitwise_or(INT a, INT b)
Performs bitwise or operation.
Arguments:
a accepts type INT
b accepts type INT
Type: Function
Returns: Int
INT bitwise_xor(INT a, INT b)
Performs bitwise xor operation.
Arguments:
a accepts type INT
b accepts type INT
Type: Function
Returns: Int
STRING bytes2string(BYTES b, ENUM {B, KB, MB, GB, TB} unit = B)
Converts the bytes b to string. unit determines the unit the converted string should be.
set req.http.size = utils.bytes2string(10000KB, GB);
Arguments:
b accepts type BYTES
unit is an ENUM that accepts values of B, KB, MB, GB, and TB with a default value of B optional
Type: Function
Returns: String
BACKEND resolve_backend([BACKEND be])
Resolve the effective backend from bereq.backend, or the supplied
argument be if present. Only directors can resolve to a different
backend, and only from a backend subroutine.
Arguments:
be accepts type BACKENDType: Function
Returns: Backend
Restricted to: backend, vcl_pipe, vcl_connect
INT backend_misses()
How many times an uncacheable object has been looked up within the ttl. Can only be called in backend subroutines.
Insert an object on the second miss. This reduces expensive cache churn on long tail content.
import utils;
sub vcl_backend_response {
if (!utils.waitinglist() && utils.backend_misses() == 0) {
set beresp.uncacheable = true;
set beresp.ttl = 24h;
}
}
Arguments: None
Type: Function
Returns: Int
Restricted to: backend
BOOL has_waited()
Whether or not the request spent time on the waiting list.
Arguments: None
Type: Function
Returns: Bool
Restricted to: client
BOOL waitinglist()
Return true if there are clients on this object’s waitinglist.
Arguments: None
Type: Function
Returns: Bool
Restricted to: backend
VOID hash_ignore_vary(BOOL flag = 1)
Set the flag to disable vary header checks during lookup. When the flag is omitted, variants are ignored. Proceed with caution, ignoring vary headers could lead to inconsistent deliveries and depending on how they are used by the origin server, security vulnerabilities. When in doubt, don’t use it.
DEPRECATED: use the req.hash_ignore_vary flag directly.
Arguments:
flag accepts type BOOL with a default value of 1 optional
Type: Function
Returns: None
Restricted to: client
VOID base64_encode(ENUM {STD, URL} alphabet = STD, BOOL pad = 1, BYTES buffer_size = 32768)
Arguments:
pad accepts type BOOL with a default value of 1 optional
buffer_size accepts type BYTES with a default value of 32768 optional
alphabet is an ENUM that accepts values of STD, and URL with a default value of STD optional
Type: Function
Returns: None
Restricted to: vcl_backend_response
INT cpu_id()
Returns the current CPU id. For older operating systems not running
on an x86_64 CPU where the C lib does not provide getcpu, this function will return 0.
Arguments: None
Type: Function
Returns: Int
INT numa_node_id()
Returns the current NUMA node id. If NUMA is not present, it will always return zero.
For older operating systems not running on an x86_64 CPU where the C lib does not
provide getcpu, this function will return 0.
Arguments: None
Type: Function
Returns: Int
BOOL force_fresh(BOOL flag = 1)
Force the subsequent backend fetch to happen on a fresh connection,
not on a reused backend connection. This function can only be called
from a backend context (ie. sub vcl_backend_*), for example from sub vcl_backend_fetch.
Setting the flag to 0 turns off the flag, which allows you to take back the previous decision about forcing a fresh connection on the next retry.
Arguments:
flag accepts type BOOL with a default value of 1 optional
Type: Function
Returns: Bool
Restricted to: backend
VOID set_pipe_timeout(DURATION timeout)
Set the pipe idle timeout. The timeout will be set for the current session only, and will silently fail if the session is not available in the current VCL sub. This function can be used for CONNECT requests too.
Arguments:
timeout accepts type DURATIONType: Function
Returns: None
VOID apply_cache_headers()
Parse the backend response headers and set beresp.ttl,
beresp.grace and beresp.keep according to them. Must be called
from vcl_backend_response.
beresp.ttl will be set to a value from one of the following HTTP
header fields, listed here in order of preference:
If neither of the above HTTP header fields are present, the
default_ttl configuration applies.
beresp.grace will have its value configured from Cache-Control: stale-while-revalidate. If that is not present the default_grace
configuration will be applied.
beresp.keep will have its default_keep configuration assigned.
If the response has a nonzero Age header, the object insertion
timestamp will be adjusted accordingly. The consequence of this is
that a configured TTL will effectively be shortened by that same
amount.
Arguments: None
Type: Function
Returns: None
Restricted to: vcl_backend_response, vcl_backend_error
INT hex2integer(STRING hex, INT fallback)
Converts the hexadecimal number hex to an integer. The hexadecimal number must
start with 0x and is case insensitive. If conversion fails, fallback will
be returned.
set req.http.integer = utils.hex2integer("0x6089F", 0);
set req.http.integer = utils.hex2integer("0x3bf8a9", 0);
Arguments:
hex accepts type STRING
fallback accepts type INT
Type: Function
Returns: Int
STRING lookup_file(STRING file)
Searches for file in the vcl_path used at VCL load time and returns its
absolute path.
Arguments:
file accepts type STRINGType: Function
Returns: String
VOID http_range_support(BOOL flag)
Overrides the parameter http_range_support on a per-request basis.
Arguments:
flag accepts type BOOLType: Function
Returns: None
Restricted to: client
IP backend_ip(BACKEND be, [BOOL prefer_ipv6], [IP fallback])
Returns the IP address of a backend. For dual-address backends (backends with
both an IPv4 address and an IPv6 address), the prefer_ipv6 option can be
used to select which address to return. If the prefer_ipv6 option is not
set, the global prefer_ipv6 cache parameter is used instead. If the backend
has neither an IPv4 nor an IPv6 address, fallback is returned instead.
Arguments:
be accepts type BACKEND
prefer_ipv6 accepts type BOOL
fallback accepts type IP
Type: Function
Returns: Ip
BYTES estimated_content_length()
Returns the expected Content-Length, calculated by taking into account a client Range header if present.
This will not work if the client response is delivered using chunked transfer encoding. For these cases, the function will return a value of -1.
If VCL tampers with req.http.Range after calling this function, the recorded result will be wrong.
Arguments: None
Type: Function
Returns: Bytes
Restricted to: vcl_deliver
DURATION backend_ttfb()
Returns an estimate of time to first byte for the backend fetch,
equivalent to the difference between the VSL timestamp records
Fetch and Beresp.
In absence of errors, the estimate also includes the time it takes for Varnish to
sub vcl_backend_response populate the beresp
structure.In practice, most of the time spent will be waiting for the backend to
respond. When bytes first arrive from the backend, they almost always
represent contain a full response, not counting the response body,
allowing sub vcl_backend_response to start immediately.
If an error occurs, and sub vcl_backend_error is called as a result
of the error, the duration returned from this function is the time
taken from the fetch was initiated to the error occured. This can be
practically zero seconds if no backend was available, to an amount of
time which depends on timeouts.
Arguments: None
Type: Function
Returns: Duration
Restricted to: vcl_backend_response, vcl_backend_error
The utils VMOD is available in Varnish Enterprise version 6.0.3r7 and later.