SearchVarnish Controller
gRPC (Google Remote Procedure Call) is an open source way of communicating between servers in a client/server setting.
The Varnish Controller Router has implemented gRPC with Flatbuffers. Flatbuffers is an open source, lightweight, efficient and cross platform serialization library.
A schema has been created using Flatbuffer’s IDL (Interface Definition Language). The schema defines the available functions and data types that are passed between the client and server via gRPC.
The router acts as a client and implements this schema in Go. The server side can implement the schema in almost any language.
The router has a common set of routing rules (LeastUtilized, History, Random, External etc.) but there are cases where the routing decision should be based on external factors such as database lookups, BGP announcements etc. To cover all these different scenarios, an external plugin via gRPC can be implemented to tackle these cases.
The gRPC service is handed the information related to the request. Such as HTTP headers, client IP, available endpoints etc. Then the service can decide which endpoint to route to, based on this information together with external data.
The Flatbuffer schema is used to generate code for specific languages. Download flatc and generate code for Go like this:
flatc --go --grpc router.fbs
Where router.fbs is the schema displayed below. This will generate a code stub to be used with Go and gRPC. This is used in an example later in this chapter.
The schema consists of two functions. One function called Check(...) for performing health checks. It aligns with the Protobuff
example. This is not used by the router itself. It enables the possibility to perform external health checks towards the service.
The function used by the router is the other function called SelectEndpoint(...). This function is called upon by the router with the EndpointRequest data structure.
This structure contains information related to the client request.
The response back from the service is an EndpointResponse structure. This should either contain an endpoint_id corresponding to one of the endpoints in the EndpointRequest
or a CustomEndpoint. If there are no healthy endpoints, the EndpointRequest.endpoints is empty.
If the gRPC service doesn’t find an endpoint that is a good fit, an empty structure should be returned, meaning endpoint_id = 0 and no CustomEndpoint defined.
Endpoint IPv4 and IPv6 address for an Endpoint is the same as configured for the agents. So it could be a comma separated list of IP addresses that has to be parsed by the gRPC service.
Note that there are 3 different ways to return data from the gRPC service.
endpoint_id=<id> (where <id> is the ID of the endpoint). The routing IP’s and/or URL will be taken from the configured endpoint (Varnish Controller Agent).namespace router;
rpc_service RouterService {
// Health-check corresponding to grpc/proto3 style
// https://github.com/grpc/grpc/blob/master/doc/health-checking.md
Check(HealthCheckRequest):HealthCheckResponse(streaming: "none");
// The method used to return an endpoint to route to.
SelectEndpoint(EndpointRequest):EndpointResponse(streaming: "none");
}
enum ServingStatus: byte {
UNKNOWN = 0,
SERVING = 1,
NOT_SERVING = 2,
}
table HealthCheckResponse {
status:ServingStatus = 1;
}
table HealthCheckRequest {
service:string;
}
table EndpointRequest {
req_type:string;
domain:string;
client_ip:string;
headers:[Header];
method:string;
uri:string;
endpoints:[Endpoint];
reject:bool; // version 5.1+
}
table Endpoint {
id:int64;
name:string;
mbps:float64;
mbps_max:float64;
ipv4:string;
ipv6:string;
tags:[Tag]; // version 5.1+
cname:string; // version 5.1+
}
table EndpointResponse {
endpoint_id:int64;
custom_endpoint:CustomEndpoint;
}
table Header {
key:string;
value:string;
}
table CustomEndpoint {
name:string;
url:string;
ipv4:string;
ipv6:string;
cname:string; // version 5.1+
status_code:int64; // version 5.1+
body:string; // version 5.1+
headers:[Header]; // version 5.1+
}
// version 5.1+
table Tag {
id:int64;
name:string;
}
This is an example of implementing a simple gRPC service using Golang. It implements both functions, but only the SelectEndpoint is required, which is
the one the router will call. Note the import named rfb should point out the output from flatc --go --grpc router.fbs command. A big part of the
SelectEndpoint function below is for debugging purposes and not required. But it gives an idea how to access the different attributes of the data structures.
The below example is for version 5.1 of Varnish Controller. Earlier versions lacks CNAME, endpoint tags, custom headers and Reject. Note that the tags for endpoints are tags accessed by the VCLGroup’s organization and may not be part of the VCLGroups deployment configurations. It may also not reflect all tags that is configured on an agent as there could be tags set by different organizations that the VCLGroup do not have access to see.
package main
import (
"fmt"
"log"
"math/rand"
"net"
"os"
context "golang.org/x/net/context"
fbs "github.com/google/flatbuffers/go"
"google.golang.org/grpc"
// This is the generated code from the schema (flatc output)
rfb "varnish/router"
)
type server struct {
rfb.UnimplementedRouterServiceServer
}
// Check responds to health checks, (Not required)
func (s *server) Check(context context.Context, in *rfb.HealthCheckRequest) (*fbs.Builder, error) {
b := fbs.NewBuilder(0)
rfb.HealthCheckResponseStart(b)
rfb.HealthCheckResponseAddStatus(b, rfb.ServingStatusSERVING)
b.Finish(rfb.HealthCheckResponseEnd(b))
return b, nil
}
// SelectEndpoint is selecting either an existing endpoint or returns a custom. (Required)
func (s *server) SelectEndpoint(context context.Context, in *rfb.EndpointRequest) (*fbs.Builder, error) {
// Print incoming EndpointRequest for debug purpose
fmt.Printf("========================================\n")
fmt.Printf("ClientIP: %v\n", string(in.ClientIp()))
fmt.Printf("Domain: %v\n", string(in.Domain()))
fmt.Printf("Type: %v\n", string(in.ReqType()))
fmt.Printf("URI: %v\n", string(in.Uri()))
fmt.Printf("Headers:\n")
for i := 0; i < in.HeadersLength(); i++ {
h := rfb.Header{}
in.Headers(&h, i)
fmt.Printf(" - %s: %s\n", string(h.Key()), string(h.Value()))
}
fmt.Printf("Endpoints:\n")
for i := 0; i < in.EndpointsLength(); i++ {
ep := rfb.Endpoint{}
in.Endpoints(&ep, i)
fmt.Printf("> ID: %#v\n", ep.Id())
fmt.Printf(" - Name: %v\n", string(ep.Name()))
fmt.Printf(" - Mbps: %v\n", ep.Mbps())
fmt.Printf(" - Mbps(Max): %v\n", ep.MbpsMax())
fmt.Printf(" - IPv4: %v\n", string(ep.Ipv4()))
fmt.Printf(" - IPv6: %v\n", string(ep.Ipv6()))
fmt.Printf(" - CNAME: %#v\n", string(ep.Cname()))
fmt.Printf(" - Tags:\n")
for n := 0; n < ep.TagsLength(); n++ {
tag := rfb.Tag{}
ep.Tags(&tag, n)
fmt.Printf(" ID: %d Name: %s\n", tag.Id(), string(tag.Name()))
}
}
fmt.Printf("========================================\n\n")
// Create the response
b := fbs.NewBuilder(0)
epID := int64(0)
var cust fbs.UOffsetT
// If it's a reject request, we just reject w/o checking any condition.
if in.Reject() {
headersm := make(map[string]string, 1)
headersm["test"] = "value"
headers := []fbs.UOffsetT{}
for k, v := range headersm {
key := b.CreateString(string(k))
value := b.CreateString(string(v))
rfb.HeaderStart(b)
rfb.HeaderAddKey(b, key)
rfb.HeaderAddValue(b, value)
headers = append(headers, rfb.HeaderEnd(b))
}
rfb.CustomEndpointStartHeadersVector(b, len(headers))
for _, h := range headers {
b.PrependUOffsetT(h)
}
headersVec := b.EndVector(len(headers))
name := b.CreateString("reject")
body := b.CreateString("<html><body><h1>You are rejected.</h1></body></html>")
url := b.CreateString("http://example.com")
ipv4 := b.CreateString("127.0.0.1")
ipv6 := b.CreateString("::1")
// Randomly set a cname or not.
cn := ""
if rand.Float32() > 0.5 {
cn = "test.example.com"
}
cname := b.CreateString(cn)
// Random status code
statusCode := int64(200 + rand.Intn(3))
rfb.CustomEndpointStart(b)
rfb.CustomEndpointAddName(b, name)
rfb.CustomEndpointAddUrl(b, url)
rfb.CustomEndpointAddIpv4(b, ipv4)
rfb.CustomEndpointAddIpv6(b, ipv6)
rfb.CustomEndpointAddCname(b, cname)
rfb.CustomEndpointAddStatusCode(b, statusCode)
rfb.CustomEndpointAddBody(b, body)
rfb.CustomEndpointAddHeaders(b, headersVec)
cust = rfb.CustomEndpointEnd(b)
} else {
// If it's not a reject request, respond with either an
// given endpoint or a custom one.
if in.EndpointsLength() > 0 {
// select a random endpoint of the received ones
id := rand.Intn(in.EndpointsLength())
ep := rfb.Endpoint{}
in.Endpoints(&ep, id)
epID = ep.Id()
} else {
// Or create a custom endpoint
name := b.CreateString("myEndpoint")
url := b.CreateString("http://varni.sh/")
ipv4 := b.CreateString("127.0.0.1")
ipv6 := b.CreateString("::1")
rfb.CustomEndpointStart(b)
rfb.CustomEndpointAddName(b, name)
rfb.CustomEndpointAddUrl(b, url)
rfb.CustomEndpointAddIpv4(b, ipv4)
rfb.CustomEndpointAddIpv6(b, ipv6)
cust = rfb.CustomEndpointEnd(b)
}
}
rfb.EndpointResponseStart(b)
// To return empty result (no decision), comment out the 2 lines below.
// Note: Remove the above code where we define 'epID' and 'cust' to make it compile.
rfb.EndpointResponseAddEndpointId(b, epID) // This line...
rfb.EndpointResponseAddCustomEndpoint(b, cust) // and this line...
b.Finish(rfb.EndpointResponseEnd(b))
return b, nil
}
func main() {
socket := "/tmp/grpc.sock"
if _, err := os.Stat(socket); err == nil {
os.Remove(socket)
}
//l, err := net.Listen("tcp", "localhost:1234")
l, err := net.Listen("unix", socket)
if err != nil {
log.Fatalf("Failed to listen: %v", err)
}
ser := grpc.NewServer(grpc.CustomCodec(fbs.FlatbuffersCodec{}))
fmt.Printf("Starting gRPC server, listening on: %s\n", socket)
rfb.RegisterRouterServiceServer(ser, &server{})
if err := ser.Serve(l); err != nil {
log.Fatalf("Failed to serve: %v", err)
}
}
For the above Golang example, follow the steps below to build your service application.
# Create a new directory for your application
mkdir myservice
cd myservice
go mod init myservice
# Create 'router.fbs' with the above schema
# then generate the code:
flatc --go --grpc router.fbs
cd router
go mod init router
go mod tidy
cd ..
# Create 'server.go' with the code from the example.
nano server.go
# make sure to update path to current path for the import:
# rfb "varnish/router"
go mod edit -replace varnish/router=./router
go get varnish/router
# Then tidy modules and build the app.
go mod tidy
go build .
# Test to run the app:
./myservice
# You should be seeing:
Starting gRPC server, listening on: /tmp/grpc.sock
To run the service in systemd, you can create the below systemd file. Replace the /tmp/myservice with
the directory of your application built from previous step.
Add the below content to /etc/systemd/system/mygrpc.service (use sudo):
[Unit]
Description=Varnish Controller gRPC Service
ConditionPathExists=/tmp/myservice
After=network.target
[Service]
Type=simple
Restart=always
RestartSec=10
WorkingDirectory=/
ExecStart=/tmp/myservice/myservice
[Install]
WantedBy=multi-user.target
Set suitable permissions to the file:
sudo chmod 664 /etc/systemd/system/mygrpc.service
Then load it into systemd.
# Reload to update systemd with the new service
sudo systemctl daemon-reload
# Start on boot
sudo systemctl enable mygrpc
# Start the service
sudo systemctl start mygrpc
# Check status with:
sudo systemctl status mygrpc
# Or
journalctl -u mygrpc
# Stop the service
sudo systemctl stop mygrpc
Either configure the gRPC routing plugin in the web UI or via the vcli. Example configuration via the vcli can be seen below:
# Create the gRPC plugin
# Expects both gRPC service and router to run on same server.
vcli pgrpc add test --uri unix:/tmp/grpc.sock --verify-tls=false
# Assign the previously created gRPC to a plugin (assumes gRPC ID = 1)
vcli rps add test --type grpc --id 1
# Now the plugin can be used in our routing rule (also add random as an example)
vcli rr add test --lookup-order=plugin:1,random --debug-headers=true
# Multiple plugins can be added to the same routing rule such as this.
# vcli rr add test --lookup-order=plugin:2,plugin:1 --debug-headers=true
# Assign the routing rule to your VCLGroup
# Assumes VCLGroupID = 1 and RoutingRule ID = 1
vcli vg update 1 --rr 1
curl can be used to verify that the plugin is being used. Look for the RouterType and Trace to see if the plugin was used for the redirect.
Note that the --debug-headers=true needs to be enabled for the RoutingRules to show this output in the redirect response from the router.
# Issue the request towards the router and with the deployed domain.
curl -v -s http://my.router.domain.tld/ -H "Host: mydomain.tld"
...
< X-Router-RouteType: plugin:1
< X-Router-Trace: [plugin:1]
...
Issues with connection or requests towards the gRPC service can be seen with either apilogs or via the gRPC list.
# See all errors for this type of error
vcli apilogs ls -fkey_type=plugingrpc -fseverity=3
# See last reported errors
vcli pgrpc ls -v
When adding a new gRPC plugin, the router will not take use of the plugin until it has a connection towards it. That means that the first client requests might not use the plugin as it is not yet connected. The reason for this is to avoid stacking up a lot of client requests until the connection is up and the gRPC service would receive all requests at once.
When updating a plugin configuration in the controller, the connection will be kept as long as the plugin’s URI isn’t changed. If a new plugin should replace the old one it is advised to add the new plugin to the lookup order while keeping the old one running until the connection is up towards the new plugin. Then remove the old plugin from the lookup order.
Example:
# Assumes two added gRPC plugins with different URI's.
# Update the routingrules by adding a new plugin with ID 2.
vcli rr update <id> --lookup-order=plugin:2,plugin:1
# Wait for the plugin:2 to be connected and routed to,
# then remove the old plugin with ID 1.
vcli rr update <id> --lookup-order=plugin:2
This will keep using plugin:1 until the router has a connection towards plugin:2. Without doing this, the default fallback rule of LeastUtilized will be used.