Search
Varnish Controller

gRPC

Introduction

gRPC (Google Remote Procedure Call) is an open source way of communicating between servers in a client/server setting.

The Varnish Controller Router has implemented gRPC with Flatbuffers. Flatbuffers is an open source, lightweight, efficient and cross platform serialization library.

A schema has been created using Flatbuffer’s IDL (Interface Definition Language). The schema defines the available functions and data types that are passed between the client and server via gRPC.

The router acts as a client and implements this schema in Go. The server side can implement the schema in almost any language.

Why Create a gRPC Routing Plugin

The router has a common set of routing rules (LeastUtilized, History, Random, External etc.) but there are cases where the routing decision should be based on external factors such as database lookups, BGP announcements etc. To cover all these different scenarios, an external plugin via gRPC can be implemented to tackle these cases.

The gRPC service is handed the information related to the request. Such as HTTP headers, client IP, available endpoints etc. Then the service can decide which endpoint to route to, based on this information together with external data.

Example Usage

router_grpc.png

  1. A client requests an URL towards the router. The router is configured to use a gRPC plugin.
  2. The router bundles all healthy endpoints and client information and send it to the gRPC service. The gRPC service looks up the client IP in a database and selects the varnish server “Varnish1” (as an example).
  3. “Varnish1” endpoint ID is returned back as a selection to the router.
  4. The router takes the BaseURL of the “Varnish1” endpoint and returns back to the client (302 HTTP redirect).
  5. The client performs the request towards the “Location” in the response header, which points to the “Varnish1” server.

Flatbuffer Schema

The Flatbuffer schema is used to generate code for specific languages. Download flatc and generate code for Go like this:

flatc --go --grpc router.fbs

Where router.fbs is the schema displayed below. This will generate a code stub to be used with Go and gRPC. This is used in an example later in this chapter.

Schema Description

The schema consists of two functions. One function called Check(...) for performing health checks. It aligns with the Protobuff example. This is not used by the router itself. It enables the possibility to perform external health checks towards the service.

The function used by the router is the other function called SelectEndpoint(...). This function is called upon by the router with the EndpointRequest data structure. This structure contains information related to the client request.

The response back from the service is an EndpointResponse structure. This should either contain an endpoint_id corresponding to one of the endpoints in the EndpointRequest or a CustomEndpoint. If there are no healthy endpoints, the EndpointRequest.endpoints is empty.

If the gRPC service doesn’t find an endpoint that is a good fit, an empty structure should be returned, meaning endpoint_id = 0 and no CustomEndpoint defined.

Note that there are 3 different ways to return data from the gRPC service.

  1. Return an ID of the endpoint to use, by specifying endpoint_id=<id> (where <id> is the ID of the endpoint). The routing IP’s and/or URL will be taken from the configured endpoint (Varnish Controller Agent).
  2. Return a custom endpoint, by specifying IPv4/IPv6 for DNS routing and/or URL for HTTP routing.
  3. Return an empty set, meaning that the router will not perform routing decision based on gRPC. The router will then perform the next in line of routing decisions configured in the RoutingRules.

Flatbuffer Schema

namespace router;

rpc_service RouterService {
  Check(HealthCheckRequest):HealthCheckResponse(streaming: "none");
  SelectEndpoint(EndpointRequest):EndpointResponse(streaming: "none");
}

enum ServingStatus: byte {
  UNKNOWN = 0,
  SERVING = 1,
  NOT_SERVING = 2,
}

table HealthCheckResponse {
  status:ServingStatus = 1;
}

table HealthCheckRequest {
  service:string;
}

table EndpointRequest {
  req_type:string;
  domain:string;
  client_ip:string;
  headers:[Header];
  method:string;
  uri:string;
  endpoints:[Endpoint];
}

table Endpoint {
  id:int64;
  name:string;
  mbps:float64;
  mbps_max:float64;
  ipv4:string;
  ipv6:string;
}

table EndpointResponse {
    endpoint_id:int64;
    custom_endpoint:CustomEndpoint;
}

table Header {
    key:string;
    value:string;
}

table CustomEndpoint {
    name:string;
    url:string;
    ipv4:string;
    ipv6:string;
}

Go Example Implementation

This is an example of implementing a simple gRPC service using Golang. It implements both functions, but only the SelectEndpoint is required, which is the one the router will call. Note the import named rfb should point out the output from flatc --go --grpc router.fbs command. A big part of the SelectEndpoint function below is for debugging purposes and not required. But it gives an idea how to access the different attributes of the data structures.

package main

import (
	"fmt"
	"log"
	"math/rand"
	"net"
	"os"

	context "golang.org/x/net/context"
	fbs "github.com/google/flatbuffers/go"
	"google.golang.org/grpc"

    // This is the generated code from the schema (flatc output)
	rfb "varnish/router" 

)

type server struct {
	rfb.UnimplementedRouterServiceServer
}

// Check responds to health checks, (Not required)
func (s *server) Check(context context.Context, in *rfb.HealthCheckRequest) (*fbs.Builder, error) {
	b := fbs.NewBuilder(0)
	rfb.HealthCheckResponseStart(b)
	rfb.HealthCheckResponseAddStatus(b, rfb.ServingStatusSERVING)
	b.Finish(rfb.HealthCheckResponseEnd(b))
	return b, nil
}

// SelectEndpoint is selecting either an existing endpoint or returns a custom. (Required)
func (s *server) SelectEndpoint(context context.Context, in *rfb.EndpointRequest) (*fbs.Builder, error) {
	// Print incoming EndpointRequest for debug purpose
	fmt.Printf("========================================\n")
	fmt.Printf("ClientIP: %v\n", string(in.ClientIp()))
	fmt.Printf("Domain: %v\n", string(in.Domain()))
	fmt.Printf("Type: %v\n", string(in.ReqType()))
	fmt.Printf("URI: %v\n", string(in.Uri()))
	fmt.Printf("Headers:\n")
	for i := 0; i < in.HeadersLength(); i++ {
		h := rfb.Header{}
		in.Headers(&h, i)
		fmt.Printf("   - %s: %s\n", string(h.Key()), string(h.Value()))
	}
	fmt.Printf("Endpoints:\n")
	for i := 0; i < in.EndpointsLength(); i++ {
		ep := rfb.Endpoint{}
		in.Endpoints(&ep, i)
		fmt.Printf("> ID: %#v\n", ep.Id())
		fmt.Printf("   - Name: %v\n", string(ep.Name()))
		fmt.Printf("   - Mbps: %v\n", ep.Mbps())
		fmt.Printf("   - Mbps(Max): %v\n", ep.MbpsMax())
		fmt.Printf("   - IPv4: %v\n", string(ep.Ipv4()))
		fmt.Printf("   - IPv6: %v\n", string(ep.Ipv6()))
	}
	fmt.Printf("========================================\n\n")

	// Create the response
	b := fbs.NewBuilder(0)

	epID := int64(0)
	var cust fbs.UOffsetT
	if in.EndpointsLength() > 0 {
		// select a random endpoint of the received ones
		id := rand.Intn(in.EndpointsLength())
		ep := rfb.Endpoint{}
		in.Endpoints(&ep, id)
		epID = ep.Id()
	} else {
		// Or create a custom endpoint
		name := b.CreateString("myEndpoint")
		url := b.CreateString("http://varni.sh/")
		ipv4 := b.CreateString("127.0.0.1")
		ipv6 := b.CreateString("::1")
		rfb.CustomEndpointStart(b)
		rfb.CustomEndpointAddName(b, name)
		rfb.CustomEndpointAddUrl(b, url)
		rfb.CustomEndpointAddIpv4(b, ipv4)
		rfb.CustomEndpointAddIpv6(b, ipv6)
		cust = rfb.CustomEndpointEnd(b)
	}

	rfb.EndpointResponseStart(b)
    // To return empty result (no decision), comment out the 2 lines below.
    // Note: Remove the above code where we define 'epID' and 'cust' to make it compile.
	rfb.EndpointResponseAddEndpointId(b, epID) // This line...
	rfb.EndpointResponseAddCustomEndpoint(b, cust) // and this line...
	b.Finish(rfb.EndpointResponseEnd(b))

	return b, nil
}

func main() {
	socket := "/tmp/grpc.sock"
	if _, err := os.Stat(socket); err == nil {
		os.Remove(socket)
	}

	//l, err := net.Listen("tcp", "localhost:1234")
	l, err := net.Listen("unix", socket)
	if err != nil {
		log.Fatalf("Failed to listen: %v", err)
	}

	ser := grpc.NewServer(grpc.CustomCodec(fbs.FlatbuffersCodec{}))

    fmt.Printf("Starting gRPC server, listening on: %s\n", socket)
	rfb.RegisterRouterServiceServer(ser, &server{})
	if err := ser.Serve(l); err != nil {
		log.Fatalf("Failed to serve: %v", err)
	}
}

Build and Run

For the above Golang example, follow the steps below to build your service application.

# Create a new directory for your application
mkdir myservice
cd myservice
go mod init myservice

# Create 'router.fbs' with the above schema
# then generate the code:
flatc --go --grpc router.fbs
cd router
go mod init router
go mod tidy
cd ..

# Create 'server.go' with the code from the example.
nano server.go

# make sure to update path to current path for the import: 
# rfb "varnish/router" 
go mod edit -replace varnish/router=./router
go get varnish/router

# Then tidy modules and build the app.
go mod tidy
go build .

# Test to run the app:
./myservice

# You should be seeing:
Starting gRPC server, listening on: /tmp/grpc.sock

To run the service in systemd, you can create the below systemd file. Replace the /tmp/myservice with the directory of your application built from previous step.

Add the below content to /etc/systemd/system/mygrpc.service (use sudo):

[Unit]
Description=Varnish Controller gRPC Service
ConditionPathExists=/tmp/myservice
After=network.target

[Service]
Type=simple
Restart=always
RestartSec=10
WorkingDirectory=/
ExecStart=/tmp/myservice/myservice

[Install]
WantedBy=multi-user.target

Set suitable permissions to the file:

sudo chmod 664 /etc/systemd/system/mygrpc.service

Then load it into systemd.

# Reload to update systemd with the new service
sudo systemctl daemon-reload

# Start on boot
sudo systemctl enable mygrpc

# Start the service
sudo systemctl start mygrpc

# Check status with:
sudo systemctl status mygrpc
# Or
journalctl -u mygrpc

# Stop the service
sudo systemctl stop mygrpc

Use The gRPC Plugin in Varnish Controller

Either configure the gRPC routing plugin in the web UI or via the vcli. Example configuration via the vcli can be seen below:

# Create the gRPC plugin 
# Expects both gRPC service and router to run on same server.
vcli pgrpc add test --uri unix:/tmp/grpc.sock --verify-tls=false

# Assign the previously created gRPC to a plugin (assumes gRPC ID = 1)
vcli rps add test --type grpc --id 1

# Now the plugin can be used in our routing rule (also add random as an example)
vcli rr add test --lookup-order=plugin:1,random --debug-headers=true

# Multiple plugins can be added to the same routing rule such as this.
# vcli rr add test --lookup-order=plugin:2,plugin:1 --debug-headers=true

# Assign the routing rule to your VCLGroup
# Assumes VCLGroupID = 1 and RoutingRule ID = 1
vcli vg update 1 --rr 1

Verification

curl can be used to verify that the plugin is being used. Look for the RouterType and Trace to see if the plugin was used for the redirect. Note that the --debug-headers=true needs to be enabled for the RoutingRules to show this output in the redirect response from the router.

# Issue the request towards the router and with the deployed domain.
curl -v -s http://my.router.domain.tld/ -H "Host: mydomain.tld"
...
< X-Router-RouteType: plugin:1
< X-Router-Trace: [plugin:1]
...

Issues with connection or requests towards the gRPC service can be seen with either apilogs or via the gRPC list.

# See all errors for this type of error
vcli apilogs ls -fkey_type=plugingrpc -fseverity=3

# See last reported errors 
vcli pgrpc ls -v

Caveats

When adding a new gRPC plugin, the router will not take use of the plugin until it has a connection towards it. That means that the first client requests might not use the plugin as it is not yet connected. The reason for this is to avoid stacking up a lot of client requests until the connection is up and the gRPC service would receive all requests at once.

When updating a plugin configuration in the controller, the connection will be kept as long as the plugin’s URI isn’t changed. If a new plugin should replace the old one it is advised to add the new plugin to the lookup order while keeping the old one running until the connection is up towards the new plugin. Then remove the old plugin from the lookup order.

Example:

# Assumes two added gRPC plugins with different URI's.
# Update the routingrules by adding a new plugin with ID 2.
vcli rr update <id> --lookup-order=plugin:2,plugin:1

# Wait for the plugin:2 to be connected and routed to,
# then remove the old plugin with ID 1.
vcli rr update <id> --lookup-order=plugin:2

This will keep using plugin:1 until the router has a connection towards plugin:2. Without doing this, the default fallback rule of LeastUtilized will be used.

Resources