In his Percona Live 2020 talk, Shlomi Noach pointed out that he sees Vitess is an infrastructure framework that would let everyone get database capabilities that you'd otherwise have to build yourself.
At Shopify, we've built our own system that lets us keep track of database health and replication lag, to throttle clients (mostly, background jobs) accordingly. It took us quite a bit of efforts to build that and to make it work at scale to handle 1000s of MySQL hosts.
I'm really excited about technologies like Vitess (and its direction as an infrastructure framework) that could bring capabilities like throttling database operations to everyone, not just to large companies like Shopify and Github that had resources to invest into a custom solution.
Not that long ago Vitess has got the throttling capability available and as soon I found about that this weekend, I was eager to try it and to sum my impressions.
Setting up the cluster
We'll follow Get Started guide for the Vitess Operator to set up a dummy Vitess cluster.
Side note: I found Vitess Operator to be very easy to use; it cover all my basic needs and its documentation is good enough to not have to go to sources.
I'm going to use a GKE cluster because that's what I'm familliar the most, but you could as well use Minikube or another Kubernetes offering.
# create a tiny cluster
$ gcloud container clusters create sample-vitess-cluster --cluster-version 1.17 --zone us-east1-b --num-nodes 5
$ git clone git@github.com:vitessio/vitess.git
$ cd vitess/examples/operator
# before running kubectl, make sure to select the context with newly created cluster
# install vitess operator
$ kubectl apply -f operator.yaml
# provision VitessCluster
$ kubectl apply -f 101_initial_cluster.yaml
# port-forward to the cluster
$ ./pf.sh
# install vtctlclient if you haven't yet
$ go get vitess.io/vitess/go/cmd/vtctlclient
# setup the schema
$ vtctlclient ApplySchema -sql="$(cat create_commerce_schema.sql)" commerce
$ vtctlclient ApplyVSchema -vschema="$(cat vschema_commerce_initial.json)" commerce
Now you have the cluster running! With pf.sh running you have ports forwarded which should allow you to connect to it with mysql -h 127.0.0.1 -P 15306 -u user and explore things a bit.
Enabling throttler
Following the throttler docs we can find that the throttler is currently disabled by default. We have to pass -enable-lag-throttler to vttablet to enable it.
The Vitess Operator makes that very easy:
diff --git a/examples/operator/101_initial_cluster.yaml b/examples/operator/101_initial_cluster.yaml
index 8df5c19c8..f2e5de108 100644
