Meta's MySQL Fork You Never Knew About

It's not very well widely known — but in the world of MySQL forks like Percona and MariaDB, Facebook's fork might be the most interesting one! Unlike Oracle or Percona, at Facebook they've been building cutting edge features into MySQL that allow them to run at a hyper scale.

Those are features are essential for a global scale platform – which I know from the fact that my team at Shopify has built ones internally. And honestly, it would save us a ton of time if we could use that Facebook's fork. Below, I'll try to capture everything novel that they've introduced in the fork.

My knowledge is incomplete and comes from commits in a public Git repo. If you're an engineer familiar with Meta's MySQL fork and can provide additional insights, email me at [email protected].

Let's start with some of the features that have been publicly talked about.

Raft

You can explore more in depth on Meta's post about the Raft consensus engine in MySQL. But in a nutshell, Meta has integrated the Raft protocol with MySQL to create a replicated state machine. This replaces the existing semisynchronous replication to enhance reliability and operational simplicity, while maintaining comparable write performance. The outcome? Faster failover, simplified operations, and a robust consistency model.

Another benefit of that work is configurable consistency. Configurable consistency allows the owners to select whether the service needs X-region quorums or quorums that ask for copies in some specific geographies (e.g., Europe and the United States). FlexiRaft has seamless support for such configurable quorums.

Imagine being able to use Cassandra-style QUORUM setting per client or per query!

MyRocks and RocksDB

RocksDB, a highly efficient embedded database written in C++, is extensively used at Facebook. Meta wanted to leverage MySQL while benefiting from RocksDB's storage efficiency, leading to development MyRocks – a new efficiency and storage optimized MySQL storage engine. MyRocks provides faster replication and data loading while saving up to 10x in storage space due to RocksDB’s efficient compression capabilities. For more details, you can check out this blog post.

MyRocks has been widely adopted by the broader MySQL comminity for both 5.7 and 8.0 versions.

Now, let's get into some more intersting features!

Vector storage

In the age of LLMs and RAG (Retrieval-Augmented Generation), you want your database to be able to store vectors that capture opaque semantic meaning about something, and to search for resources by similarity based on this opaque meaning. In 2024, all major database providers are striving to provide vector search out of the box.

Meta has integrated basic vector search functionality (e.g. kNN search) into MySQL using the FAISS library, also developed by Meta. FAISS is extensively used in the ecosystem due to its ease of embedding – think of it as LMDB for vectors. I'm familiar with it from my own pet projects and using FAISS from Ruby.

It's fascinating to see vector search support built directly into MySQL!

Here's a relevant commit.

And a DML / DDL example:

CREATE TABLE t1 (
  id BIGINT NOT NULL PRIMARY KEY,
  vector1 JSON,
  INDEX vector_key_1(vector1) FB_VECTOR_INDEX_TYPE 'flat' FB_VECTOR_INDEX_METRIC 'l2' fb_vector_dimension 1024
);

select * from t1 order by FB_VECTOR_L2(vector1, '[2, 1, 1]');

/* filter by name and order by distance using knn search */
select *, FB_VECTOR_L2(vector1, '[0.3, 0.3]') as dis from t1 where name like '%aaa%' order by dis limit 6;

Before this, I've only seen priorierary platforms like TiDB and PlanetScale offer vector search natively on MySQL.

RPC and Thrift support

Frameworks like Vitess already provide GRPC interface to query the database, but in the end, Vitess components like vtablet still need to talk MySQL protocol to MySQL Server.

In Meta’s MySQL fork, the server can accept Thrift-format RPC requests that describe a table and conditions to fetch – queried directly from RocksDB. This approach is appealing as it bypasses SQL parsing and interpretation, reducing query planner overhead and retrieving data from RocksDB efficiently.

Relevant sources:

This is very close to what I'm writing about in my B-trees as API post.

Throttling database writes

In a large scale system, we want the database to be resilient to a single workload or a tenant for causing too many writes and overloading the rest of the system. This is usually done by throttling (e.g. slowing down) writes that come from those specific clients – to provide room to the rest of workloads and to get replication lag under control.

Thanks to Raft, Meta’s MySQL server gains awareness of the topology and can track replication lag on secondary nodes (followers). This allows the server to automatically throttle write transactions if necessary to prevent overwhelming secondaries.

Here's some code examples:

https://github.com/facebook/mysql-5.6/blob/f1ec3a089dd03e0dc2abfc48b66965ab1b71313c/sql/rpl_source.cc#L337-L346
/**
  Scans through the replication lag reported by individual secondaries and
  returns replication lag for the entire topology. Replication lag for the
  topology is defined as kth largest replication lag reported by individual
  secondaries where k = write_throttle_lag_min_secondaries.
  This method is used for auto throttling write workload to avoid replication
  lag

  @retval replication_lag
*/
int get_current_replication_lag() 

/*
  check_lag_and_throttle
    Main method responsible for auto throttling to avoid replication lag.
    It checks if there is lag in the replication topology.
    If yes, it finds the entity that it should throttle. Otherwise, it
  optionally releases one of the previously throttled entities if replication
  lag is below safe threshold.
*/
void check_lag_and_throttle(time_t time_now)

This approach lets the server hold off any write transactions if secondaries are overwhelmed.

In large scale systems that I've worked on, we've had to bake this logic into the client -- which meant building the lag discovery mechanism and making all clients aware of that lag. Combined with a hundreds of shards, managing that state on the client deemed very painful and prone to staleness. Handling this server-side is indeed superior and is a direct benefit of the Raft integration.

You might look at Freno by GitHub as an example of a system that makes clients discover when they need to throttle. Vitess offers one as well.

Autosharding

Though not publicly available as source code, autosharding is mentioned across Meta's MySQL codebase. This hints at a some built-in sharding mechanism, but details remain elusive.

Wrap up

~~It's fair to mention that building this fork requires access to private repositories that are not available unless you're an employee of Meta.~~ Corrected

This fork of MySQL by Facebook/Meta is fascinating, especially when you consider the scale at which they operate and the general lack of rapid innovation in the MySQL space.

Happy hacking!

If you've enjoyed reading this, you might also be interested in reading about Mcrouter which is another piece of Facebook's stack that I wrote about.

Written in December 2024.

Kir Shatrov helps businesses to grow by scaling the infrastructure. He writes about software, scalability and the ecosystem. Follow him on Twitter to get the latest updates.

[email protected]