Sujay Jayakar

Convex MCP Server Convex now supports a powerful MCP server that lets you introspect your deployment's state, run functions, and read and write data.

check out stack.convex.dev/convex-mcp-s... for instructions for setting it up. it's still in beta, so let us know if you have any feedback.

@convex.dev now has an MCP server! the cursor AI agent can now list project's schema and API, read data from tables, and call functions.

it's insane how good it is at gluing tool calls together. here's an example of it building a histogram of a message lengths in a chat app.

Testing sync at Dropbox

we did this back at dropbox for its sync engine -- see
isaac's dropbox.tech/infrastructu... for more

inputs from the user, network, persistence -> core sync engine state machine -> output messages.

we then wire up these inputs and outputs to a real websocket, indexeddb, etc. in a client. but since we have this separation, it's trivial to reproduce any bug in a test.

been working on the @convex.dev object sync engine, and, unsurprisingly, we've been structuring it to use deterministic simulation testing.

i'm still blown away every time how *simple* writing systems as deterministic state machines makes testing complicated race conditions

agreed, but i am curious how they'll evolve their transaction size / row size limits over time. they're in a great, FoundationDB shaped, niche right now, but that niche isn't a good fit for a lot of workloads. so if they stay here, I don't see it being a fully drop in replacement for rds/aurora.

that's 100% it: we started with the database, built efficient reactivity, and are now working our way out to the clients.

excited to have it in all of your hands soon!

YouTube video by Convex Building better, faster apps with Local-First (w/Johannes Schickling)

we had a great conversation with @schickling.dev about local-first and how it's incredibly relevant to database folks. it doesn't matter how fast our databases are if this performance doesn't extend to the user experience.

www.youtube.com/watch?v=n6Mx...

at runtime, it can check to see if there's a path between a user and an object by intersecting these two sets. this operation can be made efficient using the same compressed integer set data structures used in text search indexing, something google is already very good at :)

with this assumption, their indexing system precomputes the set of highlighted nodes reachable from each user. then, for each object, it also precomputes the set of all highlighted nodes that are one step away.

intuitively, each user is a member of a group, which may be nested in another group, which may then own a project. projects may have many documents, but the set of all projects visible to a user should be relatively small.

I've highlighted the nodes *one step back* from each final object. zanzibar assumes that this set has relatively low cardinality.

I've drawn a representative access control graph, where we start with user A and traverse the graph to all of the objects reachable from A.

zanzibar addresses this with their "leopard indexing" system. the core idea here is *to make assumptions about the structure of the access control graph*, and not have to solve the general graph reachability problem.

checking path reachability is difficult: users may have access to millions+ of objects, and doing a naive depth-first search in the object graph on every access would be really expensive.

we looked at zanzibar yesterday, where a user A has access to an object B if there's a path between them in the access graph. let's walk through how they make this graph reachability problem efficient.

this approach is also flexible: it's easy to express nested hierarchies and ACL inheritance without explicit denormalization (i'm looking at you, NTFS).

the cost? implementing reachability checks efficiently is *hard*. more on that later.

this is *robust*: if user A gets kicked out of group A, they'll automatically lose access to the document. it's impossible to write a bug where we forget to remove the user from the document's ACL.

with all of that set up, we can check if user A is a reader of document A by seeing if there's a path in this graph!

the second type of edges are *derived edges* that are implicitly filled in across different objects. every admin of a group is also a member, and every owner of a document is a writer of that document, etc.

the first type of edges are *base edges* that are added explicitly into the system. user A is an admin of group A and a writer of document B, so we put heavy arrows between those nodes.

then, each (object, relationship) pair gets a node too. let's say we have one group and two documents. groups have admins and members, and documents have owners, writers, commenters, and readers. I've drawn a dotted line connecting these nodes to their shared document for clarity.

we start by filling in a red node for our user. each user gets one of these nodes in the graph.

the notation in the paper is a little confusing, so I made this diagram in my notes to keep things straight. in our example, we'll have a single user (user A) that's a member of a group (group A). group A owns document A. there's another document (document B), and user A is a writer for document B.

google's zanzibar paper (2019) provides a single, flexible model that's based on *graph traversal*. we can model if a user has access to an object by checking if there's a path in a access control graph.

authorization comes up a lot, and it's easy to get lost in the soup of different ways to express who can access what (ACLs, RBAC, ABAC, ...)

somehow this seems to be true for every system i've ever been oncall for!

we go oncall so y'all don't have to.

"convex is a machine for turning my sleep schedule into your app's reliability" - Alfréd Rényi :)

this is a little unrelated, but I always loved how SQLite on a filesystem can be faster than just using the filesystem.

www.sqlite.org/fasterthanfs...

super interesting, and that makes a ton of sense. is this implemented at the VFS layer under the hood? i.e. it loads all data pages into memory at startup and then syncs the WAL from the filesystem thread to the app thread?