Ian Macartney

I feel clever subbing `return await fn()` with `return fn()`

But gotchas don't seem worth it anymore
• Breaks try/catch/finally expectations
• Loses stack trace context

My default is now to always await. Change my mind.

Love to hear it! Hop into the agents channel in the Convex Discord if you aren't already - that's where I give updates & folks give feedback to help shape the future of it

Authorization Best Practices and Implementation Guide Learn about authorization techniques and how to implemented them with practical examples.

I've added new features to my middleware-esque helper library, and got inspired to write down everything I think about authorization:
stack.convex.dev/authorization

YouTube video by Convex How to Build Realtime AI Agents with Convex Components

A recent talk on developing the Agent Component:
www.youtube.com/watch?v=YM9n...

Documentation for the Agent component is live 🎉
-> docs.convex​.dev/agents

Odd coincidence that there's ~2300 lines of documentation & ~2300 lines of example code 🤔

...maybe more surprising there's fewer lines of React?
@​convex-dev/agent

Example:
github.com/get-convex/a...
Agent component:
convex.dev/components/a...
Rate Limiter:
www.convex.dev/components/r...

Rate limiting LLM chat per-user with `useRateLimit`

const { status } = useRateLimit(api.rl.getRateLimit)

Full code in 🧵

Algorithms:
Sending messages: fixed window
Token usage: token bucket

courtesy of
@​convex-dev/rate-limiter + @​convex-dev/agent
(components)

That sounds compelling, and yeah system prompts do feel like a pretty poor bound. Now that I think of it I agree it's pretty aligned.
What's your preferred way to handle routing / dispatching / step-by-step tool calls btw?

but I may be over-indexing on the string-based signature. Definitely on my list of things to dig deeper on. Like most things you end up having more in common than you expect from the outset

AI Agents with Built-in Memory With this new backend component, augment Agents to automatically save and search message history per-thread, providing realtime results across multipl...

Yeah DSPy came on my radar recently. It seems interesting & audacious!
My gut feel is that code is still king though - the second you want more flexibility, composability, or control over context / prompting, you'd be fighting the framework.

A la #4 here stack.convex.dev/ai-agents#go...

AI Agent Agents organize your AI workflows into units, with message history and vector search built in.

Agent component: convex.dev/components/a...
YT talk: youtube.com/watch?v=3Ydg...
Durable workflow article: stack.convex.dev/durable-work...

Recent talk on agents & agentic workflows, as well as my Convex Agent Component. Takes:
• Agentic := prompting + routing
• Prompting is input -> LLM -> output
• Routing has code at every boundary
(...even if an LLM "decides" what to do next)
Full 📺🔗->🧵

wdyt?

const messages = useThreadMessages(
api​.foo.listMessages,
{ threadId },
{ initialNumItems: 10, stream: true },
);
const sendMessage = useMutation(
api​.foo.generateText,
).withOptimisticUpdate(
optimisticallySendMessage(api​.foo.listMessages),
);

AI Agent Agents organize your AI workflows into units, with message history and vector search built in.

Agent Component: convex.dev/components/a...
Example code: github.com/get-convex/a...
Changelog: github.com/get-convex/a...

It one-ups persistent-text-streaming by syncing down only the deltas, not the full text, so you don't pay for bandwidth except proportional to the total length

Streaming LLM text using websockets + client smoothing - no HTTP necessary!

Agent v0.2.1 is out! Repo & release notes in 🧵
- Streaming text react hook + server fns
- Client-side smoothing hook
- Optimistic update helpers

YouTube video by Strange Loop Conference "Simple Made Easy" - Rich Hickey (2011)

“Simple Made Easy” is incredibly relevant nowadays where “easy” but not “simple” systems abound

youtu.be/SxdOUGdseq4?...

Convex Agent Playground Evaluate and test agents using @convex-dev/agent

Hosted playground: get-convex.github.io/agent/
Agent component / framework: github.com/get-convex/a...
Playground directory: github.com/get-convex/a...

Fun fact: It can target your @convex.dev backend if you if you expose the API, using API key auth.
Statically hosted on GitHub pages

Agent Playground for @​convex-dev/agent is live!
Investigate threads, messages, tool calls
Dial in context params
Iterate on prompting, etc.
For the @​convex-dev/agent component.
Links in 🧵

For an importance of x (0 to 1):
1. Normalize the existing vector to (1-x) and add √.x
2. Search with [...embedding, 0].
e.g.:
Say we have an embedding of 2 numbers [.6, .8]
For 50% importance: [.3, .4, .707]
For [.6, .8] we used to get 1.0.
Now we get .6*.3 + .8+.4+0 = .5🎉

My original thought was to just scale all the values, but vector search normalize the vectors for -1:+1 scores.

The trick is to add an extra number ("feature") to the embedding.
[...1536 numbers, <X>]
Then query with
[...1536 numbers, 0].
How it works: 🧵

Not all embeddings are created equal. Some represent more meaningful context. I struggled with AI Town to efficiently do vector search that also included a 1-10 "importance"
Last night I figured out a way to prioritize some embeddings over others using a "bias" feature 🧵

AI Agent Agents organize your AI workflows into units, with message history and vector search built in.

Agent component: convex.dev/components/a...
Article: stack.convex.dev/ai-agents
Code:
github.com/get-convex/a...

let textSearchMessages: Doc<"messages">[] | undefined; if (args.text) { textSearchMessages = await ctx.runQuery(api.messages.textSearch, { userId: args.userId, threadId: args.threadId, text: args.text, limit, }); } if (args.vector) { const dimension = args.vector.length as VectorDimension; if (!VectorDimensions.includes(dimension)) { throw new Error(`Unsupported vector dimension: ${dimension}`); } const vectors = ( await searchVectors(ctx, args.vector, { dimension, model: args.vectorModel ?? "unknown", table: "messages", userId: args.userId, threadId: args.threadId, limit, }) ).filter((v) => v._score > (args.vectorScoreThreshold ?? 0)); // Reciprocal rank fusion const k = 10; const textEmbeddingIds = textSearchMessages?.map((m) => m.embeddingId); const vectorScores = vectors .map((v, i) => ({ id: v._id, score: 1 / (i + k) + 1 / ((textEmbeddingIds?.indexOf(v._id) ?? Infinity) + k), })) .sort((a, b) => b.score - a.score); const vectorIds = vectorScores.slice(0, limit).map((v) => v.id); const messages: Doc<"messages">[] = await ctx.runQuery( internal.messages._fetchVectorMessages, { userId: args.userId, threadId: args.threadId, vectorIds, textSearchMessages: textSearchMessages?.filter( (m) => !vectorIds.includes(m.embeddingId!) ), messageRange: args.messageRange ?? DEFAULT_MESSAGE_RANGE, parentMessageId: args.parentMessageId, limit, } ); return messages; } return textSearchMessages?.flat() ?? [];

I do RAG via hybrid text/vector search using reciprocal rank fusion (the one-weird-trick of hybrid search imo) for my new Agent framework/component.

It's open source and the code is remarkably simple, if you're looking for an example for yourself.

I pushed a fix this morning, so if you already installed it, upgrade to `@convex-dev/agent@latest`

AI Agents with Built-in Memory With this new backend component, augment Agents to automatically save and search message history per-thread, providing realtime results across multipl...

I launched this today! Adding memory to AI SDK Agents with tools and RAG.
Article on Agentic Workflow: stack.convex.dev/ai-agents
Agent framework: convex.dev/components/a...

Agents Need Durable Workflows and Strong Guarantees Agents rely on long-lived workflows, but when happens when they fail midway through? Here are the tools you need to manage correctness and reliability...

Run workflows reliably and asynchronously, using Inngest-style code.
Why you need Durable Workflows for agentic systems:
stack.convex.dev/durable-work...
Workflow component: convex.dev/components/w...

Exciting news for Agent Workflow front:
🪨Durable Workflows🪨: Orchestrate steps async with retries, checkpointing and more, using Inngest-style syntax
🤖 Agent Framework 🤖: Define agents and use threaded memory (can hand off between agents), with hybrid text/vector search.
🧵

I reimplemented Mastra workflows and I regret it I reimplemented Mastra’s agentic workflows with durable functions in Convex, and it was the wrong decision. Look at three common strategies (reimpleme...

Hopefully you can avoid making my mistakes:

stack.convex.dev/reimplementi...

Welp not all experiments work out, but what will outlive all products is the insights you glean along the way.

I reimplemented Mastra workflows in Convex last week and I regret it. Article in 🧵

@anniesexton.com I may not have your skills, but I still had fun. One of these days I should graduate from the excalidraw center for kids who can't draw good and want to do other things good too