My co-founder and I worked as engineers at NASA and Twitter, built products with 5M+ monthly active users, and created the first AI email copilot. We're decent engineers I'd say, but somehow fell into the RAG trenches.

For our last startup, we had to set-up RAG for 9 million pages. We started out with youtube tutorials. First Langchain → Llamaindex. Got to a working prototype in a couple of days and were optimistic with the progress. We ran tests on subset of the data (100 documents) and the results looked great. We spent the next few days running the pipeline on the production dataset and got everything working in a week — incredible.

Except it wasn't, the results were subpar and only the end users could tell. We spent the following three months rewriting pieces of the system, one at a time, until the performance was at the level we wanted.

AI was at the core of our product, and we couldn't deliver a subpar experience. What started as a week long project, took 3 months — and there was so much more to do.

Our goal is that other engineers don't go through this process. They should get production-grade quality with the same effort that it takes to get a prototype working, and use this time instead to deliver features and focus on their users.

Agentset is the tool we wish we had when we started.