Mintlify replaces sandboxes with virtual filesystems to cut AI documentation assistant costs
Developers are increasingly moving away from traditional Retrieval-Augmented Generation (RAG) approaches to power AI documentation assistants, with some teams finding that virtual filesystems offer a more efficient and cost-effective alternative[2]. Mintlify, a documentation platform, replaced expensive sandbox environments with ChromaFs, a virtual filesystem built on top of their existing Chroma vector database, dramatically improving performance and reducing costs[2].
The company's motivation was clear: traditional sandbox approaches carried substantial overhead. The old system had a p90 boot time of approximately 46 seconds and would have cost over $70,000 annually at scale[1][2]. By contrast, ChromaFs—which creates the illusion of a real filesystem without requiring actual infrastructure—reduced session creation time to just 100 milliseconds[2]. Since ChromaFs reuses the documentation infrastructure already indexed in their Chroma database, the marginal per-conversation compute cost is effectively zero[2].
The technical innovation behind ChromaFs is straightforward but elegant: rather than relying on sandboxed environments, the system intercepts UNIX shell commands like grep and cat and translates them into database queries[1][2]. This approach works because AI agents don't require a genuine filesystem—they need only the functional illusion of one. Documentation that was already chunked and stored in Chroma for search capabilities could be repurposed to power the assistant's file exploration abilities[2].
The results have validated this architectural choice at scale. ChromaFs now powers Mintlify's documentation assistant for hundreds of thousands of users across more than 30,000 conversations daily[2]. Beyond performance gains, the virtual filesystem approach provided additional benefits: instant session creation, zero marginal compute cost, and built-in role-based access control (RBAC) without requiring new infrastructure[2].
This shift reflects a broader trend in AI development toward rethinking how language models interact with data. Rather than treating every assistant query as an opportunity to rebuild infrastructure, teams are finding creative ways to repurpose existing systems, whether through virtual filesystems or evolving markdown libraries maintained by AI systems themselves[2]. The approach demonstrates that sometimes the most scalable AI solutions aren't fundamentally new technologies, but rather clever reengineering of existing tools to meet new demands.