The Data Oil Blog · 28 October 2026
We got tired of keeping three systems in sync, so we built one engine
Most apps store the same data three times — rows in a database, the same text in a search engine, and again as vectors — and pay to keep all three in sync. We put SQL, documents, vector search and full-text search in one engine on object storage, so a write commits once and is searchable by keyword and by meaning straight away.
Most applications we've worked on store the same data three times. Rows in a relational database. The same text in a search engine, so people can find it. The same text again as embeddings in a vector store, so an AI feature can retrieve it.
Each of those tools is good at its job. The trouble is the space between them. A schema change becomes three schema changes. A backfill becomes three backfills. When something looks wrong, the first hour goes on working out which copy is behind. And each comes with its own bill, usually sized for capacity you might need rather than work you actually did.
What we wanted. A database platform that didn't cost the earth, and that developers could pick up with what they already know: SQL, NoSQL documents, vector search and full-text search. None of those ideas is new, and we didn't try to reinvent them. We put them together in one engine, on one platform, and we run the infrastructure so you don't have to.
SELECT title,
VECTOR_DISTANCE(embedding, EMBED('casing failure'), 'cosine') AS distance
FROM Report
WHERE SEARCH_INDEX('Report[body]', 'casing failure') = true
AND filedAt > date('2026-01-01')
ORDER BY distance
LIMIT 10;
A write commits once and is searchable by keyword and by meaning as soon as it commits. There's no sync job, because there's nothing to sync. If you'd rather use what you already have, the PostgreSQL wire protocol means psql, your usual driver or EF Core can connect.
What it costs. You pay for what you use from the first request, on a published rate card: two cents per thousand operations, with no cluster to size. On those rates, a small searchable app with 100,000 operations, 100,000 searches, 10 GiB stored and 1,000 cited answers a month comes to about $27.60, before your own model provider's charges. Every job is checked against your budget as it runs and stops above 95% of it, so a bulk load can't quietly eat the month.
Why we wrote the engine from scratch. We know "we wrote our own storage engine" makes experienced engineers nervous. It made us nervous too. We did it because we wanted all of the state to live in object storage, and bolting a vector index and a search index onto an engine that wasn't designed for that kept leaving seams. So we wrote a log-structured engine, an HNSW vector index and a BM25 full-text index together. To keep ourselves honest, 534 named reliability guarantees run on every build.
Things we care about that rarely make a feature list. Pagination uses a held cursor rather than an offset, so paging through a table that's being written to never skips or repeats a row. Errors are RFC 9457 problem documents with a stable code and a retryable flag. And before an AI answer is generated, POST /query/prepare shows you exactly what would be sent to the model, without sending it.
Where it's not the right tool. If all you need is key-value reads, a dedicated key-value store will cost less than we do, and our own pricing page says so. Data Oil earns its place when you'd otherwise run search and vectors beside your database.
Where we are. We built our own products on it first. MOODI, which analyses video frame by frame and writes millions of rows a month, went from four databases to one when it moved onto Data Oil. Mercury Echo, a multilingual workforce messaging platform, runs on it too, alongside a third product we can't name. The public clients are newer: they're on PyPI, npm and NuGet, and the MCP server is in the MCP registry. The source is closed and the team is small, so we're opening up gradually.
You don't need to talk to us to try it. The public sandbox at sandbox.dataoil.tech takes SQL over HTTP, any PostgreSQL driver, EF Core or the MCP server, with no sign-up. It runs keyword search across five languages. Semantic search and cited answers are switched off there, because they need GPU, and run on your own database.
For people who want to build on it, we're opening 50 founding builder seats. Once your product is released and in use, your free storage cap goes up to 500 GB for a year. You also get a two-year price lock, a private channel with the people who wrote the engine, a vote on the roadmap and a featured spot for your project. We ask that you ship using Data Oil within 60 days and let us show it. Apply at sandbox.dataoil.tech.