A data engineer opens their Snowflake dashboard at 9 AM. The ticket queue has 47 new requests. Three years ago, every single one would have been a SQL query request. Today, only six are.
The rest look different:
- “Find customers most similar to our top enterprise accounts”
- “Why did conversion drop last Tuesday?”
- “Show me support tickets that mention pricing, grouped by complaint type”
- “Build me a dashboard. I don’t know what tables we have.”
None of these fit the shape OLAP was built to answer. You can’t GROUP BY similarity. You can’t SUM an explanation. SQL wants precision. These questions want interpretation.
The business hasn’t stopped asking questions about data. They’ve stopped asking questions SQL can answer.
The Efficiency Problem Isn’t Speed
ClickHouse can scan billions of rows in milliseconds. Druid handles real-time ingestion at frightening throughput. StarRocks vectorizes execution paths so aggressively that simple aggregations finish before you finish typing the query.
OLAP performance is not the bottleneck. OLAP’s worldview is.
The entire analytics stack from 2010 to 2023 rested on four assumptions:
- Business needs can be structured
- Structured needs translate to SQL
- SQL operates on clean dimension tables and fact tables
- Output is a number or an aggregate
This worked when “analysis” meant arithmetic. COUNT orders. SUM revenue. GROUP BY region. Every business question eventually collapsed into a calculation.
Then 2023 happened.
LLMs went from toy to tool. RAG went from research paper to production pattern. Suddenly the questions businesses cared most about stopped being arithmetic problems:
- “Which deals are at risk and why?”
- “What do our best customers have in common?”
- “This contract feels familiar. Show me similar ones.”
These are semantic problems. Similarity problems. Explanation problems.
OLAP doesn’t answer these badly. It doesn’t answer them at all. You can force-fit them into SQL, lose information in the translation, and spend three days building a pipeline that produces numbers nobody trusts. Or you can admit the query model itself is wrong.
Vectors Become Infrastructure
Two years ago, vector databases were a separate category. Pinecone, Weaviate, Milvus, Qdrant. People treated them like Redis: a specialized cache you bolted onto the real database when you needed RAG.
That mental model is dead.
Postgres ships pgvector as a core extension. AWS Aurora has native vector indexes. BigQuery and Snowflake treat VECTOR_SEARCH as a SQL function, not an external service. Databricks runs vector queries directly on Delta Lake Parquet files.
Vectors aren’t a special data type anymore. They’re a column, same as INT or VARCHAR.
What that means in practice:
“`sql
SELECT
customer_id,
name,
total_orders,
embedding_distance(profile_vec, target_vec) AS similarity
FROM customers
WHERE region = ‘APAC’
AND total_orders > 50
ORDER BY similarity
LIMIT 100;
“`
This query mixes structured filtering (region, order count) with vector similarity. Three years ago you’d need two databases, a sync pipeline, and glue code to make this work. Today it’s one table.
LanceDB and TurboPuffer go further. They build the entire data model around vectors and structured metadata living together, not vectors-plus-structured as an afterthought.
The shift isn’t that vector search got faster. It’s that vectors stopped being an AI feature and became a data primitive. When you can join on similarity as easily as you join on foreign keys, the boundary between “structured data” and “AI data” disappears.
Natural Language Eats SQL
SQL was the universal interface for thirty years. Every BI tool, every data platform, every analytics workflow ended up generating SQL.
That assumption is breaking.
The new stack puts a semantic layer between the user and the database. Cube, dbt’s Semantic Layer, AtScale, MetricFlow. These aren’t BI tools. They’re abstraction layers that let LLMs query data without writing SQL.
Why does this matter? Because LLMs are terrible at writing SQL against raw schemas.
Point an LLM at 500 tables, 20,000 columns, and a few thousand business metrics defined across overlapping views, and it will hallucinate joins, misinterpret foreign keys, and produce queries that run but return garbage.
Point the same LLM at a semantic layer that already defines “customer,” “churn,” “active users,” and “revenue,” and accuracy jumps from 30% to 85%. The LLM only needs to understand business concepts. The semantic layer handles the translation.
The flow changes from this:
“`
User writes SQL → Database executes → User interprets result
“`
To this:
“`
User asks in plain English → LLM maps to semantic layer →
Semantic layer generates SQL → Database executes → LLM explains result
“`
Business users don’t learn SQL. Data engineers don’t write reports. The role split changes completely. Business asks questions. Engineers maintain the semantic layer.
This isn’t hypothetical. Companies building new data platforms in 2024 and 2025 are designing around this pattern from day one. Legacy BI tools like Tableau and Power BI are losing ground because their value proposition was “let non-technical users build charts.” When non-technical users can ask questions in Slack and get answers without touching a dashboard, chart-builders become optional.
Agents Query More Than Humans Do
Every query used to start with a person. Someone sat at a SQL editor, wrote a query, clicked run, and looked at results.
2026 is different. Agents are the dominant query workload.
Three examples from production systems:
A customer support agent fields a question about order status. It queries the orders table, the shipping table, the refunds table, checks geolocation data, and synthesizes an answer. That’s not one query. It’s ten queries in two seconds. No human wrote any of them.
A risk monitoring agent watches transaction streams in real time. When it spots an anomaly, it calls an LLM to classify the risk type, writes the result to a new table, and triggers a workflow. Every step involves database writes or reads. The agent orchestrates all of it.
A BI agent gets asked “Why did sales drop last week?” It generates twenty exploratory queries, finds the dimension where the drop concentrated, pulls historical comparisons, and writes a report. No human touched SQL.
Queries aren’t isolated anymore. They’re steps in workflows. The database isn’t a passive tool. It’s infrastructure that agents call hundreds of times per task.
This creates new requirements:
- Structured tool calling. Agents need function-calling interfaces to data, not just SQL endpoints.
- High concurrency, low cost. When agents query aggressively, traditional scan-and-bill pricing models break. $50 in compute for a single investigation isn’t sustainable.
- Native explainability. Agents need to know where data came from, not just what it says.
- Strong access controls. You can’t let an agent query everything. Permissions need to be first-class.
New platforms are emerging around this: Motherduck (serverless DuckDB), TurboPuffer (vector plus KV storage for agents), Databricks Agent Bricks, Snowflake Cortex Agents.
They all optimize for the same thing: agent workloads, not human workloads.
The Objections
At this point someone will say: OLAP isn’t going anywhere. It will remain core to analytics for years.
They’re half right. OLAP engines will stick around. Columnar storage, vectorized execution, and distributed query planning still matter. But OLAP as the default interface for analytics is being displaced.
The cleaner way to think about it: OLAP becomes the execution layer. Above it sits a semantic layer, vector storage, and agent orchestration. The business never touches OLAP directly. They ask questions. The new stack translates those questions into whatever combination of SQL, vector search, and LLM calls makes sense.
Another objection: natural language query accuracy isn’t reliable yet. LLMs make mistakes.
True. Text-to-SQL accuracy three years ago was 30-40%. Today, with semantic layers, chain-of-thought prompting, and self-correction loops, it’s above 85%. That’s not perfect. But for exploratory analytics, which accounts for 60-70% of business queries, 85% is good enough. Precision reporting still goes through traditional pipelines.
Last objection: mixing vector and structured queries will be slow.
Early implementations were. HNSW indexes, IVF, DiskANN, and filtered vector search have closed that gap. By 2025, pgvector performance was close enough to dedicated vector databases that many teams stopped maintaining separate systems. The performance gap is shrinking faster than most people realize.
What Data Teams Need to Learn
If you’re a data engineer who spent the last decade mastering OLAP, your skills still matter. But they’re only half the picture now.
The other half:
- Vector embeddings. How they work, when to apply them, how to tune similarity thresholds.
- Semantic layer design. Defining business concepts that LLMs can understand and query.
- Agent orchestration. Building systems where agents call databases hundreds of times per task without breaking cost or latency budgets.
- Evaluation methodology. How do you know if a natural language query returned the right answer? Traditional unit tests don’t work here.
This isn’t minor upskilling. It’s a second foundation. Your OLAP knowledge gives you the execution layer. These new skills give you the interface layer.
The good news: the companies figuring this out now have a multi-year lead. The bad news: if you wait for this to stabilize, you’ll be learning catch-up skills in 2028 while competitors are already in production.
Who Wins in This Shift
The database vendors winning right now aren’t the ones with the fastest columnar scans. They’re the ones who recognized that the interface needed to change before the business started complaining.
Snowflake added Cortex and vector support. Databricks built semantic layers and agent tooling. Motherduck made DuckDB serverless specifically for agent workloads. LanceDB designed the entire storage model around vectors and metadata coexisting.
The pattern is the same: they’re not just optimizing query speed. They’re rethinking who queries, how they query, and what a query even means.
Three years from now, “database” will mean something different. It won’t just store and retrieve data. It will interpret questions, orchestrate agents, explain results, and operate as infrastructure for AI systems that ask a thousand questions a minute.
OLAP built the first thirty years of analytics. AI-native databases are building the next thirty.
If you’re designing a data platform today, the question isn’t whether SQL will keep working. It will. The question is whether SQL will still be the interface your users see, or whether it becomes an implementation detail hidden behind semantic layers, vector search, and agents that ask their own questions.
The companies that answer correctly are the ones building for 2026, not 2015.
Stay updated with our latest AI insights
