Azure Databricks Agent Bricks - AI/BI Without Coding

Databricks has spent a decade building the platform that data engineers love. Serverless compute, Unity Catalog, Delta Lake, MLflow — these are tools designed for people who write Spark jobs and manage lakehouse architectures. THRSP904 at Ignite 2025 asked a different question: what if the people who need data analysis the most are the people who cannot write a line of code?

Session: THRSP904 Date: Tuesday, Nov 18, 2025 Time: 2:00 PM - 2:30 PM PST Location: Moscone South, The Hub, Theater A

Agent Bricks is Databricks' answer. Announced alongside a suite of platform capabilities — serverless workspaces, Lakebase, and AI/BI Genie integration with Microsoft Teams via Copilot Studio — the session painted a picture of enterprise data and AI that is accessible to business users, not just technical teams. This is a significant strategic pivot. And it raises important questions about whether "no-code data analysis" is genuinely useful or just another vendor demo that falls apart when real data gets involved.

Serverless workspaces: Removing the infrastructure tax

The friction that kills adoption

Ask any data team why their organisation is not more data-driven, and the answer is rarely "we lack data." It is almost always about infrastructure friction. Provisioning clusters, managing compute, configuring networking, handling authentication, waiting for environments — the operational overhead of getting to the point where you can actually analyse data is substantial. In many organisations, the time between "I have a question about my data" and "I have an environment where I can investigate" is measured in days or weeks.

What serverless workspaces promise:

• Launch a fully configured Databricks workspace in seconds, not hours
• No cluster management, no capacity planning, no idle compute costs
• Automatic scaling based on workload demand
• Pay only for compute consumed during actual processing

Why this matters beyond cost savings:

The real value is not the infrastructure cost reduction — though that is welcome. It is the removal of the wait time between having a question and being able to investigate it. In organisations where spinning up an analytics environment requires a ServiceNow ticket and a two-week lead time, serverless workspaces eliminate the barrier entirely. The question stops being "can I get access to the data?" and starts being "what does the data tell me?" That is a fundamentally different conversation.

The sceptic's question: Serverless Databricks is not new. What is new is the positioning as an instant-on workspace for non-technical users. The risk is that serverless simplicity at the provisioning layer masks complexity at the governance layer. Who controls costs when every business user can spin up compute on demand? Who manages access when workspaces proliferate? Who ensures data quality when analysis is performed by people who do not understand data modelling? These questions do not disappear because the workspace is serverless. If anything, the ease of provisioning makes them more urgent.

Lakebase: The operational database for AI applications

Bridging the analytics-operational gap

Databricks has traditionally been an analytics platform. You ingest data, process it in Delta Lake, run ML models, build dashboards. But the results live in the lakehouse. If you want to serve those results to an application — a recommendation engine, a real-time pricing model, an AI agent — you need a separate operational database. That architectural seam is expensive to maintain and creates governance complexity.

What Lakebase introduces:

Lakebase is Databricks' built-in serverless operational database, designed to power data and AI applications directly from the lakehouse.

• Transactional capabilities — ACID transactions for application workloads
• Low-latency serving — Millisecond response times for operational queries
• Unified governance — Same Unity Catalog controls as the rest of the lakehouse
• Native AI integration — Vector search and embedding storage for AI applications

The architectural significance:

Before Lakebase, a typical architecture for an AI-powered application looked like this:

Data ingested into Delta Lake
    --> ML model trained in Databricks
    --> Model outputs written to external database (Cosmos DB, PostgreSQL)
    --> Application queries external database
    --> Two governance models, two data copies, two sets of credentials

With Lakebase:

Data ingested into Delta Lake
    --> ML model trained in Databricks
    --> Model outputs served directly from Lakebase
    --> Application queries Lakebase
    --> Single governance model, single data platform

The reduction from five moving parts to four is not the point. The elimination of a separate governance boundary is. Every time data crosses a system boundary, you need credentials, access policies, audit logging, and lineage tracking for both sides. Removing that boundary is a genuine architectural simplification.

The competitive context: This is Databricks' direct response to Snowflake's Unistore and similar efforts by cloud data warehouse vendors to expand beyond analytics into operational workloads. The strategic intent is clear: prevent customers from needing another database vendor for the "last mile" between analytics and applications.

The honest question: Can Lakebase handle genuine operational workloads at the latency and throughput that applications demand? Analytics databases and operational databases have fundamentally different performance profiles. Write-heavy transactional workloads, point lookups at sub-millisecond latency, concurrent access from thousands of application instances — these are characteristics that analytics-optimised architectures historically struggle with. Databricks is betting that their serverless architecture gives them more flexibility than traditional approaches. History suggests this is harder than it looks, and the proof will be in production workloads, not conference demos.

Agent Bricks: No-code analysis of unstructured data

What Agent Bricks actually are

Agent Bricks are pre-built, composable AI agents designed for specific data analysis tasks. Rather than building an AI agent from scratch — defining prompts, connecting to data sources, handling errors, managing context — you assemble Agent Bricks into workflows that process data without requiring code.

Available Agent Brick types demonstrated:

• Document analysis — Extract structured information from PDFs, contracts, reports
• Data summarisation — Generate natural-language summaries of datasets
• Anomaly detection — Identify outliers and unusual patterns in data
• Data classification — Categorise records based on content analysis
• Question answering — Answer natural-language questions about datasets

The no-code proposition

The session demonstrated a business analyst using Agent Bricks to analyse a collection of customer support tickets — unstructured text data — without writing any code.

The workflow:

1. Point Agent Bricks at a collection of support tickets in the lakehouse
2. Select "Document Analysis" and "Data Classification" bricks
3. Define classification categories in natural language ("billing issue," "technical problem," "feature request," "complaint")
4. Run the analysis
5. View results as a structured dataset with classifications, sentiment scores, and summaries

What happened behind the scenes:

• Agent Bricks spun up serverless compute automatically
• Applied LLM-based classification to each document
• Stored structured results back in Delta Lake with full lineage
• Made results available for dashboarding, further analysis, or application serving via Lakebase

The demo was slick. But demos always are. The questions that matter are about what happens outside the demo environment.

The credibility test

No-code AI tools have a well-earned reputation for impressive demos that disappoint in practice. The critical questions for Agent Bricks:

Accuracy: How reliable is LLM-based classification compared to a trained ML model? For well-defined categories with clear boundaries, LLMs perform well. For nuanced categories that require domain expertise — distinguishing between a "complaint" and a "frustrated feature request," for example — accuracy degrades. The session did not share accuracy metrics, which is always a warning sign.

Scale: The demo used a manageable dataset. What happens with millions of documents? Serverless compute helps with raw processing power, but LLM inference at scale is expensive. The cost per document could make large-scale analysis prohibitively expensive compared to traditional ML approaches that are cheaper per inference after training.

Governance: When an AI agent classifies data, who validates the classification? If Agent Bricks classify a customer complaint as a "feature request," that misclassification propagates into every downstream analysis, dashboard, and business decision. Unity Catalog tracks lineage, but lineage does not prevent errors — it just makes them traceable after the damage is done.

Iteration: The first pass of any analysis is rarely correct. How easy is it to refine classifications, adjust categories, handle edge cases, and re-run analysis with Agent Bricks? The session showed a clean, first-pass workflow but did not demonstrate the iterative refinement that defines real analysis work. The difference between a demo and a useful tool is often in the iteration loop.

Reproducibility: If you run the same Agent Bricks workflow on the same data twice, do you get the same results? LLM-based classification is inherently non-deterministic unless temperature is set to zero, and even then, results can vary. For compliance, audit, or regulatory reporting use cases, non-deterministic classification is a serious concern.

AI/BI Genie + Copilot Studio: Data in Teams

The integration that changes the audience

The most strategically significant announcement in THRSP904 was the integration between AI/BI Genie and Microsoft Copilot Studio, enabling natural-language data queries directly in Microsoft Teams.

What this means practically:

A sales manager in Teams asks: "What were our top 5 products by revenue last quarter in EMEA?"

Instead of:

1. Submitting a request to the BI team
2. Waiting 2-5 days for a dashboard update
3. Receiving a static report that prompts follow-up questions
4. Submitting another request

The response is instant:

1. Copilot Studio routes the question to AI/BI Genie
2. Genie generates the appropriate SQL query against the lakehouse
3. Results are returned as a formatted table in Teams
4. The manager asks follow-up questions conversationally

The democratisation promise — and its limits

The genuine value: Removing the intermediary between business questions and data answers is transformative. Every organisation has a backlog of BI requests. Every data team is a bottleneck. If AI/BI Genie can reliably answer straightforward analytical questions, it frees data teams to focus on complex analysis while empowering business users with self-service access.

The governance challenge: When anyone can query the lakehouse via Teams, who controls what they can see? Unity Catalog provides the access control layer, but the surface area of potential queries is vastly larger than a curated set of dashboards. A natural-language interface makes it trivially easy to ask questions that the data governance framework was not designed for.

Example risk scenario:

A manager asks in Teams: "Show me individual salesperson performance rankings with compensation data."

Technically, the manager might have read access to the underlying tables. But the organisation may never have intended for that data to be surfaced casually in a Teams chat. The access control model works at the table and column level, but the intent of the query requires a higher-level policy layer that does not yet exist in most organisations.

The accuracy problem: Natural language to SQL is a well-studied problem, and it is hard. Ambiguous questions produce ambiguous queries. "What is our revenue?" could mean booked revenue, recognised revenue, pipeline revenue, or ARR depending on context. AI/BI Genie needs domain-specific configuration to handle these ambiguities correctly.

The session acknowledged this with "Genie Spaces" — curated data domains with business glossaries that provide the semantic context Genie needs to interpret questions correctly. This is the right architectural approach, but the configuration burden shifts from building dashboards to building Genie Spaces. The work does not disappear; it changes form.

-- What Genie generates behind the scenes
SELECT
    p.product_name,
    SUM(s.revenue) AS total_revenue
FROM sales.fact_sales s
JOIN products.dim_product p ON s.product_id = p.product_id
JOIN geography.dim_region r ON s.region_id = r.region_id
WHERE r.region_name = 'EMEA'
AND s.sale_date BETWEEN '2025-07-01' AND '2025-09-30'
GROUP BY p.product_name
ORDER BY total_revenue DESC
LIMIT 5;

The SQL above looks straightforward. But getting Genie to generate it correctly requires that "revenue" maps to s.revenue (not s.gross_revenue or s.net_revenue), that "last quarter" resolves to the correct date range, and that "EMEA" maps to the correct region value. Each of these mappings is a Genie Space configuration decision that requires business knowledge.

The honest assessment: This is not a replacement for a BI team. It is a force multiplier. AI/BI Genie handles the 60-70% of questions that are straightforward lookups, aggregations, and comparisons. The BI team handles the 30-40% that require complex modelling, contextual interpretation, and data quality judgment. The net effect is that the organisation gets more analytical value from its data investment.

The strategic picture: Databricks versus Fabric

The elephant in the room

The session was presented at a Microsoft conference, using Microsoft integration points (Teams, Copilot Studio), by a Microsoft partner. But the strategic subtext was unmistakable: Databricks is positioning itself as an alternative to Microsoft Fabric for enterprise analytics and AI.

Where Databricks and Fabric overlap:

• Unified data platform for analytics and AI
• Serverless compute
• Natural-language data queries
• Integration with Microsoft Teams and the broader Microsoft ecosystem
• Governance and cataloguing capabilities

Where Databricks differentiates:

• Multi-cloud by design. Databricks runs on Azure, AWS, and GCP. Fabric is Azure-only. For organisations with multi-cloud strategies, this is a decisive advantage.
• Open source foundations. Delta Lake, MLflow, and Apache Spark are open source. Fabric uses proprietary formats and engines. The lock-in risk profile is different.
• Data engineering depth. Databricks' notebook experience, Spark integration, and MLflow ecosystem are more mature for data engineering and ML workloads than Fabric's equivalents.
• Unity Catalog. The governance and cataloguing capabilities are more mature and more broadly adopted than Fabric's OneLake catalogue.

Where Fabric differentiates:

• Native Microsoft integration. Fabric integrates natively with Power BI, Synapse, Data Factory, and the rest of the Microsoft data stack. Databricks integrates via connectors and APIs.
• Licensing simplicity. Fabric's capacity-based pricing is simpler than Databricks' per-DBU model for organisations already invested in Microsoft licensing.
• End-user accessibility. Power BI is the most widely deployed BI tool in the enterprise. Fabric leverages that installed base.

The strategic question for enterprises: Do you standardise on Fabric for simplicity and Microsoft integration, or on Databricks for multi-cloud flexibility and open-source alignment? THRSP904 made the case for Databricks, but the answer depends on each organisation's priorities, existing investments, and strategic direction.

The Copilot Studio integration: Partnership or positioning?

The Teams integration via Copilot Studio is strategically interesting. Microsoft is positioning Copilot Studio as the orchestration layer for enterprise AI agents. Databricks is positioning Agent Bricks as specialist data agents within that ecosystem.

This is mutually beneficial today: Microsoft gets rich data capabilities for Copilot, Databricks gets distribution through Teams' massive enterprise footprint. But the relationship has inherent tension. Microsoft's Fabric vision includes its own natural-language data query capabilities via Copilot in Power BI. Databricks' Genie in Teams competes directly with that vision.

For enterprises, the practical question is: do you deploy Genie in Teams, Copilot in Power BI, or both? The answer will depend on where your data lives, which governance model you trust, and how much complexity you are willing to manage. Running both is technically possible but adds cognitive overhead for users who need to know which tool to use for which question.

Enterprise readiness assessment

Based on what was demonstrated in THRSP904, here is an honest assessment of each capability's production readiness:

Serverless workspaces: Production ready. This is the most mature component. The technology is proven, the economics are well-understood, and the governance model is established through Unity Catalog. Deploy with confidence, but implement cost controls and access governance from day one.

Lakebase: Early stage, approach with caution. The architectural concept is sound, and the simplification of removing an external operational database is genuinely valuable. But operational database workloads are unforgiving. Test thoroughly with production-representative load before committing. Have a fallback plan that includes an external database.

Agent Bricks: Promising, not yet proven. The no-code analysis proposition is compelling, but accuracy, scale, and governance questions remain unanswered. Deploy for exploratory analysis where accuracy is reviewed by humans before decisions are made. Do not deploy for automated workflows where incorrect classifications have operational consequences.

AI/BI Genie in Teams: Useful with significant configuration investment. The technology works for well-defined data domains with clear semantics. The Genie Space configuration investment is substantial and ongoing. Start with a single, well-understood data domain and expand based on measured accuracy and user feedback.

What to watch

Lakebase maturity: Can it genuinely handle operational workloads at production scale, or will customers still need a separate database for low-latency serving? Watch for independent benchmarks, not vendor-published numbers.

Agent Bricks accuracy at scale: Demo accuracy and production accuracy are different things. Watch for real-world case studies with accuracy metrics, particularly in regulated industries where classification accuracy has compliance implications.

Genie Space complexity: How much configuration is required to make AI/BI Genie reliably accurate for domain-specific questions? If Genie Spaces are as complex to build as traditional BI semantic models, the democratisation benefit is diminished and the effort is simply relocated.

Cost at scale: As organisations scale Agent Bricks usage and Genie queries, the cost per analysis and cost per query could become limiting factors. Watch for pricing model evolution and cost optimisation patterns.

Databricks-Microsoft relationship dynamics: As Fabric matures and its capabilities increasingly overlap with Databricks, the partnership dynamic will evolve. Watch for changes in integration depth, co-marketing, and competitive positioning that signal whether the relationship is strengthening or straining.

Bottom line

THRSP904 presented a compelling vision: enterprise data analysis accessible to everyone, powered by AI, served from a unified lakehouse platform. Serverless workspaces remove infrastructure friction. Lakebase eliminates the analytics-operational gap. Agent Bricks make unstructured data analysis possible without code. And the Teams integration puts data answers where people already work.

The vision is right. The execution is early. Serverless workspaces are the most mature component and deliver immediate value. Lakebase is architecturally interesting but unproven at operational scale. Agent Bricks need accuracy validation beyond demo scenarios. And AI/BI Genie in Teams requires careful governance to prevent data access from outrunning data policy.

For data teams evaluating their Databricks strategy, the direction is clear: Databricks is becoming a full-stack data and AI platform, not just a processing engine. That has implications for architecture decisions, vendor strategy, and team skills. The question is not whether to move in this direction, but how quickly the platform components mature enough to trust in production.

For organisations running both Databricks and Microsoft Fabric — and there are many — this session clarified the competitive overlap. The data platform decision is no longer just about processing engines and storage formats. It is about which ecosystem owns the natural-language interface to your data, which platform governs your AI agents, and which vendor's roadmap you trust to execute. Those are strategic decisions that deserve strategic attention.

Related coverage:

• Microsoft Ignite 2025 Keynote Review
• Building Multi-Agent Systems with Azure AI Foundry
• AI Agent Governance

Session: THRSP904 | Nov 18, 2025 | Moscone South, The Hub, Theater A