Topic
Databricks
7 episodes
-
Beyond Dashboards Introducing Decision Execution Platforms
Databricks introduces Decision Execution Platforms (DEPs), a new analytics category that automates the full executive decision loop—from signal detection through execution to outcome measurement—on governed Lakehouse infrastructure. The article argues that traditional BI only improves decision inputs; DEPs aim to orchestrate the entire decision workflow, with a Fortune 100 retail case study targeting a $100M+ fulfillment gap. Tyler questions whether the architecture genuinely solves the constraint-expressibility problem that kills most agent systems in practice; Pippa sees the product framing as a real reset from 'dashboards tell you what's wrong' to 'the system executes and measures what you chose.'
-
Lakeflow New Era Agentic Data Engineering
Databricks' Lakeflow introduces agentic data engineering: AI-driven pipeline development, ZeroOps automation, 100+ connectors, Kafka-free ingestion, and real-time Spark pipelines. The core claim is that AI can handle data pipeline maintenance and optimization, reducing operational overhead.
-
PixelRAG beats text parsers, cuts agent costs 10x
Justy and Cody dissect PixelRAG, a new research system that skips text parsing entirely by feeding rendered webpage screenshots directly to vision-language models. They break down the three specific failure modes of traditional parsers (parser loss, rank loss, reader loss) and discuss whether the 10x cost reduction and accuracy gains hold up against the engineering reality of managing image indices.
-
Debunking 8 Data Layout Myths Why Liquid Clustering Outperforms Partitioning
Justy and Cody dig into Databricks arguing that Liquid Clustering beats old-school partitioning for modern lakehouse tables. Cody buys some of the technical case, especially the point that modern formats prune from table metadata rather than folder paths, but he pushes on how much of the evidence is vendor-controlled and how broadly the claims travel outside Delta-heavy setups. Justy leans into who should care: teams with shifting query patterns, painful repartitioning, small-file messes, or mixed batch and real-time workloads. They land on a pretty practical verdict: this is less a universal law than a strong sign that manual partition design is becoming a tax many teams no longer need to pay.
-
How Lakebase Architecture Delivers 5x Faster Postgres Writes
Justy and Cody dig into Databricks Lakebase claiming much faster Postgres writes by turning off full page writes at the compute layer and pushing page image generation into distributed storage. Cody likes the architectural trick but questions where the complexity moved, while Justy argues the real win is for teams hitting write bottlenecks without wanting to re-architect their app.
-
Databricks tested a stronger model against its multi step agent on hybrid queries. The stronger model still lost by 21%.
Databricks' research shows multi-step agents outperform single-turn RAG systems on hybrid queries, achieving gains of 20% or more on Stanford's STaRK benchmark suite.
-
Databricks Set to Accelerate Agentic AI by Up to 100x with Mooncake
Databricks' new 'Mooncake' technology aims to revolutionize agentic AI, making it faster and more efficient. This could drastically improve various sectors by enabling smarter, real-time data-driven decisions. Hosts delve into its implications, applications, and potential impact on industries.