A few weeks ago we launched dltHub Pro. What surprised us was what people built with it. Teams didn't ask for primitives, they asked for complete solutions. So we built Blueprints.

How do you eval your agents? Agent traces are a good place to start. In this 1-hour workshop with @DataTalksClub, you'll learn how to ingest agent traces, model nested JSON into a queryable schema, and build dashboards to understand agent behavior.

AI writing pipelines is cool. AI remembering the entire workflow is better. dltHub Pro carries context from ingestion to deployment to maintenance instead of starting over every step. Try it: https://dlthub.com/products/dlthub

AI agents generate traces, but are you analyzing them?  Join Alena Astrakhantseva to learn how to turn tool calls, token usage, and outcomes into structured, queryable data with dltHub Pro. July 6, live on YouTube with DataTalks.Club Register ↓

Text-to-SQL doesn't break because models can't write SQL, it breaks because they don't know what your data means. An agent can return valid SQL and still be wrong, and a clean wrong number looks just as trustworthy as a right one.

Human in the loop shouldn't mean copy-pasting context into an agent every 5 minutes. It should mean judgment, not errands. If your agent needs a Rube Goldberg machine of prompts, tabs, and slack messages, the problem isn't human. It's missing context. https://dlthub.com/blog/context

Gave an LLM a schema: 3/10. 📉 Gave it a schema + an ontology: 10/10. 🎯 Same model, same data. The difference? It finally understood what the columns actually mean instead of just vibing off the names.

91% of the 81,000 new dlt pipelines shipped in January 2026 were built by agents. The bottleneck in data engineering is no longer implementation, it's meaning.

How much data can $1 of compute move? We benchmarked dltHub on a small worker (2 vCPU / 4 GB) loading into BigQuery: Parquet: ~170 GB Postgres: ~65 GB JSON: ~4.6 GB REST: whatever the API allows Methodology + results ↓ https://dlthub.com/blog/benchmark-dlthub

Data quality usually starts too late. By the time bad data hits a dashboard, the original business assumptions are gone. We're fixing that at the ingestion layer. 👇

Agentic Analytics Demo Night lands in San Francisco this Wednesday 🚀 We're joining @inngest.com, lightdash, and @streamkap.bsky.social for an evening of demos from teams building at the intersection of AI, analytics, and data infrastructure. 🧵

What does it take to build a product people actually keep using? On Wednesday, we're hosting Sameer Alsakran, Founder & CEO of @metabase.com, at our office in Berlin for a live conversation with Francesco Mucio from Data Berlin. 🧵

Berlin’s Applied AI Week is looking pretty special. Tomorrow, dltHub and @modal-labs.bsky.social are hosting an evening in Berlin for founders, engineers, and builders shipping agents in production. Featuring Kenny Ning, Jefferson Girao & Alena Astrakhantseva.

The AI stack is evolving fast, but reliable data movement is still the foundation. Join dltHub, LanceDB and DataHub on May 21 in Menlo Park for talks on multimodal AI storage, AI data pipelines, and trusted lineage systems. 🔗

How fast can you go from zero to a production-ready data pipeline when AI is your copilot? Next Wednesday, Elvis Kahoro joins @temporal.io alongside Melanie Warrick and @cecilphillip.bsky.social for a live Vibe Check building a GitHub-powered pipeline with AI + dlt.

Building pipelines with AI usually means losing context between tools. dltHub AI Workbench runs the full 12-step workflow as a continuous session, schemas, incrementals, traces, transformations, and notebooks share context across the stack. https://dlthub.com/blog/agentic-data-engineering-course

Explainer on ontology engineering and what we're building around it: why just clean schemas and prompts aren’t enough, and how adding a canonical model + taxonomy + ontology changes what agents can correctly compute (ARPU being the clearest example). 

We’re excited to share that Violetta Mishechkina will be speaking at GOSIM Paris 🇫🇷 Invited by probabl.ai to join the “Own Your Data Science and AI” workshop. 🎤 From Agent Traces to Analytics Agents generate code, text, telemetry, yet most teams still rely on stale datasets.

Skills that wrap a library are software. They have dependencies, need maintenance, and degrade when the product changes and no one updates them. The vendor owns the product surface. You own your integration. Same rule, new layer.

1/ dlt is now available as a Snowflake Native App. Replicate MSSQL, MySQL & PostgreSQL → Snowflake, without leaving Snowflake. Create, schedule, and monitor pipelines from the Snowflake UI. No external orchestrator. https://app.snowflake.com/marketplace/listing/GZ1MMZ1IPICU/dlthub-dlt-connector-app

Every layer of software automation was called overkill before it became the baseline. Fortran → Make → CI/CD → Docker → now agents. Code that runs is only the 10%. The other 90% is engineering judgment, boundaries, and iteration.

AI agents don’t just use your APIs, they optimize around them. Ask an agent to “build a pipeline” and it will find credentials, escalate privileges, and take the shortest path to completion. Not against your interests, just goal-driven.

Not everything that can be modeled should be. With LLMs, more context doesn’t mean a better prompt. The key is Minimum Viable Context for high-precision data models. Here’s what we learned building ontology-driven modeling. Blog by Hiba Jamal ↓

AI can generate a data pipeline in 10 minutes. But can you trust what it produces? That’s the real problem, and it’s not technical. It’s business. If you can’t trust the output, it never reaches production. 🧵

Most AI coding tools stop at “here’s your code.” But getting pipelines into production, and trusting them there, is the hard part. We built a deployment toolkit that closes that gap 🧵

Most teams still build connectors from scratch, one at a time. Different patterns. Different implementations. Accumulating tech debt. What if you built the system instead? We just released dlt Skills, a pipeline factory powered by Claude. 👇

LLMs fail at data transformation because they see isolated tables, not your business. We built the dltHub AI Workbench transformation toolkit to fix that. You feed it sources + use cases → it builds a taxonomy, business ontology, and a Canonical Data Model.

The craziest part of the new dltHub AI release? The MCP integration. Asked Claude Code for an OpenAI pipeline -> it searches the dlt context -> scaffolds the exact code with schema & incremental loading. No more starting from scratch. https://dlthub.com/blog/ai-workbench

New course: Agentic Data Engineering with dltHub 🤖 Agents can now write entire data pipelines, but writing code was never the hard part. The real challenge? Data quality, schema stability, and running reliably in production. 🧵👇

Agents can generate data pipelines. The real problem now is trusting them in production. Today we’re introducing the dltHub AI Workbench, infrastructure for generating, validating, and deploying pipelines in one workflow.

WAP (Write → Audit → Publish) has a blind spot. It assumes your data can safely land in staging. But in real pipelines, ingestion is often where things break. That’s where AWAP comes in.

Your production traces are gold, but they aren’t training data yet. We turned raw agent traces into a specialist model using: dlt → @hf.co → Distil Labs In our example, a 0.6B model beat its 120B teacher by 28 points on a specific task. Here’s how it works ↓

Curious what people here are actually using for AI coding 👀 Copilot? Cursor? Claude Code? We put together a super short (1-min) survey. 👉 https://dlthub.notion.site/3039fb8e23cf80488c45ea690778a618?pvs=105

Microsoft Fabric is great at compute & storage but data quality enforcement is on you. Use WAP to validate data before it hits the lakehouse. dlt handles schemas, business rules, uniqueness, PII, and monitoring so bad data never reaches analytics.

LLMs follow the gravity of your vocabulary, not your business logic. That's why your AI-generated stack looks great in the demo and breaks silently in prod.

❄️ Module 2 is live in our course dlt + Snowflake  Learn how to run dlt pipelines inside Snowflake using Snowpark Container Services (SPCS), enabling native execution and scheduling with no external infrastructure required. Continue the course: https://dlthub.learnworlds.com/course/dlt-snowflake ↓

We shipped a @hf.co Face datasets destination in dlt. It makes it easier to move training data between production systems and the HF Hub. Think pipelines like: raw traces → dlt → versioned datasets on Hugging Face → model training ↓

Small data teams deserve better tools. We’re opening early design partnerships for solo and small data teams to try dltHub Pro before launch. Early access, influence the roadmap, and an early-bird discount.  https://dlthub.com/solutions/for-small-data-teams

Prototyping data pipelines in @duckdb.org is great until you need to ship them to production. On Mar 16, Elvis Kahoro & @joshleecreates.bsky.social from Altinity will show how to move a DuckDB pipeline to @clickhouse.com using dlt, fully defined as Python code. Thread ↓

Spring brings good things and more than just flowers. 🌸 ❄️ Catch up on Module 1 of our dlt + Snowflake course before Module 2 drops next week! Learn nested data normalization, schema evolution, incremental loading, and merge strategies (upsert, SCD2), all in plain Python.

Tasman.ai runs data engineering projects for mid-market and enterprise clients. Their biggest challenge? Scoping. Every new client meant figuring out which APIs to connect, how long it would take, and what the data actually looked like, often before seeing a single row.

Data models describe the data, ontologies describe the world. With ontologies, an agent can reason over data as opposed to retrieving it and hallucinating meaning. The ontology-model mapping is what agents need for data literacy. This is NOW. Blog + demo

Who is the UFC GOAT? 🥊📊 We turned that curiosity into a full-stack pipeline analyzing 30+ years of UFC fights. Everything programmatic, even dashboard creation via API. Production-grade insights. Full traceability. Zero manual overhead.

Part 2 of our RAG debug series is out. We froze retrieval, prompts & dataset, and tested newer models only. Result: 3/14 → ~10/14 correct answers. 3× improvement just by upgrading the model. Same system. Same eval set. Retrieval is next. Read more 👇

AI agents fail without memory & context. @cognee.bsky.social turns data into self-improving, structured memory at scale. If you’ve built a modern data stack (ingest → transform → access), you already know the pattern. Backed by pebblebed, congrats on the $7.5M.

Ingestion shouldn’t be a maintenance trap. From Airbyte to dlt in one week. Slides 👇 https://docs.google.com/presentation/d/e/2PACX-1vQvJapgEkJxgpsWqoMlmEw-ctV3gZe0LLc5oZBHaJNezBGAYKYoyir1aQi-37tO37SjFGaYjmQJhi_r/pub?start=false&loop=false&delayms=3000&slide=id.g175a817e68e_3_932

From APIs to Warehouses 📦 On Feb 17 (16:30 CET), together with DataTalks.Club, Aashish Nair will walk through building end-to-end ingestion pipelines with dlt, from raw APIs to production-ready warehouse loads. Register here 👇

What if dimensional modeling didn’t mean hours of boilerplate SQL? We built an AI workflow that turns raw data into semantic models in minutes, powered by 20 questions. Rethinking data transformation 👇

Berlin, it’s meetup time! Join us for the dltHub Community Meetup, an evening of real-world demos, lessons learned, and conversations with builders. 📍 Rosebud, Berlin 📆 Feb 17 | 18:00 – 21:00 Curious about what we’re building at dltHub? Come by 👋

Production pipelines don’t fail loudly, they drift. Feb 12 · 16:00 CET - Online Hands-on workshop on operating pipelines in production: • schema changes • backfills • CI/CD • long-term reliability Register → https://community.dlthub.com/workshop-maintaining-servicing-production-data-pipelines