Explaining how Python engines read and write #DeltaTable is not for the faint of heart. The theory is everything will depends on the delta kernet rust for read and write, but we are not there yet github.com/djouallah/Fa... #duckdb #delta_rs #datafusion #chdb #daft #polars #rust #lakesail

Any system that allows exchanging real money for stuff with an element of chance is morally equivalent to a casino. Corollary: Pokémon cards, Roblox, Labubus, and even claw machines should all be 18+

running #duckdb at 10 TB scale using #tpch like benchmark #delta #onelake #singlenode #MicrosoftFabric datamonkeysite.com/2025/10/19/r...

you are looking at #duckdb running tpch 1 TB with only 16 cores it used to crash even with 64 pip install duckdb --upgrade is an act of faith basically

Put together a small python package duckrun :) point it at a folder of SQL/Python files, define a pipeline, and it will create Delta tables in #OneLake with #DuckDB and #delta_rs github.com/djouallah/du...

actually #Microsoftfabric Datawarehouse automatically expose an Iceberg rest Catalog thanks to #duckdb UI extension, you can see proper catalog

First Look at #onelake #apacheiceberg REST Catalog, please notice it is coming soon and not in production yet #MicrosoftFabric www.youtube.com/watch?v=_QRE...

#pyconau @mimdj.bsky.social Life Beyond Pandas: Workflows with DuckDB, Daft, Polars, and Datafusion http://youtu.be/SnogunyMnE8

2 months ago, I got access to a beta release of #onelake #Apacheiceberg REST Catalog, first thing I run it with #duckdb 😀

storage format should not be tied to #SQL logic, #duckdb got it so right !!! but a bit sad that #deltalake is left behind :(

First look at incremental framing in #PowerBI directlake mode #Parquet #OLAP #optimization #Performance #onelake datamonkeysite.com/2025/09/09/f...

first look at geometry data type in #parquet datamonkeysite.com/2025/09/07/f...

first #apacheiceberg table written by #duckdb

good news #duckdb added support for reading and writing geometry data type Bad news : other Fabric engines don't support it yet, so it is not very useful for now :(

#duckdb is the goat github.com/djouallah/Fa...

Third time is the charm ✨ With the much needed improvements to the #MicrosoftFabric scheduler, I revisited my review of Fabric F2. #duckdb #sql www.youtube.com/watch?v=tchY...

new world record 😝 using #duckdb and #ducklake 22 cents for the 3 B rows, coffee benchmark :) not bad at all for a single node :) www.linkedin.com/posts/mimoun...

Writing #ApacheIceberg in Azure is not particularly hard, but you do need a catalog (essentially a database). For simple tests, you can use an in-memory DB #ADLS #opentableformat #PyIceberg.

mssql-python vs pyodbc: Benchmarking SQL Server Performance - devblogs.microsoft.com/python/mssql... A pretty big rewrite, actually

I hope this is a fair subjective assessment of the current state of #Python data processing engines

#gpt-oss 20B vs #qwen for Text to #Sql datamonkeysite.com/2025/08/07/u...

first impression of #qwen3 30B-A3B-Instruct-2507 Local analyzing a wide table in my laptop it still feels strange that all that knowledge is encoded in one file in my hard drive :) www.youtube.com/watch?v=gmk7... #mcp #nl2sql #sql

I see a new destination for Dataflows Gen2 in Fabric! #SharePoint destination is in preview! This opens up so many possibilities for business applications! You can save your prepared and transformed data in CSV or Excel file formats. #MicrosoftFabric #DataFactory #Dataflows

I think this whole #mcp thing will change data analytics as we know it datamonkeysite.com/2025/08/01/a...

Does a Single-Node #Python Notebook Scale? datamonkeysite.com/2025/07/21/d...

#ducklake , #pyiceberg and #deltalake for single writer www.youtube.com/watch?v=SUU0...

POC, export #ducklake metadata to #deltatable format, #apacheSpark and Fabric DWH, #PowerBI can read it github.com/djouallah/Fa...

first look at #ducklake youtu.be/LD0Qh0Lmsec

The first solar farm I worked on , 8 years ago :) #PowerBI #Python djouallah.github.io/AEMO-POWERBI/

Some Observations on Running #TPCH 1 TB on #MicrosoftFabric datamonkeysite.com/2025/05/25/s...

Watched the original Highlander with my 10yo last week and though it was a joy to share, I'm excited it's going to get a refresh.

Stress Testing #apacheIceberg shortcut in #Onelake datamonkeysite.com/2025/05/07/s...

A Non-scientific Benchmark of Text to #SQL using Small Language Models using Qwen3, 4B, 8B and 14B, takeaway, maybe I need a better laptop datamonkeysite.com/2025/05/05/a...

#Powerbi compression is insane !!!

Vibe Coding a Simple #SQL orchestrator in a #MicrosoftFabric #Python Notebook AI code assistant works very well datamonkeysite.com/2025/04/27/v...

An #Excel User’s Perspective on #Lakehouse Architecture datamonkeysite.com/2025/04/14/a...

Experimenting with Text-to-SQL: Lessons from Optimizing Product Return Analysis TL:DR ; don't join on text and LLMs maybe generate correct #SQL queries but with very bad performance #PowerBi #duckdb datamonkeysite.com/2025/04/10/e...

LLMs are getting really good at working with structured data 🚀 I tried a quick experiment using the latest update of GPT-4o. Instead of loading an existing semantic model from PowerBI, Chatgpt built a semantic model from scratch and then used it to analyze the data. www.youtube.com/watch?v=oN4F...

A new blog post, in which I dust off my ETL and data modelling skills, and build a data pipeline with @duckdb.org: rmoff.net/2025/03/20/b... #dataBS

controversial Take : Metastore at the storage layer make a lot of sense, Engines have a lot of incentives to be a neutral player.

🚀 Querying #MicrosoftFabric DWH Data from Your Laptop with #DuckDB UI 🦆💻 I am not saying it is a good idea to do so, but you can if you want. www.youtube.com/watch?v=q8Dp...

How to Read a #DeltaTable with Deletion Vectors and Column Mapping in #Python datamonkeysite.com/2025/03/19/h... #Microsoftfabric #onelake #duckdb #deltatlake

Real-time analytics with Microsoft Fabric Learn how to stream, process, and visualize live data using Fabric RTI, Notebooks (Python), Real-Time Dashboard, EventStream, KQL, and #PowerBI. Automate alerts with Data Activator. youtu.be/pT4xawPV3sM #MicrosoftFabric

Now you can load arbitrary files (csv, json, parquet etc) to #onelake using a standard API, same code will works for S3, GCP etc thanks to the excellent rust package object_store now ported to #Python #obstore datamonkeysite.com/2025/03/16/u...

how to connect to #onelake using duckdb UI step 1 : install azure cli in your laptop step 2 : login to your account step 3 : load credential in Duckdb CREATE or replace SECRET onelake( TYPE azure, PROVIDER credential_chain, CHAIN 'cli', ACCOUNT_NAME 'onelake' ); enjoy

LLMs Are Getting Better at #SQL if you have a well defined semantic model datamonkeysite.com/2025/03/07/l...

one of the simplest trick to reduce #Microsoftfabric consumption (or any data platform for that matter) is just to reduce data refresh frequency. just ask your audience if it is really needed #Powerbi #Python datamonkeysite.com/2025/03/05/o...

Improve #PowerQuery performance on CSV files containing Date columns blog.crossjoin.co.uk/2025/02/09/i...

ready #BigQuery #iceberg table in #Microsoftfabric #onelake make sure your iceberg metadata is up to date. datamonkeysite.com/2025/02/04/r...

Limit the impact of expensive #PowerBI queries on your capacity by reducing the query timeout blog.crossjoin.co.uk/2025/02/02/l...