Any patterns or recurring topics you want me to cover? Thanks for filling this 1-minute survey 🙏. docs.google.com/forms/d/e/1F...

New chapter out about Data Engineering Workspace Packaging.

Random shoutout - I just started to read Data Engineering Design Patterns by @ssp.sh and it is full of tremendous inspiration, not only from a technical perspective, but also on how to effectively communicate data challenges. Getting pumped abt returning to work after paternity leave dedp.online

What should be next?

Latest pattern published.

I am happy to share that I released my second pattern: «Data-Asset Reusability Pattern», focusing on #EfficientDataReuse. This chapter explores reusability in data engineering with dedicated sub-patterns. But first, what is data-asset reusability?

The history and state of data engineering have changed drastically over the years. Let's explore what and how it changed. Much has been written about data engineering, but how long has it existed, and where does it come from? 🌐 dedp.online/part-1/1-int... (direct link)

The book on the front page on Hacker News.

📖 New chapter: Evolution of Schema Change and database change management. > #SchemaEvolution: Managing database changes while preserving integrity > #NoSQL: Flexible schemas; speed over consistency > #DataContracts: Producer-consumer agreements with automated validation dedp.online/part-2/4-ce/...

After years in #dataengineering, I've noticed we focus too much on tools & tech, missing the fundamental challenges. It's not about picking the right stack—it's about mastering the complete data lifecycle. The reality: We can't control source systems and their upstream data yet.

A recent review I've got 🥰. > what a great content! "awesome, fresh view. I shown it to my fiancee & said "that's what wisdom is, like I would meet an 60 year old druid living in a forest and doing DE pipelines work". > I'm patiently waiting to read more, It's a great piece &work that inspires me.

Data Engineering Design Patterns. This book aims to provide a practical guide based on convergent evolution, resulting in design patterns applicable for navigating the data engineering lifecycle—timeless practices for data engineers.