Latest pattern published.

I am happy to share that I released my second pattern: «Data-Asset Reusability Pattern», focusing on #EfficientDataReuse. This chapter explores reusability in data engineering with dedicated sub-patterns. But first, what is data-asset reusability?

The history and state of data engineering have changed drastically over the years. Let's explore what and how it changed. Much has been written about data engineering, but how long has it existed, and where does it come from? 🌐 dedp.online/part-1/1-int... (direct link)

The book on the front page on Hacker News.

📖 New chapter: Evolution of Schema Change and database change management. > #SchemaEvolution: Managing database changes while preserving integrity > #NoSQL: Flexible schemas; speed over consistency > #DataContracts: Producer-consumer agreements with automated validation dedp.online/part-2/4-ce/...

After years in #dataengineering, I've noticed we focus too much on tools & tech, missing the fundamental challenges. It's not about picking the right stack—it's about mastering the complete data lifecycle. The reality: We can't control source systems and their upstream data yet.

A recent review I've got 🥰. > what a great content! "awesome, fresh view. I shown it to my fiancee & said "that's what wisdom is, like I would meet an 60 year old druid living in a forest and doing DE pipelines work". > I'm patiently waiting to read more, It's a great piece &work that inspires me.

Data Engineering Design Patterns. This book aims to provide a practical guide based on convergent evolution, resulting in design patterns applicable for navigating the data engineering lifecycle—timeless practices for data engineers.