Since 2005, I’ve immersed myself in the dynamic world of data and its modeling. It’s been fascinating to witness how data modeling has evolved over the years, navigating through its challenges and successes. Throughout this journey, advocating for its significance has been a constant endeavour, emphasizing its crucial role not just as a practice, but…
Category: Data Engineering
Looking back to Kimball’s approach to Data Warehousing
Over time, it’s fascinating to witness how certain concepts, approaches, or visions age. Time, in its inexorable flow, often brings justice to ideas and individuals alike. Such a reflection is pertinent when considering the enduring impact of Ralph Kimball’s groundbreaking book on dimensional modeling for data warehousing and business intelligence.
Playing with some Pandas functions and Airflow operators
Recently, I was dealing with a task where I had to import raw information into a staging PostgreSQL Database as part of a task in the Airflow DAG pipeline. I would like to share the approach used and exposes the valuable Pandas.json_normalize function and how it helps to prepare the data before importing. Reading the…
Getting the last modified directory with Python
Working with os paths is a prevalent task, especially when working on a console application in Python. Recently, I had to add a change to an existing feature where it was mandatory to return the last modified directory on a specific target. In Python, we have the opportunity to count with different libraries; in this…