TueAM5: Agile Data Warehousing

For decades now, data warehouses were extremely “brittle” in the face of new or changing business requirements. Once loaded, their data repositories could not be repurposed without converting the existing data, a practice that can be ruinously expensive. As a result data warehouse designers were forced to doom their projects to “death by architecture,” in which they insisted on perfect, up-front designs before data transform programming could begin. Luckily, the new data modeling paradigms of hyper normalizing and hyper generalizing allow teams to readily adapt established warehouses in new directions without conversion script programming. Teams can now start with a simple design that meets current business requirements, then steadily evolve the database as new business needs emerge. These novel techniques have hundreds of successful implementations and strong support by vendors and practitioners. In this presentation, we will consider a couple of such case studies, then touch upon:

Why traditional data models become so expensive to update
Why “database refactoring” leaves much to be desired
Variations upon standard normal forms that allow agile data engineering
How highly generalized data models enable model-driven development
Where tools can amplify the power of these new modeling techniques

Class Reviews:

"Great job covering lots of info effectively."

"Good presenter! Engaged the audience."

"I hate to give 10's but I have to."

"This was the best session of the entire conference. It taught me a great deal about data warehousing design processes."

"Very timely, challenging & great techniques."

"A good blend of agile terminology aligned with practical day-2-day use."

"Great! Great! Great!"

"It was a great introduction to using agile for warehouse development."

Ralph Hughes serves as Chief Systems Architect for Ceregenics, a Denver data analytics consulting firm. He has been building data warehouses since the mid-1980s, starting with mainframe computers, and has led numerous BI programs and projects for Fortune 500 companies in aerospace, telecom, government, and life sciences. He authored the industry's first book agile data management, and his third book, "Agile Data Warehousing for the Enterprise," was just released in October 2015. He is a faculty member at The Data Warehousing Institute, a certified Scrum Master, PMI Project Management Professional, and has coached over 1,200 BI professionals worldwide in the discipline of incremental and iterative delivery of large data management systems. He holds BA and MA degrees from Stanford University in computer modeling and econometric forecasting. He can be contacts at ralph.hughes@ceregenics.com or 303.274.9101.