CIOReview | | 9 DECEMBER 2022future target state, and then the gaps, and roadmap. ADM also includes maintaining requirements in an artifact repository (which originally were tons of static diagrams -now most modern data tools can produce automatically).Big "A" architects hiding away an ivory tower, then publishing static artifacts of the ADM in a big bang approach, without delivering value-adding solutions along the way is a bad idea.Fortunately, many tools in the modern data stack can be spun up pretty quickly and relatively cheaply (at least in the beginning), are easy to use, and incredibly powerful. So it is an understandable choice to select toolsets that can solve perhaps a local problem, and cycle through some of the ADM steps quickly (or skip some entirely).But data is an enterprise asset, so for solutions that can and should impact the enterprise, it is important to have applicable processes, standards, principles, and policies in place. TOGAF can help as a starting point. Lean IX is also a fantastic approach worth revisiting. DAMA DMBOK (Data Management Association, Data Management Body of Knowledge) Even though the DAMA DMBOK was last updated in 2017, and a lot has changed since then, it is still an excellent and relevant framework. Data is an enterprise asset, so data governance, data quality, and data privacy always has been and always will be important. Especially in a distributed ecosystem, there are a higher quantity and diversity of data producers and data consumers, so managing ownership of the data supply chain can be challenging. Those in a data engineering role, then try to turn application exhaust into reusable, scalable data products. In an attempt to combat this quickly, organizations add new, more powerful data technologies and vendors with the intent to simplify and/or add more capabilities to data engineering. However, this can add even more complexity!Having a response for each segment in the DAMA wheel, along with enterprise architecture, will set you on the right path forward. Fortunately, many tools in the modern data stack can provide good answers, especially for metadata management, observability, security and access controls. Leverage the DAMA fundaments to identify, understand, and communicate how each tool helps or challenges your efforts to manage your data as an enterprise asset. Supplement existing frameworks Many fundamentals of business relationship management, enterprise architecture, and data management are timeless. Aspects of each can be applied no matter the situation or department. However, there are many aspects of more modern, domain driven, and distributed architectures that the legacy frameworks do not address, exactly and should be supplemented with more timely, relevant and technical deep dives.Below are a few recent books that have helped supplement my understanding of the fundamentals-Fundamentals of Data Engineering by Reis &Housely is an excellent overview of the stages of data engineering in an easy-to-read format. Very well written and will prepare aspiring data engineers and remind seasoned data engineers how to handle legacy AND modern data technologies.Data Management at Scale by Strengholt, provides more detail, context and reference architecture around the patterns, principles, and `gotchas' managing the full data supply chain. From RDS architecture, and how to link data to ownership via golden data sets, to interoperability and eventual consistency, CQRS and service contracts, this book bridges the gap from legacy frameworks on data management.Software Architecture: The Hard Parts by Ford, Richards, Sadalage & Dehghani. While there is only one chapter on managing analytical data, the book provides a fantastic framework for evaluating the trade-offs with different architectural approaches (primarily concerning distributed application architecture). But not only is it a great idea for data engineers to understand and empathize with their upstream friends, but it also provides excellent guidance and structure data engineers can and should apply.Gavin Hupp's views and opinions are his own, gathered from technology, data, and product leadership experience across multiple industries, working for some of America's most well-known brands, startups, collaborating with peers, vendors, and solution providers. He is an official member of Forbes Technology Council and Vation Innovation Council. The `Modern Data Stack' became more prominent over the last few years, with a proliferation of cloud technologies and vendors fueled by high demand and VC funds
< Page 8 | Page 10 >