Capturing Data Analytics Workflows and System Requirements

Implement an effective and repeatable strategy for documenting data analytics workflows and capturing system requirements

“Data analytics applications involve more than just analyzing data, particularly on advanced analytics projects. Much of the required work takes place upfront, in collecting, integrating, and preparing data and then developing, testing, and revising analytical models to ensure that they produce accurate results. In addition to data scientists and other data analysts, analytics teams often include data engineers, who create data pipelines and help prepare data sets for analysis.” — TechTarget

Introduction

Successful consultants, project managers, and product owners use well-proven and systematic approaches to achieve desired outcomes, including successful customer engagements, project results, and product and service launches. Modern data stacks and analytics workflows are increasingly complex. This technology-agnostic discovery process aims to help an organization efficiently and repeatably capture a concise record of existing analytics workflows, business and technical goals and constraints, and measures of success. If applicable, the discovery process is also used to compile and clarify requirements for new data analytics workflows. It takes a ‘product’ versus ‘product’ approach.

The Process

The discovery process starts by it identifies the current analytics workflows, examining the six stages of generate, collect, prepare, store, and analyze, and deliver. The process then works backward by identifying existing goals and desired outcomes. It then identifies existing and anticipated future constraints. Finally, it examines the existing and future measures of success.

Specifically, the process identifies and documents the following:

1. Existing analytics workflows: tools, techniques, procedures, and organizational structure
2. Inputs (datasources)
3. Outputs (deliverables)
4. Data producers and…