In a recent discussion, we were asked why a CXO should care about the modern data ecosystem. The short answer is, they shouldn't. Instead, they should care about driving business analytics that are delivered quickly, cheaply, iteratively, and with trust and transparency. Let's delve into why this shift in focus is critical and how it addresses the biggest complaints from businesses regarding data and analytics.
The Common Business Complaints About Data and Analytics
Over the past 30 years, Iโve heard these three major complaints from businesses regarding their data and analytics (D&A):
- Outdated and Untrusted Data: โWhy can't we have up-to-date, understandable, and trusted data?โ
- Slow Delivery: โWhy does it take so long to deliver my use-case?โ
- Friction in Changes: โWhy does changing my mind cause so much friction and delay?โ
The root cause of these issues often lies in the traditional data-only view, where lengthy data production lines precede the actual use-case development. This approach leads to several inefficiencies:
- Long Onboarding Times: Data can take weeks or even months to be onboarded.
- Lack of Iterative Flexibility: Data platforms struggle to iterate, especially for data science use cases, which require lots of experimental cycles.
- Brittle Data Models: Cross-use-case data structures often break with new analytics requests, making them fragile and hard to maintain.
- Delayed Production: After prototyping, getting models or dashboards into production can take weeks or months, and sometimes it doesnโt happen at all.
A New Approach: Business-Driven Data Products
To overcome these challenges, a new approach is neededโone that focuses on delivering direct business value through data products. This approach should aim to:
- Deliver End-to-End Business Use Cases: Focus on implementation that solves specific business problems.
- Ensure Fast Time-to-Market: Accelerate the delivery of analytics solutions.
- Enable Iterative Development: Allow the system to adapt quickly to market and business changes.
- Reduce Implementation Costs: Lower the time and resources needed for each use case.
- Facilitate Genuine Self-Service: Enable use-case developers to onboard, prototype, play, and productionize data products independently.
The Three-Stage Process for Effective Data Products
1. Prototype
The use-case developer creates a data product prototype, including visualizations, and publishes it to the infrastructure in "draft mode." This allows for one-click deployment in seconds, fostering direct collaboration and fast feedback with business, governance, and other teams.
- Capture: Quickly gather data from the data estate, external sources, and the catalogue into sandboxes. Use other data products (metrics/models) as dependencies.
- Understand: Collaborate with data owners and product owners to understand the data and products. Link or join data as needed, using annotation and markup rather than transformation.
- Model: Build data product-specific structures with just enough information for the actual models.
- Visualize and Iterate: Present prototypes to users and gather feedback early and often.
2. Industrialize
Once the data product prototype is accepted by the business, it enters the pre-production phase.
- Engineer: Harden the code and pipelines, implement CI/CD, and version the data product into run-times for execution on the platform.
- Test: Conduct model and system testing with production data and control groups.
- Control: Add data quality rules, policies, and map back to data dictionaries/taxonomies for re-use.
- Publish: Package and publish the data product to the ecosystem for others to see and use.
3. Operate
In the final stage, the use-case developers and domain support teams manage the data product run-times on the ecosystem infrastructure.
- Execute and Adapt: Run data product run-times on platform clusters with processes for emergency fixes.
- Monitor: Continuously monitor data quality, model performance, system errors, resource usage, and other metrics.
- Retire: Retire data products as needed, ensuring the ecosystem remains dynamic and responsive.
Conclusion
CXOs should care about implementing a business-driven analytics approach that prioritizes quick, iterative, and transparent delivery of data products. This approach not only addresses the common complaints businesses have about D&A but also ensures that analytics are aligned with business goals and can adapt to market changes swiftly. By focusing on the value delivered through data products, organizations can achieve faster time-to-market, reduced implementation costs, and a genuinely self-service analytics environment.
If youโre interested in exploring how this approach can transform your data strategy, feel free to reach out for a deeper dive into our Data Product operating model.