IDP delivers data-driven marketing solutions with a focus on in-database processing and the Analytic Sandbox.
And with in-database processing, marketers do these in seconds, not hours.
With 3,500 [Tesco] stores and each store stocking an average of 40,000 products, tracking them involves the creation of over 100 million data points. This is where in-database analytics comes into play – deploying analytics where data is stored, rather than moving the data for external analytics. — Forbes, Nov 2016
Successful in-database analytics depend on rock-solid:
✓ analytical data sets and
✓ analytic sandboxes
both built to specific marketing directives.
The Analytical Data Set
The foundation to deep-dive analytics is the analytical data set. This is the final data table used to deliver analytics.
Far too much handcrafted work — ‘data wrangling’, ‘data munging’ and ‘data janitor work’ — is required. Data scientists spend from 50% to 80% of their time mired in this mundane labor of preparing unruly data, before it can be explored. – NYT, Aug 2014.
Supporting analytics are largely-hidden data management and data mining activities. These represent 75% of the entire analytics process and can easily employ a billion data rows. And significantly impact the time spent on analytics. During this process, new data relationships are revealed. New data transformations are created. And the best analytical approaches are discovered. The end result is an actionable, marketable, analytical data set.
The Analytic Sandbox
The analytic sandbox is physical space inside a production database. Within the sandbox, analysts create analytical data sets. And build, test and run analytics. Initial gains from the analytic sandbox are speed – impressive speed – literally turning hours of effort into seconds!
✓ Gone are data dumps!
✓ Gone are data transfers across networks!
✓ Gone are complex joins and aggregations on workstations!
With an analytic sandbox, the data warehouse manages data. And literally, hours of effort drop to seconds. Managing data, or exploring data relationships and interactions are much faster inside a database.
Another gain is the ability to audit analytics. Business analysts notoriously collect rogue data marts or ‘spreadmarts’. These datasets come from outside sources and are stored as text files or in Excel. They’re inefficient and generate unreliable reports. And there’s no way to verify reporting accuracy. As a result, marketing executives lose confidence in both reports and data.
Migrating ‘spreadmarts’ from desktops and into analytic sandboxes produces datasets that can be audited. And gives all business analysts equal access to once isolated data. And as data migrates to the sandbox, business rules and processes also migrate. Complete analytic processes can be audited for accuracy and consistency.
The Analytic Sandbox — IDP
IDP builds analytic sandboxes that preserve the single-view of data and speed up deep-dive analytics. New analytics can be developed, tested and converted into production applications quickly, giving a wider user group quick access to new analytics.
With an analytic sandbox, the process of analytics, from data preparation to final reports can be audited giving everyone greater confidence in the results. And the analytic sandbox becomes more valuable as data volumes explode and data structures become more complex.