How can a modern data intelligence service optimize its ability to effectively manage and commercialize big data? Here I present “OPRA Swarm” (pronounced like “Oprah” as in Oprah Winfrey the American media tycoon), an organizational theory that leverages the principles of: 1) Objective + Parameters + Rationale + Aid (OPRA) – slight renaming of my previously introduced management technique which received positive feedback, and 2) swarm intelligence. My wife, a practicing artist and an expert on the intersection between art, branding, and technology, inspired key aspects of this theory through her work on a project that remains undisclosed as of this writing.
The Challenge for DaaS Organizations
The leveraging of numerous large, third-party, and often disparate data sets to inform business decision-making is becoming ever-more prevalent. The use of “alternative data” such as B2B transactions, credit card transactions, point-of-sale (POS) data, supply chain linkages, real property data, shipping data, smartphone geolocation, private flight itineraries, satellite imagery…etc. to inform investment strategies, operational decisions, and marketing campaigns has moved beyond early adopters to the mainstream. In response, data-as-a-service (DaaS) organizations, whether in-house or third-party, have emerged to collect data, and build various products for their internal or external client(s). These products can come in a variety of form factors such as research reports for executive audiences, application programming interfaces (API) to deliver structured information feeds to client computer systems, and software applications targeting either technical “power users” or mass market professionals.
The challenge is when the number of data sets increase to unmanageable quantities. The organizational structure and business processes to manage the ingestion, processing, productization, and commercialization of a single structured data set, is very different than doing so for 1,000, 100,000, or even millions of unstructured data sets spanning all format types and update frequencies, especially if one is trying to package elements of multiple independent data sets into differentiated products. Vendor management, data error correction, collaboration between the large teams of data managers, product developers, sales engineers, marketers, salespeople, and customer service reps, and very importantly, coming up with ideas on how to build differentiated products to reach new markets. This is not hypothetical; this is a very real challenge being faced by leading DaaS providers.
OPRA Swarm is an executive-coordinated but operationally de-centralized strategy to empower and incentivize organizational resources to proactively innovate and collaborate cross-functionally for business competitive advantage and market leadership.
Objective + Parameters + Rational + Aid (OPRA)
For a more comprehensive introduction into OPRA, please refer to this prior blog of mine. But at this time, it’s important to explain why the name change from Objective + Parameters + Rationale + Guidance (OPRG) to Objective + Parameters + Rationale + Aid (OPRA).
There are two reasons for why I’ve updated OPRG to OPRA. The first is driven by branding. It’s easier, more memorable, and more appropriately connotative to say “Oprah” and “Oprah Swarm” than coming with a clever way to pronounce O-P-R-G. I must also thank Oprah Winfrey for having a name that sounds like “opera” (hence the title of this blog “help your DaaS organization sing!”), and for her success as a business tycoon, media icon, and philanthropist – which helps with positive connotations. Secondly, the use of the work “aid” rather than “guidance” could reduce the unfortunately too common stigma associated with asking for and giving help in a professional setting. Since successful implementation of an OPRA paradigm rests so much on people freely and proactively asking for and offering help, it would be in-line with the framework’s mission to emphasize this in its name.
Swarm Intelligence
In a February 2018 article appearing in Singularity Hub, its author Scott Simonsen opened with the following:
“As a group, simple creatures following simple rules can display a surprising amount of complexity, efficiency, and even creativity. Known as swarm intelligence, this trait is found throughout nature, but researchers have recently begun using it to transform various fields such as robotics, data mining, medicine, and blockchains.
Ants, for example, can only perform a limited range of functions, but an ant colony can build bridges, create superhighways of food and information, wage war, and enslave other ant species—all of which are beyond the comprehension of any single ant. Likewise, schools of fish, flocks of birds, beehives, and other species exhibit behavior indicative of planning by a higher intelligence that doesn’t actually exist.
It happens by a process called stigmergy. Simply put, a small change by a group member causes other members to behave differently, leading to a new pattern of behavior.”
Applied to human organizations, it boils down to this – can an operational paradigm and set of relatively simple instructions be given to a team of research, product, sales, marketing, customer service, finance, admin, human resources, and management professionals, each individual having relatively simple capabilities, so that together, they can accomplish relatively complex objectives that are beyond the comprehension of any single individual?
A Simple (Overly So) Example Using 3 Basic “Alternative Data” Sets
Let’s imagine there are in existence, the following 3 structured data sets:
1) Property Records Data – address, longitude/latitude, year built, occupied/unoccupied, and sell price history of all residential and commercial buildings in the United States. 6 data fields (simplified).
2) Utility Usage Data – monthly billings and received payments on water, electricity, and heating for all residential buildings as identified by address in the United States. 8 data fields (simplified).
3) Zoning and Parcel Data – sizes, zoning type, address, as well as current and past ownership of land tracts in the United States. 5 data fields (simplified).
I’m sure most readers can imagine scenarios where these individual data sets have value and can be commercialized, so I won’t go into that here. Instead let’s look at how subsets of these individual data sets can be combined into a fourth derived data set that results in a novel application.
Illegal cultivation of cannabis in urban areas can be “detected” by monitoring electricity usage. Growing cannabis in cities often means setting up potted plants in closets or storage rooms and exposing the plants to constant specialized lighting. This leads to significantly elevated electricity use. As such, we can combine sub-components of each of the 3 data aforementioned data sets:
- the address, occupied/un-occupied data fields of Property Records Data (2 of 6 data fields),
- the monthly billings on electricity and address fields of Utility Usage Data (3 of 8 data fields),
- the address and ownership data fields of Zoning and Parcel Data (2 of 5 data fields),
…in order to discover buildings where illicit cannabis growing may be taking place and identify some of the real estate stakeholders which law enforcement may want to investigate and/or collaborate with to shut down tenant growers.
Notice that for this 4th law-enforcement application, not all the data fields for each of the 3 separate data sets are necessary. For example, neither longitude/latitude from Property Records Data, water usage from Utility Usage Data, nor land parcel sizes were used in this example.
Visualizing the “Total Opportunity Space” and “Distinct Application Instance”
Using the above example, we can describe the different combinations of data fields that can be made using those 3 data sets using a 3-dimensional rectangular cube – I’m going to avoid overly technical terminology like “rectangular parallelepiped” given the intended audience of this writing. The volume of the rectangular cube is 6 Property Records data fields times 8 Utility Usage data fields times 5 Zoning and Parcel data fields = 240 (6 x 8 x 5). There are 240 ways we can combine one or more of the 19 data fields (6 + 8 + 5) from the 3 data sets.
Each of these 240 combinations, or “Total Opportunity Space” are now each represented by a single 1 by 1 by 1 cube (“unit cube”), of which there are 240 in this 3-dimension rectangular structure.
We can now represent the cannabis law enforcement application by highlighting the unit cubes that represent the 2 out of 6 Property Records data fields, 3 out of 8 Utility Usage data fields, and 2 out of 5 Zoning and Parcel data fields used in the application. This should result in a highlighting of 12 (2 x 3 x2) unit cubes out of the total 240 possible combinations in the Total Opportunity Space. These 12 highlighted cubes represent the cannabis law enforcement’s “Distinct Application Instance”.
Any time an application is discovered using some combination of subsets of data fields, that can be represented as a Distinct Application Instance within the Total Opportunity Space. For each Distinct Application Instance, we can append meta information like customer(s), data usage frequency, cost to maintain associated DaaS product, revenue from each customer(s)…etc. The result is that we now have an intuitive visual representation of everything we can do with our base data assets, the construction of each DaaS product, and the commercial aspects of each DaaS product.
Combining Swarm Intelligence with OPRA
Using the TOS and DAI visualization frameworks, we can now measure the utilization of data assets in terms of:
1) what is the addressable market in terms of how many applications, clients, and potential revenue?
2) what is our current penetration of the addressable market?
3) how many DAIs do each of our data sets occupy?
4) which of our data sets occupy the most DAIs? which the least?
5) how should we explore “empty” regions of the TOS to discover new DAIs?
These many other questions can now be readily answered and or explored using Swarm Intelligence. For example, an Objective + Parameters + Rationale + Aid (OPRA) approach to a business mission might be:
“Team, our mission is to maximize the revenue-weighted opportunities in our TOS by discovering novel DAIs that generate at least $5,000,000 in total annual revenue, with a preference closer to discovering 10 new DAIS that generate $500,000 each, rather than 1 DAI that generates $5,000,000 (objective).
We calculate this is what it will take to ensure our investments in acquiring, processing, and distributing those 3 data sets will pay off and we achieve true profitability as an organization (rationale).
Each of the product, customer support, and sales teams for the 3 distinct data sets should feel free to collaborate, share ideas, in addition to individual efforts, in making this happen (parameters).
To help you in this process, 1) this 3-dimensional visualization of our TOS and the various DAIs will be made available to everyone to facilitate collaboration – executive management will be responsible for keeping it updated in near-time as ideas are tested and implemented, and 2) any time a team can put together a revenue-generating DAI, the individuals involved in that process will receive company recognition and be invited to present to the rest of the company; if a team achieves at least $500,000 for their concept, all members of the team will be entitled to share in $15% of that product’s revenue for the next 3 years; if as a company we hit our $5,000,000 in new revenue goal, then the entire company will be eligible for a “success bonus”. Finally, our executive team has done some initial investigation into the market and competitive landscape, and we believe the aforementioned “dead zones” you see in the visualization are not likely to result in useful DAIs – feel free to explore if desired, but that’s our guidance (aid).”
With this simple set of OPRA instructions, and the OPRA tools like the Total Opportunity Space and Distinct Application Instance visualization, one can now set loose the team of data, research, product, sales, marketing, customer service, human resources, finance, admin, and operations professionals to “swarm” the problem – drastically reducing expensive, difficult, and slow top-down management overhead. Rather than relying on the genius, passion, and coordination capabilities of just those at the top of the management hierarchy, this decentralized approach harnesses the dynamic creativity and properly incentivized passion of the entire company to create emergent capabilities arising from swarm intelligence.
Much More Complicated in Practice
The fictional use case presented here, while effective in conveying the concepts of TOS, DAI, and OPRA Swarm, is grossly simplistic compared to the nature of modern DaaS organizations.
For example, visualizing a 3-dimensional rectangular cube TOS is intuitive, but what happens when the number of data sets grows into the thousands, hundreds of thousands, and millions? That would imply a TOS resembling a geometric hyper-structure of thousands or millions of dimensions. Additionally, the example uses each data set’s number of data fields as a scalar value to calculate cube volume and thus TOS. There may be content sets where this kind of scalar representation of the various data elements may not be appropriate.
If defining the TOS and DAI can get complicated, the visualization of them, which is a critical tool to be used by the organization to manage an executive-coordinated but operationally decentralized swarm also becomes a challenge. A good DaaS organization, however, should welcome this challenge, as solving this problem will probably position the DaaS organization to be much more effective at implementing client data intelligence solutions!
Operationally, there are also plenty of details…
1) the reality is that not all people are suited for an OPRA environment, it is quite a human resource challenge alone to find a critical mass of individuals with the appropriate skills;
2) traditional incentive structures may be inadequate for an OPRA Swarm organization. Novel organizational performance solutions, like adjustable-frequency, personalized, extensively-scalable incentive technologies like that offered by www.newbreedw.com, may be an important supplement;
3) there will be business functions that require a more stovepipe kind of mindset, which can cause confusion and friction if not properly managed against a cross-functional OPRA Swarm culture;
4) this TOS and DAI models will become valuable intellectual property; how to responsibly protect that while maintaining the necessary informational transparency to be effective?
It would be fascinating to see how a bold DaaS company handles these and other challenges in a successful implementation of OPRA Swarm.
#DataIntelligence #DaaS #OrganizationalDesign #OrganizationalBehavior #SwarmIntelligence #IncentiveManagement #OrganizationalPerformance #Leadership #Singularity #Decentralized
Comments