Big Grease, Big Data, and Big Apple - manholes and MDM

"Yellow grease" - the waste resulting from deep frying prawns and french fries and the kind - is yucky substance. In a large city, with thousands of restaurants, there are literally tons of it created every day. And restaurants need to get rid of it somehow.

An easy way to dispose of yellow grease is to tip it down a manhole. Needless to say, this is illegal. In sewage system the stuff creates a horde of problems: it blocks drains, smells foul, and can even cause fires, as explained in a current Financial Times article "Why grease is the word in New York" by Gillian Tett.

Restaurants do have legal options. In New York City it means using licensed disposal companies. But many restaurants choose to go with the manhole approach. It costs less and the risk of getting caught is small. The city has had little means to catch the rule breakers: too many restaurants, too many manholes, not enough inspectors.

But enter Big Data. By analyzing the data that already could be found from city's numerous databases (quoting the article):

"..the results were striking: suddenly they could spot which restaurants were likely to be dumping grease, and the inspectors’ success in catching offenders soared." 

The article gave credit to the new profession of "data scientist", who deep dived into and mined the city's 60 or so data sources. The article went on…:

"Combining these databases was nightmarishly hard; in the matter of yellow grease, for example, information on pollution, manholes, restaurants’ licences and waste companies were in different (incompatible) files." 

…however, is this really primarily just about "big data" or "data science"? I'd argue this is more of an example of a traditional Master Data Management topic. Yes, there is a lot of big data involved, requiring skills in predictive analytics etc, while combing through massive sets of data. But the work starts with solving the baseline of having solid foundations about Master Data about manholes, restaurants, disposal licenses, and so on.

Indeed, the article recognizes this, explaining that the "geeks must be empowered to break down departmental silos". Also, what Ms. Tett well recognizes, this is difficult. It is about human processes of crossing the silos, arriving at common definitions on which to build the next steps.

- - -

The nexus between big data and MDM is getting more attention. In an excellent book published in January 2013 by Sunil Soares, "Big Data Governance: An Emerging Imperative", the role of Master Data Management gets its due attention. The connection between the two is mirrored even to selecting Chief Research Officer Aaron Zornes from MDM Institute to write the foreword for the book.

I see this as a healthy development: emphasizing that "big data" is not a separate island of expertise or separate set of technologies, but rather it is built on top of many Data Management disciplines.

The challenge for the practitioners is that amidst all the marketing hype around "big data", it is not always easy to remind that the realization of business case often starts from simple things.

Like knowing where the manholes are.

Simplification via animation - making Information Management tangible

Explaining the intricacies of Information Management can be a challenge. The concepts lack concreteness: for example "Master Data Management" or - God forbid! - "Big Data" have a wide variety of meanings, depending on the context.

How can Information Management be explained in an understandable way?

We decided to come up with a piece of animation, explaining certain perspectives IM, namely  Product Information Management (PIM): (Note: it is in Finnish language - a synopsis for non-Finnish speakers below the video)

Synopsis:

• The animation starts by setting the context and explaining the pain. A fictitious Product Manager tells about the difficulties he daily faces in managing Product Information
• He  briefly goes through how he has seen the requirements for Product Information growing via e-commerce and emerging use of analytics, among other factors.
• The animation proceeds then to outlining what ought to be done, emphasizing three simplified steps: 1. have your  processes&responsibilities defined, 2. have clear information content/model, and 3. have supporting technologies in place.
• Next, the video describes the "playing field":  there are number of different perspectives to Product Information because for example RnD, Logistics, Marketing, Sales or external stakeholders all tehdn to look at it from slightly different angles.
• Then the video dives into the technical solution that we at Datpro, the company I've co-founded, represents, i.e. IBM MDM Collaborative Edition. It is done via zoomable screenshots to end'user's UI.

This video tells the story from the perspective of a single product manager.  IBM's own video behind this Youtube link well explains PIM in a larger context.

- - -

"Storification" in its various formats, in animations or via infographs or extended use of analogies in explaining different phenomena, will shape how complexity is managed in the future. Last autumn I blogged about tackling complexity with storytelling and "thought clarifier-ship". I firmly believe the future is not in extensive whitepapers, but in shorter format, more compact, ways of attempting to make oneself understood. Like the Bard himself said:

"Therefore, since brevity is the soul of wit,

And tediousness the limbs and outward flourishes,

I will be brief."

—Shakespeare (Hamlet)

Credits: anyone who has read The Art of Explanation by Lee LeFever may notice that the advice given in the book is taken into account in creating the storyline - if some advice has been misunderstood, the fault is entirely mine. The content of the animation is teamwork by Datpro colleagues, Pertti Karhu, Robert Dahlström, and Mikko Toivonen. Our ad agency Campmedia did great work in creating the video.

The problem is managerial, not technical

“It was obvious to me they were going to crash because they had 30 people doing something that could be done better by three.”

…is a quote by the father of Unmanned Aerial Vehicles (UAVs), Abe Karem, in The Economist early December. He continues: "Doing things with the absolute smallest team increases the chance that you’re not going to screw up". 

In addition to having the smallest possible team, Mr. Karem also emphasizes the need for short time from idea to completion.

Again to the point. While UAVs (more commonly known as "drones") have transformed contemporary warfare, the management philosophy Mr. Karem represents has the potential to transform the ways many businesses operate. Small team delivering something tangible quickly is the best way to get things done.

This applies also Master Data Management: a small team with authority & accountability to deliver tangible results quickly is the key, whatever data domain we're talking about. Large committees and endless validation rounds with people who have not participated do not yield to meaningful results.

Final quote by Mr. Karem: "The problem is managerial, not technological". 

With Tiger's clubs, you'll golf better - and what it means to Product Information Management

You play golf better if you think Tiger has used your club. You are willing to pay 2000+% premium on a worthless object if there is a good story behind it. What is going on and is there a lesson for Product Information Management?

- - -

The value of a physical product is ever more closely tied to information about it. Thomas Redman uses the term of "informationalization" to describe the phenomenon in an HBR.org post "Integrate Data into Products or Get Behind" in late June. To quote Mr. Redman's words on informationalization:   

"The basic idea is simple: Make existing products and services more valuable to your customers by building in more data and information. [..] Virtually every product and service can be made more valuable through informationalization. [..] Coors has informationalized the beer can. The mountains turn blue when the beer is cold enough to drink."

Adding one peculiar example of informationalization: The New York Times recently run an article "He Takes Stuff Seriously". I'll summarize some of its core messages to a few lines: 

• nearly worthless objects (figurines, buttons, etc, of max value of couple of dollars) were collected from flea markets and thrift stores 
• short stories were created around these objects, in essence, the objects were "storified" 
• ..the outcome was that nearly worthless objects with a "story" became much more valuable

This enforces the message of informationalization of products to the extreme. The product is nothing without the information. Even fictitious information breaths new life to the product.

Another, perhaps even more peculiar example, is the study conducted by Dr. Sally Linkenauger and referenced in an HBR article "You'll golf Better If You Think Tiger Has Used Your Clubs". The result of the study was as the title implies. An amateur golfer plays better if he believes a pro has played with the same equipment.

The effect is named "positive contagion": a johndoe golfer thinks that the skills of Tiger have somehow an effect on a club that Tiger has used. Meaning that  he at least subconsciously assumes Tiger has left the essence of himself on the club. Rather magical, yes, but somehow real for the individual.

The two phenomena, the effects on storification of flea market object as well positive contagion of Tiger's clubs, fall into the realms of psychology (and marketing, good marketers have understood them since long...). Nevertheless, assuming they are real, they touch Product Information Management. Stories on the background of products should be managed systematically. And both on general level and on level of particular instance of a product.

Many leading companies have recognized this. Stories are embedded in products: for example name of the designer, time of the original design, famous events of people connected to the product, are made available to potential consumer. Engaging stories can then be told about these people and events related to the product. Web allows a platform to transmit the stories, in many formats. Old-fashioned paper leaflet in the physical box of a product is not dead either.

But do organizations manage this information as part of product information model, as part of systematic product information management? Or is it a separate island, disconnected from the  main processes of PIM?

In many organizations it is the latter.

If it were managed in a structured way, product information model would cover the elements related to "storification". There would be ability to systematically connect descriptive texts, binaries, perhaps something harvested from the streams of social media to the product.

In any way, the scope Product Information Management, throughout the lifecycle of products, becomes ever broader. Traditional supply chain centric or technical perspectives on product information is not enough. Incorporation of marketing oriented information, including "stories"of different kind, should become ever more integral part of managing product information. Master Data Management systems beware: there's going to be need for storing and managing stories soon.

Tackling complexity with storytelling and "thought clarifier-ship"

"We deal with that world by constructing simplifying narratives. We do this not because we are stupid, or irrational, or have forgotten probability 101, but because storytelling is the best means of making sense of complexity. The test of these narratives is whether they are believable." 

The quote above is from "When storytelling leads to unhappy endings", a Financial times article on August 7th, 2012. I feel it demonstrates masterful insight by its author, John Kay (@JohnKayFT). The paragraph is in the nexus of two phenomena shaping modern management: complexity and "storification".

Increasing complexity of businesses combined with ever faster ticking clock, largely caused by 24/7 e-mail and social media communication explosion, makes thorough sense-making to become luxury. For the very simple reason that true understanding requires time and effort. (Yes, I believe that Nicholas Carr has a point in his book Shallows and in his famous argument "Google makes us stupid").

Nevertheless, certain degree of understanding is required for organisations to function well. A multinational attempting to expand its market share and profitability endangers reaching its goals if complexity impairs its people from understanding what they should really do. And the answer is construction of simplifying narratives, to put it bluntly, telling stories. Those who can create stories that make people work towards the same goal, will lead businesses.

Today I bumped into an inspiring tweet by James Kobielius (@jameskobielus). It has the same point as Mr. Kay in bit different words:

"Thought leadership. Often, it's "thought clarifier-ship." If you've expressed old idea more clearly than others, you can lead the discussion"

How to be clear, cut through the clutter, is about solving tomorrow's issues. I believe this is especially true for a cross-organisational, cross-process management topics such as my own favourite, Data Management. However, it is more universal rule and applicable to other disciplines as well.

From High Frequency Trading to lessons in Data Governance

A rogue algorithm spread turmoil on the trading floors of Wall Street last week. It started to sell and buy large volumes of stock on its own, bringing trading volumes artificially high. Price volatility went through the roof.

The outcome of the algorithm-gone-wild was a loss of 440 million USD to the company responsible, Knight Capital, as reported by the Financial Times in "Knight shows danger of automation frenzy". The number of court cases to follow can just be guessed.

This is not an isolated incident. There have been a worrying number of high-visibility cases of trading software going awry. These are discussed in further detail in both the FT article as well as in the New York Times coverage in "Flood of Errant Trades Is a Black Eye for Wall Street".

What's wrong - is Skynet taking over? Or are self-aware algorithms such as in Robert Harris' novel "The Fear Index" come live? (Sorry for spoiler, but I guess anyone really interested has already read the book. Btw: not the best by Harris, but still rather readable)

Probably Skynet is not here, though. More likely that the glitches - that in this case happened to coincide with a software upgrade - are old-fashioned errors. Sufficient software and process testing may have been omitted, instructions not followed, software (c)rushed into production use too quickly etc.

The effects of errors are augmented by the removal of human element from the process. Quoting another FT article, "Knight Capital - a Disturbing Pattern":

"But the obsession with HFT and algorithms that their designers claim mimic the behaviour of investors is in danger of taking the human dimension out of the stock market. Worse, HFT renders even its creators redundant if it decides to go berserk: nobody at Knight apparently had the sangfroid to turn the switch to off."

Further discussion in a NYT article by Joe Nocera, "Frankenstein Takes Over the Market", where already the title tells what viewpoint the writer represents.

- - -

And indeed. the inertia we flesh'n'blood beings have, the slowness of reasoning and imperfect logic, all have their positive sides. Although everything can go berserk under human control as well, at least errors happen slower...

But what is the lesson for Data Management professionals? I'd say that there's really no fundamentally new lesson. However, the meaning of old lessons is greatly enforced: when automation increases and the computer systems are ever more integrated, the importance of having all the basics in place is more important than ever: roles and responsibilities must be defined, escalation paths clear when something unexpected happens, solid testing practices must be in place prior to doing anything important, etc. 

For example a Master Data Management solution, fully integrated within a company and possibly reaching through the supply chain, can propagate errors far quicker than has previously been possible. The effects of these MDM related glitches rarely are as spectacular as those of High Frequency Trading, but they're very real problems to companies experiencing them.

Have good Data Governance practices in place.

Analogies galore!

Data Management blogosphere is full of analogies and metaphors. Ever more innovative comparisons between data and you-name-what come up. A few examples:

- the discovery of the Higgs Boson triggered the creativity of many eloquent Data Management souls. One example is "Quality is the Higgs Field of Data" by Jim Harris (@ocdqblog) where lines are drawn between the Higgs Field and quality of data. Quoting Jim: "By using data, we give data its quality. The quality can not be separated from its use any more than the particles of the universe can be separated from the Higgs Field".

- two splendid discourses on linking data to wine and applications to fish by @monkchips a.k.a James Governor and @chuckhollis. Both blogs named "Why Applications Are Like Fish And Data Is Like Wine" (Link to James' blog and to Chuck's blog here). (The original idea of this particular comparison seems to be older than either of the blogs, though.)

- A blog tweeted by @IBMinitiate,  "Healthy Data: A Diet of Thoughtful Consumption", finds similarities between data, health, and food. In a similar way that food fuels our bodies, data fuels the IT systems and organizations using them. Yes, in these times of "food wars" and passionate health care arguments one could have guessed nutritional metaphors entering the game.

- Another exotic approach explained "big data" via movie plots, including Minority Report (=predictive analytics), Black Swan (=unprecedented insights), The Gold Rush by Charles Chaplin (=unlocking the intrinsic potential in the data), to name a few examples in the blog post titled "Big Data  Explained in 10 movies".

- ..and one industrious blogger @hlsdk (in www.liliendahl.com) produces thought-provoking linkages between data and other phenomena at an amazing frequency. The latest one by Henrik goes back to Biblical times, to the big tower of Babel.

Indeed, it seems that any discussion between data aficionados seems to dive into the world of metaphors. Just as one example, I collected a list of analogies from a discussion thread in LinkedIn about two years ago: comparisons between data and Galileo, Einstein, Plato & Socrates, Holy Grail, George Orwell, and Greek mythology were made, to name a few.

Highly contextual nature of data management means that each analogy is at its best imperfect. It may be that Einstein's wisdom "Combinatory play seems to be the essential feature in productive thoughts" is the best explanation for the flood of analogies. Playing with seemingly unrelated ideas help us to make sense of the role of data (which arguably is unambiguous..) in an enterprise or society.

However, is the flood of comparisons between data & something else, raging through philosophy, history, physics, literature, and popular arts, too much for an uninitiated? Does one need to be a polymath to understand Data Management? Need to continue on this in a later blog post.

- - -

PS. Jim Harris shed light on the question on metaphors in  "Metaphorically Blogging" post about in 2010. Recommended read.