16th Sept : Supriya Gurtu (Team J)

Unstructured Data and Role of Analytics

Unstructured data is information that either does not have a pre-defined data model and/or does not fit well into relational tables.  Unstructured data can be textual or non-textual.  Textual unstructured data is generated in media like email messages, PowerPoint presentations, Word documents, collaboration software and instant messages.  Non-textual unstructured data is generated in media like JPEG images, MP3 audio files and Flash video files. Unstructured data can be of various forms such as data on various social media platforms such as Facebook, Twitter, flicker, pinterest, youtube etc. It can be present in the form of traditional media in both online and offline formats. Industry Analyst Reviews is also a form of unstructured data. In an organization unstructured data can be present in the form of emails, Chats or IMs, files – word doc, presentations, pdf documents etc. Call center transcripts, qualitative surveys and consumer reviews also form a part of unstructured data. These data can be in text, images, audio and video formats.

Although unstructured data has a lot of untapped insights, it also brings with it lot of challenges –

a)      Scale: The amount of unstructured data being generated is huge. It is difficult to estimate the exact amount of data currently present with the varied streams and formats in which this data is getting generated. It is rightly being called as ‘Big Data’ because of the problems in capturing, storing, searching, sharing, analyzing and visualization. Managing unstructured data is an IT responsibility and a challenge for the IT professionals that handle it.

b)      Speed: Unstructured data is being generated at an alarming pace. Every day professionals add more unstructured data to business files. The knowledge and insights accumulate day by day with each keystroke. The speed at which the amount of unstructured data grows is far greater than the speed at which people can build tools to use it. According to Gartner, IT Market Clock for Storage, 2011, Sept 6, 2011, rate of growth of unstructured data in an enterprise is up to 80%. Another challenge here is that this data is very dynamic in nature.

c)      Noise: This data is so enormous it is difficult to assess what is relevant and what is not. There is a lot of junk which is present in this. It is intricate to make some sense out of the data.

d)     Less explored: Unstructured data is still very less explored by the industry players. There have been a lot of tools available in the market to analyze unstructured data, but none is a one stop solution for unstructured data.

 

Why unstructured data should be analyzed?

The new age consumer is empowered. Marketers are not the only influencers to a consumers buying decision. In the age of ‘Google’, the consumer today has access to a lot of information. They consult with family and friends, go to review forums online, follow what industry analysts are talking about and hence make intelligent and well informed decisions. In the bid to influence the consumers, marketers have no choice but to get into the social media bandwagon in accession to traditional marketing tactics. And why not, social media is where consumers spend most of their time; it is free of cost and has a farther and faster reach. The companies get a first hand feedback of consumers on these platforms. 

All this unstructured data help the companies to -

- Monitor health of the business

- Identify problems and fix them quickly

- Recognize opportunities and capitalize on them

Other than the marketing perspective, a lot of competitive intelligence can be gathered from unstructured data which can be valuable to make strategic decisions. Business functions like Human resources, Finance, Supply Chain etc. can leverage the unstructured data to optimize their decision making process.

 Traditionally analytics has focused on structured data which can consist of anything which can be put in a tabular format like sales reports, customer details, market data, supply chain data etc. But in recent times the focus has shifted to unstructured data which can consist of social media, traditional media, industry analyst reviews, consumer reviews, emails &IMs etc.

Organizations have originally used extensive structured data sources to enable decision making, but with the present scenario in which there are rich sources of unstructured data present across different media sources, we need to  understand various business priorities and problems unstructured data analytics can help augment and tackle; determine infrastructure, framework and algorithmic needs; do a footprint exercise pan research organization and eventually develop a roadmap for development of unstructured data analytics.