Data Visualization is the process of representing data in a visual format; e.g. graphs, pie charts, heat maps, distribution curves, scatter charts, Venn Diagrams, Tree Diagrams, Sunbursts. The old adage “a picture is worth a thousand words” is most apt when applied to data analysis – a pie-chart is any day easier to interpret versus a thousand data points listed in a few columns of a spreadsheet/table. In today’s world, the adage could be rephrased as “a picture is worth a billion bytes”.
So where lays the problem? Excel is probably one of the most prolific tools that we are all used to when attempting to visually represent our data. But then we are limited by the maximum number of rows a spreadsheet can handle, our ability to manipulate large worksheets etc. Imagine trying to load your website’s raw logs into MS-Excel and trying to make sense of it visually or otherwise… you get the drift.
Enter the realm of Big Data – that’s where the Data Visualization Tools come into play. Big data can yield a treasure trove of valuable insights into customer and market behavior. Of course, as the name suggests, the data volumes are big, the data is typically unstructured, and mining this data for information is beyond the capability of traditional tools like Excel. So the need for big data engines coupled with data visualization tools made for this very purpose. These tools are made specifically to load large volumes of data (talk terabytes or more), comb through them, allow you to make sense of them, and actually display the results in a visual format that allows you to quickly spot trends, opportunities, problem areas, and whatever you may be looking for.
The Big Data Visualization process typically looks as below:
In the above process, you may be tempted to comment that it looks remarkably similar to the traditional data warehousing and reporting approach. Not really; hidden in this process are a number of steps where the raw data is sifted and explored using a Big Data engine; only data that shows promise of hidden opportunities is promoted to the transformation stage.
Before I get into telling you about your options for data visualization engines/tools – one of the key things to bear in mind around being successful in using Big Data and using visualization tools is to understand that Big Data is vast, changes rapidly, and cannot be dealt with at the same speed as traditional data warehousing / data analysis. The opportunities inside of Big Data are fleeting and so require a rapid response approach by IT in being able to collect and provide the data to the business for spotting “opportunities”. The business has to be equally skilled to spot these “opportunities” so as to take only the relevant subset of data to the Transformation stage in preparation for the data visualization tools to actually represent it visually. Though you may hear people use the cliché “Big Data = Big Insights”, the corrected formula would be:
Big Data + Rapid Action = Big Insights
Key features to look for in Data Visualization Tools
- Ability to visually explore a database
- Allow interaction with mobile devices –your mobile workforce is a key user of real-time intelligence
- Ability to interact with touch based devices
- Intuitive Report building
- Multiple display formats to visually represent your data
- Intuitive UI to interface with the tool
- Flexibility to customize the UI and screen elements like labels, positioning etc.
- Stability, scalability, memory management
Key vendors / products
Gartner’s 2013 Magic Quadrant lists IBM, TIBCO Spotfire, Microsoft, Tableau Software, Oracle, SAS and SAP as some of the major players who also incidentally fall under the “Leaders” quadrant.
While these vendors and their products are definitely safe bets, they all have their plusses and minuses. Choosing the right product is critical and must align with your organization’s specific needs. This I can say with confidence as I have helped clients evaluate and pick some of these products.
I have also done technical book-reviews for publishers on some of these products. TIBCO Spotfire is one that I can say is a great product. Check out the book “TIBCO Spotfire for Developers” that I did a technical review for. Its a great technical resource for those of you who have decided to make TIBCO Spotfire your tool of choice.
Thanks for taking the time to read my post. Feel free to subscribe to my blog to be automatically notified on my next post. If you liked this post and would care to rate it, please use the star rating on the top of the post. It’s just a click. Till my next post, be safe and happy holidays.