In a recent media interview I was asked about whether the requirements for data visualization had changed. The questions were focused around whether users are still satisfied with dashboards, graphs and charts or do they have new needs, demands and expectations.

Arguably, Ancient Egyptian hieroglyphics were probably the first real "commercial" examples of data visualization (though many people before the Egyptians also used the same approach — but more often as a general communications tool). Since then, visualization of data has certainly always been both a popular and important topic. For example, Florence Nightingale changed the course of healthcare with a single compelling polar area chart on the causes of death during the Crimean War. 

In looking at this question of how and why data visualization might be changing, I identified at least 5 major triggers. Namely:

  • Increasing volumes of data. It's no surprise that we now have to process much larger volumes of data. But this also impacts the ways we need to represent it. The volume of data stimulates new forms of visualization tools. While not all of these tools are new (strictly speaking), they have at least begun to find a much broader audience as we find the need to communicate much more information much more rapidly. Time walling and infographics are just two approaches that are not necessarily all that new but they have attracted much greater usage as a direct result of the increasing volume of data.
  • More complexity in data relationships. More traditional computer-based data visualization approaches were fairly simplistic in their ability to represent the complex relationships within data. A chart based on an Excel spreadsheet for example, represented the data contained only within the spreadsheet's own rows and columns. With a greater emphasis on integrating data from a wide variety of information sources, new methods of visualization need to be used. These again include infographics but also interactive bubble charts, 3D data "landscapes" and semantic analysis maps.
  • Interactivity. Users now expect that they can completely interact with data, not just visualize it. Static reports simply don't cut it anymore. This means that the visualization tools must understand the context of the data and be able to dynamically adapt the navigation, look and feel and even the core functionality as user manipulate and immerse themselves in the information.
  • Gamification. As per the previous bullet point, static representations of data in charts and graphs no longer hold the attention of users. Interactivity does go part of the way. However, we're now seeing much more emphasis on introducing "game play" as part of data visualization. This can be as simple as including some level of interactivity to allow the user to drill down into information, but it can also go to the extreme of integrating real-time, multi-player simulations. These are common in highly regulated or life-threatening work environments where attention spans may mean the difference between life and death.   
  • Cognitive computing. Smart "thinking machines" appear to be becoming the norm. But unfortunately it is still just an illusion (yes, including IBM's Watson). Though our smartphones and tablets may know exactly where we are (by the inbuilt GPS location services) they can also know much richer information about what we're doing — using things like the information available in our calendars for example. Data visualization in this kind of context means that complex data can be represented in some new and compelling ways.

In my Google Nexus device, it uses my current location and the contents of my calendar to pop up an infographic panel. This infographic panel can contain the current time, alternative route maps, traffic loads on those routes, weather information and expected travel times to ensure that I make it comfortably to my next appointment in plenty of time. You could certainly make the argument that this is not necessarily a new form of visualization, but a collection of various existing data visualization approaches. And that's exactly my argument. While many people fail to see this scenario as data visualization, to me, it's a very good example of how this market really is changing. It's certainly no longer just about historic transactional charts and graphs that the user has to read and interpret. Or even drill down BI. The new approach to data virtualization is both dynamically generated and autonomically optimized. Most of all, it answers very complex questions with a simple graphical response. Where do I have to be next and how long will it take me to get there? Which route should I take and when do I need to leave to compensate for traffic? Very complex questions, but a very simple graphical response.      

But are the old approaches to data visualization completely inadequate today? Well, it's highly unlikely that the "old ways" will ever disappear completely. In fact, in most cases they will likely remain as relevant as ever. But there's one major difference. It's no longer us humans that will decide the most appropriate and meaningful visual representation of data. The machines will increasingly do that for us — at least as a starting point. We will then get the opportunity to interact with the information and further interact with it and refine it as needed.
 
The challenge for most organizations today is to choose the right data visualization tool and strategy based on achieving the best outcome. For example, a sales person might still find comparative sales charts the most effective way of representing sales data. But a service rep might find a semantic information map a better way to navigate the vast number of possibilities when diagnosing equipment failure. 
 
There are many, many ways of visualizing data. And today, it remains more of an art than a science. There are specialist companies and individuals that do this and nothing else. There is actually plenty of science behind it but few organizations pursue it in this way. Generally enterprises choose a tool — or set of tools — and then implement their data visualization solutions based on the capabilities of those tools. It's very rare to see organizations giving much consideration to identifying the very best way of representing each individual data requirement and then finding specific solutions to fit.
 
Surprisingly, one form of data visualization tool that is commonly and consistently overlooked is….colour. Many organizations fail to use even basic color representations to communicate things like timeliness, accuracy or the security requirements for business intelligence reports. They will generally define and create the report structures, run the reports over their data sets, and then produce the generated report results, either through a portal or sent to a user via email. In these cases, the user typically has little information about the context of the report they're looking at. Is it verified information? Is it live information or a snapshot generated at a point in time? Who should have access to this information and how can they be allowed to use it? Many BI implementations that I've seen try to embed complex master data or meta data definitions into the BI reports so that users are clear on validity, accuracy, security and timeliness. It's confusing. There's too much data and not enough understanding. However, the simple color coding of a report can communicate a whole complex set of data without ever having to write even a single additional word or character on the page. 
 
The market for data visualization tools remains very fragmented — and we don't see this changing any time soon. There are a myriad of options and vendors available and it really is a case of "horses for courses". Most organizations still want an "out-of-the-box" solution for data visualization though. But by doing so, they also miss some broader opportunities to apply the technology more intelligently. Others do go the best of breed approach but then waste many hundreds of man hours just getting the basics right, assembling the data to get it into the right format to suit the specific visualization tools. In my experience, this will continue to remain the case for some time yet. Only a few leading organizations are really great at data visualization today. 
 
So what should CIOs do? Lead by example. Constantly and critically look at how information is being presented within the organization and ask themselves and others "Is this the best way to represent this data? Can we do something radically different to increase the impact and value of this information to the business?" A great way of getting started is by taking a short course provided by some of the specialists companies on infographics and data visualization techniques. It helps train your brain to think more critically about the way the information is being presented. Rather than focusing on the quality of the data within the cells of an Excel spreadsheet or the rows of a database, the CIO can help the business start to think creatively and differently. Look what that approach has done for Apple.
 
Let me know if you have some really great examples — or some war stories — on data visualization of your own. It would be great to collect some references for some of our research.