We’ve all heard the aphorism “a picture is worth a thousand words.” These days, that’s certainly true of the balance between content and behavior that modern application developers face. There’s long been a certain amount of creative tension between designers and developers, but good developers generally appreciate the value of effective visualization.

This week I’m yielding my soapbox to a guest blogger: Rowan Curran. Rowan is a research associate on the application development and delivery role team, and I often enjoy his tweets about his own particular interests in the digital media space (follow him at @shortpierreview). His remarks below about his most recent vacation day are a good reminder that the changing nature of print and digital experiences will place increasing demands on developers to blend the real and the digital. Devs might even find themselves spending more time with designers and (gasp) artists as the real and the digital converge.

 

 

If you could see Siri as well as talk to her, what might she look like? I recently attended a panel of digital artists the MIT Media Lab who are struggling to answer questions like this. Their works ranged from algorithm-generated mosaics to more traditional digital photo-stitching. But the most surprising and interesting medium that they were working in was big data and visualization. The most poignant realization of this was Joshua Davis’s work on the visualization of IBM’s Watson.

Watson, as many of you are aware, was the name given to the instance of IBM’s Deep Q&A engine that played and won Jeopardy! against the game show’s greatest human players. Unlike previous celebrity software like Deep Blue, Watson was embodied in the form of a dynamic image. This physical realization allowed for the communicationed of multiple metastates that would not have been possible with just voice or text.

The heart of Watson’s avatar is the IBM Smarter Planet logo, a simple but effective nod to the usefulness of trustmarks. The floating balls around Watson’s core sphere expressed a combination of certainty in its answers as well as a variety of other states, such as whether the question was a Daily Double. All together, the cues produced a “face” wherein you could understand Watson’s expressions of its “emotional states.” This relatively concise and efficient design tacks closely to the theory of media naturalness: provided with more natural cues (body language, facial language, and tone of voice), we are more effective and less biased communicators.

The novel and nuanced perspectives that artists can bring to digital technology are not limited to photo manipulation or big data visualization — they can inform more unusual ideas, such as how we experience urban, inorganic settings. Pieces like Patterned By Nature could provide the cognitive benefits afforded in attention restoration theory (continued research supports the basic idea of this theory). The Energy of the Nation could provide a model for a new generation of public works: local monuments of emotion.

New sensory interactions like the heads-up-displays in products like Google Glass or motion control in Leap Motion will present new opportunities and challenges. As we move into an era where wearable devices and augmented reality are the platforms for application development, designers and developers will have to work together to build innovative, trustworthy interfaces that take advantage of the new opportunities the platforms provide. Digital artists can provide unique perspectives in the construction of these experiences that might be overlooked by developers with their heads in the weeds or executives with their heads in the clouds.

New frontiers in user interface design are bringing us to a realm where the question is not so much “How can I interact with this data?” but “How can I express this data in the most meaningful way for myself or these particular groups of people?”  We can already see the first stirrings of interesting answers in the wave of infographics produced the past several years, but these are just our first tentative steps in a future that blends the real and digital. The future is bright! And full of information to be digested.