Streaming Analytics Captures Real-Time Intelligence

Streaming AnalyticsMost enterprises aren’t fully exploiting real-time streaming data that flows from IoT devices and mobile, web, and enterprise apps. Streaming analytics is essential for real-time insights and bringing real-time context to apps. Don’t dismiss streaming analytics as a form of “traditional analytics” use for postmortem analysis. Far from it —  streaming analytics analyzes data right now, when it can be analyzed and put to good use to make applications of all kinds (including IoT) contextual and smarter. Forrester defines streaming analytics as:

Software that can filter, aggregate, enrich, and analyze a high throughput of data from multiple, disparate live data sources and in any data format to identify simple and complex patterns to provide applications with context to detect opportune situations, automate immediate actions, and dynamically adapt.

Forrester Wave™: Big Data Streaming Analytics, Q1 2016

To help enterprises understand what commercial and open source options are available, Rowan Curran and I evaluated 15 streaming analytics vendors using Forrester’s Wave methodology. Forrester clients can read the full report to understand the market category and see the detailed criteria, scores, and ranking of the vendors. Here is a summary of the 15 vendors solutions we evaluated listed in alphabetical order:

  • Cisco Systems’ streaming solution starts at the edge of IoT. Cisco Systems has made a name for itself in the networking and device space for decades, and it is poised to capitalize on this by providing streaming analytics for IoT applications. The vendor’s acquisitions of ParStream and Truviso give it the power to collect data as close to the edge as possible and to efficiently parse and pass it back to the center for analysis. Cisco plans to leverage its wide installation base of networking customers.
  • Data Artisans engineered Apache Flink for all data flows. Berlin-based data Artisans is the commercial force behind the open source Apache Flink project for distributed stream and batch processing. A tiny, young company of fewer than 15 engineers, data Artisans is only beginning to think about how to evolve into a company that can support enterprise customers. Its current modus operandi is to engineer the best open source streaming analytics (that can also do batch processing). Apache Flink is squarely set against the popular Apache Spark project that came out of UC Berkeley’s AMPLab. While Spark uses micro-batches to enable fast processing, Flink is a true streaming engine that can also do batch processing by treating a stream of events as a data set with a beginning and an end.
  • DataTorrent aims for enterprise and open source dominance. DataTorrent is the streaming startup to beat in Silicon Valley. The Yahoo-trained founders built a streaming platform to handle the world’s biggest, fastest data. But enterprises have additional needs, and DataTorrent is delivering them. In addition to providing a distributed streaming analytics platform, the vendor also delivers accruements including a visual development tool and a library of over 400 operators. The core of DataTorrent is now open sourced as Apache Apex.
  • EsperTech provides an enterprise-grade open source CEP engine. EsperTech’s event processing offering provides enterprises with a flexible basis for building applications that require complex pattern matching with sophisticated time windows. Its engine is open source and battle-tested. Customers appreciate the flexibility and freedom the platform gives them to build customizations — and the price.
  • IBM Streams enables cognitive solutions. Cognitive computing encompasses all of intelligence — natural interfaces, situation awareness, smart decisions, and learning to become more effective. Streams can ingest and understand the always-on stream of data from applications and IoT devices needed to make the decisions that underlie cognitive solutions. IBM’s architecture can flex to handle any streaming challenge, and the development environment provides one of the richest set of operators in the market.
  • Impetus Technologies future-proofs Apache Storm. Instead of offering a core streaming engine, Impetus’ solution abstracts the details of deploying, management, and building applications that run on Apache Storm, Apache Spark, and an open source CEP engine. Its StreamAnalytix product includes a visual user interface that hides the gory details underneath, but it also provides a mechanism for developers to add custom code. Their strategy and architecture is to make its solution pluggable with other open source processing engines as they become popular. StreamAnalytix is a relatively new product, but is backed by the 1,000-plus-employee Impetus Technologies, a software development and services company.
  • Informatica delivers streaming within a rich business rules engine. Informatica’s streaming capabilities come in the form of a real-time rules engine. Streaming data is handled by the vendor’s rules engine, which includes enterprise capabilities around security, encryption, operational management, and data lineage. The vendor has a proven track record in a large number of industries, with customers as varied as oil and gas, finance, and high-tech equipment providers.
  • Oracle offers streaming business intelligence and starter solutions. Oracle’s streaming solution includes two distinct pieces that are critical for the future of analytics: Stream Explorer for ingesting and interrogating data as it lands in the cloud or the enterprise; and Oracle Edge Analytics (OEA) for preprocessing data on IoT devices. Oracle offers customers libraries that provides analytics to detect patterns in streams for verticals, such as utilities and finance. The vendor empowers business users with a user-friendly interface to explore streams in real time.
  • SAP’s smart data streaming becomes an engine for IoT. Smart data streaming (SDS) is available as an integrated add-on to SAP Hana or as stand-alone software. Customers can build sophisticated end-to-end applications with a full set of development tools. SAP currently includes two machine learning algorithms — one supervised, one unsupervised — which can incrementally train on data running through the system. The company has also released Streaming Lite, a small-footprint version of the engine that customers can deploy on IoT devices at the edge.
  • SAS goes real time. Although SAS launched its event stream processor in 2014, it has been developing it and using it behind the scenes as an engine for other SAS solutions for several years. The product’s architecture focuses tightly on low-latency, high-throughput complex analytics, so it is well positioned to embed many of SAS’s highly regarded advanced analytics algorithms, including text analytics and machine learning. SAS has also developed a smaller-footprint version that can bring the same sophisticated analytics to edge IoT devices.
  • Software AG’s Apama powers real-time, digital business transformations. Software AG’s Apama product delivers the scalability, management, operators, and application development tools that world-class enterprises need to make the real-time decisions. Long-running pattern detection and stream enrichment is also well supported via integration with Software AG’s own in-memory data grid, Terracotta, and its integration platform, webMethods. The vendor offers a comprehensive set of capabilities for companies that wish to undergo a fast digital transformation and build IoT applications.
  • SQLstream makes cities smarter and heads to the cloud. SQLstream’s Blaze delivers a solid platform for companies to build real-time applications, especially for customers that have a lot of machine data and prefer a declarative SQL syntax to operate on streaming data. Blaze’s StreamLab gives app developers a robust, easy-to-use interface for ingesting, analyzing, and acting on streaming data. The company has found success in delivering applications for governments and municipal bodies that want to make their city’s traffic systems smarter.
  • Striim’s platform captures and analyzes data as it is born. Striim is a valley startup founded in 2012 by former executives from GoldenGate Software. Its platform focuses equally on the continuous capture of data at its point of origin and on the upstream real-time analytics. It can ingest streaming data from many sources, including streaming change data capture (CDC) from transactions in databases. Developers use SQL that has been extended to include streaming semantics to create continuous queries for both in-stream ETL and analytics. Striim’s initial customer success has been in financial services and large telecoms — a great proving ground for new technologies. Striim’s philosophy, to provide insights the moment they are born, positions it well for IoT applications.
  • TIBCO Software sees fast data in every solution. Most streaming solutions gloss over the human insights needed to inspire great streaming applications. Not TIBCO. It recognizes that ideas come from human insight and domain knowledge. That’s why its streaming solution includes LiveView, a real-time view of streaming data. TIBCO has an impressive list of global, complex IoT implementations across multiple industries. The company has end-to-end analytical tools and a track record of delivering mission-critical solutions.
  • WSO2 is a one-stop shop for application middleware, including event processing. WSO2 is an open source middleware provider that includes a full spectrum of architected-as-one components such as application servers, message brokers, enterprise service bus, and many others. Its streaming analytics solution follows the complex event processor architectural approach, so it provides very low-latency analytics. Enterprises that already use WSO2 middleware can add CEP seamlessly. Enterprises looking for a full middleware stack that includes streaming analytics will find a place for WSO2 on their shortlist as well.