Continuous Applications: Matei’s Vision for Next-gen Big Data Applications - Seguridad Mania.com - España y América Latina

Developers who build applications with Apache Spark love the single programming interface to SQL, data transformation, machine learning, and stream analysis. The next generation of big data applications will react to data in real time while blurring the lines between batch and stream processing. Live data streams can be joined to historical batch data to produce aggregates, then queried interactively on demand. Or developers can build machine learning models from historic data and apply them to online streams. The key to achieving this vision is the new Structured Streaming API, now available as an alpha in Spark 2.0. Our long-term vision is to enable every library in Spark to work incrementally over Structured Streaming, giving developers a single interface to build applications that work across batch and real-time processing.