Survey: Streaming Data And The Future Tech Stack

Insights From Over 800 It Professionals on Their Use of Data Stream Processing


Grab your copy

How, and why, do enterprises adopt streaming data applications in the cloud?

To better understand this trend and the underlying software stack needed to do it right, Lightbend has partnered with The New Stack for its second charity survey on the trends around cloud-native, real-time streaming data applications.

A total of 804 IT professionals provided details about their prototypes, projects, and production applications that use stream processing at their organizations–and created over $800 in charitable funds towards Girls Who Code.

Download this report to see insights that reveal:

  • Artificial Intelligence and Machine Learning are overtaking early adopters’ use cases
    The use of stream processing for AI/ML applications increased four-fold in two years. Those already using streaming for AI/ML expect this trend to continue with even broader use in the coming year.
  • Early adopters are concerned about unknowns
    Developer experience, familiarity with tools, and technical complexity are barriers to adoption. Concern about scalability, latency and other technical challenges increases as the number of workloads utilizing stream processing rises.
  • “State” concerns lessen as more applications use stream processing
    Persisting data in a microservices architecture becomes less of an issue as users gain more experience with containers, microservices and modern databases. Architects see a future where stream processing and microservices are deployed in the same container-based infrastructure stack.
  • Technologists are looking beyond Apache Kafka for advanced use cases
    While Kafka is sufficient for ETL and messaging, it faces robust vendor competition among streaming platforms for advanced use cases such as IoT pipelines and recommendation engines