DataTaunew | comments | leaders | submitlogin
Why we need a SQL-like Query Language for Real-time Streaming Analytics? (kdnuggets.com)
3 points by hemapani 3329 days ago | 5 comments


1 point by conflag 3328 days ago | link

I implemented the same idea with Storm and Esper and can recommend this combination. The key to scale is, as explained in the slideshow, to engineer the aggregate functions of your Storm topology so that they fit your aggregations in Esper.

-----

1 point by hemapani 3328 days ago | link

We have done the same, but using WSO2 CEP instead of Esper. Also, one key thing is we have done a layer that accepts queries from the user, partition them using hints from the user, and automatically create and run the Storm topology. Basically, everything come out of the box.

-----

1 point by kiyoto 3329 days ago | link

You might find this project interesting for SQL-on-Stream: https://norikra.github.io

-----

1 point by hemapani 3329 days ago | link

Actually there are lot of tools from CEP world that does this (http://www.complexevents.com/2014/12/03/cep-tooling-market-s...), it is just that big data world does not know about those much.

-----

1 point by hemapani 3329 days ago | link

Big Data Realtime Analytics world mostly doing and discussing counting usecases and implementing them by hand (e.g. using Storm). However, realtime usecases like windows, temporal event patterns etc are too complicated to be rewritten by every programmer. We need higher level abstractions, and learning from Hive and CEP, SQL like languages might fit the bill.

-----




RSS | Announcements