I have been working with Amazon Kinesis Data Firehose for over a year, and I'm pretty happy with the way how it works. In a nutshell, it's a service that does writing to Amazon S3 with custom data transformation and buffering rules. My current use case is simple -- write events into Amazon S3 for further processing with Apache Spark .
Unfortunately, the more events I have, the more small files land on S3, and processing time with Spark slow down.
That is where Fireblender comes in. The idea of the project is simple -- given time range, join all data files into bigger chunks to allow faster processing and return URL.