Building real time data pipelines with Kafka Streams I

Building real time data pipelines with Kafka Streams I

Many interesting projects have been developed within the stream processing field in the last years. Most of us could name open source projects (Apache Spark, Apache Storm, Apache Flink) or proprietary services (Google DataFlow or AWS Lambda) that are very well-known solutions for real time scenarios. However, the objective of this post to present Kafka […]

Futurakka Volume II

Futurakka Volume II

In the previous post, we reviewed the features of Futures in Scala and the most important methods used in the API. In this post, we’ll review another interesting API for handling asynchrony with Scala together with the use of Futures. Promises provide an alternative way to create Future instances. A Promise has to be fulfilled […]

Futurakka, Volume I

Futurakka, Volume I

The objective of this post is to explain the nuts and bolts of dealing with Futures in Scala and Akka. Futures allow to perform many operations in parallel in an efficient and non-blocking way, but dealing with multiple operations can be a real headache. On this post, we’ll review the main APIs and we’ll see […]

ELASTICSEARCH UNIVERSE: FLYING OVER THE DATA-I

ELASTICSEARCH UNIVERSE: FLYING OVER THE DATA-I

Hi all again!! This is the third tech-paper of this serie dedicated to the optimization of an Elasticsearch Cluster. Here we  can find how to configure your searches in order to make them faster and more efficient. Avoid big http requests In search  tasks Elasticsearch needs to fetch the Id of all the documents, this […]

GOOGLE DATAFLOW AND APACHE BEAM (II)

GOOGLE DATAFLOW AND APACHE BEAM (II)

What is Apache Beam? Apache beam is an open source, unified programming model that defines and executes data processing pipelines. These pipelines can be both batch and streaming. It is exposed via several sdks that allow to execute a pipeline in different processing engines, aka, runners. The supported runners so far are: Apache Spark Apache […]