Googlielmo's blog

AI, Machine Learning and Deep Learning are getting an hype nowadays even if most part of the algorithms and models at their core are around since long time: 1805 Least Squares 1812 Bayes' Theorem 1913 Markov Chains 1950 Turing's Learning Machine 1957 Perceptron 1967 Nearest Neighbor 1970 Automatic Differentiation 1972 TF-IDF 1980 Neocognitron 1981 Explanation Based Learning 1982 Recurrent Neural Network 1970 Back Propagation 1989 Reinforcement Learning 1995 Random Forest Algorithm 1995 Support Vector Machines 1997 LSTM So what are the reasons that speed up and accelerated the implementation and made possible today for the theory to become reality? There are several factors:  - Cheaper computation: in the past hardware was a constraining factor for AI/ML/DL. Late advance in hardware (coupled with improved tools and software frameworks) and new computational models (in particular around GPUs) have accelerated AI/ML/DL adoption.  - Cheaper storage: the increased number of availab...

Do you believe that Java is dead? Please join the Java with the Best online conference on April 17th-18th and then probably you will change your mind. Of course if you are a Java technology passionate you need to attend as well ;) 2 days, 50 talks, 3 parallel sessions (Core Java, Big Data, Machine Learning and Cloud Development with Java, Java Frameworks, Libraries and Languages).  I hope you will get a chance to attend my talk:

An interesting conference on Scala is going to happen in Kiev (Ukraine) on April 20th-21st: https://www.scalaua.com/ Some early bird tickets should be still available. Here's a list of the speakers already confirmed for this event. Have a look at this short video to get some insight from the 2017 edition.

The second post of this series focuses on how to run a Livy server instance and start playing with its REST APIs. The steps below are meant for a Linux environment (any distribution). Prerequisites The prerequisites to start a Livy server are the following: The JAVA_HOME env variable set to a JDK/JRE 8 installation. A running Spark cluster. Starting the Livy server Download the latest version ( 0.4.0-incubating at the time this post is written) from the official website and extract the archive content (it is a ZIP file). Then setup the SPARK_HOME env variable to the Spark location in the server (for simplicity in this post I am assuming that the cluster is in the same machine as for the Livy server, but in the next post I will go through the customization of the configuration files, including the connection to a remote Spark cluster, wherever it is). By default Livy writes its logs into the $LIVY_HOME/logs location: you need to manually create this directory. Finally ...

I have started doing evaluation of Livy for potential case scenarios where this technology could help and I'd like to share some findings with others who would like to approach this interesting Open Source project. It has been started by Cloudera and Microsoft and it is currently in the process of being incubated by the Apache Software Foundation. The official documentation isn't comprehensive at the moment, so I hope my posts on this topic could help someone else. Apache Livy is a service to interact with Apache Spark through a REST interface. It enables both submissions of Spark jobs or snippets of Spark code. The following features are supported: The jobs can be submitted as pre-compiled jars, snippets of code or via Java/Scala client API. Interactive Scala, Python, and R shells. Support for Spark 2.x and Spark1.x, Scala 2.10 and 2.11. It doesn't require any change to Spark code. It allows long running Spark Contexts that can be used for multiple Spark jobs, by...

My latest article on DZone is online. There you can learn how to configure Telegraf to pull metrics from a Java application running inside a Docker container. The Telegraf Jolokia plugin configuration presented as example in the article is set up to collect metrics about the heap memory usage, thread count and class count, but these aren't the only metrics you can collect this way. When running a container hosting the Java app with the Jolokia agent, you can get the full list of available metrics through the following GET request: curl -X GET http://<jolokia_host>:<jolokia_port>/jolokia/list and pick up their names and attributes to be added to the plugin configuration.

Few days ago Pat Patterson published an excellent article on DZone about Evolving Avro Schemas With Apache Kafka and StreamSets Data Collector. I recommend reading this interesting article. I followed this tutorial and today I want to share the details on how I quickly setup the environment for this purpose, just in case you should be interested on doing the same. I did it on a Linux Red Hat Server 7 (but the steps are the same for any other Linux distro) and using only images available in the Docker Hub . First start a Zookeeper node (which is required by Kafka ): sudo docker run --name some-zookeeper --restart always -d zookeeper and then a Kafka broker, linking the container to that for Zookeeper: sudo docker run -d --name kafka --link zookeeper:zookeeper ches/kafka Then start the Confluent Schema Registry (linking it to Zookeeper and Kafka): sudo docker run -d --name schema-registry -p 8081:8081 --link zookeeper:zookeeper --link kafka:kafka confluent/schema-registry ...