Googlielmo's blog

I have just released the version 0.2 of jolokiaui ( https://github.com/virtualramblas/jolokiaui ), a web client for the Jolokia JVM agent ( https://jolokia.org/agent/jvm.html ) to monitor local and remote JVMs. It is implemented in R ( https://www.r-project.org/ ) using the Shiny framework ( http://shiny.rstudio.com/ ).  The current release is still an alpha, but new features would be added in the next months. Any feedback is welcome. Enjoy it.

One of the most useful things introduced by Java 8 is the Optional class ( https://docs.oracle.com/javase/8/docs/api/java/util/Optional.html ). It is a container object that wraps either a value or null , to represent the absence of a value. You can quickly understand the benefits coming from it: a better way to handle the existence of null values with no need to explicitly check for nulls and possibly less chance to have NullPointerException exceptions. Some code like this // The method call below could return null User currentUser = getUser(id); Role role = null; if(currentUser != null) {   role = getRole(currentUser); } becomes: Optional<User> currentUser = Optional.ofNullable(getUser(id)); Optional<Role> role = currentUser.map(u -> getRole(currentUser));

Apache Drill ( https://drill.apache.org/ ) is an Open Source framework that supports data-intensive distributed applications for interactive analysis of large scale datasets. It supports several kinds of filesystems and NoSQL databases, including HDFS and MongoDB. In this short post I want to show an exception you could deal with if your Drill installation has the MongoDB storage plugin enabled. If suddenly, after more than 24 of uptime, any SQL query (not only those against a MongoDB database) issued through Drill throw the following exception: SYSTEM ERROR: IllegalStateException: state should be: open [Error Id: 57a02508-1920-4360-a111-c2a55a7af15c on hostname:31010]] this should be related to a connection cache that expires and isn't automatically reset. This issue affects in particular the latest release (1.1.0) of Drill and it is still marked as unresolved at the time of this post writing ( https://issues.apache.org/jira/browse/DRILL-3522 ), but there is a patch available...

Apache MRUnit ( https://mrunit.apache.org/ ) is an Open Source library that allows unit-testing for Hadoop Mappers, Reducers, and MapReduce programs. It provides a convenient integration between MapReduce and standard testing libraries such as JUnit and Mockito and helps (providing a set of interfaces and test harnesses) bridging the gap between MapReduce programs and those traditional libraries. It doesn't replace JUnit, but works on top of it. Before reading further, please be aware that knowledge of Hadoop MapReduce and JUnit is required for a better understanding of this post. The three core classes of MRUnit are the following: • MapDriver : the driver class responsible for calling the Mapper’s map() method. • ReducerDriver : the driver class responsible for calling the Reducer’s reduce() method. • MapReduceDriver : the combined MapReduce driver responsible for calling the Mapper’s map() method first, followed by an in-memory Shuffle phase. At the end of this phase the ...

Just a quick note about Scrum (and Agile in general) on a concept that is often ignored. Scrum is intended for those kinds of work that defined processes have often failed to manage: uncertain requirements combined with unpredictable technology implementation risks (this distinction is not a blasphemy in the Agile context). These conditions usually exist during a new product (in particular software products) development: no matter how carefully you plan the future, it can never be more than a dream, unless you adjust your plan every day. So don't be scared by uncertainty, be ready to live with it and work hard to reduce it. This is what makes the difference between a good member of an Agile Team and a bad one: the good one manages and doesn't ignore uncertainty. This is true at each level of any self proclaimed agile company.

An appendix to complete the tutorial about the Shiny Server (OS edition) installation: a quick reference to its configuration. The configuration file of a Shiny Server is stored in the /etc/shiny-server/shiny-server.conf file. It comes in a format like this: run_as shiny; server {   listen 3838;   location / {     site_dir /srv/shiny-server;         log_dir /var/log/shiny-server;         directory_index on;   } } Here's the explanation of the parameters: run_as : to set the system user to run R/Shiny applications. Don't use root. server : this block defines a single server details (see the following properties). listen : the server listening port. The default value is 3838. location : defines a location available at the base URL. site_dir : the directory hosting the Shiny web apps. log_dir : the directory where the Shiny Server logs are stored. directory_index : to enable ( on ) ...

RHadoop ( https://github.com/RevolutionAnalytics/RHadoop/wiki ) is a collection of five R packages (rhdfs, rmr2, rhbase, ravro, plyrmr) that allow users to manage and analyze data with Hadoop. Running any MapReduce function, also this simple one     from.dfs(mapreduce(to.dfs(1:100)))  through RHadoop on Linux servers you could face this exception: 2015-10-20 08:39:41,722 ERROR [main] org.apache.hadoop.streaming.PipeMapRed: configuration exception java.io.IOException: Cannot run program "Rscript": error=2, No such file or directory     at java.lang.ProcessBuilder.start(ProcessBuilder.java:1059)     at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)     at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.jav...

Shiny Server (https://www.rstudio.com/products/shiny/shiny-server/ ) is a server to put Shiny ( http://shiny.rstudio.com/ ) applications or interactive documents available over the web. The official documentation seems quite confusing to me, so I want to share this quick tutorial for people approaching it for the first time. The procedure explained in this post has been tested and replicated on several Linux Red Hat Servers 6.6+ 64-bit, but it should work on any Linux distribution that allows RPM management. The overall procedure could be of course automated, but for better understanding of this tutorial I am going to describe any manual step involved. Installation Connect to your Linux server as root and download the latest RPM for your specific architecture: wget https://download3.rstudio.org/centos5.9/x86_64/shiny-server-1.4.0.721-rh5-x86_64.rpm and then install it using yum: yum install --nogpgcheck shiny-server-1.4.0.721-rh5-x86_64.rpm At the end of the installation p...