Skip to main content

Posts

Exporting InfluxDB data to a CVS file

Sometimes you would need to export a sample of the data from an InfluxDB table to a CSV file (for example to allow a data scientist to do some offline analysis using a tool like Jupyter, Zeppelin or Spark Notebook). It is possible to perform this operation through the influx command line client. This is the general syntax: sudo /usr/bin/influx -database '<database_name>' -host '<hostname>' -username '<username>'  -password '<password>' -execute 'select_statement' -format '<format>' > <file_path>/<file_name>.csv where the format could be csv , json or column . Example: sudo /usr/bin/influx -database 'telegraf' -host 'localhost' -username 'admin'  -password '123456789' -execute 'select * from mem' -format 'csv' > /home/googlielmo/influxdb-export/mem-export.csv
6 comments
Read more

All Day DevOps 2017 is coming!

All Day DevOps 2017 online conference is coming on October 24th 2017. Last year I had a chance to attend it and I can confirm that the overall quality of the talks was excellent. Looking at this year's agenda it seems more promising than 2016's. I heartily recommend attending it to everyone interested in DevOps matters.
Post a Comment
Read more

Publish to Kafka Jenkins plugin talk @ Dublin Jenkins Meetup

I am going to present a preview of the next release 1.5 and insights about the future release 2.0 of the publishtokafka Jenkins plugin at the Dublin Jenkins meetup on August 31st. Please come in if you're in the Dublin area to learn more about this plugin of mine and to have a chat about Jenkins stuff. Here's the map for the event: https://goo.gl/maps/BGT7MRig1Tv
Post a Comment
Read more

Streamsets Data Collector pipeline execution scheduling through the SDC REST APIs

A hot topic in the sdc-user group during the past weeks has been about how to schedule the start and stop of SDC pipelines. Usage of the SDC REST APIs has been suggested in some threads, but because the general impression I have is that the audience doesn't have a clear idea about them, I decided to write an article on DZone to help and clarify once and for all how to do it. Enjoy it!
Post a Comment
Read more

Unit testing Spark applications in Scala (Part 2): Intro to spark-testing-base

In the first part of this series we became familiar with ScalaTest . When it comes to unit test Scala Spark applications ScalaTest isn't enough: you need to add to the roster spark-testing-base . It is an Open Source framework which provides base classes for the main Spark abstractions like SparkContext, RDD, DataFrame, DataSet and Streaming. Let's start to explore all of the facilities provided by this framework and how it works along with ScalaTest with some simple examples. Let's consider the following Scala word count example found on the web: import org.apache.spark.{SparkConf, SparkContext} object SparkWordCount {    def main(args: Array[String]) {     val inputFile = args(0)     val outputFile = args(1)     val conf = new SparkConf().setAppName(" SparkWordCount ")     // Create a Scala Spark Context.     val sc = new SparkContext(conf)     // Load our input data....
4 comments
Read more