My book "Hands-on Deep Learning with Apache Spark" is finally available on Packt. Here's the final cover:
This is the book content:
1: THE APACHE SPARK ECOSYSTEM
2: DEEP LEARNING BASICS
3: EXTRACT, TRANSFORM, LOAD
4: STREAMING
5: CONVOLUTIONAL NEURAL NETWORKS
6: RECURRENT NEURAL NETWORKS
7: TRAINING NEURAL NETWORKS WITH SPARK
8: MONITORING AND DEBUGGING NEURAL NETWORK TRAINING
9: INTERPRETING NEURAL NETWORK OUTPUT
10: DEPLOYING ON A DISTRIBUTED SYSTEM
11: NLP BASICS
12: TEXTUAL ANALYSIS AND DEEP LEARNING
13: CONVOLUTION
14: IMAGE CLASSIFICATION
15: WHAT'S NEXT FOR DEEP LEARNING?
DeepLearning4J (Scala), but also Keras and TensorFlow (Python) are the reference frameworks.
More topics on Deep Learning on the JVM and Spark would be covered in the next months in this blog.
This is the book content:
1: THE APACHE SPARK ECOSYSTEM
2: DEEP LEARNING BASICS
3: EXTRACT, TRANSFORM, LOAD
4: STREAMING
5: CONVOLUTIONAL NEURAL NETWORKS
6: RECURRENT NEURAL NETWORKS
7: TRAINING NEURAL NETWORKS WITH SPARK
8: MONITORING AND DEBUGGING NEURAL NETWORK TRAINING
9: INTERPRETING NEURAL NETWORK OUTPUT
10: DEPLOYING ON A DISTRIBUTED SYSTEM
11: NLP BASICS
12: TEXTUAL ANALYSIS AND DEEP LEARNING
13: CONVOLUTION
14: IMAGE CLASSIFICATION
15: WHAT'S NEXT FOR DEEP LEARNING?
DeepLearning4J (Scala), but also Keras and TensorFlow (Python) are the reference frameworks.
More topics on Deep Learning on the JVM and Spark would be covered in the next months in this blog.
This comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteHi Guglielmo,
ReplyDeleteI am trying the code in the chapter 4.
I am not a Java or Scala developer and I'm feeling loss with the errors that I found executing the jar that I have compiled with Maven project.
I appended to the pom.xml file the scala plugin needed for scala code build and a shaded plugin needed to solve an error about Hadoop filesystem not found .The following is the build part that I have appended
(build)
(plugins)
(!-- Configure maven-compiler-plugin to use the desired Java version --)
(plugin)
(groupId)org.apache.maven.plugins(/groupId)
(artifactId)maven-compiler-plugin(/artifactId)
(version)3.8.0(/version)
(configuration)
(source)${java.version}(/source)
(target)${java.version}(/target)
(/configuration)
(/plugin)
(!-- Use build-helper-maven-plugin to add Scala source and test source directories --)
(plugin)
(groupId)org.codehaus.mojo(/groupId)
(artifactId)build-helper-maven-plugin(/artifactId)
(version)3.0.0(/version)
(executions)
(execution)
(id)add-source(/id)
(phase)generate-sources(/phase)
(goals)
(goal)add-source(/goal)
(/goals)
(configuration)
(sources)
(source)src/main/scala(/source)
(/sources)
(/configuration)
(/execution)
(execution)
(id)add-test-source(/id)
(phase)generate-test-sources(/phase)
(goals)
(goal)add-test-source(/goal)
(/goals)
(configuration)
(sources)
(source)src/test/scala(/source)
(/sources)
(/configuration)
(/execution)
(/executions)
(/plugin)
(!-- Use scala-maven-plugin for Scala support --)
(plugin)
(groupId)net.alchim31.maven(/groupId)
(artifactId)scala-maven-plugin(/artifactId)
(version)3.2.2(/version)
(executions)
(execution)
(goals)
(!-- Need to specify this explicitly, otherwise plugin won't be called when doing e.g. mvn compile --)
(goal)compile(/goal)
(goal)testCompile(/goal)
(/goals)
(/execution)
(/executions)
(/plugin)
(plugin)
(groupId)org.apache.maven.plugins(/groupId)
(artifactId)maven-shade-plugin(/artifactId)
(version)2.3(/version)
(executions)
(execution)
(phase)package(/phase)
(goals)
(goal)shade(/goal)
(/goals)
(configuration)
(transformers)
(transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/)
(/transformers)
(/configuration)
(/execution)
(/executions)
(/plugin)
(/plugins)
(/build) --)
The following is the command line that I use to submit
ReplyDelete$SPARK_HOME/bin/spark-submit --master local --class org.googlielmo.sparkstreamingkafka.DirectKafkaDataVec --jars /data/handsonspark/Chapter04/sparkstreamingkafka/target/sparkstreamingkafka-1.0.jar /usr/apache/spark-2.2.3/jars/*
Now I am stuck on this error :
Exception in thread "main" scala.MatchError: [Ljava.lang.String;@4a94ee4 (of class [Ljava.lang.String;)
at org.googlielmo.sparkstreamingkafka.DirectKafkaDataVec$.main(DirectKafkaDataVec.scala:30)
at org.googlielmo.sparkstreamingkafka.DirectKafkaDataVec.main(DirectKafkaDataVec.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:775)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Please help because I am stuck from 2 days and I need to go on
Regards
Herman
This has been solved here: https://github.com/PacktPublishing/Hands-On-Deep-Learning-with-Apache-Spark/issues/1
Delete