Few weeks ago we had a need to move to Scala in order to do some things in Spark using APIs available for that language only. In order to speed up things while starting to learn it and minimize the impact on the existing components and the ongoing CI process, we found the dirty way discussed in this post to quickly start with. Started to use the Scala IDE for Eclipse and did the following actions to create new Scala projects:
- Create a new Scala project (File -> New -> Scala Project).
- Add the Maven nature to it (Configure -> Convert to Maven Project). All of our existing projects in the same area are built through Maven and our Jenkins CI servers use Maven to build after any code change and do a lot of other actions (Unit Tests execution, static analysis of the code, code coverage and many others) through it. That's the main reason we are not using sbt. The m2eclipse-scala and the m2e plugins are bundled with the Scala IDE, so no need to install them.
- Remove the reference to the maven-compiler-plugin in the project POM file and replace it with the scala-maven-plugin:
- Update the project to apply the new changes to the POM file: right click on the project name and then click on Maven -> Update Project. There is a bug in the Maven plugin that resets the Java library for the project to 1.5. After this step you need to restore the version you really need to use.
- Manually create the src/main/scala directory and replace the original value in the POM file:
Do the same for the src/test/scala directory.
- Remove the src source folder in the project build path (Properties -> Java Build Path -> Source tab) and add the new ones (src/main/scala and src/test/scala).
- Add the Scala dependency to the project POM file:
- Add Spark and all of the other dependencies you need to the POM file the usual way.
- Create a new Scala project (File -> New -> Scala Project).
- Add the Maven nature to it (Configure -> Convert to Maven Project). All of our existing projects in the same area are built through Maven and our Jenkins CI servers use Maven to build after any code change and do a lot of other actions (Unit Tests execution, static analysis of the code, code coverage and many others) through it. That's the main reason we are not using sbt. The m2eclipse-scala and the m2e plugins are bundled with the Scala IDE, so no need to install them.
- Remove the reference to the maven-compiler-plugin in the project POM file and replace it with the scala-maven-plugin:
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<version>3.1.3</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>testCompile</goal>
</goals>
</execution>
</executions>
</plugin>- Update the project to apply the new changes to the POM file: right click on the project name and then click on Maven -> Update Project. There is a bug in the Maven plugin that resets the Java library for the project to 1.5. After this step you need to restore the version you really need to use.
- Manually create the src/main/scala directory and replace the original value in the POM file:
<build>
<sourceDirectory>src/main/scala</sourceDirectory>
<plugins>
<plugin>
<groupId>net.alchim31.maven</groupId>
...Do the same for the src/test/scala directory.
- Remove the src source folder in the project build path (Properties -> Java Build Path -> Source tab) and add the new ones (src/main/scala and src/test/scala).
- Add the Scala dependency to the project POM file:
<properties>
<scala.version>2.10.6</scala.version>
</properties>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>${scala.version}</version>
</dependency>
<dependencies>- Add Spark and all of the other dependencies you need to the POM file the usual way.
Comments
Post a Comment