I am new to spark and is trying to build a simple project in java with IDEA. This test project simply read txt file as javaRDD and display its content.
When I am trying to build my project, I got error
java: cannot access scala.Cloneable
class file for scala.Cloneable not found
at line
SparkConf conf = new SparkConf().setAppName("test_app").setMaster("local[*]");
and
java: cannot access scala.Serializable
class file for scala.Serializable not found
at
for(String line:lines.collect()){
System.out.println(line);
}
From my understanding this means IDEA do not have scala dependency. However, in my pom file, spark dependency shows
Dependency 'org.apache.spark:spark-core_2.13:3.3.1' not found
My pom.xml looks like this.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>spark_test_project</artifactId>
<version>1.0</version>
<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<spark.version>3.3.1</spark.version>
<java.version>11.0.16</java.version>
<scala.version>2.11.12</scala.version>
<java.compat.version>11.0.16</java.compat.version>
</properties>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.13</artifactId>
<version>3.3.1</version>
</dependency>
</dependencies>
</project>
How should I solve this issue?
I tried to install jar dependency manually in IDEA but this would not solve the issue. My current scala version is 2.11.12.
Related
I've started to create a java project using the dropwizard framework and maven. My pom.xml file looks like
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.dexcane</groupId>
<artifactId>dexcane-backend</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<dropwizard.version>2.1.4-SNAPSHOT</dropwizard.version>
</properties>
<dependencies>
<!-- https://mvnrepository.com/artifact/io.dropwizard.archetypes/dropwizard-archetypes -->
<dependency>
<groupId>io.dropwizard</groupId>
<artifactId>dropwizard-core</artifactId>
<version>2.1.4-SNAPSHOT</version>
<type>pom</type>
</dependency>
</dependencies>
</project>
I'm following the dropwizard tutorial. When I hover over the "2.1.4-SNAPSHOT" dependency Intellij says "Dependency 'io.dropwizard:dropwizard-core:2.1.4-SNAPSHOT' not found ". Does anyone know what's wrong with the import? The tutorial says the current version of dropwizard is 2.1.4-SNAPSHOT.
You shouldn't add it with the type of pom as its packaging is jar.
I'm trying to execute the following project from GitHub:
https://github.com/natzei/jvrp
I'm trying to install Install on a local repository.
I create a maven project on Eclipse IDE.
In my pom.xml field I put the following code :
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>it.unica.informatica.ro</groupId>
<artifactId>jvrp</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<group-id>it.unica.informatica.ro</group-id>
<artifact-id>jvrp</artifact-id>
<version>0.0.1</version>
</dependency>
</dependencies>
</project>
But I'm getting the followings errors:
Are these the steps I need to take? If you have an idea please help
I am trying to add an external jar file according to the article
https://www.tutorialspoint.com/maven/maven_external_dependencies.htm
but idea can't resolve a class from the library after I took all steps article.
Although if I move the cursor in pom.xml to "systemPath" and press cntrl+right mouse button idea points to jar file in tree project.
I've tried to press File->invalidate cashes/reset
What have not I done yet?
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.name</groupId>
<artifactId>generalTwo</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>lib_v1</groupId>
<artifactId>lib_v1</artifactId>
<version>1.0</version>
<systemPath>${project.basedir}/src/lib/lib_v1.jar</systemPath>
<scope>system</scope>
</dependency>
</dependencies>
Screenshot from idea, structure of project
Also there is my lib in "project structure->modules->dependency" with name "maven:lib_v1:lib_v1:1.0"
This is my pom file
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany</groupId>
<artifactId>mavenproject2</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>net.sourceforge.owlapi</groupId>
<artifactId>owlapi-distribution</artifactId>
<version>5.1.0</version>
</dependency>
<dependency>
<groupId>net.sourceforge.owlapi</groupId>
<artifactId>org.semanticweb.hermit</artifactId>
<version>1.3.8.500</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
</project>
I just want to make simple examples like this:
OWLOntology o = manager.loadOntologyFromOntologyDocument(file);
System.out.println("--------------------------");
OWLReasonerFactory rf = new ReasonerFactory();
OWLReasoner r = rf.createReasoner(o);
r.precomputeInferences(InferenceType.CLASS_HIERARCHY);
But ReasonerFactory is not recognized by the system.
I downloaded the jar file, installed maven, opened a netbeans project using a maven project. Why it doesn't works??
Use OWLAPI version 5.1.7 and HermiT version 1.4.3.517.
The problem here is that in the version you used ReasonerFactory is an inner class in the Reasoner class, but your test code relies on it being a standalone class. That happened in a later version of HermiT.
1.4.3.517 is the most recent release and includes a few bug fixes, hence I recommend using it in place of all the other OWLAPI 5 compatible versions.
My pom.xml file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.name.MyProject</groupId>
<artifactId>MyProject</artifactId>
<version>0.0.1-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>example-jetty-embedded</artifactId>
<version>9.1.0.v20131115</version>
</dependency>
</dependencies>
</project>
and I got a problem:
Missing artifact
org.eclipse.jetty.tests:test-mock-resources:jar:9.1.0.v20131115
I see that mvnrepository does not contain this jar file (http://mvnrepository.com/artifact/org.eclipse.jetty.tests/test-mock-resources)
How can I fix this problem?
Test resources are not deployed to maven central by the Jetty project.
And example projects are usually not considered dependencies.