Missing plugins found while loading a transformation on Kettle - java

I receive this error whenever I run my extraction from the command line, not in the Spoon UI.
Missing plugins found while loading a transformation
Step : MongoDbInput
at org.pentaho.di.job.entries.trans.JobEntryTrans.getTransMeta(JobEntryTrans.java:1200)
at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:643)
at org.pentaho.di.job.Job.execute(Job.java:714)
at org.pentaho.di.job.Job.execute(Job.java:856)
... 4 more
Caused by: org.pentaho.di.core.exception.KettleMissingPluginsException:
Missing plugins found while loading a transformation
My maven dependencies is as follows.
<dependency>
<groupId>rhino</groupId>
<artifactId>js</artifactId>
</dependency>
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-core</artifactId>
</dependency>
<dependency>
<groupId>pentaho-kettle</groupId>
<artifactId>kettle-engine</artifactId>
</dependency>
<dependency>
<groupId>pentaho-library</groupId>
<artifactId>libbase</artifactId>
<version>5.1.0.0-752</version>
</dependency>
<dependency>
<groupId>pentaho</groupId>
<artifactId>pentaho-big-data-plugin</artifactId>
<version>5.1.0.0-751</version>
</dependency>
<dependency>
<groupId>pentaho</groupId>
<artifactId>pentaho-mongodb-plugin</artifactId>
<version>5.1.0.0-751</version>
</dependency>
<dependency>
<groupId>org.mongodb</groupId>
<artifactId>mongo-java-driver</artifactId>
<version>2.11.1</version>
</dependency>
<dependency>
<groupId>pentaho</groupId>
<artifactId>metastore</artifactId>
<version>5.1.0.0-751</version>
</dependency>
<dependency>
<groupId>pentaho-library</groupId>
<artifactId>libformula</artifactId>
</dependency>
<dependency>
<groupId>simple-jndi</groupId>
<artifactId>simple-jndi</artifactId>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-compiler</artifactId>
</dependency>
<dependency>
<groupId>org.drools</groupId>
<artifactId>drools-core</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
</dependencies>
Am I missing anything?

before init kettleEnviroment
KettleEnviroment.init();
you should add this code like
StepPluginType.getInstance().getPluginFolders().add(new PluginFolder("your plugins path", false, true));

Having the MongoDB plugin as a dependency only works for compilation. When PDI runs it looks for a plugins/ folder under the current directory, and will load all plugins from there. You can override the location by setting the KETTLE_PLUGIN_BASE_FOLDERS system property. Either way, you'll want a plugins/ folder and to unzip the MongoDB plugin package (not the JAR) into plugins/. That should put a folder called pentaho-mongodb-plugin under plugins/, and if PDI is pointing at that plugins/ folder, it should load and use the plugin successfully at runtime.

You must run Kitchen from Kettle installation directory.
As written here:
http://wiki.pentaho.com/display/EAI/Kitchen+User+Documentation
Please make sure that you are positioned in the Kettle directory
before running the samples below. If you put these scripts into a
batch file or shell script, simply do a change directory to the
installation directory

I had the same error message after an update from version 7 to 8.3. In my case, it was the step 'Abort' and 'Attend' that caused the error. In version 8 these two steps aren't part of the library "kettle-engine-8.3.0.0-371.jar" anymore, they are now part of the library "pdi-core-plugins-impl-8.3.0.0-371.jar" in a new plugin. I had to add the plugin in the maven-dependencies:
<dependency>
<groupId>org.pentaho.di.plugins</groupId>
<artifactId>pdi-core-plugins-impl</artifactId>
<version>8.3.0.0-371</version>
</dependency>
But that was still not everything, I still got the error message. The reason: "To use non-native plugins with an embedded Pentaho Server, you must configure the server to find where the plugins reside" (https://help.hitachivantara.com/Documentation/Pentaho/8.0/Developer_Center/PDI/Embed).
So above the line KettleEnvironment.init() in my java-code, I added this one:
System.setProperty("KETTLE_PLUGIN_CLASSES", "org.pentaho.di.trans.steps.abort.AbortMeta,org.pentaho.di.trans.steps.append.AppendMeta");
KettleEnvironment.init();
This way worked for me. There's also a suggestion using a plugin-folder (see the link or the other answers) and working with KETTLE_PLUGIN_BASE_FOLDER instead.
Unfortunately, the error message doesn't mention which plugin is missing. I deleted and added them (in my local Spoon application) until I found exactly the one that was missing.

Just edit spoon.sh and add
OPT="$OPT -DKETTLE_PLUGIN_BASE_FOLDERS=$BASEDIR/plugins"
right after the line where the OPT variable is set.
This is actually the solution from mattyb. Editing spoon.sh will fix the problem for good.
Another solution is to create a symbolic link in your ${KETTLE_HOME}/.kettle directory which points to the plugins folder.

Related

Wrong repository - Maven

I'm trying to configure some dependencies in a Java project using Maven:
<dependencies>
<dependency>
<groupId>com.rometools</groupId>
<artifactId>rome</artifactId>
<version>1.12.0</version>
</dependency>
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.5.2</version>
</dependency>
<dependency>
<groupId>com.basho.riak</groupId>
<artifactId>riak-client</artifactId>
<version>2.0.7</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
</dependencies>
They are not being recognised using IntellIJ so I've tried running this from console:
mvn clean install -U
It didn't work but I've managed to see that Maven is looking for the dependencies at the private Nexus of my Company (totally unrelated). So, I guess my Maven was configured to search in this repo instead of the public ones maybe? How can I change this configuration to get this dependencies correctly?
I'm using macOS 10.13.6.
Maven is a bit peculiar about configuration which typically happen for everything in ~/.m2/settings.xml. Unfortunately the XML-language is not smart enough to include actual code which mean that you cannot set things conditionally like "Use this mirror when I'm on a company network" (which you appear to have done earlier).
You can, however, rename it to for example ~/.m2/company-settings.xml for now, so Maven will use its built-in defaults, and rename it back when you need the current functionality again.
$ mvn install -s /path/to/my-settings.xml
use specific settings xml file can make you skip your company's private repo.

java.lang.ClassNotFoundException: com.mysql.jdbc.driver

I have seen many solutions in Stack Overflow but none of them worked for me.
So I am getting this exception. My pom.xml code is here
<!-- https://mvnrepository.com/artifact/mysql/mysql-connector-java -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.38</version>
</dependency>
I have added jar file in tomcat/lib folder also. But nothing is working for me.
You should rebuild your Maven project after adding new dependencies, either directly on Eclipse if you have the m2e built-in support (which you should, as explained and recommended in this question),
By right-clicking on the project name and then doing Maven->Update project (as #BackSlash suggests)
Also, looking at your pom.xml you are not nesting that <dependency> inside the <dependencies> tag, my db connector (postgress) looks something like this:
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.0.0.jre7</version>
</dependency>
... more <dependency> here...
</dependencies>
You can check this great link that explains how to write pom.xml files when creating and building Java projects with Maven.
Edit:
Remember to call the driver exactly like "com.mysql.jdbc.Driver", with capital 'd'.

NoClassDefFoundError: Mockito Bytebuddy

I recently added Mockito to a maven project on eclipse, by adding the external jar "mockito-core-2.0.53-beta.jar", and upon attempting to create my first mock object (Line two in the function)
And upon running it, the console prints out the first line, then throws this error:
It seems like previously there was a similar issue, but it was supposedly fixed internally. https://github.com/raphw/byte-buddy/issues/99
What is going wrong here?
You simply forgot to add the dependencies to your project which are according to the pom file:
<dependency>
<groupId>net.bytebuddy</groupId>
<artifactId>byte-buddy</artifactId>
<version>1.3.16</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.objenesis</groupId>
<artifactId>objenesis</artifactId>
<version>2.1</version>
<scope>runtime</scope>
</dependency>
In other words you need to add byte-buddy 1.3.16 and objenesis 2.1 to your project too.
More details here
Instead adding
mockito-core
better option will to add
mockito-all
Refer to this link https://mvnrepository.com/artifact/org.mockito/mockito-all/2.0.2-beta
There is a post that explain the problem pretty well, you can find it here:
https://solidsoft.wordpress.com/2012/09/11/beyond-the-mockito-refcard-part-3-mockito-core-vs-mockito-all-in-mavengradle-based-projects/
If you are not using gradle or maven and you are just using mockit-core you should add these dependencies:
<dependency>
<groupId>net.bytebuddy</groupId>
<artifactId>byte-buddy</artifactId>
<version>1.7.9</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.objenesis</groupId>
<artifactId>objenesis</artifactId>
<version>2.4</version>
<scope>runtime</scope>
</dependency>
java.lang.NoClassDefFoundError
This indicates that in your .jar (org.mockito), you don't have that class.
This usually occurs when you have more than one .jar (with a different version) in your classpath. You could check that.

Assertion Error when running `SolrTestCaseJ4` tests: Fix your classpath to have tests-framework.jar before lucene-core.jar

When trying to run simple tests using Solr's testing framework I get the following Error message:
java.lang.AssertionError: fix your classpath to have tests-framework.jar before lucene-core.jar
at __randomizedtesting.SeedInfo.seed(...)
I'm using the IntelliJ IDEA IDE. How do I fix the classpath to have tests-framework.jar before lucene-core.jar?
Even though #mindas's answer above pointed me to it, it is not in a sense, a direct answer to the question. I faced this exact issue today, and found out that the lucene-core.jar wasn't being pulled itself. I added it to my pom.xml and now I don't get the above mentioned error anymore. Here is what the <dependencies> section of my pom.xml now looks like:
<dependencies>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
<dependency>
<groupId>org.apache.solr</groupId>
<artifactId>solr-test-framework</artifactId>
<version>4.6.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.lucene</groupId>
<artifactId>lucene-core</artifactId>
<version>4.6.0</version>
<scope>test</scope>
</dependency>
</dependencies>
Note: Make sure that your lucene-core and solr-test-framework versions match.
Hope this helps someone else landing on this page.
In certain situations, the order of libraries may be important. <..>
For the compilation to succeed, certain libraries (e.g. the ones having to do with the JDK itself and its extensions) must be included in the module bootclasspath. So, such libraries must appear in the list before the JDK.
For more details see relevant chapter of IntelliJ Web help.

Maven error when resolving dependency

I am new to Maven and am trying to set up one of my first POMs. My application will cache using EhCache. Going to Maven Central Repo (link here) I copy-n-pasted the <dependency> tag and copy it into my pom.xml like so:
...many dependencies above this point
<dependency>
<scope>compile</scope>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
<version>4.0.1.Final</version>
</dependency>
<dependency>
<scope>compile</scope>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache</artifactId>
<version>2.5.0</version>
</dependency>
<dependency>
<scope>compile</scope>
<groupId>jasperreports</groupId>
<artifactId>jasperreports</artifactId>
<version>3.5.3</version>
</dependency>
...many dependencies below this point
When I save the changes, Eclipse builds the workspace and gives me an error on the opening <dependency> tag for EhCache 2.5:
Missing artifact net.sf.ehcache:ehcache:jar:2.5.0
So I figured that perhaps v.2.5.0 has something wrong with it, and repeated the same for 2.4.7 (the last 2.4.x release before 2.5.0). Same deal.
Since I'm so new to Maven, I don't even know where to begin looking. I tried Project >> Clean and even restarted Eclipse to see if it was just a typical Eclipse "quirk". Nope.
I am thinking:
Could EhCache be publishing bad JARs to the Maven repo?
Could Maven Repo have something wrong with it?
Could this be due to something else configured wrong in my pom.xml?
Could this be a "JAR hell" issue where I have a conflict somewhere on my dependency graph?
How would SO start tackling this problem? Thanks in advance!
It is usually safer to refer to search.maven.org. Dependency from there:
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache</artifactId>
<version>2.5.0</version>
<type>pom</type>
</dependency>
Mind type pom. From module's pom:
This is a pom artifact to pull in ehcache-core and ehcache-terracotta
for clustering. Make sure to set 'type' to 'pom' in your dependency.
Aparently when someone does not need terracotta, ehcache-core will do perfectly fine as other answer states.
They use ehcache-core in the official documentation. Maven Central does not have a jar artifact for ehcache 2.5 which explains your error message.
Using ehcache-core changes the dependency to:
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-core</artifactId>
<version>2.5.0</version>
</dependency>
Which successfully downloads on my machine (ehcache does not).
I dropped this into IntelliJ and it found it. I suspect there is something wrong with your settings. Try creating a project with only this dependency in it. If it fails to download, I would check your setting. e.g. .m2/settings.xml Are you using a Nexus server or maven proxy/cache?
BTW: A simpler way to search for JARs is to use http://mvnrepository.com/ which will find all the available versions and show you the XML you need to add.

Categories

Resources