I try to deploy a program as windows service with apache commons daemon. i have everything set up fine beside the classpath.
Classpath=C:\test\test-service\lib\*
specifies where the service itself lies, but the service needs some configuration files to run and these are under
Classpath=C:\test\test-service\conf\*
now i struggle to get it to work so that the program uses C:\test\test-service\* as classpath and not just one of the both specified above. sadly C:\test\test-service\* does not work and throws a ClassDefNotFoundException for the starter class. that error is solved by C:\test\test-service\lib\* but then i get the error that my config files can't be read. therefore i thought, why not also add the config path to the classpath like this:
Classpath=C:\test\test-service\lib\*;C:\test\test-service\conf\*
but this still throws the FileNotFoundException. does anyone has a solution to this?
So I was able to solve this by myself. Following works:
Classpath= C:\test\test-service\conf\;C:\test\test-service\lib\*
it takes all files in conf folder by default as well as all files from lib as the * states
Related
How to set the HADOOP_CLASSPATH for using the local filesystem with a local job runner?
How to set the input and output path from local directories?
ClassNotFoundException arises for mapper and reducer classes when I try to run with the following command.
hadoop WordCount input/sample.txt output
current value is:
: hadoop classpath
/usr/local/hadoop/hadoop-3.2.1/etc/hadoop:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/common/lib/*:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/common/*:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/hdfs:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/hdfs/lib/*:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/hdfs/*:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/mapreduce/lib/*:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/mapreduce/*:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/yarn:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/yarn/lib/*:/usr/local/hadoop/hadoop-3.2.1/share/hadoop/yarn/*:/usr/local/hadoop/hadoop-3.2.1/etc/hadoop/usr/local/hadoop/hadoop-3.2.1/share/hadoop/common/*.jar
versions:
Apache hadoop-3.2.1 ,
openjdk 11.0.5.
please help. It is useful for debugging. Thanks in advance.
I would believe if you had FileNotFound Exception, but your classpath seems fine, so I have a hard time seeing you would get ClassNotFound Exception
Although, this path seems wrong /usr/local/hadoop/hadoop-3.2.1/etc/hadoop/usr/local/hadoop/hadoop-3.2.1/share/hadoop/common/*.jar.
I would suggest moving all files under hadoop-3.2.1 up into /usr/local/hadoop, or at the very least, rename hadoop-3.2.1 directory to just /usr/local/hadoop/3.2.1/
By default, Hadoop jobs use file:// paths as your fs.defaultFS (defined in core-site.xml)
Otherwise, if you have changed that to use hdfs://, then you can still use local files like so
hadoop fs -ls file://
To run jobs, I would suggest using yarn jar, not hadoop <name>. And you need to shade your Java application into an uber-jar, or use the existing hadoop-examples JAR to run WordCount
I uploaded a Spring application to Heroku but the application crashed with the following error:
java.io.FileNotFoundException: class path resource [com/myname/myapp/config/dao-context.xml
The file is definitely there, and it is in GIT, and the app runs successfully locally.
Any ideas what is happening here?
I suspect that when you are running locally, it is picking up the file on the classpath as a regular file on the filesystem (i.e. not inside of a JAR).
On Heroku, it is probably inside of a JAR file, which means it is not a regular file, and must be read as an input stream, which might look like this:
ClassLoader cl = this.getClass().getClassLoader();
InputStream inputStream = cl.getResourceAsStream("com/myname/myapp/config/dao-context.xml");
You can probably reproduce the problem locally by running the same command that's in your Procfile.
If this is not the case, then make sure the file exists on Heroku by running this command:
$ heroku run ls com/myname/myapp/config/dao-context.xml
For future visitors to this question, I overcame the problem by converting my DAO XML config file to the Java Config method, therefore Spring no longer required that XML file. This didn't directly solve the issue of being unable to find the XML file, but the added benefit is that I am now using the more modern and less verbose Java Config method.
I have a java code snippet
TitanGraph g = TitanFactory.open("titan-all-0.4.4/conf/titan-berkeleydb.properties");
where titan-berkeleydb.properties is the in build config file that comes with Titan db installation.
On execution, it throws an exception
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.commons.lang.StringUtils.isNotBlank(Ljava/lang/String;)Z
at com.thinkaurelius.titan.graphdb.configuration.KCVSConfiguration.<init>(KCVSConfiguration.java:40)
at com.thinkaurelius.titan.diskstorage.Backend.initialize(Backend.java:273)
at com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1174)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.<init>(StandardTitanGraph.java:75)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:40)
at com.thinkaurelius.titan.core.TitanFactory.open(TitanFactory.java:29)
Is there a way to resolve this issue?
Check your classpath for another commons-lang-*.jar. According to the Apache Commons documentation, the isBlank() method is new since 2.0.
https://commons.apache.org/proper/commons-lang/javadocs/api-2.5/org/apache/commons/lang/StringUtils.html#isBlank%28java.lang.String%29
Titan 0.4.4 ships with commons-lang-2.5.jar in the lib directory. Your original post didn't mention anything else about the runtime environment. I would suspect that if you had, for example, commons-lang-1.0.1.jar on the classpath, it would be picked up before commons-lang-2.5.jar and you would see the error.
If you're using a web application with Tomcat, you have to add the jar file into WEB-INF/lib folder of the application (or into $TOMCAT_HOME/lib folder, if you have more webapps using it).
Either create a library with this jar and add it to the project class path.
If you done all this and it still not working, you can change the jar extension to zip or rar and open it to see if the requested classes you need are inside.
I have build a jar file of classes and configuration files. The configuration.yml file is located in root of the jar. When I try to run the application using the following command:
java -jar target/drop-wizard-0.0.1-SNAPSHOT.jar server configuration.yml
I get the exception below. How can I specify file located in jar from command prompt?
Exception in thread "main" java.io.FileNotFoundException: File configuration.yml not found <br>
at io.dropwizard.configuration.FileConfigurationSourceProvider.open(FileConfigurationSourceProvider.java:14)<br>
at io.dropwizard.configuration.ConfigurationFactory.build(ConfigurationFactory.java:75)<br>
at io.dropwizard.cli.ConfiguredCommand.parseConfiguration(ConfiguredCommand.java:114)<br>
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:63)<br>
at io.dropwizard.cli.Cli.run(Cli.java:70)<br>
at io.dropwizard.Application.run(Application.java:72)<br>
at com.flightnetwork.dropwizard.example.HelloWorldApplication.main(HelloWorldApplication.java:10)<br>
It is possible to load a yaml file from a class path since Dropwizard 0.9.1 version.
Just configure Application with ResourceConfigurationSourceProvider in the similar manner:
public class MicroUsersApplication extends Application<MicroUsersConfiguration> {
#Override
public void initialize(Bootstrap<MicroUsersConfiguration> bootstrap) {
bootstrap.setConfigurationSourceProvider(
new ResourceConfigurationSourceProvider());
}
}
And for configuration.yml in the root of a jar:
java -jar target/drop-wizard-0.0.1-SNAPSHOT.jar server configuration.yml
For configuration.yml in the /com/some/configuration.yml from the jar root
java -jar target/drop-wizard-0.0.1-SNAPSHOT.jar server com/some/configuration.yml
Please, note — there is not a leading / in the path com/some/configuration.yml. Looks like, this behaviour will be kept till 1.1.0 release: Fix #1640: ResourceConfigurationSourceProvider - process a path to the resource in the more sophisticated way.
If you use more recent Dropwizard version, you can implement your own ResourceConfigurationSourceProvider — it is pretty simple.
configuration.yml is expected to be in the working directory i.e. on the filesystem, because this is how you try to read it. If you want to read it from jar file or from classpath in general you need to use getResource or getResourceAstream methods. (please see this similar question and answer)
EIDT
If you want to read the config from a resource inside your jar then you might want to configure your application to use UrlConfigurationSourceProvider instead of FileConfigurationSourceProvider and pass it the URL which you can obtain from getResource, the open method of the underlying interface expects a String as parameter, so you will need to use URL#toString on the result of getResource.
Try executing the command with absolute paths instead, apparently configuration.yml isn't in the run folder.
Example when configuration.yml is in /tmp
java -jar target/drop-wizard-0.0.1-SNAPSHOT.jar server /tmp/configuration.yml
I don't think it's possible to feed dropwizard a yml file within jar. It needs to be on the file system AFAIK. UPDATE: it looks like it's possible but I'm leaving this as it's still a solution.
If you don't want to expose configuration details to outside, then you need to configure it using the Configuration class, by setting the default values in the constructor. It gets quite ugly though since it's so nested. I don't like how dropwizard enforces the yml dependency myself.
An example I was using for my tests:
public class TestConfiguration extends Configuration {
public TestConfiguration() {
super();
// The following is to make sure it runs with a random port. parallel tests clash otherwise
((HttpConnectorFactory) ((DefaultServerFactory) getServerFactory()).getApplicationConnectors().get(0)).setPort(0);
// this is for admin port
((HttpConnectorFactory) ((DefaultServerFactory) getServerFactory()).getAdminConnectors().get(0)).setPort(0); } }
I am using one third party jar in my code. In the jar file , in one of the classes, when I opened the class using de-compiler, the code below is written:
java.net.URL fileURL = ClassLoader.getSystemResource("SOAPConfig.xml");
Now I am using this in my webapplication, where should I place this SOAPConfig.xml so that it will find the fileURL.
Note: I have tried putting this XML in WEB-INF/classes folder. But it is not working. Your help will be appreciated.
In Addition: In the explaination you have given, It is telling me not to use this code snippet inside the third party jar in this way...What is the exact usage of this statement
ClassLoader.getSystemResource will load the resource from the system classloader, which uses the classpath of the application as started from the command line. Any classloaders created by the application at runtime (i.e. the one that looks in WEB-INF/classes) are not on the system classpath.
You need to
Look through the script that starts your server, find out which directories are on the classpath there, and put your SOAPConfig.xml in one of those. If necessary, change the classpath in the script to look in a separate directory that's just used for your config file.
Track down the person who used ClassLoader.getSystemResource in the library, kick them squarely in the nuts, and tell them never to do that again.