I uploaded a Spring application to Heroku but the application crashed with the following error:
java.io.FileNotFoundException: class path resource [com/myname/myapp/config/dao-context.xml
The file is definitely there, and it is in GIT, and the app runs successfully locally.
Any ideas what is happening here?
I suspect that when you are running locally, it is picking up the file on the classpath as a regular file on the filesystem (i.e. not inside of a JAR).
On Heroku, it is probably inside of a JAR file, which means it is not a regular file, and must be read as an input stream, which might look like this:
ClassLoader cl = this.getClass().getClassLoader();
InputStream inputStream = cl.getResourceAsStream("com/myname/myapp/config/dao-context.xml");
You can probably reproduce the problem locally by running the same command that's in your Procfile.
If this is not the case, then make sure the file exists on Heroku by running this command:
$ heroku run ls com/myname/myapp/config/dao-context.xml
For future visitors to this question, I overcame the problem by converting my DAO XML config file to the Java Config method, therefore Spring no longer required that XML file. This didn't directly solve the issue of being unable to find the XML file, but the added benefit is that I am now using the more modern and less verbose Java Config method.
Related
I am trying to run Spark locally in my Intellij and passing in an external jar with sparkConf:
sparkConf.set("spark.jars", "C:\\path\\to\\my\\jar.jar");
I see in the console when running that the jar has been added:
INFO org.apache.spark.SparkContext:57 - Added JAR C:\\path\\to\\my\\jar.jar at spark://some spark path
I also see the jar in Spark UI underneath Classpath Entries:
some-spark-path Added By User
However I am running into two issues.
When I tried to call upon a resource file in the external jar via code below, I get a null for the Inputstream, path would be like "/file/i/need.json":
try(InputStream is= SomeClass.class.getResourceAsStream(path))
When I am also passing a className (this class exists in the external jar) as string to a method which then creates an instance, I get ClassNotFoundException. ClassName would be like "class.i.need":
Class<?> classes = Class.forName(className);
Any advice would be helpful. Thanks!
Edit: Running on Spark 3.0.0.
I have also tried with "spark.driver.extraClassPath" to no avail.
My Springboot Application is running fine in IDE but when I create fat jar file and run on docker it gives the error. I am connecting my application with firebase so i want to include the serviceAccountKey.json file from the resource folder. The application runs fine in my ide, but while deploying it over the docker container it gives the error of file not found. Though when include the file and print it path it doesn't give any sort of error . But when i give the file path to fileInputStream it produces the error. I have tried multiple ways but nothing seems to work. I'm including the file using classLoader.getResource("filename.json").
I tried to skip the inclusion of file and do it by saving files content in a string and then sending it to stream but this method crashes the server whenever i query firebase.
this is the code where error is occurring. Notice that I'm printing the file path and it gets printed in the output before showing error. I have also tried file.getAbsoluteFile instead of path but doesn't work. Probably I'm doing it wrong or probably i have to mention the path in some other place as well which i don't know about. If anyone has done this before then please help me on this.
File path is getting printed but FileStream can't get it
An alternative to having a credential in a json file that is packaged with the application, would be that you set an environment variable upon starting the application, and instead load the key via application.yml. This way, you don't need to package the secrets into the application jar file.
1. create config class
#Configuration
#ConfigurationProperties(value = "my-custom-config")
public class MyConfig {
private String serviceAccountKey;
// getters/setters etc. if not using lombok
}
2. use above config
in other spring beans by injecting the config class, and retrieve the secret with getter, e.g. config.getServiceAccountKey();
3. Add config to your application.yml file
# application.yml
my-custom-config:
serviceAccountKey: ${ENV_VARIABLE_NAME} # <---- this way you can bind an env variable to your config.
4. Define env variable in container on startup
On instantiating the docker container, provide env option and define an environment variable.
docker run -env ENV_VARIABLE_NAME=<value> ...
For context, I'm trying to configure my Spark session to use fair scheduling. For that reason I have a file, fairscheduler.xml, at the 'root' of my resources folder. This file is correctly packaging into my build folder in the location I'd expect.
To further prove that's working, I have this line:
val mypath = getClass.getResource("/fairscheduler.xml")
That is working and returns exactly the path I'd expect: /<some path>/<my jar>/fairscheduler.xml
The following is throwing a FileNotFound exception:
sessionBuilder
...
.config("spark.scheduler.mode", "FAIR")
.config("spark.scheduler.allocation.file", mypath.toString) <- THIS LINE
.config("spark.scheduler.pool", "fair_pool")
...
.getOrCreate
Just for sanity, I have logged out mypath.toString.
Why is it that the spark session builder can't recognize a resource file that otherwise seems valid? Does it expect "spark configuration" files to exist somewhere specific? Or am I completely missing some dumb little thing here?
This is not a Apache Spark limitation but an expected behaviour of how resource files are handled in Java JAR files. Question How to get a path to a resource in a Java JAR file provides more details and a workaround of persisting the content in a temp file.
I am running a project using $ activator "run 8081". In the application, I try to read an xml file using the following command:
resource = ClassLoader.getSystemClassLoader().getResource(filePath)
The problem is that the resource is null. After debuging, I noticed that the running ClassLoader.getSystemClassLoader().getURLs() returns a single jar file. Which means that the above code is trying to load filePath from inside this jar file.
Any ideas on what I can do to fix this problem and load the filePath? Note that I have to keep the ClassLoader.getSystemClassLoader().getResource(filePath) as is, but I can change the value of filePath and environment variables, etc.
EDIT:
How do I set the classpath for activator when I run it in dev mode: $ activator "run 8081"?
Answering this can help answer my original question.
Iam using spring boot org.springframework.boot.loader.JarLauncher to run my self executable spring boot app jar.
Now, I have my self executable jar packed like this (For brevity Iam just adding the files only that are needed to show the problem):
Main.jar
---META-INF
---lib
---a.jar
---com
---comp
---Abc.class
---folder1
---1.txt
---2.txt
---b.jar
and so on.
In my Abc.class Iam trying to load the resource 1.txt which is working fine when I run it in eclipse; but the story starts when I run using command line as self executable jar. I was not able to read that resource and ends up with null pointer error.
After reading few threads, I learnt that my IDEs does some magic on Class Loaders and so it was working fine, which will not be the case when I run it as executable jar
This is how I was Loading the files and the all the possible options I have tried to no luck.
Option 1:
Abc.class.getResourceAsStream("\folder1\1.txt")
Option 2:
Abc.class.getClassLoader().getResourceAsStream("folder1\1.txt")
Option 3: Reading thread, I have tried considered the current thread class loader context as below too, But to no luck again
Thread.currentThread().getContextClassLoader().getResourceAsStream("folder1\1.txt")
Can any one advise. How should I write my code to be able to read my resource that is in the jar and by the class that is in the same jar ?
PS: Spring boot is not adding class-path entry to MANIFEST.MF and if I have to do something around that, how do I do that ? In-fact I tried -cp when running my jar setting it to current directory, but that have not worked either
In Spring, the best way to access a Resource is via Resource APIs. For a Classpath resource what you should use a ClassPathResource and it would look something like this:
Resource resource = new ClassPathResource("/my/resource.json", this.getClass());
After obtaining a Resource you can either get a File reference via getFile() or get an InputStream straight off by calling getInputStream().
Spring provides quite a few different implementations of the Resource interface, take a look at the list of known implementations in the docs
Use Spring's class ClassPathResource to load file from classpath.
For example you can read file into String like this from classpath:
String fileContent = FileUtils.readFileToString(
new ClassPathResource("folder1" + File.separator + "1.txt").getFile());
It doesn't matter what kind of build tool or project structure you are using as long as that file is on classpath.