Our tool (http://plse.cs.washington.edu/daikon) calculates program invariants by inserting instrumentation into the Java byte codes for a program. The user code is instrumented during runtime via the normal ClassFileTransformer::transform method.
It is also necessary to track value flows through JDK methods. Thus, we need to instrument the Java runtime as well. We cannot use transform, because hundreds of runtime methods are loaded prior to the first time we get control at transform.
Prior to Java 9, we handled this in an offline step that reads rt.jar, instruments its methods, and writes out a modified version as dcomp-rt.jar. The user placed dcomp-rt.jar on the bootclasspath to ensure our modified Java runtime methods were loaded instead of the standard ones. The user program invocation would look something like:
java -cp .:.../daikon/daikon.jar \
-Xbootclasspath/p:.../daikon/java/dcomp_rt.jar:.:.../daikon/daikon.jar \
-javaagent:.../daikon/java/dcomp_premain.jar={various dcomp arguments} \
{user program} {user program arguments}
Now to Java 9+. Our first approach was to read in and instrument the class files within the Java runtime jmod files (via the new jrt:/ file system) and create a dcomp_rt.jar as before. The problem we are experiencing is that we cannot get the system to use the contents of this jar instead of jrt:/java.base (for example). We tried various --module-path and -Xbootclasspath (only /a is available now, might be part of problem) options to no avail. Still hoping there might be a way to do this?
If not, I'm guessing we need to make modified versions of each of the interesting runtime jmods and then use a --patch-module argument for each of them. Would this ensure our modified code is loaded instead of the standard runtime?
Any thoughts/suggestions?
Well it looks like --patch-module does the trick. I made the same dcomp_rt.jar but with only classes from java.base.jmod. Then used:
--patch-module java.base={full path}/dcomp_rt.jar
Running java with -verbose:class showed all base classes being loaded from my jar.
Is this the best way to accomplish my goal?
Related
I have to run a java task, with a very large number of classpath (1000, totaling 150k characters if concatenated).
The problem is that java returns an error when I try to execute this class:
/jdk/JAVA8/bin/java: Argument list too long
The error code is 7
I've tried to put the classpaths using "export CLASSPATH=CLASSPATH:....." and so I shouldn't specify them through the -cp java parameter, but it returned the same error.
I'm pretty sure that the problem revolves round a classpath's limit, because if I delete some of the classpath, the error disappears (but then I will have logical errors in the execution, because I need all the classpaths)
You could use classpath wildcards. Especially if many of your jars/class files are in the same directory, this would help a lot.
It could be environment variable size limit or command-line size limit as well rather than javac classpath arg limit.
javac takes arguments from file input as well. You can add all your arguments to this file and pass this file argument to command. Refer this for more.
You didn’t hit a java-specific limitation, but a system dependent limit. This is best illustrated by the fact, that the attempt to set the CLASSPATH variable fails as well, but setting an environment variable via export name=value in the shell isn’t related to Java.
As said by others, you could try to use wildcards for jar files within the same directory, but you have to care that Java does the expansion rather than the shell, as in the latter case, it would again yield a too long command line. So you have to escape the * character to ensure it will not be processed by the shell.
javac supports reading the command line arguments from an external file specified via #filename, but unfortunately, the java launcher doesn’t support this option.
An alternative would be to create symbolic links pointing to the jar files, having shorter paths and specifying these. You could even combine the approaches by creating one directory full of symbolic links and specifying that/directory/* as class path.
But there seems to be a logical error in the requirement. In a comment, you are mentioning “code analysis” and an analyzing tool should not require having the code to analyze in its own application class path. You can access class files via ordinary I/O, if you want to read and parse them. In case you want to load them, e.g. for using the builtin Reflection, you can create new ClassLoader instances pointing to the locations. So the tool doesn’t depend on the application class path and could read the locations from a configuration file, for example.
Using distinct class loaders has the additional advantage that you can close them when you’re done.
JVM does not limit classpath length. However, there is a hard OS limit on command line length and environment variables size.
On Linux check getconf ARG_MAX to see the limit.
On older kernel versions it is only 128KB. On newer kernels it is somewhere around 2MB.
If you want to set really long classpaths, you may need a JAR-Manifest trick. See this question for details.
I am a little bit confused...
I know that classes are loaded by the class loader only when they are needed,that is when we are trying to use static variables or when we are creating instance of that class.Thus if we have for e.g. 3 classes in our program and we are going to use only one,then only that particular class will be loaded and rest are not,but when we run the java compiler,it will create 3 .class files,I know these 3 .class files are byte code files,but then what is this byte code and what is the difference between loading a class and generating bytecode of a class?Where is the use of this byte code?If we are not going to use a particular class,then what is the need of generating a bytecode for that class?
Java is a compiled language. The purpose of compiling into bytecode is to allow the code to run on the JVM on any platform. Platform independence is a feature built into java.
Furthermore, you don't have to compile all three class files unless they have inter-dependencies. You can specify which specific files to compile in the console javac command. If you are using an IDE, check your settings or exclude the undesired class from the project.
Loading a class happens at runtime, when you're preparing to invoke whatever properties the class has.
Generating the bytecode of a class happens at compile time. This allows the code to be run on the virtual machine.
Java is a compiled language, and it runs on top of the Java Virtual Machine. Compiling bytecode translates whatever higher level code (be it Java, Scala, or Clojure) into machine-independent instructions to be read by the JVM. This is why that your (backend-specific) program will generally run without modification on Linux, Windows, and Mac OS X.
The Java language will compile any classes that have dependencies on each other within the path, so if you have a class but it is not used, chances are it will not be compiled. There may be tools that override that, so if you find yourself not using a class, then remove the class so that unnecessary bytecode is not generated.
Difference between languages like C++ and java is byte code. C++ binaries(compiled,assembled,linked) will have the machine(op) codes for the OS it got compiled for. In the case of java the byte code is the target for JVM. Byte code will have the opcodes for JVM. JVM in turn will initiate the respective os calls. So bytecode and JVM makes java programs independent of os.
Reg loading class loading,it happens when the program needs it. This is at runtime. JIT will do the second compilation of class when needed.
When we compile .java we get .class file.
The .class file is called byte code.
The Byte code in Java is nothing but a .class file which is not understandable by humans i.e (00110011). These .class files are generated only after the compilation of .java.
These .class file can be used to run on any platform.
So I have a java project with multiple java files.
I know that is almost straight forward to start a java application using batch file. But that is for a pretty simple java program with a single class.
However I am wondering if it is possible to do that with in a scale of a project that you usually create using eclipse. A large project with multiple packages, classes and multiple java files.
My try was to write a script and apply on the main class as following
set path = C:\Program Files\Java\jdk1.7.0_25\bin
javac -classpath twitter/twitter4j-stream-3.0.5.jar;twitter4j-core-3.0.5.jar" sourcepath="lib/twitter4j-core-4.0.1.jar;lib/twitter4j-core-4.0.1.jar;lib/twitter4j-stream-4.0.1.jar;svm_light_lib Program.java
java Program
However when I start the .bat file it automatically closes.
Any Ideas ?
Thanks in advance
First, never overwrite the environment variable path, not even
temporarily. Append your folder instead: set "path=%path%;%mypath%" or set "path=%mypath%;%path%".
(There exists a particular path command but I'm not sure about right syntax: path=%path%;%mypath% with = assignment or path %path%;%mypath% without it).
Use full path to a program if you know it, e.g. "%mypath%\javac".
For better readability, values for -classpath and -sourcepath options are stored to the environment variables mycpth and mysrcp, respectively. Note and use proper " quotation and no spacing around = to avoid any leading and trailing spaces in all set commands.
pause to see all the javac output. Displays the message Press any key to continue . . .
Next code should be (syntax) error-free. However, success depends (among others) on classpath and sourcepath entries visibility as well...
set "mypath=C:\Program Files\Java\jdk1.7.0_25\bin"
set "path=%path%;%mypath%"
set "mycpth=twitter/twitter4j-stream-3.0.5.jar;twitter4j-core-3.0.5.jar"
set "mysrcp=lib/twitter4j-core-4.0.1.jar;lib/twitter4j-core-4.0.1.jar;lib/twitter4j-stream-4.0.1.jar;svm_light_lib"
"%mypath%\javac" -classpath "%mycpth%" -sourcepath "%mysrcp%" Program.java
pause
java Program
However I am wondering if it is possible to do that with in a scale of a project that you usually create using eclipse. A large project with multiple packages, classes and multiple java files.
Of course it is possible!
In this case, I suspect the problem is that you java command doesn't have a "-cp" argument. The java command is probably failing because it can't find twitter classes ... at runtime.
Remember to include "." on the classpath ... or else java won't find the file that you just compiled.
#JB Nizet's suggestion is also very important advice for finding out what is actually happening.
I have a C binary that calls out to Java via JNI. I set CLASSPATH to somedir/* to pick up all the jars in somedir.
When I run the binary, a required class definition cannot be found. When I run
java that.class's.name
from the same command line, the class is successfully found. If I explicitly add all the jars in somedir/ to the classpath, everything works great, but that leads to a very long classpath which I'd like to avoid.
Does a JVM executed via JNI honour wildcard expansion of the classpath? Can it be made to do so?
I figured out the answer by reading the hotspot source code.
Only paths passed via either CLASSPATH or -cp / -classpath are subject to wildcard expansion. These are then passed as a system property to the running JVM via -Djava.class.path.
You tell a JNI-invoked JVM about a classpath via a JVMOptions structure, which may include -Djava.class.path but -classpath will not necessarily be honoured (and in practice, isn't by the hotspot implementation). Since java.class.path is directly passed to the JVM as a system property, it doesn't get wildcard expanded and therefore wildcards won't work.
No. No, it cannot. Using JNI doesn't help.
The way you would do this is by implementing your own class loader (in Java), but that class loader would have to be in the wildcard-free CLASSPATH.
You could, of course, set the CLASSPATH to its expanded form before invoking the JVM. That would work and could be done via a shell script (no JNI needed).
I'm trying to run .class file from command line. It works when I manually move to the directory it's stored in, but when I try something like this:
java C:\Peter\Michael\Lazarus\Main
it says it can't find the main class. Is there any solution to this other than making a .jar file (I know that .jar is the best solution, but at this moment isn't the one I'm looking for)?
The Java application launcher (a.k.a java.exe or simply java) supports up to four different ways to specify what to launch (depending on which Java version you use).
Specifying a class name is the most basic way. Note that the class name is different from the file name.
java -cp path/to/classFiles/ mypackage.Main
Here we start the class mypackage.Main and use the -cp switch to specify the classpath which is used to find the class (the full path to the class mypackage.Main will be path/to/classFiles/mypackage/Main.class.
Starting a jar file.
java -jar myJar.jar
This puts the jar itself and anything specified on its Class-Path entry on the class path and starts the class indicated via the Main-Class entry. Note that in this case you can not specify any additional class path entries (they will be silently ignored).
Java 9 introduced modules and with that it introduce a way to launch a specific module in a way similar to how option #2 works (either by starting that modules dedicated main class or by starting a user-specified class within that module):
java --module my.module
Java 11 introduces support for Single-File Source Code Programs, which makes it very easy to execute Java programs that fit into a single source file. It even does the compile step for you:
java MyMain.java
This option can be useful for experimenting with Java for the first time, but quickly reaches its limits as it will not allow you to access classes that are defined in another source file (unless you compile them separately and put them on the classpath, which defeats the ease of use of this method and means you should probably switch back to option #1 in that case).
This feature was developed as JEP 330 and is still sometimes referred to as such.
For your specific case you'd use option #1 and tell java where to look for that class by using the -classpath option (or its short form -cp):
java -classpath C:\Peter\Michael\Lazarus\ Main
If your Main.java contains the entirety of your source code (and it is in the same directory), then you can use option #4, skip the compile step and directly compile-and-execute it:
java c:\Peter\Michael\Lazarus\Main.java
Assuming that Main.class does not have a package declaration:
java -cp C:\Peter\Michael\Lazarus\ Main
Java looks for classes in a "classpath", which can be set on the command line via the -cp option.
I just had the same issue, I tried running java hello.class, this is wrong.
The command should be java hello.
Do not include the file extension. It is looking for a class file, and will add the name on its own.
So running 'java hello.class' will tell it to go looking for 'hello.class.class' file.
Try this:
java -cp C:\Peter\Michael\Lazarus Main
You need to define the classpath.