Importing "google/protobuf/descriptor.proto" in java protocol buffers - java

I have a .proto file definition which needs to import "google/protobuf/descriptor.proto" because I use Custom Options.
So in my .proto file I do:
import "google/protobuf/descriptor.proto";
package ...;
...
Since my file didn't compile complaining about the dependency, I got a copy of the descriptor.proto file placing it in the same directory my proto file was.
This solved the problem but I don't believe this is the correct way. Now the descriptor.proto gets compiled together with my .proto file resulting in having 2 compiled descriptor.proto at runtime:
the one shipped with the protobuf-java-2.5.0.jar file
the one which was compiled together with my .proto file
I think the --proto-path option should be used somehow but not entirely sure what is the correct way.
Thanks for the best practise tip here!

When I have used descriptor in a .proto, I have used it like
import "google/protobuf/descriptor.proto";
message AddressBook {
required google.protobuf.FileDescriptorSet proto_files = 1;
Then to generate the java (on windows) with addressbookSD.proto in the default directory:
protoc addressbookSD.proto --java_out=./ --proto_path=./ --proto_path=<protobuf-install-directory>\src
where <protobuf-install-directory> is the protocol buffers install directory.
The key point is descriptor.proto is in
<protobuf-install-directory>\src\google\protobuf
The levels in an protobuf import stament must match directories in the File system just like they would in java.
So I use <protobuf-install-directory>\src as the import directory, The directory structure must be
<protobuf-install-directory>\src
+-- google
+-- protobuf
+-- descriptor.proto

Just for the record: I had the same problem recently, and in my case my project depends on protobuf-lite. Apparently, protobuf needed the 'full' protobuf for the custom extensions, and therefore I had to add the dependency, as shown here:
dependencies {
protobuf "io.grpc:grpc-protobuf:${grpcVersion}"
...
}
Note that I depend on gRPC in my project, but that works with com.google.protobuf, too.

Your protoc is not able to find the files in the default include folder for your system
https://github.com/golang/protobuf/issues/694
apt install protobuf-compiler does not put it in include folder
Use this if you are having errors like in linux machines
google/protobuf/descriptor.proto: File not found.
google/protobuf/duration.proto: File not found.
google/protobuf/timestamp.proto: File not found.
For Correct installation on linux systems
PROTOC_ZIP=protoc-3.7.1-linux-x86_64.zip
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/$PROTOC_ZIP
sudo unzip -o $PROTOC_ZIP -d /usr/local bin/protoc
sudo unzip -o $PROTOC_ZIP -d /usr/local 'include/*'
rm -f $PROTOC_ZIP

Related

Generated JAR throws ClassNotFoundException for main class

I'm using IntelliJ IDEA to create a JAR. I selected "JAR from modules with dependencies" and "extract to the target JAR" for library JARs - the generated JAR looks just fine:
myJar.jar
|
+- META-INF
| +- MANIFEST.MF
+- com
| +- my
| +- package
| +- Main.class
+- some dependencies...
I checked twice: all the needed dependencies are present. The Main-Class field in MANIFEST.MF points to the correct main class (com.my.package.Main in this example). I've opened the file in my archive tool and inspected it using
jar tf myJar.jar
and both show that the classes needed are available (also Main.class). I've left the Classpath field in IntelliJ's wizard empty, as I do not need any external libraries. However, when I issue
java -jar myJar.jar
it throws the following exception:
Error: could not find or load main class com.my.package.Main
Caused by: java.lang.ClassNotFoundException: com.my.package.Main
I've followed this guide by JetBrains
, and IntelliJ automatically added all dependencies from the Maven pom.xml to be extracted into the JAR, which is correct.
Am I missing something I should configure here and how comes that this does not work?
(Note: Generating a JAR from that project also does not work when I choose "copy to output folder and link via manifest" for the library JARs.)
(Note 2: Using maven-assembly-plugin is sadly not an option since I referenced other IntelliJ workspace modules which would then not get included.)
Update: That it should work as-is shows also the following phenominum: When I unzip my JAR and do
java -cp . com.my.package.Main
in the created directory, it works without any problems, which is strange given that Java refuses to load it...
In my case, my JAR did not work as expected due to some signed JAR dependencies.
These JARs bring a signature and key file, which also gets extracted when you embed the signed JARs. There is basically no risk in removing them, but they are one of the many possible reasons your JAR may not work.
How to remove such signatures from a JAR?
I'm describing the following steps for Linux/Darwin, but I think that there is a similar way for Windows.
Unpack you JAR.
Since JARs are nothing more than simple ZIP archives, you can use unzip:
mkdir temporaryDirectory
unzip myJar.jar -d temporaryDirectory/
The -d option is optional, but it helps keeping your directory structure clean because it sets the target directory.
Locate the signature files.
The signature files (or keys) are located in the META-INF/ directory, so change there:
cd temporaryDirectory/META-INF/
Next, we need to find the troubling files. They have the file extensions .SF (for the signature) and .DSA for the key file:
ll | grep '.DSA\|.SF'
Delete (or rename) the signature and key files.
Renaming these files bares the benefit that you can restore them later (for whatever reason), deleting them also does the trick but is a bit more risky:
Deleting:
rm signature.DSA signature.SF
# Either enter the name of the files
# instead of 'signature' or use * to delete any:
# rm *.DSA *.SF
Renaming:
rename 's/\.DSA$/.DSA.old/' * # Append ".old" to all .DSA files
rename 's/\.SF$/.SF.old/' * # Append ".old" to all .SF files
Repacking your JAR.
Still in temporaryDirectory/, we can repack our JARs to make them work:
cd ../ # If you're still in temporaryDirectory/META-INF
jar cfm ../myWorkingJar.jar ./META-INF/MANIFEST.MF -C ./ .
Explanation:
jar cfm
jar is Java's built-in JAR builder. Invoking it with c means that we want to create a JAR, f stands for the output file (which we specify next) and m is for the MANIFEST.MF we want to use. If you omit m, jar will write an empty MANIFEST.MF to the JAR.
../myWorkingJar.jar
This is the path we want to output our JAR to. It belongs to f which we specified earlier.
./META-INF/MANIFEST.MF
This is the manifest file we want to use. It belongs to the m of cfm.
-C ./
This means that our .class files are located in this directory (.).
. (last argument)
This specifies that we want to add this directory to the JAR.
If you want a detailed description of what jar is doing, you can issue the command from above with cvfm instead of cfm (v stands for verbose).
Verify that it works.
All set, now you can check that your JAR is working as intended by issuing
java -jar myWorkingJar.jar
Removing the temporary directory.
Since you're finished repairing your JAR, you can safely delete the temporary directory we created (and/or the "broken" JAR).
I've created a simple BASH script which "automates" this process a little bit:
#!/bin/bash
JARNAME=myJar.jar # enter your JAR's name here
OUT_JARNAME=myJar_out.jar # enter your output JAR's name here
# Creating the directory, unpacking the JAR, entering META-INF/
mkdir temp
unzip $JARNAME -d temp
cd temp/META-INF
# Renaming the troublemakers.
rename 's/\.DSA$/.DSA.old/' *
rename 's/\.SF$/.SF.old/' *
# Reassembling the JAR
cd ../
jar cfm ../$OUT_JARNAME ./META-INF/MANIFEST.MF -C ./ .
cd ../
# Uncomment this line if you wish to delete the temp directory upon finish.
# rm -r temp
I hope this helps people also encountering this issue.

jdeprscan throws cannot find class error

I'm trying jdeprscan on my CentOS system. Here are the commands that I'm executing:
export classpath=<PATH/*>:<ANOTHER/PATH/*>:<SOME/OTHER/PATH/*>
jdeprscan --for-removal --verbose --class-path $classpath --release 9 <ANOTHER/PATH>/MyProject.jar
In spite of providing the classpath, I'm getting multiple errors of this sort
Processing class <some/class/in/MyProject.jar>...
error: cannot find class <some/class/in/a/different/jar>
error: cannot resolve Methodref <some/class/in/a/different/jar>.<method>:()Ljava/lang/String;
I've verified that the class mentioned in the error message are pretty much among one of the jars provided in the classpath.
Strangely, it's not the case that I'm getting this error for every other classes referred in the jar that I'm scanning.
Few points for your consideration:
The are 50+ jars in the paths provided in the classpath
The jar that I'm trying to scan is residing in one of the paths mentioned in the classpath
I've tried jdeprscan available in JDK 9 & JDK 10 and getting the same errors
I've tried replacing the * in classpath with *.jar, doesn't help!
Is there anything wrong in the syntax that I'm following or is this a known bug in jdeprscan?
Some preliminary notes on jdeprscsan from Oracle JDK 11 under Windows (I know the question related to JDK 9 on CentOS, but maybe the following applies as well...):
use wildcard "path/to/lib/*.jar" (it will not work without the ".jar" extension)
having a wildcard with more than one path is not supported (i.e. --class-path dir1/;dir2/*.jar throws a parse exception on the wildcard)
JARs in the directory specified by --class-path are added to the classpath and analyzed in alphabetical order, which may cause some error : cannot find class X messages because a JAR a.jar may depend on a JAR b.jar which is not yet loaded.
Based on the above, I found the 3 solutions below. Note that I did the same experiment with jdeprscan from the Oracle JDK 12.0.2 without any improvement.
Solution 1: all classes on the classpath
unzip all JARs in a specific directory (ignore overwritted files such as META-INF/maven/pom.xml), e.g. mylib (note that at this stage, the mylibs directory contains only classes organized by directory packages, and no JARs).
run the following command:
jdeprscan --for-removal --class-path /path/to/mylib /path/to/my-application.jar
Advantage: fast (very manual operations)
Drawback: only analyses the JAR file that has been specified on the command line (my-application.jar)
Solution 2: all JARs + a fat JAR on the classpath
copy all JARs libs into a single mylib directory
extract all files from JARs of mylib (ignore duplicate files) and repack them into a big JAR mylib/00lib.jar (a simple ZIP file renamed to .jar makes the trick)
copy the mylib/00lib.jar to mylib/01lib.jar to ensure that it will be analysed
run the following command:
jdeprscan --for-removal --verbose --class-path path/to/mylib/*.jar path/to/my-application.jar
Advantage: fast (only a few manual operations) + analyses the JARs from mylib
Drawback: some of the jdeprscan messages will be related to the fat JAR 00lib.jar so you will not be able to determine immediately which library uses classes that are deprecated or removed from Java 9 or 11, but you can do it indirectly by looking at the class name, e.g. (a class from the com.atomikos library requires the missing class javax.jms.JMSException):
Processing class com/atomikos/datasource/xa/jms/JmsTransactionalResource...
error: cannot find class javax/jms/JMSException
Solution 3: reorder the JAR files on the classpath
copy all JARs libs into a single mylib directory
run the following command:
jdeprscan --for-removal --class-path path/to/mylib/*.jar path/to/my-application.jar
inspect the log to see error: cannot find class messages that are not supposed to be raised because the JAR exists in the lib directory. For each such library, copy the library with a name that is before the library name which reference it alphabetically speaking. For example, I have a JAR in the lib dir alpha.jar which depends on commons-lang-3.0.jar (which is not yet loaded in the classpath), so I copy commons-lang-3.0.jar to a0commons-lang-3.0.jar so that it will be loaded before alpha.jar. It is important to copy the JAR and not to rename it otherwise it may not be analyzed by jdeprscan (does not occur on every JAR). Once done, go back to step 2 until no error messages produced by library dependencies occurs.
Advantage: gives a clear view of which JAR uses deprecated/removed classes.
Drawback: takes a lot time (manual copy of each individual JARs which cause a classloading issue).
Conclusion
I use jdeprscan with Solution 2. This should be considered as a workaround (is it an incomplete tool documentation or a bug, I don't know...).
If you happen to have a folder in PATH which has classes and JARs, it won't work. You'd have to specify the JARs individually (or get rid of the classes).

Package does not exist error when package was added to classpath

Note, linked solutions (ex. Fatal Error: Unable to find package java.lang in classpath or bootclasspath) do not work.
I get this error, but the package is imported (commons... .jar)
org.apache.commons.lang3.tuple //does not exist import
org.apache.commons.lang3.tuple.MutableTriple
Source code
import org.apache.commons.lang3.tuple.MutableTriple;
import org.apache.commons.lang3.tuple.Pair;
import org.apache.commons.lang3.tuple.Triple;
Build code:
export
JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home
/Library/Java/JavaVirtualMachines/jdk1.8.0_25.jdk/Contents/Home/bin/javac
-target 1.8 -source 1.8 -classpath \ "../lib/commons-lang3-3.4.jar;../lib/httpclient-4.5.jar;../lib/httpcore-4.4.1.jar;../lib/org.json-20120521.jar;../lib/pdfbox-app-2.0.0-20150606.170503-1383.jar;../src/:../lib/commons-lang3-3.4-javadoc.jar;../lib/pdfbox-app-2.0.0-20150606.170503-1383-sources.jar" \ -d output \ ../src/com/tymaf/pdf/*.java
How to fix this problem?
Double check your classpath. Looks like you mixed delimiters ; and :.
Also instead of including jar with compiled classes (library itself). You've included java-docs and sources that are useless in classpath.
../src/:
../lib/commons-lang3-3.4-javadoc.jar;
../lib/pdfbox-app-2.0.0-20150606.170503-1383-sources.jar
Here is my suggestion
How to compile and use .jar extension
.jar extension can be imported different ways depending on your environment and IDE.
Here how it work as native mode from console.
Download the .jar.zip library from
http://www.java2s.com/Code/Jar/c/Downloadcommonslang333jar.htm
Create a folder in your working (project) directory call it libs
Unzip the downloaded file and copy commons-lang3-3.3.jar to your working directory libs
I have also created a class just for testing call it TheNewWork.java and added the 3 imports.
Now from your working directory c:\projects for Compile:
javac -classpath "/Projects/libs/commons-lang3-3.3.jar;" TheNewWork.java
And for running it:
java -classpath "/Projects/libs/commons-lang3-3.3.jar;" TheNewWork
If you have more than one .jar just add ; for Windows and : for Linux. Btw I use windows 10 cmder console and java jdk1.8.0_66. In other OS console you might need to put .:Projects...etc in stead of /Projects...etc. but the idea is the same.
UPDATE
In windows it is possible to set classpath like
set CLASSPATH=%CLASSPATH%;C:\Projects\libs\commons-lang3-3.3.jar
OR in Linux
export CLASSPATH=".:/Projects/libs/commons-lang3-3.3.jar"
Then you can run javac TheNewWork.java but it is personal taste to do it this or the other way. Some things similar is also possible to do in other OS.
Last thing, if you lazy and do neither want to write a full command line nor create a classpath, you could create a batch file with the full command line and run it that way in stead ;)
Some references:
http://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html
https://www3.ntu.edu.sg/home/ehchua/programming/howto/JDK_Howto.html
https://www.chilkatsoft.com/java-classpath-Windows.asp
I hope this solves your problem
Before the solution
After the solution
NOTE
In addition thanks to #MarkPeters notified me on my previous answer: Adding application dependencies directly to the JRE libs is not a good approach, as it makes the JRE suitable for running only one Java application, rather than being a generic runtime. Plus it would complicate whatever deployment the OP wants to do. lib/ext is made for extending the core Java APIs, as described here: docs.oracle.com/javase/tutorial/ext/basics/install.html. Not for normal application dependencies.

j2objc Xcode build rules, not recognizing imports

I am using j2objc to compile and use a java library in iOS. I followed the processes:
http://j2objc.org/docs/Xcode-Build-Rules.html
http://j2objc.org/docs/Required-Link-Flags.html
I do not get any build errors until I start importing the header files to use them:
#import "org/library/Class.h"
The files are not found. What I am missing?
On the other hand, I tried to use the manually translated library files (using the terminal j2objc commands). If I pùt the .h and .m files into the j2objc-dist/include folder they are recognized by the editor and I can use the classes without compile errors. But, when I try to build the project it finds errors of the type _OBJ_CLASS_$_Java. I tried to include the files to the compile list in Xcode and I verified the path of libjre_emul.a but I still get the error.
My library contains packages so it has multiple folders in a tree.
My preference will be to use the first method (original Java sources)
DATA FOR JAVA SOURCES CASE:
Build rules:
Java source files
Custom script:
/Users/xgibert/Desktop/Orekit_iOS/j2objc-dist/j2objc -d ${DERIVED_FILES_DIR} -sourcepath ${PROJECT_DIR}/src/ ** \ --no-package-directories ${INPUT_FILE_PATH};
Output files:
${DERIVED_FILES_DIR}/${INPUT_FILE_BASE}.h
${DERIVED_FILES_DIR}/${INPUT_FILE_BASE}.m
Build phases
Link Binary with libraries:
libicucore.dylib
Security.framework
libz.dylib
Build Settings
Linking - Other Linker Flags:
-ljre_emul -L /Users/xgibert/Desktop/Orekit_iOS/j2objc-dist/lib -force_load /Users/xgibert/Desktop/Orekit_iOS/j2objc-dist/lib/libjre_emul.a -l jre_emul -ObjC
Search Paths - Library Seargh Paths:
/Users/xgibert/Desktop/Orekit_iOS/j2objc-dist/lib
Search Paths - User Header Search Paths:
/Users/xgibert/Desktop/Orekit_iOS/j2objc-dist/include
My java library files are in Project_root/src. The tree looks like this:
root/
src/
org/
orekit/
data/
time/
...
apache/
...
In my ViewController.m file I try to import with the following line without success (file not found):
#import "org/orekit/data/DataProvidersManager.h"
Xcode assumes all sources are in a top-level directory, so imports such as you describe fail. As described in the Xcode-Build-Rules page, the --no-package-directories flag is needed, as it outputs all generated files to the specified build directory without sub-directories, and generates correct #import directives.
A sample project that demonstrates including Java sources with packages is j2objc-sample-reversi. Its UI sucks (I wrote it, sigh), but the game engine is decent; if you need a tougher opponent, crank up the engine strength here.

Create debian package from executable jar source without using any tools and scripts

I have created a package based on http://packaging.ubuntu.com/html/packaging-new-software.html sample. In this sample, sources are c++ files. I want to create my new package from executable jar files source.
I found maven, ANT and dhBuild tools but I don't want to use this tools. So I need a way to create my package with command line. please give me some hints or samples to know most about that.
The goal is to create a package that simply puts a shell script where I want it.
Create a directory to build your packages in. Some use "deb" and others use "packages". Others create a directory structure for making multiple packages under "deb" (or whatever).
mkdir deb
Create the directory structure in deb that represents where you want the script to be placed1
mkdir -p ./deb/usr/local/bin
3.Copy the script into your new directory
cp /path/to/my/script/myscript.sh ./deb/usr/local/bin/
4.Make a subdirectory called "DEBIAN", this will host the package control file.
mkdir -p ./deb/DEBIAN
Create a control file.
touch ./deb/DEBIAN/control
5.Open the control file and enter the text below.
Package: myPackagename (no spaces or underscores allowed)
Priority: optional
Section: misc
Maintainer: Maintainer Name <user#mail.com>
Architecture: all
Version: 1.0
Depends: package1, package2, .........
Description: short description here
long description here (don't remove space at the beginning of
line(replace this with an empty line)
Change ownership:
sudo chown -R root:root ./deb
6.Create the debian package.
dpkg -b ./deb /my/output/destination/packagename.deb
i fail to understand how "not using debhelper" and "creating a package from the cmdline" are contradictory.
however, since you explicitely asked about creating packages "without using any tools and scripts", here you go:
dissect an existing package
a .deb file is really just a renamed archive, so you can do
$ ar xf /path/to/package.deb
study the contents of the package
$ ls
debian-binary control.tar.gz data.tar.gz
the debian-binary file contains the format-version of the package (usually 2.0)
the data.tar.gz file contains the actual data files.
the control.tar.gz contains control information about the package, including it's name (and description), it's dependencies; but also pre-installation and post-installation scripts and the like.
once you've learned the inner workings of a .deb-file, you can create one by inversing the above process:
tar your data into a data.tar.gz
create the control information for your deb and tar it into a control.tar.gz
create a debian-binary file containing the binary-version your are using:
$ echo "2.0" > debian-binary
create a deb out of these files by running:
$ ar q /tmp/mypackage.deb debian-binary control.tar.gz data.tar.gz
however, you should do the above only if you really know what you are doing.
creating packages is not a trivial task.
that's why people have created a number of tools to help with the repetitive tasks.
use them!

Categories

Resources