I created a new Android Studio Project with C++ support and did not add anything to it. The project builds, compiles and runs with CMake. I then added the following two lines of code to CMakeLists.txt at the bottom of the file and I get a CMake build error.
find_package(Java COMPONENTS Development) # Line 47
find_package(JNI REQUIRED) # Line 48
And this is the error I get when building the project
-- Found Java: /usr/lib/jvm/java-10-oracle/bin/java (found version "10.0.1") found components: Development
-- Configuring incomplete, errors occurred!
CMake Error at /home/xxxxx/Android/Sdk/cmake/3.6.4111459/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find JNI (missing: JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY
See also "/home/xxxxx/AndroidStudioProjects/MyApplication/app/.externalNativeBuild/cmake/release/x86_64/CMakeFiles/CMakeOutput.log".
JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH)
Call Stack (most recent call first):
/home/xxxxxx/Android/Sdk/cmake/3.6.4111459/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
/home/xxxxxx/Android/Sdk/cmake/3.6.4111459/share/cmake-3.6/Modules/FindJNI.cmake:314 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:48 (find_package)
External native generate JSON release: JSON generation completed with problems
CMake runs when being called form the command line, but fails inside Android Studio for some reason and I am not sure why.
EDIT 1
Here is the CMakeOutput.log file
EDIT 2
This is all that is in CMakeLists.txt file. It compiles in cmd with the command cmake.
cmake_minimum_required(VERSION 3.4.1)
find_package(Java COMPONENTS Development)
find_package(JNI REQUIRED)
This is the cmd output
E:\Users\xxxxx\AndroidStudioProjects\MyApplication\app\build>cmake ..
-- Configuring done
-- Generating done
-- Build files have been written to: E:/Users/xxxxx/AndroidStudioProjects/MyApplication/app/build
When I try to add it compile it in Android Studio using gradle I get the following error on Windows 10 pro 64 bit
Microsoft Windows [Version 10.0.17134.112]
(c) 2018 Microsoft Corporation. All rights reserved.
E:\Users\xxxxx\AndroidStudioProjects\MyApplication>gradlew build --stacktrace
> Task :app:generateJsonModelDebug
External native generate JSON debug: starting JSON generation
External native generate JSON debug: using platform version 24 for ABI ARMEABI_V7A and min SDK version 24
External native generate JSON debug: noticing that build file 'E:\Users\xxxxx\AndroidStudioProjects\MyApplication\app\CMakeLists.txt' is out of date with respect to E:\Users\xxxxx\AndroidStudioProjects\MyApplication\app\.extern
alNativeBuild\cmake\debug\armeabi-v7a\android_gradle_build.json
External native generate JSON debug: rebuilding JSON E:\Users\xxxxx\AndroidStudioProjects\MyApplication\app\.externalNativeBuild\cmake\debug\armeabi-v7a\android_gradle_build.json due to:
External native generate JSON debug: - a dependent build file changed
External native generate JSON debug: keeping json folder 'E:\Users\xxxxx\AndroidStudioProjects\MyApplication\app\.externalNativeBuild\cmake\debug\armeabi-v7a' but regenerating project
External native generate JSON debug: executing cmake Executable : E:\Android\SDK\cmake\3.6.4111459\bin\cmake.exe
arguments :
-HE:\Users\xxxxx\AndroidStudioProjects\MyApplication\app
-BE:\Users\xxxxx\AndroidStudioProjects\MyApplication\app\.externalNativeBuild\cmake\debug\armeabi-v7a
-DANDROID_ABI=armeabi-v7a
-DANDROID_PLATFORM=android-24
-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=E:\Users\xxxxx\AndroidStudioProjects\MyApplication\app\build\intermediates\cmake\debug\obj\armeabi-v7a
-DCMAKE_BUILD_TYPE=Debug
-DANDROID_NDK=E:\Android\SDK\ndk-bundle
-DCMAKE_CXX_FLAGS=
-DCMAKE_TOOLCHAIN_FILE=E:\Android\SDK\ndk-bundle\build\cmake\android.toolchain.cmake
-DCMAKE_MAKE_PROGRAM=E:\Android\SDK\cmake\3.6.4111459\bin\ninja.exe
-GAndroid Gradle - Ninja
jvmArgs :
CMake Error at E:/Android/SDK/cmake/3.6.4111459/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find JNI (missing: JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY
JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH)
Call Stack (most recent call first):
E:/Android/SDK/cmake/3.6.4111459/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
E:/Android/SDK/cmake/3.6.4111459/share/cmake-3.6/Modules/FindJNI.cmake:314 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
CMakeLists.txt:4 (find_package)
-- Configuring incomplete, errors occurred!
See also "E:/Users/xxxxx/AndroidStudioProjects/MyApplication/app/.externalNativeBuild/cmake/debug/armeabi-v7a/CMakeFiles/CMakeOutput.log".
Could NOT find JNI (missing: JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY
Solved with
sudo apt-get install -y openjdk-8-jdk
sudo apt-get install -y default-jdk
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
As the CMake version bundled with Android tries to tell you, it can't find the JNI package because some parts were missing:
missing: JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH
Checking the documentation for FindJNI (https://cmake.org/cmake/help/latest/module/FindJNI.html) these variables are set to the locations of libraries and headers not shipped with the Android version of JNI (to little surprise, Android does not included the AWT library for instance).
When running find_package(JNI REQUIRED), the FindJNI code checks if these variables are set and if not, issues an error.
A workaround is to set these variables yourself, before calling find_package:
# We are only interested in finding jni.h: we do not care about extended JVM
# functionality or the AWT library.
set(JAVA_AWT_LIBRARY NotNeeded)
set(JAVA_JVM_LIBRARY NotNeeded)
set(JAVA_INCLUDE_PATH2 NotNeeded)
set(JAVA_AWT_INCLUDE_PATH NotNeeded)
find_package(JNI REQUIRED)
Be aware though, that your code will only be able to use jni.h and its functionality: if it tries to access any other part of the JNI package it will fail (probably at compile time) because essentially, you have tricked CMake into thinking that the entire package was found, when in reality only a part of it exists in the Android setup.
I found a solution that works for me:
cmake_minimum_required(VERSION 3.10)
project("ktaglib")
set(JAVA_AWT_LIBRARY "$ENV{JAVA_HOME}/lib/libjawt.so")
set(JAVA_JVM_LIBRARY "$ENV{JAVA_HOME}/lib/server/libjvm.so")
set(JAVA_INCLUDE_PATH "$ENV{JAVA_HOME}/include")
set(JAVA_INCLUDE_PATH2 "$ENV{JAVA_HOME}/include/linux")
set(JAVA_AWT_INCLUDE_PATH "$ENV{JAVA_HOME}/include")
find_package(JNI REQUIRED)
You must set the environment variable JAVA_HOME="/usr/lib/jvm/java-11-openjdk-amd64" beforehand.
More info: https://github.com/dockcross/dockcross/issues/576
Same on alpine:edge docker image, fixed using:
ENV JAVA_HOME=/usr/lib/jvm/java-1.8-openjdk
RUN apk add --no-cache openjdk8
ref: https://pkgs.alpinelinux.org/contents?branch=edge&name=openjdk8&arch=x86_64&repo=community
Related
Yoooo!
Scope
I am trying to deploy a Quarkus based application to a Raspberry Pi using some fancy technologies, my goal is to figure out an easy way to develop an application with Quarkus framework, subsequently deploy as native executable to a raspberry device with full GPIO pins access. Below I will provide you will see requirements that I set for myself and my environment settings to have a better picture of the problem that I faced.
Acceptance Criteria
Java 17
Build native executable using GraalVM
Execute native executable in a micro image on raspberry's docker
Target platform can vary
Be able to use GPIO, SPI, I2C and etc. interfaces of the raspberry
Environment
Development PC
Raspberry Pi Model 3 B+
os: Ubuntu 22.04.1 LTS
os: DietPy
platform: x86_64, linux/amd64
platform: aarch64, linux/arm64/v8
Prerequisites
Java: diozero a device I/O library
Docker: working with buildx
Quarkus: build a native executable
How I built ARM based Docker Images for Raspberry Pi using buildx CLI Plugin on Docker Desktop?
Building Multi-Architecture Docker Images With Buildx
Application
source code on github
As for project base I used getting-started application from
https://github.com/quarkusio/quarkus-quickstarts
Adding diozero library to pom.xml
<dependency>
<groupId>com.diozero</groupId>
<artifactId>diozero-core</artifactId>
<version>1.3.3</version>
</dependency>
Creating a simple resource to test GPIO pinspackage org.acme.getting.started;
import com.diozero.devices.LED;
import javax.ws.rs.Path;
import javax.ws.rs.QueryParam;
#Path("led")
public class LedResource {
#Path("on")
public String turnOn(final #QueryParam("gpio") Integer gpio) {
try (final LED led = new LED(gpio)) {
led.on();
} catch (final Throwable e) {
return e.getMessage();
}
return "turn on led on gpio " + gpio;
}
#Path("off")
public String turnOff(final #QueryParam("gpio") Integer gpio) {
try (final LED led = new LED(gpio)) {
led.off();
} catch (final Throwable e) {
return e.getMessage();
}
return "turn off led on gpio " + gpio;
}
}
4.Dockerfile
```
# Stage 1 : build with maven builder image with native capabilities
FROM quay.io/quarkus/ubi-quarkus-native-image:22.0.0-java17-arm64 AS build
COPY --chown=quarkus:quarkus mvnw /code/mvnw
COPY --chown=quarkus:quarkus .mvn /code/.mvn
COPY --chown=quarkus:quarkus pom.xml /code/
USER quarkus
WORKDIR /code
RUN ./mvnw -B org.apache.maven.plugins:maven-dependency-plugin:3.1.2:go-offline
COPY src /code/src
RUN ./mvnw package -Pnative
# Stage 2 : create the docker final image
FROM registry.access.redhat.com/ubi8/ubi-minimal:8.6-902
WORKDIR /work/
COPY --from=build /code/target/*-runner /work/application
# set up permissions for user 1001
RUN chmod 775 /work /work/application \
&& chown -R 1001 /work \
&& chmod -R "g+rwX" /work \
&& chown -R 1001:root /work
EXPOSE 8080
USER 1001
CMD ["./application", "-Dquarkus.http.host=0.0.0.0"]
```
Building image with native executable
Dockerfile based on quarkus docs, I changed image of the build container to quay.io/quarkus/ubi-quarkus-native-image:22.0.0-java17-arm64 and executor container to registry.access.redhat.com/ubi8/ubi-minimal:8.6-902, both of these are linux/arm64* compliant.
Since I am developing and building in linux/amd64 and I want to target linux/arm64/v8 my executable must be created in a target like environment. I can achieve that with buildx feature which enables cross-arch builds for docker images.
Installing QEMU
sudo apt-get install -y qemu-user-static
sudo apt-get install -y binfmt-support
Initializing buildx for linux/arm64/v8 builds
sudo docker buildx create --platform linux/arm64/v8 --name arm64-v8
Use new driver
sudo docker buildx use arm64-v8
Bootstrap driver
sudo docker buildx inspect --bootstrap
Verify
sudo docker buildx inspect
Name: arm64-v8
Driver: docker-container
Nodes:
Name: arm64-v80
Endpoint: unix:///var/run/docker.sock
Status: running
Platforms: linux/arm64*, linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6
Now looks like we're ready to run the build. I ended up with the following command
sudo docker buildx build --push --progress plain --platform linux/arm64/v8 -f Dockerfile -t nanobreaker/agus:arm64 .
--push - since I need to deploy a final image somewhere
--platform linux/arm64/v8 - docker requires to define target platform
-t nanobreaker/agus:arm64 - my target repository for final image
It took ~16 minutes to complete the build and push the image
target platform is linux/arm64 as needed
59.75 MB image size, good enough already (with micro image I could achieve ~10 MB)
After I connected to raspberry, downloaded image and run it
docker run -p 8080:8080 nanobreaker/agus:arm64
Pretty nice, let's try to execute a http request to test out gpio pins
curl 192.168.0.20:8080/led/on?gpio=3
Okey, so I see here that there are permission problems and diozero library is not in java.library.path
We can fix permission problems by adding additional parameter to docker run command
docker run --privileged -p 8080:8080 nanobreaker/agus:arm64
PROBLEM
From this point I do not know how to resolve library load error in a native executable.
I've tried:
Pulled out native executable from final container, executed on raspberry host os and had same result, this makes me think that library was not included at GraalVM compile time?
Learning how library gets loaded https://github.com/mattjlewis/diozero/blob/main/diozero-core/src/main/java/com/diozero/util/LibraryLoader.java
UPDATE I
It looks like I have two options here
Figure out a way to create configuration for the diozero library so it is properly resolved by GraalVM during native image compilation.
Add library to the native image and pass it to the native executable.
UPDATE II
Further reading of quarkus docs landed me here https://quarkus.io/guides/writing-native-applications-tips
By default, when building a native executable, GraalVM will not include any of the resources that are on the classpath into the native executable it creates. Resources that are meant to be part of the native executable need to be configured explicitly. Quarkus automatically includes the resources present in META-INF/resources (the web resources) but, outside this directory, you are on your own.
I reached out #Matt Lewis (creator of diozero) and he was kind to share his configs, which he used to compile into GraalVM. Thank you Matt!
Here’s the documentation on my initial tests: https://www.diozero.com/performance/graalvm.html
I stashed the GraalVM config here: https://github.com/mattjlewis/diozero/tree/main/src/main/graalvm/config
So combining the knowledge we can enrich pom.xml with additional setting to tell GraalVM how to process our library
<quarkus.native.additional-build-args>
-H:ResourceConfigurationFiles=resource-config.json,
-H:ReflectionConfigurationFiles=reflection-config.json,
-H:JNIConfigurationFiles=jni-config.json,
-H:+TraceServiceLoaderFeature,
-H:+ReportExceptionStackTraces
</quarkus.native.additional-build-args>
Also added resource-config.json, reflection-config.json, jni-config.json to the resource folder of the project (src/main/resources)
First, I will try to create a native executable in my native os ./mvnw package -Dnative
Fatal error: org.graalvm.compiler.debug.GraalError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of java.lang.ProcessHandleImpl are allowed in the image heap as this class should be initialized at image runtime. To see how this object got instantiated use --trace-object-instantiation=java.lang.ProcessHandleImpl.
Okey, so it failed, but let's trace object instantiation as recommended, maybe we can do something in configs to get around this. I added --trace-object-instantiation=java.lang.ProcessHandleImpl to the additional build args.
Fatal error: org.graalvm.compiler.debug.GraalError: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: No instances of java.lang.ProcessHandleImpl are allowed in the image heap as this class should be initialized at image runtime. Object has been initialized by the java.lang.ProcessHandleImpl class initializer with a trace:
at java.lang.ProcessHandleImpl.<init>(ProcessHandleImpl.java:227)
at java.lang.ProcessHandleImpl.<clinit>(ProcessHandleImpl.java:77)
. To fix the issue mark java.lang.ProcessHandleImpl for build-time initialization with --initialize-at-build-time=java.lang.ProcessHandleImpl or use the the information from the trace to find the culprit and --initialize-at-run-time=<culprit> to prevent its instantiation.
something new at least, let's try to initialize it first at build time with --initialize-at-build-time=java.lang.ProcessHandleImpl
Error: Incompatible change of initialization policy for java.lang.ProcessHandleImpl: trying to change BUILD_TIME from command line with 'java.lang.ProcessHandleImpl' to RERUN for JDK native code support via JNI
com.oracle.svm.core.util.UserError$UserException: Incompatible change of initialization policy for java.lang.ProcessHandleImpl: trying to change BUILD_TIME from command line with 'java.lang.ProcessHandleImpl' to RERUN for JDK native code support via JNI
Okey, we're not able to change the initialization kind and looks like it won't give us any effect.
I found out that with -H:+PrintClassInitialization we can generate a csv file with class initialization info
here we have two lines for java.lang.ProcessHandleImpl
java.lang.ProcessHandleImpl, RERUN, for JDK native code support via JNI
java.lang.ProcessHandleImpl$Info, RERUN, for JDK native code support via JNI
So it says that class is marked as RERUN, but isn't this the thing we're looking for? Makes no sense for me right now.
UPDATE III
With the configs for graalvm provided by #Matt I was able to compile a native image, but it fails anyways during runtime due to java.lang.UnsatisfiedLinkError, makes me feel like the library was not injected properly.
So looks like we just need to build a proper configuration file, in order to do this let's build our application without native for now, just run it on raspberry, trigger the code related to diozero, get output configs.
./mvnw clean package -Dquarkus.package.type=uber-jar
Deploying to raspberry, will run with graalvm agent for configs generation (https://www.graalvm.org/22.1/reference-manual/native-image/Agent/)
/$GRAALVM_HOME/bin/java -agentlib:native-image-agent=config-output-dir=config -jar ags-gateway-1.0.0-SNAPSHOT-runner.jar
Running simple requests to trigger diozero code (I've connected a led to raspberry on gpio 4, and was actually seeing it turn off/on)
curl -X POST 192.168.0.20:8080/blink/off?gpio=4
curl -X POST 192.168.0.20:8080/blink/on?gpio=4
I've published project with output configs
One thing I noticed that "pattern":"\\Qlib/linux-aarch64/libdiozero-system-utils.so\\E" aarch64 library gets pulled while running on py which is correct, but when I build on native OS I should specify 'amd64' platform.
Let's try to build a native with new configs
./mvnw package -Dnative
Successfully compiled, let's run and test
./target/ags-gateway-1.0.0-SNAPSHOT-runner
curl -X POST localhost:8080/led/on?gpio=4
And here we have error again
ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-0) HTTP Request to /led/on?gpio=4 failed, error id: b0ef3f8a-6813-4ea8-886f-83f626eea3b5-1: java.lang.UnsatisfiedLinkError: com.diozero.internal.provider.builtin.gpio.NativeGpioDevice.openChip(Ljava/lang/String;)Lcom/diozero/internal/provider/builtin/gpio/GpioChip; [symbol: Java_com_diozero_internal_provider_builtin_gpio_NativeGpioDevice_openChip or Java_com_diozero_internal_provider_builtin_gpio_NativeGpioDevice_openChip__Ljava_lang_String_2]
So I finally managed to build native image, but for some reason it didn't resolve JNI for native library.
Any thoughts on how to properly inject diozero library into native executable?
UPDATE IV
With help of #matthew-lewis we managed to build aarch64 native executable on amd64 os. I updated the source project with final configurations, but I must inform you that this is not a final solution and it doesn't cover all the library code, also according to the Matt's comments this might not be the only way to configure the graalvm build.
I've created a very simple Quarkus app that exposes a single REST API to list the available GPIOs. Note that it currently uses the mock provider that will be introduced in v1.3.4 so that I can test and run locally without deploying to a Raspberry Pi.
Running on a Pi would be as simple as removing the dependency to diozero-provider-mock in the pom.xml - you would also currently need to change the dependency to 1.3.3 until 1.3.4 is released.
Basically you need to add this to the application.properties file:
quarkus.native.additional-build-args=\
-H:ResourceConfigurationFiles=resource-config.json,\
-H:JNIConfigurationFiles=jni-config.json,\
-H:ReflectionConfigurationFiles=reflect-config.json
These files were generated by running com.diozero.sampleapps.LEDTest with the GraalVM Java executable (with a few minor tweaks), i.e.:
$GRAALVM_HOME/bin/java -agentlib:native-image-agent=config-output-dir=config \
-cp diozero-sampleapps-1.3.4.jar:diozero-core-1.3.4.jar:tinylog-api-2.4.1.jar:tinylog-impl-2.4.1.jar \
com.diozero.sampleapps.LEDTest 18
Note a lot of this was based my prior experiments with GraalVM as documented here and here.
The ProcessHandlerImpl error appear to be related to the tinylog reflect config that I have edited out.
Update 1
In making life easy for users of diozero, the library does a bit of static initialisation for things like detecting the local board. This causes issues when loading the most appropriate native library at most once (see LibraryLoader - you will notice it has a static Map of libraries that have been loaded which prevents it being loaded at runtime). To get around this I recommend adding this build property:
--initialize-at-run-time=com.diozero.sbc\\,com.diozero.util
Next, I have been unable to resolve the java.lang.ProcessHandleImpl issue, which prevents reenabling the service loader (diozero uses service loader quite a bit to enable flexibility and extensibility). It would be nice to be able to add this flag:
quarkus.native.auto-service-loader-registration=true
Instead I have specified relevant classes in resource-config.json.
I download the source code of OpenJDK14 and put them in \home\yuanfang\jdk14 and after running bash configure --disable-warnings-as-errors and make images, I build OpenJDK14 successfully, The newly built JDK is in home\yuanfang\jdk14\build\linux-x86_64-server-release\jdk. By the way, I am using Ubuntu 20.04 LTS.
Then I want to debug OpenJDK14 using CLion IDE. I was using CLion 2020.2 and I take the following steps:
After open CLion I choose Create New CMake Project from Sources and choose the directory of \home\yuanfang\jdk14, which is the root directory of the jdk project.
I alter the Run/Debug Configurations to make it look like this:
CLion create a CMakeLists.txt automatically but the file doesn't work, so after googling I find find the correct CMakeLists.txt here at https://github.com/ojdkbuild/ojdkbuild/blob/master/src/java-14-openjdk/CMakeLists.txt. I then rewrite the old CMakeLists.txt using the correct one.
I download the whole repository(that is, github.com/ojdkbuild/ojdkbuild), unzip it and put it into \home\yuanfang\jdk14.
It looks like as follow, the ojdkbuild-master is the newly added folder.
I reload the CMake project, but some CMake error occurs(as follow), why can't CLion find those files? javaI googled but can't find any effective solution. Is there anything I can do or refer to? Thanks in advance.
CMake Error at CMakeLists.txt:19 (include):
include could not find load file:
/home/yuanfang/jdk14/../../resources/cmake/ojdkbuild_common.cmake
CMake Error at CMakeLists.txt:21 (include):
include could not find load file:
/home/yuanfang/jdk14/../../resources/cmake/version.cmake
CMake Error at CMakeLists.txt:93 (add_subdirectory):
add_subdirectory given source
"/home/yuanfang/jdk14/../../deps/rhino/scripting_tasks" which is not an
existing directory.
CMake Error at CMakeLists.txt:98 (ojdkbuild_add_subdirectory):
Unknown CMake command "ojdkbuild_add_subdirectory".
-- Configuring incomplete, errors occurred!
See also "/home/yuanfang/jdk14/cmake-build-debug/CMakeFiles/CMakeOutput.log".
Cannot get compiler information:
Compiler exited with error code 1: /usr/bin/cc -xobjective-c -I/home/yuanfang/jdk14/build/linux-x86_64-server-release/hotspot/variant-server/gensrc/adfiles......./loading/LibraryLoader/jar_src -g -fpch-preprocess -v -dD -E
Using built-in specs.
COLLECT_GCC=/usr/bin/cc
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-10ubuntu2' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)
COLLECT_GCC_OPTIONS='-I' '/home/yuanfang/jdk14/build/linux-x86_64-server-release/hotspot/variant-server/gensrc/adfiles' '-I' '......'-g' '-fpch-preprocess' '-v' '-dD' '-E' '-D' '___CIDR_DEFINITIONS_END' '-mtune=generic' '-march=x86-64'
cc1obj -E -quiet -v #/tmp/cci3XM6r -imultiarch x86_64-linux-gnu -D ___CIDR_DEFINITIONS_END /tmp/compiler-file5929385022787926768 -mtune=generic -march=x86-64 -fpch-preprocess -g -fworking-directory -fasynchronous-unwind-tables -fstack-protector-strong -Wformat -Wformat-security -fstack-clash-protection -fcf-protection -dD
cc: fatal error: cannot execute 'cc1obj': execvp: No such file or directory
compilation terminated.
[Failed to reload]
step: check out this projecdt :https://github.com/ojdkbuild/ojdkbuild.git
step: put the source code of OpenJDK14 in ojdkbuild/src/
step: copy reousrces directory and CMakelists.txt to OpenJDK14
step： recomile openjdk and import project to CLin
I am attempting to setup Travis CI so that it runs instrumentation tests on an emulator running API 25. Travis runs the tests to completion on API 4/10/19, but fails to startup the emulator for API 25, with the following message:
$ echo no | android create avd --force -n test -t $ANDROID_TARGET --abi $ANDROID_ABI
Valid ABIs: no ABIs.
Error: Invalid --abi armeabi-v7a for the selected target.
The output of android list targets shows that the API 19 emulator has a Tag/ABI, whereas the API 25 emulator does not:
id: 7 or "android-19"
Name: Android 4.4.2
Type: Platform
API level: 19
Revision: 4
Skins: HVGA, QVGA, WQVGA400, WQVGA432, WSVGA, WVGA800 (default), WVGA854, WXGA720, WXGA800, WXGA800-7in
Tag/ABIs : default/armeabi-v7a
id: 11 or "android-25"
Name: Android 7.1.1
Type: Platform
API level: 25
Revision: 3
Skins: HVGA, QVGA, WQVGA400, WQVGA432, WSVGA, WVGA800 (default), WVGA854, WXGA720, WXGA800, WXGA800-7in
Tag/ABIs : no ABIs.
How can I fix this so that the API 25 emulator launches and runs the tests?
The full .travis.yml file is as follows:
language: android
android:
components:
- tools
- platform-tools
- tools # appears twice as per Travis docs
- build-tools-23.0.1
- build-tools-25.0.2
- android-4
- android-10
- android-19
- android-23
- android-25
- extra-android-m2repository
- sys-img-armeabi-v7a-android-25
env:
matrix:
- ANDROID_TARGET=android-4 ANDROID_ABI=armeabi
- ANDROID_TARGET=android-10 ANDROID_ABI=armeabi
- ANDROID_TARGET=android-19 ANDROID_ABI=armeabi-v7a
- ANDROID_TARGET=android-25 ANDROID_ABI=armeabi-v7a
before_script:
# Create and start emulator
- android list targets
- jdk_switcher use oraclejdk8
- echo no | android create avd --force -n test -t $ANDROID_TARGET --abi $ANDROID_ABI
- emulator -avd test -no-skin -no-audio -no-window &
- adb wait-for-device
- while [[ `adb shell pm path android` == 'Error'* ]]; do sleep 2; done
- adb shell input keyevent 82 &
script: ./gradlew --info connectedAndroidTest
sudo: false
Note: emulator version 26.1.4+ versions can introduce or fix bugs. If a reproducible ci build is required, the best option is to download the desired sdk-tools version no via android components.
Note: emulator version 26.1.3 is here including interesting fixes. I'm busy to revise my answer now:
Related issue using emulator 26.1.2 here. Relaunch last build like me to use 26.1.3 and pray, green
I am attempting to setup Travis CI so that it runs instrumentation
tests on an emulator running API 25. Travis runs the tests to
completion on API 4/10/19, but fails to startup the emulator for API
25...
How can I fix this so that the API 25 emulator launches and runs the
tests?
Update: I confirmed that ARM ABIs for APIs 23, 25 and 26 are also missing in Android Studio, so:
It's not a Travis-ci issue. I reproduced it locally as shown in the next image.
You can use ARM ABI for API-24. I forked your project and it worked in build #6 (next image).
You can use Google APIs image for API-25 as I explain below using the new sdkmanager: 1
You can use Google APIs image for API-25 using legacy tools if you know the img name: 1,2,3
We need to ask to Google or someone else why the ARM ABIs for these APIs are missing.
Update: A related bug was already reported to Google and discussed here, please, star it:
For some reason it's marked as being only available on windows. I'm
not sure if this is a problem with ADRT itself or the configuration
for this package, but from a quick look it seems like it's probably a
problem with adrt itself.
I'm not sure if the api-25 issue is related but I expect that it will work combining several solutions:
Use system images with google_apis to simplify the build matrix and avoid missing ABIs.
Use the new sdkmanager and avdmanager required by the recent Sdk Tools.
Use the new Trusty images including these tools to simplify the Travis-ci configuration or alternatively use dist: precise as explained here.
See the links below to working samples using a build matrix that includes Api-25 emulator.
I updated the api 23 system image one time on a macbook via Android Studio and appeared a message about the new version was not supported by my computer, so I use the google_apis one.
I offer a possible solution to the issue commented by Ashish Pathak (I'm also ardock) here.
I need to update my response to support the new Travis-ci images that comes with new SDK tools.
The new pre-installed Sdk Tools will simplify the solution and reduce the YAML file in size.
As announced in this blog post by Carmen Andoh and Dominic Jodoin on 19 Jun 2017:
Updates are coming to all Ubuntu Trusty 14.04 images
This new generation of Trusty images is already available in
production and can be used by adding group: edge in your .travis.yml
file:
sudo: required
dist: trusty
group: edge # Add this
And as explained here:
As a bit more information, this new Android image comes with:
Android SDK 25.2.3 build-tools-25.0.2
The new sdkmanager tool - a command line tool that allows you to view,
install, update, and uninstall packages for the Android SDK. Replaces
the previous android tool, see
https://developer.android.com/studio/tools/help/android.html
Also, the new Android image should be retro-compatible. See the full
list of Android SDK components that can be specified in the
.travis.yml file, including build-tools-26.0.0-preview.
But they are not using the new avdmanager, I'll try to fix it this weekend.
Alternatively, you can use the previous images by adding dist: precise.
The lines below are part of my answer on a related question: prior to the images update:
Full working sample using constraint-layout codelab repository for Android API level 22 to 25:
Two more samples using sdkmanager and avdmanager:
Android Maps Utils library - Google
Dexter library - Karumi
References
Official documentation related to Auto-download missing packages with Gradle
The new Emulator options are explained in Start the Emulator from the Command Line
avdmanager explained here replaces android avd since SDK tools version 25.3.0
sdkmanager explained here also enhanced to view and accept all licenses from the command line
I'm trying to build the following hadoop version on development computer with Windows 10 Home Edition
hadoop-2.7.3-src
Here are the details about my local development environment:
-Windows 10 Home Edition
-Intel Core i5-6200U CPU #2.30GHz
-RAM 16 GB
-64-bit Operating System, x64-based processor
-Microsoft Visual Studio Community 2015 Version 14.0.25431.01 Update 3
-Also added MSBUILD location as C:\Program Files (x86)\MSBuild\14.0\Bin\amd64 to Windows System Environment Variable Path
-.NET Framework 4.6.01586
-cmake version 3.7.2
-CYGWIN_NT-10.0 LTPBCV82DUG 2.7.0(0.306/5/3) 2017-02-12 13:18 x86_64 Cygwin
-java version "1.8.0_121"
-Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
-Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
-Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T11:41:47-05:00)
-Google Protocol Buffers protoc --version libprotoc 2.5.0
Also, I've created and system environment variable called Platform and set it to x64
I opened up Developer Command Prompt for Visual Studio 2015 (VS2015)
c:\hadoop\hadoop-2.7.3-src> mvn package -Pdist,native-win -DskipTests -Dtar -X
Unfortunately, I'm getting the following error:
[C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
ZlibDecompressor.c
c:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
Done Building Project "C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj" (default targets) -- FAILED.
Done Building Project "C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.sln" (default targets) -- FAILED.
Build FAILED.
"C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.sln" (default target) (1) ->
"C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj" (default target) (2) ->
(ClCompile target) ->
c:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
c:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hadoop\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj]
0 Warning(s)
2 Error(s)
Time Elapsed 00:00:02.49
The aforementioned error has to do with zlib tool.
After researching online, someone said that the following Visual Studio solution file needs to be built successfully in Visual Studio:
....\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\native\native.sln
Using Visual Studio 2015 in Administrator mode, I opened up the native.sln file, and immediately saw an error:
enter image description here
Could someone please tell me what steps I have to take to resolve said error?
So there were quite a few steps I had to take in order to resolve the problems.
Within the ....\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\winutils directory, I opened up the following solution in Visual Studio 2015:
winutils.sln
Within .....\hadoop-2.7.3-src\hadoop-common-project\hadoop-common\src\main\winutils\libwinutils.c , I commented out the following line of code, and made a modified copy of it as shown below:
//const WCHAR* wsceConfigRelativePath = WIDEN_STRING(STRINGIFY(WSCE_CONFIG_DIR)) L"\\" WIDEN_STRING(STRINGIFY(WSCE_CONFIG_FILE));
const WCHAR* wsceConfigRelativePath = WIDEN_STRING("../etc/hadoop") L"\\" WIDEN_STRING("wsce-site.xml");
Also, In the winutils solution's property window, I had to set the platform value to x64 as the screenshot below shows:
Next, I opened Dos command prompt, and checked the exact version of my Windows OS:
ver
Microsoft Windows [Version 10.0.14393]
Also, I opened up the property window of the libwinutils project, and ensured that properties that are marked in the following snapshot had the proper values:
Also, I took the same steps for the properties of the winutils project:
(Sorry, stackoverflow would not allow me to place another picture snapshot, but all you basically have to do is make sure the the winutils project's property are set properly )
I downloaded zlib version 1.2.11 source code. Using Developer Command Prompt for VS2015 ( Visual Studio 2015 ) I built zlib from zlib version 1.2.11 source code using cmake
c:\zlib\zlib-1.2.11>cmake -G "Visual Studio 14 2015" -A x64 c:\zlib\zlib-1.2.11\
-- The C compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of off64_t
-- Check size of off64_t - failed
-- Looking for fseeko
-- Looking for fseeko - not found
-- Looking for unistd.h
-- Looking for unistd.h - not found
-- Configuring done
-- Generating done
-- Build files have been written to: C:/zlib/zlib-1.2.11
Finally, run the build with cmake
c:\zlib\zlib-1.2.11>cmake --build .
In Windows System Variables I have the following variable defined:
ZLIB_HOME is set to C:\zlib\zlib-1.2.11
I have a native library for my Android app that uses libjpeg-turbo. I am compiling the library and libjpeg-turbo using NDK. It all seems to be compiling and installing fine (I see all .so files on the phone in /data/data/com.company.app/lib ) and I am calling
System.loadLibrary("jpeg");
System.loadLibrary("zmq");
System.loadLibrary("MySuperLib");
System.loadLibrary("jnilibwrapper");
E/AndroidRuntime(8186): java.lang.UnsatisfiedLinkError: dlopen failed:
could not load library "libMySuperLib.so" needed by "libjnilibrapper.so";
caused by could not load library "libjpeg.so.62" needed by
"libMySuperLib.so"; caused by library "libjpeg.so.62" not found
I found this workaround, making a symlink of the lib like so gets me passed this error
root#klteatt:/data/data/com.company.app/lib # ln -s libjpeg.so libjpeg.so.62
but of course that is not a solution.
Where could we be going wrong? Why is it wanting libjpeg.so.62 specifically?
The issue is caused by MySuperLib being linked with libjpeg-turbo's versioned soname at compile time. Android's JNI runtime does not currently support versioned dynamic libraries, which is why it cannot "find" libjpeg.so, since it only expects an unversioned soname.
One possible way to compile libjpeg-turbo without versioning information is to modify libjpeg-turbo's Makefile.am *LDFLAGS variable, which is passed to libtool. By changing the *LDFLAGS variable to contain libtool's -avoid-version flag, libtool will try to avoid creating libraries with the version information embedded. Creating unversioned shared libraries is usually a terrible idea, but since you can't actually "install" shared libraries on Android at this point, it is fairly safe since your MySuperLib library is the only one that will ever be using the unversioned libjpeg.so.
Here is an example of a patch to libjpeg-turbo's Makefile.am that will produce an unversioned libjpeg.so:
--- Makefile.am 2014-08-28 09:57:37.000000000 -0700
+++ Makefile.am.patched 2014-08-28 09:57:20.000000000 -0700
## -1,5 +1,5 ##
lib_LTLIBRARIES = libjpeg.la
-libjpeg_la_LDFLAGS = -version-info ${LIBTOOL_CURRENT}:${SO_MINOR_VERSION}:${SO_AGE} -no-undefined
+libjpeg_la_LDFLAGS = -avoid-version -no-undefined
include_HEADERS = jerror.h jmorecfg.h jpeglib.h
if WITH_TURBOJPEG
Here is a link to libtool's link-mode manual, for more options and information: http://www.gnu.org/software/libtool/manual/html_node/Link-mode.html