Fortify Source Analyzer and Apache Lenya - java

I am trying to use Fortify Source Code Analyzer for a research project at my school to test the security for open source Java web applications. I am currently working on Apache Lenya. I am working with the last stable release (Lenya v2.0.2).
Inside the root directory there is a file named build.sh. This file is called to build Lenya using the version of Ant that ships with the release (in the tools/bin folder). I can build Lenya just fine when I run ./build.sh. So, it would be assumed that running the following command in Fortify would work :
sourceanalyzer -b lenya -Xmx1200M touchless ./build.sh
However, when I try and run:
sourceanayzer -b lenya -Xmx1200M -scan -f lenya.fpr
I get:
build id Lenya not found.
I looked at the buid.sh file and noticed that it was just resetting the current ant home, classpath, and ant options variables, running the ant build command, and resetting the values back to their defaults. So I reset all of the variables manually (without the script) instead of running the script and ran:
sourceanalyzer -b lenya -Xmx1200M touchless tools/bin/ant -logger org.apache.tools.ant.NoBannerLogger
Then I ran :
sourceanalyzer -b lenya -Xmx1200M -scan -f lenya.fpr
but I got the same error. I'm not sure if this is because I am doing something wrong or if it is something that Fortify is not doing correctly. Any insight will be great.

I'm not sure whether you have access to the Fortify documentation, but that will definitely help. You should refer to the SCA User's Guide to understand how to use the sourceanalyzer executable.
To cut things short, there are two ways of getting the FPR file:
(The long way) Write a script to translate and analyse the source code by providing the path of the source code and the classpath to the sourceanalyzer executable.
(The short way) Use the SCACompiler instead of javac as the compiler. You'll need to modify the build script for this.
I prefer the former due to its customizability when handling large code bases.
PS: Which version of Fortify is this?

Don't use the touchless command, that is for C/C++ Integration. Since Lenya is written in Java, you're better off with other commands. Try this for your first translation step (run from your base lenya dir):
sourceanalyzer -b lenya -Xmx1200M -source 1.5 -cp "**/*.jar" "**/*"

The command you used actually could work with Java builds, except that it has some limitations. Using the touchless build wrapper creates a number of compiler wrappers, e.g. for javac, and puts the wrappers at the front of the PATH environment variable.
If your build.sh script contains fully qualified references to javac, for example /usr/java/bin/javac, then the touchless build integration will not work.
user233276's instructions are the most broadly useful. If you want to experiment with Fortify SCA build integration, I would suggest the technique would be to modify build.sh (see http://svn.apache.org/viewvc/lenya/trunk/build.sh?view=markup&pathrev=400414) and change line 43 from:
"$ANT_HOME/bin/ant" -logger org.apache.tools.ant.NoBannerLogger -emacs $#
to:
sourceanalyzer -b Lenya "$ANT_HOME/bin/ant" -logger org.apache.tools.ant.NoBannerLogger -emacs $#
See the Fortify SCA User Guide for the three types of ant integration:
Override the build.compiler property:
ant -lib sourceanalyzer.jar {Fortify ant options} {ant options}
Shortcut to the above:
sourceanalyzer -b {Fortify options} ant {ant options}
Or, if you make a custom build.xml as shown in the appendix:
ant -lib sourceanalyzer.jar {ant options}

Related

How can I see the javac command IntelliJ IDEA uses to compile my code?

When I write a Java code in IntelliJ IDEA and runs it, IntelliJ compiles the Java file, a class file is extracted and then the class file is run.
How can I see the javac command line that IntelliJ runs. I ask it so I can see whether IntelliJ adds some flags to the javac command.
IntelliJ IDEA doesn't run javac, therefore you can't see the command line.
Compiler API is used directly from Java code. If you enable debug logging for build.log file, you may find some more details how the modified and dependent files are compiled and what options are used.
Sample debug log line:
[ 41011] DEBUG - s.incremental.java.JavaBuilder - Compiling chunk [stopme] with options: "-g -deprecation -proceedOnError -encoding UTF-8 -source 1.8 -target 1.8 -proc:none"
IDEA is not running java binary, so there is no way to see the commands. Instead, IDEA uses Java compiler API directly.
If you want a raw representation of what is done to build the project,
you can use Build | Generate Ant build. Examine the build file or run
it from the command line via Ant to see what happens and what
options/commands are invoked.
Look in /Users/itsabhiaryan/Library/Logs/IdeaIC2017.1/build-log
When you build inside IDEA, it writes a log to build.log in this directory.
IDEA is not running java binary, so there is no way to see the commands.

Android building: JarJar can't find or load main class

I'm trying to build AOSP from source (With a few modifications) but my build stops with
[ 0% 1/35196] JarJar: out/target/common/obj/JAVA_LIBRARIES/conscrypt_intermediates/classes-jarjar.jar
FAILED: /bin/bash -c "java -jar out/host/linux-x86/framework/jarjar.jar process external/conscrypt/jarjar-rules.txt out/target/common/obj/JAVA_LIBRARIES/conscrypt_intermediates/classes-full-debug.jar out/target/common/obj/JAVA_LIBRARIES/conscrypt_intermediates/classes-jarjar.jar"
Error: Could not find or load main class com.tonicsystems.jarjar.Main
ninja: build stopped: subcommand failed.
make: *** [build/core/ninja.mk:149: ninja_wrapper] Error 1
and i can't get to fix it. My source is hosted here and the path of external/jarjar is the stock one on googlesource.
Can anyone help?
I had the same problem until I realized that it's due to the wrong path where com.tonicsystems.jarjar.Main is located. At ~/android/system, JarJar's com.tonicsystems.jarjar class is located in:
external/jarjar/src/main/com/tonicsystems
When it should be
external/jarjar/src/main/java/com/tonicsystems
Like it's source (from GitHub)
src/main/java/com/tonicsystems/jarjar
P.S: I had to create the external/jarjar/src/main/java directory and then move the source.
I've run into this with an AOSP fork (I think Qualcomm's tree) before - that was caused by the Makefile fragments that generate the file lists for the jar files not dealing properly with localized versions of the "sort" utility (causing removal of important class files in addition to the duplicates that were supposed to be removed).
I don't remember all the details or the proper fix, but the workaround that got me going initially was simply disabling localization while building.
rm -rf out
export LANG=C
export LC_ALL=C
export LC_COLLATE=C
. build/envsetup.sh
lunch whatever
make droidcore -j8
What fixed this for me was something completely unrelated to jarjar itself.
I was building AOSP 7.1.1 on Ubuntu 20.04. The default python command on Ubuntu 20.04 points to python3 [1], but AOSP 7.1.1 builds using python (which is really python 2.7.5).
I updated my system with sudo apt install python which linked the python command to python2 correctly instead of python3. After this, I built successfully.
AOSP is shipped with python under the prebuilts/python directory and I'm still confused as to why the build system doesn't point to that python version.
Edit
I had more trouble with adjusting my system to use the correct version of python, ultimately I had to add python and python3 as alternatives.
$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python2 1
$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 2
$ sudo update-alternatives --config python
Select python2

Javapackager tool from command-line on OSX?

javapackager and javafxpackager don't seem to be recognised on the command line for me. They don't show up in the terminal, even after I installed the latest 1.8 SDK. (Even 'echo $JAVA_HOME' seems to be drawing a blank, though java -version seems to work fine.)
If I look under /Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/bin/ I can see the javapackager and javafxpackager tools are present, but if I follow /usr/libexec/java_home back to it's origin in /System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/ , there's no sign of them.
The only other discussions I could find on the subject were a blog post and mailing list from about a year and a half back:
https://devreboot.wordpress.com/2014/11/26/java-desktop-app-packaging-automation/
http://lists.apple.com/archives/java-dev/2015/Nov/msg00009.html
I realise OSX hasn't come with java 'by default' for some time, but that's kinda why I was hoping to release my application as a self-contained bundle with it's own VM. Would it be standard procedure to update my bash profile to point at the tool explicitly? Is there something screwy about my personal setup, or is there some gap in the tool support on OSX?
PS: I realise there's an older thread on this subject below:
What is the best way to deploy JavaFX application, create JAR and self-contained applications and native installers
However, that describes javapackager as a .jar file, and whatever I've got doesn't seem to be a .jar file. I'm legitimately confused about what I'm supposed to do with it.
EDIT: Thanks to everyone for the tips- I think I have enough info to go on for now.
You could try this gist for building, packaging and running a test install on Java client apps from the OS X command line. No guarantees it will work for you, it was just something I whipped up for personal development purposes a long time ago. But, the info in there may help in resolving packager tool locations from the command line and also in performing other packaging related functions.
The key part for locating (and using) the javapackager is:
# select java version
export JAVA_HOME=`/usr/libexec/java_home -v 1.8`
$JAVA_HOME/bin/java -version
...
# make an executable jar file
$JAVA_HOME/bin/javapackager -createjar -srcdir . -appclass start.HelloWorldSwing -srcfiles HelloWorldSwing.jar -outdir . -outfile HelloWorld.jar
# package the jar and java runtime as a native application with installer
$JAVA_HOME/bin/javapackager -deploy -srcdir . -srcfiles HelloWorld.jar -outdir . -outfile HelloWorld -appclass start.HelloWorldSwing -native -name HelloWorld
Note, the above is for packaging a Swing application. Packaging a JavaFX application will use slightly different command line options for the packager.
Doing things this way from command line scripts is decidedly old school, usually maven or gradle is used.
My personal preference would be to just use Ant, but I guess that's only slightly less old-school?
Yes, not that there is anything wrong with that. Documentation on using Ant to package java client applications is provided by Oracle.

Ant unable to launch, giving a main class error

I'm running Elementary OS (Ubuntu 12 based), and I'm having issues running apache ant. It was working earlier before a restart, so I'm not sure what would've changed.
I've defined environment variables in /etc/environment as follows:
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:$ANT_HOME/bin"
JAVA_HOME="/usr/lib/jvm/java-1.7.0-openjdk-i386"
ANT_HOME="/opt/ant"
ANT_OPTS="-Xms256M -Xmx512M"
So my Java and Ant environment variables should be set. I'm trying to deploy with ant, with 'ant clean deploy', but I get an error in my terminal:
Error: Could not find or load main class org.apache.tools.ant.launch.Launcher
I've tried "source /etc/environment". Running 'echo $ANT_HOME' shows the correct path. I've tried moving ant to a different location and resetting the variables. Nothing. I'm kind of lost. Please help!
you should define $ANT_HOME before using it in your $PATH
JAVA_HOME="/usr/lib/jvm/java-1.7.0-openjdk-i386"
ANT_HOME="/opt/ant"
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:$ANT_HOME/bin"
double check that you have ant-launcher.jar under $ANT_HOME/lib
As others have stated, it looks like ANT_HOME isn't being set before you declare it in your path.
I take a slightly different approach to these things. Instead of constantly updating my PATH, I usually set my path to something fairly simple:
PATH="/usr/local/bin:/bin:/usr/bin/:$HOME/bin"
Then, I make hard links to various binaries into my /usr/local/bin directory.
$ cd $ANT_HOME/bin # All the Ant binaries
> for binary in *
> do
> sudo ln -s $PWD/$binary /usr/local/bin
> done
Now, I don't have to add $ANT_HOME/bin to my $PATH. Even better, if I include /usr/local/bin before /usr/bin and /bin, I am picking the binary in /usr/local/bin first. This way, I can ensure I run the version I installed over the default. For example, my machine comes with Ant 1.7 in /usr/bin/ant but I want to use Ant 1.9.1. I can install Ant 1.9.1, and that will be my default version.
I had same error when install ant with npm install. When I tried install from official repository throw pacman -S apache-ant(apt-get install apache-ant -- for Debian/Ubuntu) it's start working proper for me.
I'm experiencing this bug with jdk 1.8. But, I came across this RHEL 6.5 bug for OpenJDK 1.8 which may be related:
https://bugzilla.redhat.com/show_bug.cgi?id=1149605
Essentially, it's fixed in jpackage-utils-1.7.5-3.13 which is included with RHEL 5.7.
The problem, as Javier Ramirez said in the bug comments:
Your script /usr/share/java-utils/java-functions has problems with
"openjdk version" because it expects "java version" as Java 7 does.
$ mkdir /usr/share/java-1.8.0
$ mkdir /usr/lib/java-1.8.0
$ diff /usr/share/java-utils/java-functions.orig /usr/share/java-utils/java-functions
149,150c149,150
< -e '/java \(full \)*version "/s/'$re'/<<<\1>>>/' \
< -e '/java \(full \)*version "/s/.*<<<\([^>]\{1,\}\)>>>.*/\1/p')
---
> -e '/[java|openjdk] \(full \)*version "/s/'$re'/<<<\1>>>/' \
> -e '/[java|openjdk] \(full \)*version "/s/.*<<<\([^>]\{1,\}\)>>>.*/\1/p')
------
I am using RHEL which comes with ant (in /usr/bin/ant) and the ant libraries in
/usr/share/ant. As suggest above, ant -version gave the "could not find" error.
I installed my own version of ant (in /home/Ant since I have a lot of space
in /home) and put ANT_HOME in my .bash_profile and $ANT_HOME/bin in my PATH
and $ANT_HOME and $ANT_HOME/bin (for good measure) in my CLASSPATH. Then ant
worked (when used as myself, not as root).

Install Protocol Buffers on Windows

I am unable to find clear instructions to install Google Protocol Buffers (including compiler) on Windows x64 platform.
I went through the instructions README file for compiler and source:
For Compiler: To install, simply place this binary somewhere in your PATH
I added system variable to Path:
PROTOC 'C:\dev_tools\protoc-2.4.1-win32'
I am stuck on installing Protocol Buffers source using Cygwin. I tried following
Unix instructions provided in the readme file:
To build and install the C++ Protocol Buffer runtime and the Protocol
Buffer compiler (protoc) execute the following:
$ ./configure
$ make
$ make check
$ make install
If make check fails, you can still install, but it is likely that
some features of this library will not work correctly on your system.
Proceed at your own risk.
make install may require superuser privileges.
For advanced usage information on configure and make, see INSTALL.txt.
** Hint on install location **
By default, the package will be installed to /usr/local. However, on many platforms, /usr/local/lib is not part of LD_LIBRARY_PATH. You can add it, but it may be easier to just install to /usr> instead. To do this, invoke configure as follows:
./configure --prefix=/usr
I get
-bash: ./configure: No such file or directory'
Can some one provide clear and detailed steps to make this work?
UPDATE
I switched to using MSYS/MINGW32 instead and
I followed instructions given in this link. Now I am stuck with following:
When I run the './configure' command I get following error:
checking how to run the C++ preprocessor... /lib/cpp
configure: error: C++ preprocessor "/lib/cpp" fails sanity check'
As a result, none of the make, make install commands work. For eg:
make: No targets specified and no makefile found. Stop.
If you just want to compile ProtoBuf definitions, you can download precompiled binaries of protoc for all platforms right on the ProtoBuf GitHub releases page.
They had precompiled binaries at least since 2015, but it's easy to overlook them in between the many downloads.
For installing proto buff in windows
Go to https://github.com/protocolbuffers/protobuf/releases
Scroll down and download the zip for windows compatible
Binary File
Once download the zip, extract it to C:/
Just copy the path, C:\protoc-3.17.3-win64\bin and set this as path to an environment variables
path setup
Hope, this helps!
There is a whole documentation file for compiling protobuf on Windows :
https://github.com/google/protobuf/blob/master/src/README.md#c-installation---windows
https://github.com/google/protobuf/blob/master/cmake/README.md
You'll need 7-zip, Cmake and Visual Studio.
Anyway, one of the unexpected side-effects of using a Continuous Integration tool (like Travis or Appveyor) is that there is always a up-to-date and working build script available. I happen to always look at appveyor.yml and travis_config.yml files whenever they exists.
>>> git clone -b v3.1.0 https://github.com/google/protobuf.git
>>> cd protobuf
>>> curl -L -o release-1.7.0.zip https://github.com/google/googlemock/archive/release-1.7.0.zip
>>> 7z x release-1.7.0.zip
>>> del /Q release-1.7.0.zip
>>> rename googlemock-release-1.7.0 gmock
>>> curl -L -o release-1.7.0.zip "https://github.com/google/googletest/archive/release-1.7.0.zip"
>>> 7z x release-1.7.0.zip
>>> del /Q release-1.7.0.zip
>>> rename googletest-release-1.7.0 gtest
>>> move gtest gmock
>>> set generator=Visual Studio 12 Win64
>>> set vcplatform=x64
>>> mkdir build_msvc
>>> cd build_msvc
>>> cmake -G "%generator%" -Dprotobuf_BUILD_SHARED_LIBS=%BUILD_DLL% -Dprotobuf_UNICODE=%UNICODE% ../cmake
>>> msbuild protobuf.sln /p:Platform=%vcplatform% || goto error
You'll need curl (Git Bash has it) as well as resolving paths for the 7z.exe and Msbuild.exe executables.
I successfully managed to build the protobuf compiler on a Windows 10 x64 machine with Visual Studio 2015.
Use chocolatey
choco install protoc
Download protoc-2.5.0-win32.zip from
https://github.com/protocolbuffers/protobuf/releases/tag/v2.5.0
Then to install, simply place this binary somewhere in your PATH
I was build protobuf v2.4.1 on Windows 10 as follows:
git clone https://github.com/protocolbuffers/protobuf.git;
cd protobuf;
git checkout v2.4.1;
cd vsprojects
open protobuf.sln in Visual Studio 2019
Press build solution and take many errors: min undefined and max undefined
Add in file protobuf/stubs/common.h next code:
#if defined(_WIN32) && !defined(min)
#define min(a,b) __min(a,b)
#define max(a,b) __max(a,b)
#endif
Press build solution and take error: fatal error C1189: “#error: hash_map is deprecated and will be REMOVED….”
Add compile definition -D_SILENCE_STDEXT_HASH_DEPRECATION_WARNINGS solved problem
Next I had error “error C2664: "std::pair std::make_pair(_Ty1 &&,_Ty2 &&) noexcept”…”
In file src\google\protobuf\compiler\command_line_interface.cc I modified string “proto_path_.push_back(make_pair< string, string >(virtual_path, disk_path));” to “proto_path_.push_back(make_pair(virtual_path, disk_path));”
Press build solution. All build succeed.(Tests projects not build without test framework)
INSTALL:
Run extract_includes.bat to copy all the public headers into a separate "include" directory (under the top-level package directory).
Copy the contents of the include directory to wherever you want to put headers
Copy protoc.exe wherever you put build tools
copy libprotobuf.lib, libprotobuf-lite.lib, and libprotoc.lib wherever you put libraries.
I installed it with chocolatey and it worked perfectly.
choco --install -y protoc
I'd recommend using vcpkg tool on windows. Here is step by step manual.
Regarding protobuf, firstly check what options you have (in cmd):
vcpkg search protobuf
Next install the required package:
vcpkg install protobuf:x64-windows-static
Notice x64-windows-static after the colon - this is the triplet. Check vcpkg help triplet for all of them.
Then go to your_path\vcpkg-master\packages\protobuf_x64-windows-static\
Now you can set your environment variables.
Now protobuf is a NuGet package in Visual Studio. Just go get that.
just easy ref
choco install protoc --pre
choco install protoc --pre worked for, make sure your in an elevated state.
this is the output when I ran protoc from gitbash
$ protoc
Usage: C:\ProgramData\chocolatey\lib\protoc\tools\bin\protoc.exe [OPTION] PROTO_FILES
Parse PROTO_FILES and generate output based on the options given:
-IPATH, --proto_path=PATH Specify the directory in which to search for
adding it to my $path environment variables didn't work from me as expected.

Categories

Resources