Develop Reference

Jmeter linux shell cannot start server

Situation:
I have installed Jasper Reports Library (V6.5.1) on my local Linux server which generates PDF reports (Data is dumped in a temp Oracle DB table for the reporting engine).
It then serves this PDF back to the website from which I kick off the process.
Goal:
Install Jmeter to analyse performance / possible bottlenecks of "Jasper Reports Library" (aka Report Generation) on my local linux server (I cannot access this server via GUI, only shell).
I understand I have to connect my local Windows 10 machine (running same Jmeter 4.0) with this local server. On the server I have to start Jmeter 4.0 Server (via jmeter-server command) however I get an error and am stuck (have not found anything online or even people with the same goal unfortunately...)
Steps I have taken:
Download latest (4.0) bin from here
Extracted on local linux server in /opt/dlins/apache-jmeter-4.0bin
Trying to start server with /usr/lib/jvm/jdk1.8.0_102/bin/java jmeter-server (the default java version is 6 so through this I can run this app with java 8) - Instructions found here
-> Getting error: "Error: Could not find or load main class jmeter-server"
Any help regarding above or even any other tool you may use are appreciated (Maybe there is a preferable way to test performance for the above scenario)

There are 2 aspects related to your issue and screenshot:
1) Using java 8 instead of 6 - This can be done in several ways, depending on your needs and restrictions, such as the need to have Java 6 globally available for other applications and using 8 just to run JMeter, or just replacing 6 with 8 entirely. For the sake of brevity, I'll assume the first scenario, but there's documentation available for both and Dmitri T has partially explained it already.
Anyway, the same JMeter doc link you used, describes (just scroll down a few times) how to create a setenv.sh script in the bin directory and configure JAVA_HOME or JRE_HOME depending on your needs.
To set those variables permanently, you can place them in a file called setenv.sh in the bin directory. This file will be sourced when running JMeter by calling the jmeter script.
You seem to be wanting a JDK, so create the script and add inside JAVA_HOME=/usr/lib/jvm/jdk1.8.0_102, save and exit.
2) Running JMeter - To clarify a minor confusion, java MyCompiledClass instructs java to load and execute the "program" defined in MyCompiledClass, which is not what you want to do, because jmeter-server is a shell script. If you open it, you'll see that it calls the jmeter shell script which will do some configuration, end eventually call (in short) java -jar ApacheJMeter.jar with some arguments and options.
So, to run JMeter make sure your scripts are executable with chmod, and simply run from command line ./jmeter-server. From the same link:
Un*x script files; should work on most Linux/Unix systems:
jmeter - run JMeter (in GUI mode by default). Defines some JVM settings which may not work for all JVMs.
jmeter-server - start JMeter in server mode (calls jmeter script with appropriate parameters)
jmeter.sh - very basic JMeter script (You may need to adapt JVM options like memory settings).
mirror-server.sh - runs the JMeter Mirror Server in non-GUI mode
shutdown.sh - Run the Shutdown client to stop a non-GUI instance gracefully
stoptest.sh - Run the Shutdown client to stop a non-GUI instance abruptly

Amend your PATH environment variable so Java 8 bin would be before Java 6 bin like:
PATH=/usr/lib/jvm/jdk1.8.0_102/bin:$PATH && export PATH
Once done you should be able to just launch the jmeter-server script like
pushd /opt/dlins/apache-jmeter-4.0bin/bin && ./jmeter-server
More information:
Remote Testing
JMeter Distributed Testing Step-by-step
How to Get Started With JMeter: Part 1 - Installation & Test Plans

To debug map reduce jobs in eclipse

I want to debug Map-reduce jobs (pig,hive) using eclipse. That is, to set break points in the hadoop source java file and to inspect the elements while running map-reduce jobs. To do this, I started all the services using eclipse and I can debug some class files. But I cant create an entire debug environment. Can anyone tell me how?

I do not know of an eclipse tool that can do what you are looking for. If you are looking for a possible solution the following will work in java.
import java.util.logging.Logger;
For debugging java map reduce files you can use the java logger for each class( driver, mapper, reducer ).
Logger log = Logger.getLogger(MyClass.class.getName());
To inspect elements/variables just use :
log.info( "varOne: " + varOne );
These log lines with be printed in the administration page for your job.

The basic thing to remember here is that debugging a Hadoop MR job is going to be similar to any remotely debugged application in Eclipse.
As you would know, Hadoop can be run in the local environment in 3 different modes :
Local Mode
Pseudo Distributed Mode
Fully Distributed Mode (Cluster)
Typically you will be running your local hadoop setup in Pseudo Distributed Mode to leverage HDFS and Map Reduce(MR). However you cannot debug MR programs in this mode as each Map/Reduce task will be running in a separate JVM process so you need to switch back to Local mode where you can run your MR programs in a single JVM process.
Here are the quick and simple steps to debug this in your local environment:
Run hadoop in local mode for debugging so mapper and reducer tasks run in a single JVM instead of separate JVMs. Below steps help you do it.
Configure HADOOP_OPTS to enable debugging so when you run your Hadoop job, it will be waiting for the debugger to connect. Below is the command to debug the same at port 8080.
(export HADOOP_OPTS=”-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8008“)
Configure fs.default.name value in core-site.xml to file:/// from hdfs://. You won’t be using hdfs in local mode.
Configure mapred.job.tracker value in mapred-site.xml to local. This will instruct Hadoop to run MR tasks in a single JVM.
Create debug configuration for Eclipse and set the port to 8008 – typical stuff. For that go to the debugger configurations and create a new Remote Java Application type of configuration and set the port as 8080 in the settings.
Run your hadoop job (it will be waiting for the debugger to connect) and then launch Eclipse in debug mode with the above configuration. Do make sure to put a break-point first.
Thats it.

I created an eclipse project to debug generic mapreduce program, for example WordCount.java, running standalone hadoop in Eclipse. But I did not try hive/pig specific mapreduce jobs yet. The project locates at https://github.com/drachenrio/hadoopmr, it can be download using
git clone https://github.com/drachenrio/hadoopmr
This project was created with Ubuntu 16.04.2, Eclipse Neon.3 Release (4.6.3RC2), jdk1.8.0_121, hadoop-2.7.3 environments.
Quick setup:
1) Once project imported into Eclipse, open .classpath,
replace /j01/srv/hadoop-2.7.3 with your hadoop installation home path
2) mkdir -p /home/hadroop/input
copy src/main/resources/input.txt to /home/hadoop/input/
It is ready to run/debug WordCount.java mapreduce job.
Read README.md for more details.
If you prefer to manually create the project, see my another answer in stackoverflow

How do I add a jar to jconsole classpath in windows?

I'm trying to invoke a JMX MBean via Jconsole, but the method that I'm calling receive an object that needs to be on jconsole classpath in order to work.
I've tried this and the jconsole does not open (and no error is shown):
jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;path_to_newjar_\newjar.jar

Take jconsole.jar out of the -J classpath. It's already set in the jconsole launcher.
===== Update =====
Hmmm.... I take back my suggestion. I have a windows batch file I use [which works] which adds a JAR to the classpath. The intent is to add the JMXMP JMX client into jconsole and then launch to the argument specified JMX Service URL. It looks like this:
#echo off
start /B jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;MY-JAR-PATH\jmx-optional-1.0-b02-SNAPSHOT.jar service:jmx:jmxmp://localhost:%1
If a command like that does not work (i.e. jconsole still will not launch, but does not error out either), then you need to figure out where it is stalling. 2 suggestions for this:
Launch another jconsole and attach to the stalled jconsole by PID, switch to the Threads tab and eyeball what's going on in the main thread.
Or, (since it looks like you're in windows) hit Ctrl-Break and hopefully it will print out a thread dump to the console and you can then eyeball the main thread going-ons there.
Post back if you get anything (or if you don't....)

If you want to have access to both local & remote processes, the above answers are still missing a step.
From the Java 8 oracle docs:
If the JMX agent uses a connector which is not included in the Java platform, you need to add the connector classes to the class path when you run the jconsole command, as follows.
$ jconsole -J-Djava.class.path=JAVA_HOME/lib/jconsole.jar:JAVA_HOME/lib/tools.jar:connector-path
In the command above, connector-path is the directory or the Java archive (Jar) file containing the connector classes that are not included in the JDK, that are to be used by JConsole.
In your case, then, the command would be:
$ jconsole -J-Djava.class.path=%JAVA_HOME%\lib\jconsole.jar;%JAVA_HOME%/lib/tools.jar:path_to_newjar_\newjar.jar
When you leave off the JAVA_HOME/lib/tools.jar, local processes are no longer available.

Solution in Windows is to use quotes on your classpath, for example:
jconsole -J-Djava.class.path="%JAVA_HOME%\lib\jconsole.jar;path_to_newjar_\newjar.jar"

eclipse: debug programmatically initiated process

I'm programmatically executing a java process via another java process in eclipse:
Process process = Runtime.getRuntime().exec(command, envp, dir);
Is there any way to tell eclipse to debug the child process?
Alternatively, I could solve this by chaining multiple launch configurations, ie launch process A, on completion launch process B - provided B could be launched in debug mode.

If you use the "Remote Debug" feature in Eclipse then you can point it at any JVM instance. You just need to make sure that each instance is told to use a unique JDWP port. This is how you'd do it from the command line:
java -Xdebug -Xrunjdwp:transport=dt_socket,address=8998,server=y
So, change the 'address' part to whatever port you want, then you can point Eclipse at that port. Here's some more information:
http://java.dzone.com/articles/how-debug-remote-java-applicat

Add a command line option to the child process as follows:
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=9999
(you can use a different port number than 9999, it's up to you, but best choose a number above 1024)
Then in Eclipse, create a Debug Configuration of type "Remote Java Application". Set the host as localhost and the port as 9999.
This developerWorks article has plenty of extra information.
EDIT: Incidentally for more flexibility in specifying the command line and environment of your child process, use the java.lang.ProcessBuilder class instead of Runtime.exec(...).

Remote debugging a Java application

I have a java application running on linux machine. I run the java application using the following:
java myapp -Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=4000, suspend=n
I have opened port 4000 for TCP on this Linux machine. I use eclipse from Windows XP machine and try to connect to this application. I have opened the port in windows also.
Both machines are on the LAN but I can't seem to connect the debugger to the Java application. What am I doing wrong?

Edit: I noticed that some people are cutting and pasting the invocation here. The answer I originally gave was relevant for the OP only. Here's a more modern invocation style (including using the more conventional port of 8000):
java -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=n <other arguments>
Original answer follows.
Try this:
java -Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=4000,suspend=n myapp
Two points here:
No spaces in the runjdwp option.
Options come before the class name. Any arguments you have after the class name are arguments to your program!

For JDK 1.3 or earlier :
-Xnoagent -Djava.compiler=NONE -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=6006
For JDK 1.4
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=6006
For newer JDK :
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=6006
Please change the port number based on your needs.
From java technotes
From 5.0 onwards the -agentlib:jdwp option is used to load and specify
options to the JDWP agent. For releases prior to 5.0, the -Xdebug and
-Xrunjdwp options are used (the 5.0 implementation also supports the -Xdebug and -Xrunjdwp options but the newer -agentlib:jdwp option is preferable as the JDWP agent in 5.0 uses the JVM TI interface to the
VM rather than the older JVMDI interface)
One more thing to note, from JVM Tool interface documentation:
JVM TI was introduced at JDK 5.0. JVM TI replaces the Java Virtual Machine Profiler Interface (JVMPI) and the Java Virtual Machine Debug Interface (JVMDI) which, as of JDK 6, are no longer provided.

Answer covering Java >= 9:
For Java 9+, the JVM option needs a slight change by prefixing the address with the IP address of the machine hosting the JVM, or just *:
-agentlib:jdwp=transport=dt_socket,server=y,address=*:8000,suspend=n
This is due to a change noted in https://www.oracle.com/java/technologies/javase/9-all-relnotes.html#JDK-8041435.
For Java < 9, the port number is enough to connect.

Steps:
Start your remote java application with debugging options as said in above post.
Configure Eclipse for remote debugging by specifying host and port.
Start remote debugging in Eclipse and wait for connection to succeed.
Setup breakpoint and debug.
If you want to debug from start of application use suspend=y , this will keep remote application suspended until you connect from eclipse.
See Step by Step guide on Java remote debugging for full details.

I'd like to emphasize that order of arguments is important.
java -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000 -jar app.jar command opens debugger port,
but java -jar app.jar -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000 command doesn't. It will pass everything after app.jar as command-line arguments.

This is how you should setup Eclipse Debugger for remote debugging:
Eclipse Settings:
1.Click the Run Button
2.Select the Debug Configurations
3.Select the “Remote Java Application”
4.New Configuration
Name : GatewayPortalProject
Project : GatewayPortal-portlet
Connection Type: Socket Attach
Connection Properties:
i) localhost ii) 8787
For JBoss:
1.Change the /path/toJboss/jboss-eap-6.1/bin/standalone.conf in your vm as follows:
Uncomment the following line by removing the #:
JAVA_OPTS="$JAVA_OPTS -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n"
For Tomcat :
In catalina.bat file :
Step 1:
CATALINA_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8000,server=y,suspend=n"
Step 2:
JPDA_OPTS="-agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n"
Step 3: Run Tomcat from command prompt like below:
catalina.sh jpda start
Then you need to set breakpoints in the Java classes you desire to debug.

for everybody that has the problem that if you really do remote debugging from 1 machine to the other then using :
-agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=n
is not enough because that binds now (at least on unix/osx machines) to localhost so you can only connect to it from localhost.
If you try to remote debug this then you will get connection refused for this. From i think Java 9 on you need to do:
-agentlib:jdwp=transport=dt_socket,server=y,address=*:8000,suspend=n
or give an ip that it needs to bind on for hat *