Develop Reference

Java remote debugging - how can I keep debugger listening?

I'm using IntelliJ IDEA to remote debug a Java CLI program with the debugger listening for connections.
This works fine for the first invocation, but the debugger stops listening after the CLI program disconnects. I want the debugger to keep listening since multiple CLI invocations will be made (in sequence, not in parallel) and only one of these will trigger the breakpoint I've set.
Here's my client debug config:
-agentlib:jdwp=transport=dt_socket,server=n,address=5005,suspend=y
Is it possible to keep the debugger listening?

Well since your CLI program terminates, debugger also stops. If you still want to continue debugger session for multiple runs of CLI program, then you can try as below,
Write a wrapper program from which you invoke your CLI program multiple times and debug the wrapper program instead of your CLI program.
Something like this,
public class Wrapper {
public static void main(String[] args) {
YourCLIProgram yp = new YourCLIProgram();
// First Invocation
String[] arg1 = { }; // Arguments required for your CLI program
yp.main(arg1);
// Second Invocation
String[] arg2 = { }; // Arguments required for your CLI program
yp.main(arg2);
// Third Invocation
String[] arg3 = { }; // Arguments required for your CLI program
yp.main(arg3);
// Fourth Invocation
String[] arg4 = { }; // Arguments required for your CLI program
yp.main(arg4);
}
}
I hope it works.

It depends also what you are trying to achieve.
If you want to check what parameters are passed to your CLI you can just log them to the file or save any information that you need in DB (or file as well).

In JPDA by specification transport service could support or not multiple connections.
For example, in Eclipse it doesn't. I suppose the same for IDEA.

When setting up your run configuration, did you select the "Listen" Debugger mode? The command line arguments you show look like the normal "Attach" settings, whereas the arguments for "Listen" look like this:
-agentlib:jdwp=transport=dt_socket,server=n,address=yourhost.yourdomain:5005, suspend=y,onthrow=<FQ exception class name>,onuncaught=<y/n> (Specifically, your arguments are missing the address for the application - your CLI program - to connect to IDEA at on start-up.)
I read a post that suggests the "onthrow" argument may not be necessary for general debugging, but I haven't tried it myself.

Try with suspend=n:
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005
On my local app (tomcat web app), even though I run on JDK8, I still use the older way of doing it and it works fine (another thing you could try):
-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005

How does Android determine which buffer to use for logcat messages?

By standard, the Android logcat system have 4 different ring buffers:
main
system
radio
events
# and alias & groups:
all -- all available logs
default -- main
crash -- n/a
However, on AOS 6+ there seem to be other buffers as well:
# logcat --help
...
-b <buffer> Request alternate ring buffer, 'main', 'system', 'radio',
'events', 'crash' or 'all'. Multiple -b parameters are
allowed and results are interleaved. The default is
-b main -b system -b crash.
...
and the android logcat.cpp source code, seem to imply there are still other ones, such as:
security
kernel
Normally in java apps, the way to put a message in main logcat is by using: Log.i(TAG, msg).
So the question is:
How does Android determine which buffer to use for the various logcat messages?
(Specific references to the AOS source code would be especially appreciated.)
Then a natural follow up question would be, how can you see or enable the other hidden buffers?

I don't like to answer my own questions, but I've found a few answers, and also some very good notes.
First, the various log related (Java) source files are located in:
platform_frameworks_base/core/java/android/util/ and in:
platform_frameworks_base/telephony/java/android/telephony/.
EventLog.java # EVENT Log: These diagnostic events are for system integrators, not application authors.
Log.java # MAIN Log: Where user app logcat goes from: Log.v() Log.d() Log.i() Log.w() and Log.e()
Slog.java # SYSTEM Log: Primarily for use by coding running within the system process.
Rlog.java # RADIO Log: All radio, wifi, bluetooth etc. related logs. Also scrubs personal info from appearing in logs.
with the related files:
EventLogTags.java # Deprecated! (Use EventLog)
LocalLog.java # log(), dump(), reverseDump(), ReadOnlyLocalLog()
LogPrinter.java # decides what buffer to print to: LOG_ID_<buffer_name>
LogWriter.java # decides priority and TAG
TimingLogger.java # A utility class to help log timings splits throughout a method call.
The log buffers are identified in Log.java by:
public static final int LOG_ID_MAIN = 0;
public static final int LOG_ID_RADIO = 1;
public static final int LOG_ID_EVENTS = 2;
public static final int LOG_ID_SYSTEM = 3;
public static final int LOG_ID_CRASH = 4;
In the OS, the logging is governed by the properties:
setprop log.tag.<YOUR_LOG_TAG> <LEVEL>
and in the file /data/local.prop by:
log.tag.<YOUR_LOG_TAG>=<LEVEL>
Furthermore, I found the interesting comments.
The MAIN log:
The order in terms of verbosity, from least to most is
ERROR, WARN, INFO, DEBUG, VERBOSE. Verbose should never be compiled
into an application except during development. Debug logs are compiled
in but stripped at runtime. Error, warning and info logs are always kept.
Tip: Don't forget that when you make a call like
Log.v(TAG, "index=" + i);
that when you're building the string to pass into Log.d, the compiler uses a
StringBuilder and at least three allocations occur: the StringBuilder
itself, the buffer, and the String object. Realistically, there is also
another buffer allocation and copy, and even more pressure on the gc.
That means that if your log message is filtered out, you might be doing
significant work and incurring significant overhead.
The EVENT log:
Access to the system diagnostic event record. System diagnostic events are
used to record certain system-level events (such as garbage collection,
activity manager state, system watchdogs, and other low level activity),
which may be automatically collected and analyzed during system development.
This is not the main "logcat" debugging log ({#link android.util.Log})!
These diagnostic events are for system integrators, not application authors.
Events use integer tag codes corresponding to /system/etc/event-log-tags.
They carry a payload of one or more int, long, or String values. The
event-log-tags file defines the payload contents for each type code.

How to get the automatically defined port for a Spark Java Application?

In the API documentation for Java Spark (not Apache spark), you can specify a port of 0 to have it automatically select a port. Great!
However, I cannot figure out how to get that port after the server is started. I can see it in the logs:
15:41:12.459 [Thread-2] INFO spark.webserver.JettySparkServer - >> Listening on 0.0.0.0:63134
But I need to be able to get to it programmatically, so that my integration tests are able to run reliably every time.
So how do I get that port?

I could find no way to get this information in the API, and so I filed an issue on their github.
I was able to get at it via an ugly pile of reflection:
/**
* Meant to be called from a different thread, once the spark app is running
* This is probably only going to be used during the integration testing process, not ever in prod!
*
* #return the port it's running on
*/
public static int awaitRunningPort() throws Exception {
awaitInitialization();
//I have to get the port via reflection, which is fugly, but the API doesn't exist :(
//Since we'll only use this in testing, it's not going to kill us
Object instance = getInstance();
Class theClass = instance.getClass();
Field serverField = theClass.getDeclaredField("server");
serverField.setAccessible(true);
Object oneLevelDeepServer = serverField.get(instance);
Class jettyServerClass = oneLevelDeepServer.getClass();
Field jettyServerField = jettyServerClass.getDeclaredField("server");
jettyServerField.setAccessible(true);
//Have to pull in the jetty server stuff to do this mess
Server jettyServer = (Server)jettyServerField.get(oneLevelDeepServer);
int acquiredPort = ((ServerConnector)jettyServer.getConnectors()[0]).getLocalPort();
log.debug("Acquired port: {}", acquiredPort);
return acquiredPort;
}
This works well for me in our integration tests, but I'm not using https, and it does reach about two levels deep into the API via reflection grabbing protected fields. I could not find any other way to do it. Would be quite happy to be proven wrong.

This will work on Spark 2.6.0:
public static int start (String keystoreFile, String keystorePw)
{
secure(keystoreFile, keystorePw, null, null);
port(0);
staticFiles.location("/public");
get(Path.CLOCK, ClockController.time);
get(Path.CALENDAR, CalendarController.date);
// This is the important line. It must be *after* creating the routes and *before* the call to port()
awaitInitialization();
return port();
}
Without the call to awaitInitialization() port() would return 0.

How to read Windows Security events possibly with JNA?

I am reading Windows event logs with JNA as below:
EventLogIterator iterator = new EventLogIterator("Security");
while (iterator.hasNext()) {
EventLogRecord curRecord = iterator.next();
}
But an exception is thrown (other channels like Application are read successfully):
Exception in thread "main" com.sun.jna.platform.win32.Win32Exception: A required privilege is not held by the client.
at com.sun.jna.platform.win32.Advapi32Util$EventLogIterator.<init>(Advapi32Util.java:1929)
at com.sun.jna.platform.win32.Advapi32Util$EventLogIterator.<init>(Advapi32Util.java:1922)
at com.emc.windowsagent.EventLogReader.getEventsByIterator(EventLogReader.java:155)
at com.emc.windowsagent.WindowsAgentMain.main(WindowsAgentMain.java:17)
Is there a way to obtain privilege to read Security events?

Run the application as Administrator.

Hi there Follow simple steps
To get the security logs from the windows we need to change the security permisions via regedit`.
Steps to follow
Type regedit in windows+run
Find
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\policies\system
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Policies\system
In the right pane of System key, find EnableLUA DWORD and double click on it.
Then restart the operating system.And then add the java code
it will work.

How to get shutdown hook to execute on a process launched from Eclipse

I have a shutdown hook in my application (created using Runtime.getRuntime().addShutdownHook). However if I launch the application from within Eclipse, when it is shut-down the shutdown hook doesn't execute.
I think this is because Eclipse sends the equivalent of a force-kill signal to the process, which doesn't cause the shut-down hook to execute (equivalent of taskkill /F on Windows or kill -p on Linux), though I'm not absolutely sure.
Does anyone know how to get around this? I'm running Windows (Vista), and I've a feeling it may be a Windows-specific issue, but I'm not sure.

I used the following hack at the end of my main method to get around the problem:
if (Boolean.parseBoolean(System.getenv("RUNNING_IN_ECLIPSE"))) {
System.out.println("You're using Eclipse; click in this console and " +
"press ENTER to call System.exit() and run the shutdown routine.");
try {
System.in.read();
} catch (IOException e) {
e.printStackTrace();
}
System.exit(0);
}

First off, is your application ending or are you forcibly ending it? If you are forcibly ending it (via the stop button), this Eclipse bug report has details about why this might not be working.
Failing that, you may be correct about this being Windows-specific behavior. I'm on a Mac so I can't confirm, sorry. However, I can tell you that the following test program does execute the shutdown hooks as expected.
public class MyShutdownHook
{
public static void main( String[] args )
{
System.out.println( "Entering main." );
Runtime.getRuntime().addShutdownHook(
new Thread(
new Runnable() {
public void run() {
System.out.println( "Shutdown hook ran." );
}
}
)
);
System.out.println( "Exiting main." );
}
}
The Javadocs for Runtime#addShutdownHook do mention that shutdown hooks do not run when the JVM is aborted, not exited normally, so again, you are probably correct in your assumptions. That being said, here are some things to try. Again, sorry I can't confirm these ahead of time -- no Windows here. (Blessedly!)
Make sure the JVM does not include the -Xrs option. This has a side-effect on shutdown hooks.
Try launching the JVM using either the -server or -client option. It's been my experience that edge-case behavior can be effected by the choice of VM mode. (Especially with regard to threading -- at least on Mac.)
Are you launching your app under a profiler or the like? Loading any native libraries?
Are you getting some fatal error and missing it? E.g., OutOfMemoryError or the like?
Try checking/unchecking the Launch in background option from your application's Run Configuration dialog in Eclipse.
Last but not least: Invoke System.exit() manually if you have the opportunity. :)

Here's a script that you can run outside of eclipse to list the available processes running under Eclipse that you could kill.
#!/bin/bash
set -o nounset # Treat unset variables as an error
PROCESSES=$(ps axo pid,ppid,command)
# Find eclipse launcher PID
LAUNCHER_PID=$(echo "$PROCESSES" | grep "/usr/lib/eclipse/eclipse" |grep -v "launcher"|awk '{print $1}')
echo "Launcher PID $LAUNCHER_PID"
# Find eclipse PID
ECLIPSE_PID=$(echo "$PROCESSES" | egrep "[[:digit:]]* $LAUNCHER_PID " | awk '{print $1}')
echo "Eclipse PID $ECLIPSE_PID"
# Find running eclipse sub-process PIDs
SUB_PROCESS=$(echo "$PROCESSES" | egrep "[[:digit:]]* $ECLIPSE_PID " | awk '{print $1}')
# List processes
echo
for PROCESS in $SUB_PROCESS; do
DRIVER=$(ps --no-headers o pid,ppid,command $PROCESS | awk '{print $NF}')
echo "$PROCESS $DRIVER"
done
echo "Kill a process using: 'kill -SIGTERM \$PID'"

On Windows to gracefully stop a java application in a standard way you need to send Ctrl + C to it. This only works with console apps, but Eclipse uses javaw.exe instead of java.exe. To solve this open the launch configuration, JRE tab and select "Alternative JRE:". The "Java executable" group box appears and allows to enter the alternate executable "java".
Now we need an external program to send Ctrl-C to a process with a hidden console. I found hints here and here. Our program attaches to the console of the desired process and sends the console event.
#include <stdio.h>
#include <windows.h>
int main(int argc, char* argv[])
{
if (argc == 2) {
unsigned pid = 0;
if (sscanf_s(argv[1], "%u", &pid) == 1) {
FreeConsole(); // AttachConsole will fail if we don't detach from current console
if (AttachConsole(pid)) {
//Disable Ctrl-C handling for our program
SetConsoleCtrlHandler(NULL, TRUE);
GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0);
return 0;
}
}
}
return 1;
}
Test java program:
public class Shuthook {
public static void main(final String[] args) throws Exception {
Runtime.getRuntime().addShutdownHook(new Thread() {
#Override
public void run() {
System.out.println("Shutting down...");
}
});
String sPid = ManagementFactory.getRuntimeMXBean().getName();
sPid = sPid.substring(0, sPid.indexOf('#'));
System.out.println("pid: " + sPid);
System.out.println("Sleeping...");
Thread.sleep(1000000);
}
}
Terminating it:
C:\>killsoft.exe 10520
Test program output in Eclipse:
pid: 10520
Sleeping...
Shutting down...

I'm not sure how to fix this but IntelliJ added a separate button to their 'Run' dialog which will shutdown the VM in a way that calls the Shutdown Hooks. Their debugger does not have this feature.

I'm stuck in websphere at the moment, and don't see what I'm looking for.
But I do remember eclipse having a run configuration option related to launching the application in the same VM.
Is it possible your launching your java application in the same VM as the eclipse VM?
But the option slips my mind.

Sairam is right here, By calling System.exit(0) we can terminate eclipse JVM and can see Shutdown hook results