Develop Reference

Connecting JMSToolbox to JMS app on OpenLiberty

I am trying to connect JMSToolbox to an app that is driven by JMS queues running on OpenLiberty.
I am using Open liberty version 22. Specifically 22.0.0.11-202210101601
As far as I can tell, the correct documentation to follow is https://github.com/jmstoolbox/jmstoolbox/wiki/2.2-Setup-for-IBM-LibertyProfile
The installed features I have on the Open Liberty server from the documentation are as follows:
restConnector-2.0 (note restConnector-1.0 as specified in the
documentation does not seem to be available)
appSecurity-2.0
wasJmsClient-2.0
wasJmsServer-1.0
Note I was not able to install restConnector-1.0 from the documentation as I could only find restConnector-2.0.
For the extra jars, I was only able to find restConnector.jar
I could not find the other jars specified in the documentation:
com.ibm.ws.ejb.thinclient_x.y.z.jar (from <was_full_home>/runtimes)
com.ibm.ws.orb_x.y.z.jar (from <was_full_home>/runtimes)
com.ibm.ws.sib.client.thin.jms_x.y.z.jar (from
<was_full_home>/runtimes) (tested with x.z.y ==8.5.5.0+, 9.0.0.0)
Where do I get these jars from? I'm not sure what WAS Full Home means. Am I supposed to take them from a copy of WAS? Are these Jars proprietary?
Thanks,
John

"WAS full" refers to "traditional" WebSphere Application Server. You can download it following this page https://www.ibm.com/cloud/blog/websphere-trial-options-and-downloads
WAS full home is shorthand for WAS installation directory, typically /IBM/WebSphere/AppServer.
These jars are included in the /runtimes subdirectory after you installed the product.
So typical approach following above page would be:
download InstallationManager
install InstallationManager
install developers version either v9 (http://www.ibm.com/software/repositorymanager/com.ibm.websphere.ILAN.v90) or v8.5.5 (https://www.ibm.com/software/repositorymanager/com.ibm.websphere.DEVELOPERSILAN.v85)
copy required jars from the installation directory
... but that would take a while...
So alternatively you could (if you have docker), which should be much faster than whole mumbo-jumbo with installation:
pull WAS v8.5 or v9 version from here https://hub.docker.com/r/ibmcom/websphere-traditional
start container: docker run --name was-server -p 9043:9043 -p 9443:9443 -d ibmcom/websphere-traditional
locate required files:
$ cd opt/IBM/WebSphere/AppServer/runtimes/
$ ls -la
total 343540
com.ibm.jaxrs1.1.thinclient_9.0.jar
com.ibm.jaxrs2.0.thinclient_9.0.jar
com.ibm.jaxws.thinclient_9.0.jar
com.ibm.ws.admin.client.forJython21_9.0.jar
com.ibm.ws.admin.client_9.0.jar
com.ibm.ws.ejb.embeddableContainer_9.0.jar
com.ibm.ws.ejb.embeddableContainer_nls_9.0.jar
com.ibm.ws.ejb.portable_9.0.jar
com.ibm.ws.ejb.thinclient_9.0.jar
com.ibm.ws.jpa-2.0.thinclient_9.0.jar
com.ibm.ws.jpa-2.1.thinclient_9.0.jar
com.ibm.ws.messagingClient.jar
com.ibm.ws.orb_9.0.jar
com.ibm.ws.sib.client.thin.jms_9.0.jar
com.ibm.ws.sib.client_ExpeditorDRE_9.0.jar
com.ibm.ws.webservices.thinclient_9.0.jar
com.ibm.xml.thinclient_9.0.jar
endorsed
properties
sibc.jmsra.rar
sibc.nls.zip
copy required files from the container:
docker cp <containerID>:/opt/IBM/WebSphere/AppServer/runtimes/xyz.jar .

Cannot connect to Wildfly in Dockerfile

I'm creating a custom Dockerfile with extensions for official keycloak docker image. I want to change web-context and add some custom providers.
Here's my Dockerfile:
FROM jboss/keycloak:7.0.0
COPY startup-config.cli /opt/jboss/tools/cli/startup-config.cli
RUN /opt/jboss/keycloak/bin/jboss-cli.sh --connect --controller=localhost:9990 --file="/opt/jboss/tools/cli/startup-config.cli"
ENV KEYCLOAK_USER=admin
ENV KEYCLOAK_PASSWORD=admin
and startup-config.cli file:
/subsystem=keycloak-server/:write-attribute(name=web-context,value="keycloak/auth")
/subsystem=keycloak-server/:add(name=providers,value="module:module:x.y.z.some-custom-provider")
Bu unfortunately I receive such error:
The controller is not available at localhost:9990: java.net.ConnectException: WFLYPRT0053: Could not connect to remote+http://localhost:9990. The connection failed: WFLYPRT0053: Could not connect to remote+http://localhost:9990. The connection failed: Connection refused
The command '/bin/sh -c /opt/jboss/keycloak/bin/jboss-cli.sh --connect --controller=localhost:9990 --file="/opt/jboss/tools/cli/startup-config.cli"' returned a non-zero code: 1
Is it a matter of invalid localhost? How should I refer to the management API?
Edit: I also tried with ENTRYPOINT instead of RUN, but the same error occurred during container initialization.

You are trying to have Wildfly load your custom config file at build-time here. The trouble is, that the Wildfly server is not running while the Dockerfile is building.
Wildfly actually already has you covered regarding automatically loading custom config, there is built in support for what you want to do. You simply need to put your config file in a "magic location" inside the image.
You need to drop your config file here:
/opt/jboss/startup-scripts/
So that your Dockerfile looks like this:
FROM jboss/keycloak:7.0.0
COPY startup-config.cli /opt/jboss/startup-scripts/startup-config.cli
ENV KEYCLOAK_USER=admin
ENV KEYCLOAK_PASSWORD=admin
Excerpt from the keycloak documentation:
Adding custom script using Dockerfile
A custom script can be added by
creating your own Dockerfile:
FROM keycloak
COPY custom-scripts/ /opt/jboss/startup-scripts/
Now you can simply start the image, and the built features in keycloak (Wildfly feature really) will go look for a config in that spedific directory, and then attempt to load it up.
Edit from comment with final solution:
While the original answer solved the issue with being able to pass configuration to the server at all, an issue remained with the content of the script. The following error was received when starting the container:
=========================================================================
Executing cli script: /opt/jboss/startup-scripts/startup-config.cli
No connection to the controller.
=========================================================================
The issue turned out to be in the startup-config.cli script, where the jboss command embed-server was missing, needed to initiate a connection to the jboss instance. Also missing was the closing stop-embedded-server command. More about configuring jboss in this manner in the docs here: CHAPTER 8. EMBEDDING A SERVER FOR OFFLINE CONFIGURATION
The final script:
embed-server --std-out=echo
/subsystem=keycloak-server/theme=defaults/:write-attribute(name=cacheThemes,value=false)
/subsystem=keycloak-server/theme=defaults/:write-attribute(name=cacheTemplates,value=false)
stop-embedded-server

WildFly management interfaces are not available when building the Docker image. Your only option is to start the CLI in embedded mode as discussed here Running CLI commands in WildFly Dockerfile.
A more advanced approach consists in using the S2I installation scripts to trigger CLI commands.

Connect Java Mission Control to Wildfly 16

I try to connect Java Mission Control (JMC) with Wildfly 16. Application server lays on Docker.
I successfully connected to wildfly via jconsole, to manage it I followed steps described here.
Unfortunately, I have no luck to connect via JMC. The URL which I use looks like this:
service:jmx:remoting-jmx://192.168.99.100:9990
I tried to set Xbootclasspath to jboss-cli-client.jar as it was described here, but I just get Unable to connect error.
I set the same jars, which are used for jconsole, but still I got Unable to connect.
I gave a try to adding flags on container site, as it was shown here, but with these flags, even wildfly haven't started.
Then, I found here the idea to hardcode some jboss classes to enable connection via remoting-jmx. I changed version of jars, according to these provided by wildfly16 and put it to jmc.ini like this.
-Xbootclasspath/a:"C:/Program Files/Java/jdk-10.0.2/lib/missioncontrol/dropins/jboss-cli-client.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/remoting-jmx/main/remoting-jmx-3.0.1.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/remoting/main/jboss-remoting-5.0.8.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/logging/main/jboss-logging-3.3.2.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/xnio/main/xnio-api-3.6.5.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/xnio/nio/main/xnio-nio-3.6.5.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/marshalling/main/jboss-marshalling-2.0.6.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/marshalling/river/main/jboss-marshalling-river-2.0.6.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/as/cli/main/wildfly-cli-8.0.0.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/staxmapper/main/staxmapper-1.3.0.Final;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/as/protocol/main/wildfly-protocol-8.0.0.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/dmr/main/jboss-dmr-1.5.0.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/as/controller-client/main/wildfly-controller-client-8.0.0.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/threads/main/jboss-threads-2.3.3.Final.jar;C:/wildfly-16.0.0.Final/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.7.Final.jar"
After that, finally, I have another error, which is
Could not initialize class org.jboss.remotingjmx.RemotingConnector
I added dependencies of remoting-jmx-3.0.1.Final to Xbootclasspath, but I got still the same error.
My question is, have you got any idea, how to make this connection works ? Maybe someone have done it in different way ?
Any advices how can i debug this problem, will be priceless? Because I'm lack of ideas how to solve it.

In %WILDFLY_HOME%\bin\standalone.conf.bat
put:
set "JAVA_OPTS=%JAVA_OPTS% -XX:+FlightRecorder"
In jmc.ini below -vmargs put
-Xbootclasspath/a:C:\%wildfly_home%\bin\client\jboss-cli-client.jar
(%wildfly_home% is different of course, or just copy jboss-cli-client.jar to another directory and correct the path)
3. Run JMC, then Create New Connection - in Connection Properties pane push the button "Custom JMX service URL", put:
service:jmx:http-remoting-jmx://localhost:9990
In the credentials fields just put user and password, they should be created for Realm Management (e.g. using %wildfly_home%\bin\add-user.bat)
Hope this helps someone.

Solution doesn't work on java 11 for me. Mission control fails on connect to wildfly with error:
Caused by: java.lang.NoClassDefFoundError: org/ietf/jgss/GSSManager
at java.base/java.lang.Class.getDeclaredConstructors0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredConstructors(Class.java:3137)
at java.base/java.lang.Class.getConstructor0(Class.java:3342)
at java.base/java.lang.Class.getConstructor(Class.java:2151)
at java.base/java.security.Provider.newInstanceUtil(Provider.java:152)
at java.base/java.security.Provider$Service.newInstance(Provider.java:1824)
at org.wildfly.security.WildFlyElytronBaseProvider$ProviderService.newInstance(WildFlyElytronBaseProvider.java:218)
at org.wildfly.security.sasl.util.SecurityProviderSaslClientFactory.createSaslClient(SecurityProviderSaslClientFactory.java:94)
at org.wildfly.security.sasl.util.AbstractDelegatingSaslClientFactory.createSaslClient(AbstractDelegatingSaslClientFactory.java:66)
at org.wildfly.security.sasl.util.ProtocolSaslClientFactory.createSaslClient(ProtocolSaslClientFactory.java:50)
at org.wildfly.security.sasl.util.AbstractDelegatingSaslClientFactory.createSaslClient(AbstractDelegatingSaslClientFactory.java:66)
at org.wildfly.security.sasl.util.ServerNameSaslClientFactory.createSaslClient(ServerNameSaslClientFactory.java:50)
at org.wildfly.security.sasl.util.AbstractDelegatingSaslClientFactory.createSaslClient(AbstractDelegatingSaslClientFactory.java:66)
at org.wildfly.security.sasl.util.ServerNameSaslClientFactory.createSaslClient(ServerNameSaslClientFactory.java:50)
at org.wildfly.security.sasl.util.FilterMechanismSaslClientFactory.createSaslClient(FilterMechanismSaslClientFactory.java:102)
at org.wildfly.security.sasl.util.AbstractDelegatingSaslClientFactory.createSaslClient(AbstractDelegatingSaslClientFactory.java:66)
at org.wildfly.security.sasl.util.LocalPrincipalSaslClientFactory.createSaslClient(LocalPrincipalSaslClientFactory.java:76)
at org.wildfly.security.sasl.util.PrivilegedSaslClientFactory.lambda$createSaslClient$0(PrivilegedSaslClientFactory.java:64)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at org.wildfly.security.sasl.util.PrivilegedSaslClientFactory.createSaslClient(PrivilegedSaslClientFactory.java:64)
at org.wildfly.security.auth.client.AuthenticationConfiguration.createSaslClient(AuthenticationConfiguration.java:1545)
at org.wildfly.security.auth.client.AuthenticationContextConfigurationClient.createSaslClient(AuthenticationContextConfigurationClient.java:430)
at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:419)
at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:244)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)
Besides, jmc that was embedded to jdk 8 isn't able to start flight recording for java 11 process.
So after investigation i found out that this class is loaded with bootstrap classloader. According to https://openjdk.java.net/jeps/261
jdk.security.jgss module isn't defined to bootstrap classloader. But classes in jboss-cli-client.jar(it originates from wildfly-elytron project) need jgss classes in runtime.
So i found out dirty workaround for this problem: bootstrap needed classes from jre 8 in jmc.ini. Full option for linux is:
-vmargs -Xbootclasspath/a:<path_to_wildfly>/jboss-cli-client.jar:<path_to_jdk8>/jre/lib/rt.jar
And for windows:
-vmargs -Xbootclasspath/a:<path_to_wildfly>\jboss-cli-client.jar;<path_to_jdk8>\jre\lib\rt.jar
after this jmc(run on 11 jdk) succesfully connects to wildfly(run on 11 jdk) and can start and analyze flight recordings.

How to troubleshoot Microsoft Azure WebApp on java? How to access logs?

I have a java-application (standard springboot from default tutorial: https://spring.io/guides/gs/spring-boot-for-azure/ ) that I "successfuly" deploy to my WebApp (created during deployment) via Eclipse/maven plugin azure-webapp:deploy
Once deployed, files are inside the WebApp, I can see them. If start-up is successful I do get running application, but if it is not - I do not know how to troubleshoot. I don't know where to find error logs, what caused the problem and as consequence, how to solve it.
as an example of how to make it fail, add this line:
throw new RuntimeException("Doomed to fail");
I tried enabling logs from "diagnostic logs tab" and expected to see them under LogFiles/Applications but that folder remains empty.
How do I troubleshoot java-application that fails to start in WebApps of Azure?
edit: additional example of Exception to troubleshoot:
public static void main(String[] args) {
throw new RuntimeException("start failure #21");
//SpringApplication.run(Application.class, args);
}

It sounds like you followed the springboot tutorial Deploying a Spring Boot app to Azure to build the GitHub project microsoft/gs-spring-boot and deploy to Azure, but not works.
Here is my steps which I also followed the tutorial, but deployed via my own way.
I created a directory SpringBoot on my local machine, and to do the commands cd SpringBoot and git clone https://github.com/microsoft/gs-spring-boot.
Then, to build it via commands cd gs-spring-boot/complete and mvnw clean package
Note: I reviewed the sections of the tutorial under Create a sample Spring Boot web app which seems to do on Linux, but the web.config file in microsoft/gs-spring-boot/complete is ready to Azure WebApp for Windows. However, there is not any comments to describe the deployment target that be Azure WebApp for Windows or Linux.
So I used my existing WebApp for Windows to test my deployment. I open my Kudu console in browser via the url https://<webapp name>.scm.azurewebsites.net/DebugConsole and drag the files complete/web.config and complete/target/gs-spring-boot-0.1.0.jar to site/wwwroot as the figure below. Then, I started my webapp, and it works fine.
Note: Please check the JAVA_HOME environment variable which has been configured on Azure via command echo %JAVA_HOME% as the figure below.
If not, you need to set Java runtime in the Application settings tab of Azure portal.
Or you can also configure the web.config file to replace the reference %JAVA_HOME% with an existing Java runtime installed in the path D:\Program Files\Java of Azure WebApp, as below.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<handlers>
<add name="httpPlatformHandler" path="*" verb="*" modules="httpPlatformHandler" resourceType="Unspecified"/>
</handlers>
<!-- <httpPlatform processPath="%JAVA_HOME%\bin\java.exe" -->
<httpPlatform processPath="D:\Program Files\Java\jre1.8.0_181\bin\java.exe"
arguments="-Djava.net.preferIPv4Stack=true -Dserver.port=%HTTP_PLATFORM_PORT% -jar "%HOME%\site\wwwroot\gs-spring-boot-0.1.0.jar"">
</httpPlatform>
</system.webServer>
</configuration>

I didn't manage to find logs in windows based machine, but If you enable logs on linux-based-machine you will see them in the "Diagnostic logs" output. There is a catch though.
There is 230 timeout. It will wait full "timeout" time, until producing logs in the log file, after which you can access them via log file or through "Diagnostic logs". Make sure to enable logging before you start application. This applies to linux based machine, I don't know if it can apply to windows based machine.
Then it waits for an answer trigger. Answer trigger, as it turns out, is a phrase in console output "Application is started in X seconds". I increased the timeout to 500 seconds, because although it starts in 60 seconds on my local machine, it takes 430 seconds on remote-linux-based machine of Microsoft Azure*
Second, I changed the name of my main class from "GameStart" to "Application" and after that it actually caught the trigger. After that the application started. Nowhere in manuals did I find the "until timeout - no logs" and "trigger phrase" clauses mentioned.
ps: For reference, it took me 20 to upload application, 5 minutes to start it. I was using centralUS server, cuz centralEU did not work out for me, as it was even longer, although I'm in central EU myself
-
*using test account. It might happen that on payed account number are either different or similar.

What is a simple, effective way to debug custom Kafka connectors?

I'm working a couple of Kafka connectors and I don't see any errors in their creation/deployment in the console output, however I am not getting the result that I'm looking for (no results whatsoever for that matter, desired or otherwise). I made these connectors based on Kafka's example FileStream connectors, so my debug technique was based off the use of the SLF4J Logger that is used in the example. I've searched for the log messages that I thought would be produced in the console output, but to no avail. Am I looking in the wrong place for these messages? Or perhaps is there a better way of going about debugging these connectors?
Example uses of the SLF4J Logger that I referenced for my implementation:
Kafka FileStreamSinkTask
Kafka FileStreamSourceTask

I will try to reply to your question in a broad way. A simple way to do Connector development could be as follows:
Structure and build your connector source code by looking at one of the many Kafka Connectors available publicly (you'll find an extensive list available here: https://www.confluent.io/product/connectors/ )
Download the latest Confluent Open Source edition (>= 3.3.0) from https://www.confluent.io/download/
Make your connector package available to Kafka Connect in one of the following ways:
Store all your connector jar files (connector jar plus dependency jars excluding Connect API jars) to a location in your filesystem and enable plugin isolation by adding this location to the
plugin.path property in the Connect worker properties. For instance, if your connector jars are stored in /opt/connectors/my-first-connector, you will set plugin.path=/opt/connectors in your worker's properties (see below).
Store all your connector jar files in a folder under ${CONFLUENT_HOME}/share/java. For example: ${CONFLUENT_HOME}/share/java/kafka-connect-my-first-connector. (Needs to start with kafka-connect- prefix to be picked up by the startup scripts). $CONFLUENT_HOME is where you've installed Confluent Platform.
Optionally, increase your logging by changing the log level for Connect in ${CONFLUENT_HOME}/etc/kafka/connect-log4j.properties to DEBUG or even TRACE.
Use Confluent CLI to start all the services, including Kafka Connect. Details here: http://docs.confluent.io/current/connect/quickstart.html
Briefly: confluent start
Note: The Connect worker's properties file currently loaded by the CLI is ${CONFLUENT_HOME}/etc/schema-registry/connect-avro-distributed.properties. That's the file you should edit if you choose to enable classloading isolation but also if you need to change your Connect worker's properties.
Once you have Connect worker running, start your connector by running:
confluent load <connector_name> -d <connector_config.properties>
or
confluent load <connector_name> -d <connector_config.json>
The connector configuration can be either in java properties or JSON format.
Run
confluent log connect to open the Connect worker's log file, or navigate directly to where your logs and data are stored by running
cd "$( confluent current )"
Note: change where your logs and data are stored during a session of the Confluent CLI by setting the environment variable CONFLUENT_CURRENT appropriately. E.g. given that /opt/confluent exists and is where you want to store your data, run:
export CONFLUENT_CURRENT=/opt/confluent
confluent current
Finally, to interactively debug your connector a possible way is to apply the following before starting Connect with Confluent CLI :
confluent stop connect
export CONNECT_DEBUG=y; export DEBUG_SUSPEND_FLAG=y;
confluent start connect
and then connect with your debugger (for instance remotely to the Connect worker (default port: 5005). To stop running connect in debug mode, just run: unset CONNECT_DEBUG; unset DEBUG_SUSPEND_FLAG; when you are done.
I hope the above will make your connector development easier and ... more fun!

i love the accepted answer. one thing - the environment variables didn't work for me... i'm using confluent community edition 5.3.1...
here's what i did that worked...
i installed the confluent cli from here:
https://docs.confluent.io/current/cli/installing.html#tarball-installation
i ran confluent using the command confluent local start
i got the connect app details using the command ps -ef | grep connect
i copied the resulting command to an editor and added the arg (right after java):
-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005
then i stopped connect using the command confluent local stop connect
then i ran the connect command with the arg
brief intermission ---
vs code development is led by erich gamma - of gang of four fame, who also wrote eclipse. vs code is becoming a first class java ide see https://en.wikipedia.org/wiki/Erich_Gamma
intermission over ---
next i launched vs code and opened the debezium oracle connector folder (cloned from here) https://github.com/debezium/debezium-incubator
then i chose Debug - Open Configurations
and entered the highlighted debugging configuration
and then run the debugger - it will hit your breakpoints !!
the connect command should look something like this:
/Library/Java/JavaVirtualMachines/jdk1.8.0_221.jdk/Contents/Home/bin/java -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005 -Xms256M -Xmx2G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+ExplicitGCInvokesConcurrent -Djava.awt.headless=true -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dkafka.logs.dir=/var/folders/yn/4k6t1qzn5kg3zwgbnf9qq_v40000gn/T/confluent.CYZjfRLm/connect/logs -Dlog4j.configuration=file:/Users/myuserid/confluent-5.3.1/bin/../etc/kafka/connect-log4j.properties -cp /Users/myuserid/confluent-5.3.1/share/java/kafka/*:/Users/myuserid/confluent-5.3.1/share/java/confluent-common/*:/Users/myuserid/confluent-5.3.1/share/java/kafka-serde-tools/*:/Users/myuserid/confluent-5.3.1/bin/../share/java/kafka/*:/Users/myuserid/confluent-5.3.1/bin/../support-metrics-client/build/dependant-libs-2.12.8/*:/Users/myuserid/confluent-5.3.1/bin/../support-metrics-client/build/libs/*:/usr/share/java/support-metrics-client/* org.apache.kafka.connect.cli.ConnectDistributed /var/folders/yn/4k6t1qzn5kg3zwgbnf9qq_v40000gn/T/confluent.CYZjfRLm/connect/connect.properties

Connector module is executed by the kafka connector framework. For debugging, we can use the standalone mode. we can configure IDE to use the ConnectStandalone main function as entry point.
create debug configure as the following. Need remember to tick "Include dependencies with "Provided" scope if it is maven project
connector properties file need specify the connector class name "connector.class" for debugging
worker properties file can copied from kafka folder /usr/local/etc/kafka/connect-standalone.properties