Develop Reference

Different behavior of OkHttp3 on Linux and Windows

I have some code which works ok on windows platform, however, the code gives a different behavior on Linux.
I have used the following code to submit a request to an HTTP server to get some messages. what I have done as follows
deploy the code on my local windows machine, then trigger a request and get the server response.
parameters:
{"articleid":"","endtime":"2019-10-29T18:00:00","starttime":"2019-10-29T16:00:00","areaid":"","title":"","pageIndex":"1"}
server response:
{"result":1,"errorcode":"","message":"","pageindex":1,"nextpage":2,"pagesize":100,"data":[...
some data here ...]}
deploy the code on a Linux server, trigger the request with the same parameters in step 1, however, the server response is different.
parameters:
{"articleid":"","endtime":"2019-10-29T18:00:00","starttime":"2019-10-29T16:00:00","areaid":"","title":"","pageIndex":"1"}
server response:
{"result":1,"errorcode":"","message":"","pageindex":1,"nextpage":null,"pagesize":0,"data":[]}
We have looked through the code but can not find what causes the different behaviors.
I suppose there may exist one/some java class files with the same name in different jars, and windows/Linux load different class files then cause the problem, but after looking through the jar file, I also have no ideas. the okhttp related jar files are as following:
okhttp-3.10.0.jar
okio-1.14.0.jar
netty-codec-http-4.1.31.Final.jar
httpcore-nio-4.4.10.jar
httpcore-4.4.10.jar
httpclient-4.5.6.jar
httpasyncclient-4.1.4.jar
public static String okHttpPost(String requestUrl,Map<String,String> map,String orgId,String taskID) throws IOException {
String exceptionMessage="";
String responseResult="";
try {
FormBody.Builder newFormBody = new FormBody.Builder();
Set<String> keys = map.keySet();
for(String key:keys){
newFormBody.add(key,map.get(key));
}
RequestBody body = newFormBody.build();
log.info("server url : "+requestUrl+";paramters："+new ObjectMapper().writeValueAsString(map));
Request request = new Request.Builder()
.url(requestUrl)
.post(body)
.build();
Call call = okHttpClient.newCall(request);
Response response = call.execute();
if (response.code() != 200) {
exceptionMessage = "request failed, taskID：" + taskID + "orgid：" + orgId + "response mesage："+response.toString();
log.info(exceptionMessage);
}
responseResult = response.body().string();
log.info("server url : " + requestUrl + ", reponse messages："+responseResult);
return responseResult;
} catch (IOException e) {
e.printStackTrace();
}finally {
if (!responseResult.contains("token")) {
do some thing;
}
}
return null;
}
Can give any ideas on why the same code behaviors different on windows and Linux platform?
How to change the code to let it works well on Linux?

MDC logging for SSHD server with custom, per-company file-system

We are using Apache-Mina SSHD 1.7 to expose a SFTP server that uses a custom file-system implementation which creates a file system per company. So users of the same company (or more precisely for the same connector) will access the same file system while a users of an other company will access a filesystem unique to their company. The file-system is moreover just a view on a MySQL database and will write uploaded files after some conversions directly into the DB and read files on download from the DB.
The setup of the server looks like the excerpt below
void init() {
server = MessageSftpServer.setUpDefaultServer();
server.setPort(port);
LOG.debug("Server is configured for port {}", port);
File pemFile = new File(pemLocation);
FileKeyPairProvider provider = new FileKeyPairProvider(pemFile.toPath());
validateKeyPairProvider(provider.loadKeys(), publicKeyList);
server.setKeyPairProvider(provider);
server.setCommandFactory(new ScpCommandFactory());
server.setPasswordAuthenticator(
(String username, String password, ServerSession session) -> {
...
});
PropertyResolverUtils.updateProperty(server, ServerAuthenticationManager.MAX_AUTH_REQUESTS, 3);
SftpSubsystemFactory sftpFactory = new SftpSubsystemFactory.Builder()
.withShutdownOnExit(false)
.withUnsupportedAttributePolicy(UnsupportedAttributePolicy.Warn)
.build();
server.setSubsystemFactories(Collections.singletonList(sftpFactory));
// add our custom virtual file system to trick the user into believing she is operating against
// a true file system instead of just operating against a backing database
server.setFileSystemFactory(
new DBFileSystemFactory(connectorService, companyService, mmService, template));
// filter connection attempts based on remote IPs defined in connectors
server.addSessionListener(whitelistSessionListener);
}
Within the file system factory we basically just create the URI for the file system provider and pass it to the respective method of it
#Override
public FileSystem createFileSystem(Session session) throws IOException {
SFTPServerConnectorEntity connector =
connectorService.getSFTPServerConnectorForUser(session.getUsername());
if (null == connector) {
throw new IOException("No SFTP Server connector found for user " + session.getUsername());
}
String ip = CommonUtils.getIPforSession(session);
URI fsUri = URI.create("dbfs://" + session.getUsername() + "#" + ip + "/" + connector.getUuid());
LOG.debug("Checking whether to create file system for user {} connected via IP {}",
session.getUsername(), ip);
Map<String, Object> env = new HashMap<>();
env.put("UserAgent", session.getClientVersion());
try {
return fileSystemProvider.newFileSystem(fsUri, env);
} catch (FileSystemAlreadyExistsException fsaeEx) {
LOG.debug("Reusing existing filesystem for connector {}", connector.getUuid());
return fileSystemProvider.getFileSystem(fsUri);
}
}
and within the provider we simply parse the values from the provided URI and environment variables to create the final filesystem if none was yet available within the cache
#Override
public DBFileSystem newFileSystem(URI uri, Map<String, ?> env) throws IOException {
LOG.trace("newFileSystem({}, {}))", uri, env);
ConnectionInfo ci = ConnectionInfo.fromSchemeSpecificPart(uri.getSchemeSpecificPart());
String cacheKey = generateCacheKey(ci);
synchronized (fileSystems) {
if (fileSystems.containsKey(cacheKey)) {
throw new FileSystemAlreadyExistsException(
"A filesystem for connector " + ci.getConnectorUuid()
+ " connected from IP " + ci.getIp() + " already exists");
}
}
SFTPServerConnectorEntity connector =
connectorService.get(SFTPServerConnectorEntity.class, ci.getConnectorUuid());
List<CompanyEntity> companies = companyService.getCompaniesForConnector(connector);
if (companies.size() < 1) {
throw new IOException("No company for connector " + connector.getUuid() + " found");
}
DBFileSystem fileSystem = null;
synchronized (fileSystems) {
if (!fileSystems.containsKey(cacheKey)) {
LOG.info("Created new filesystem for connector {} (Remote IP: {}, User: {}, UserAgent: {})",
ci.getConnectorUuid(), ci.getIp(), ci.getUser(), env.get("UserAgent"));
fileSystem = new DBFileSystem(this, connector.getUsername(), companies, connector,
template, ci.getIp(), (String) env.get("UserAgent"));
Pair<DBFileSystem, AtomicInteger> sessions = Pair.of(fileSystem, new AtomicInteger(1));
fileSystems.put(cacheKey, sessions);
}
}
if (null == fileSystem) {
throw new FileSystemAlreadyExistsException(
"A filesystem for connector " + ci.getConnectorUuid()
+ " connected from IP " + ci.getIp() + " already exists");
}
return fileSystem;
}
#Override
public DBFileSystem getFileSystem(URI uri) {
LOG.trace("getFileSystem({}))", uri);
String schemeSpecificPart = uri.getSchemeSpecificPart();
if (!schemeSpecificPart.startsWith("//")) {
throw new IllegalArgumentException(
"Invalid URI provided. URI must have a form of 'dbfs://ip:port/connector-uuid' where "
+ "'ip' is the IP address of the connected user, 'port' is the remote port of the user and "
+ "'connector-uuid' is a UUID string identifying the connector the filesystem was created for");
}
ConnectionInfo ci = ConnectionInfo.fromSchemeSpecificPart(schemeSpecificPart);
String cacheKey = generateCacheKey(ci);
if (!fileSystems.containsKey(cacheKey)) {
throw new FileSystemNotFoundException(
"No filesystem found for connector " + ci.getConnectorUuid() + " with connection from IP "
+ ci.getIp());
}
Pair<DBFileSystem, AtomicInteger> sessions = fileSystems.get(cacheKey);
if (!sessions.getKey().isOpen()) {
throw new FileSystemNotFoundException(
"Filesystem for connector " + ci.getConnectorUuid() + " with connection from IP " + ci
.getIp() + " was closed already");
}
int curSessions = sessions.getValue().incrementAndGet();
LOG.info("Added further session to filesystem for connector {}. Current connected sessions: {} (Remote IP: {}, User: {})",
ci.getConnectorUuid(), curSessions, ci.getIp(), ci.getUser());
return sessions.getKey();
}
private String generateCacheKey(String user, String ip, String connectorUuid) {
return connectorUuid + "_" + ip + "_" + user;
}
private String generateCacheKey(ConnectionInfo ci) {
return generateCacheKey(ci.getUser(), ci.getIp(), ci.getConnectorUuid());
}
This works out really well, however, as more and more users get added to the SFTP server the monitoring of the performed actions is suffering a bit due to the lack of propper MDC logging. Simply adding MDC logging isn't working cleanly as Mina or SSHD in particular share the threads among connected users which lead to the MDC context printing the wrong information at times which further lead to confusion on analyzing the log. As a temporary solution we removed it currently from the project.
We also tried to customize Nio2Session (and a couple of other classes) in order to intervene into the threading creation, though this classes were obviously not designed for inheritance which later lead to problems down the road.
Is there a better strategy to include propper MDC logging in our particular scenario where not one file system is used but a filesystem per company approach?

How to learn system (Windows) non-proxy hosts in Java

I'm writting a Java (1.7) application to be running on Windows. The application is accessing additional services running on the same host and other ones running in the Internet. The application can be run in two environments where in one, proxy settings must be specified (there is proxy when accessing the Internet); while in the other environment, the proxy settings must not be specified (there is no proxy).
I want the application to be simple and don't want its users bother with specification of the proxy settings on cmd-line (-Dhttp.proxyHost, etc.) - the application should learn the proxy settings from Windows system settings (IE / Tools / Internet Properties / Connections / LAN Settings).
I have written a piece of code that is supposed to learn that settings, see below. The trouble is that this piece of code does not identify localhost, 127.0.0.1 and my-computer-name (where my-computer-name is the name of my computer) as URLs where proxy should be by-passed when being accessed (yes, I do have 'Bypass proxy server for local addresses' checked). As a result, the application tries to access local services through the proxy which is wrong.
So far I've found out that one way to teach JVM not to use proxy for 'local addresses' is to list the strings (localhost, 127.0.0.1, my-computer-name) in Proxy Settings / Exceptions (Do not use proxy server for addresses beginning with). Obviously, this is not a good solution as usually no one is listing these strings there (the first check-box is enough for non-Java applications).
Second (trivial) solution would be just to count with these strings in my piece of code and do not use proxy settings for them even when JVM thinks otherwise. I don't think this is a good solution and if this is the only solution, IMHO, there is a defect in JVM.
I've found many resources in the Internet how to learn System proxy settings. But how to learn the non-proxy settings?
Thanks,
PP
public static final String HTTP_PROXY_HOST_KEY = "http.proxyHost";
public static final String HTTPS_PROXY_HOST_KEY = "https.proxyHost";
public static final String HTTP_PROXY_PORT_KEY = "http.proxyPort";
public static final String HTTPS_PROXY_PORT_KEY = "https.proxyPort";
public static final String NO_PROXY_HOSTS_KEY = "http.nonProxyHosts";
// provide list of urls which are to be accessed by this application and return proxy and non-proxy settings
private Properties getSystemProxyConfiguration(String[] urls) {
log.debug("Getting system proxy");
Properties properties = new Properties();
SortedSet<String> nonProxyHosts = new TreeSet<>();
for (String url : urls) {
URI uri;
try {
uri = new URI(url);
} catch (URISyntaxException e) {
throw new IllegalStateException(e);
}
InetSocketAddress address = getSystemProxy(uri);
if (address != null) {
if (url.toLowerCase().startsWith("https")) {
properties.put(HTTPS_PROXY_HOST_KEY, address.getHostString());
properties.put(HTTPS_PROXY_PORT_KEY, ""+address.getPort());
//todo verify that all previous URLs in this array are using the same proxy
log.debug("HTTPS proxy: " + address.getHostString() + ":" + address.getPort());
} else {
properties.put(HTTP_PROXY_HOST_KEY, address.getHostString());
properties.put(HTTP_PROXY_PORT_KEY, ""+address.getPort());
//todo verify that all previous URLs in this array are using the same proxy
log.debug("HTTP proxy: " + address.getHostString() + ":" + address.getPort());
}
} else { //todo DEFECT -> this does not find the non-proxy hosts (even though specified in IE Internet settings)
nonProxyHosts.add(uri.getHost());
}
}
if (nonProxyHosts.size() > 0) {
String nonProxyHostsString = nonProxyHosts.first();
nonProxyHosts.remove(nonProxyHostsString);
for (String nonProxyHost : nonProxyHosts) {
nonProxyHostsString = nonProxyHostsString + "|" + nonProxyHost;
}
properties.put(NO_PROXY_HOSTS_KEY, nonProxyHostsString);
log.debug("Non HTTP(S) proxy hosts: "+nonProxyHostsString);
} else {
log.debug("No non HTTP(S) proxy hosts set");
}
return properties;
}
private InetSocketAddress getSystemProxy(URI uri) {
List<Proxy> proxyList;
proxyList = ProxySelector.getDefault().select(uri);
if (proxyList != null && proxyList.size() > 0) { //todo DEFECT - this never returns DIRECT proxy for localhost, 127.0.0.1, my-computer-name strings
Proxy proxy = proxyList.get(0);
if (proxyList.size() > 1) {
log.warn("There is more " + proxy.type() + " proxies available. Use "+PROXY_PROPERTIES_FILE_NAME+" to set the right one.");
}
InetSocketAddress address = (InetSocketAddress) proxy.address();
return address;
}
return null;
}

error while creating adminclient for websphere 7.0.0.11

I need to develop an application for managing WebSphere Application Server v7.0.0.11. I explored a bit and found out that we can use Mbeans. Actually I need to create something similar as Web-sphere's web console.
My problem is that the application should be in C# .net so is there any connector/Adapter to invoke web-sphere's management API. Please point me in right direction.
I am a C#.net developer and a total newbie in java/websphere, I tried creating Admin Client Example from IBM site by using packages found at IBM/Webshpere/Cimrepos directory. The name of Jar file is com.ibm.wplc.was_7.0.0.11.jar I unzipped that jar file in the same folder.
So now My App is starts, connects to websphere successfully and finds mbean on the nodeAgent. The problem I am facing in invoking mbean. I am getting following error message.
exception invoking launchProcess : javax.management.ReflectionExcetion: Target Method not found com.ibm.ws.management.nodeagent.NodeAgent.launchProcess
I am using following url for list of mbean
http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=/com.ibm.websphere.javadoc.doc/web/mbeanDocs/index.html
i tried using different methods from nodeAgent mbean but no joy , I am always getting same exception "method not found".
Following is the code snipped for invoking launchprocess
private void invokeLaunchProcess(String serverName)
{
// Use the launchProcess operation on the NodeAgent MBean to start
// the given server
String opName = "launchProcess";
String signature[] = { "java.lang.String" };
String params[] = { serverName };
boolean launched = false;
try
{
Boolean b = (Boolean)adminClient.invoke(nodeAgent, opName, params, null);
launched = b.booleanValue();
if (launched)
System.out.println(serverName + " was launched");
else
System.out.println(serverName + " was not launched");
}
catch (Exception e)
{
System.out.println("Exception invoking launchProcess: " + e);
}
}
Full Code could be found on following link
http://pic.dhe.ibm.com/infocenter/wasinfo/v6r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftjmx_develop.html
Please let me know what I am doing wrong, do i need to include some other package ? I browsed com.ibm.wplc.was_7.0.0.11.jar, there isn't any folder named nodeagent in com\ibm\ws\managemnt. I found the same jar file in Appserver\runtimes library.
Any help is greatly appreciated, Thanks in Advance.
Getting Mbean
private void getNodeAgentMBean(String nodeName)
{
// Query for the ObjectName of the NodeAgent MBean on the given node
try
{
String query = "WebSphere:type=NodeAgent,node=" + nodeName + ",*";
ObjectName queryName = new ObjectName(query);
Set s = adminClient.queryNames(queryName, null);
if (!s.isEmpty())
nodeAgent = (ObjectName)s.iterator().next();
else
{
System.out.println("Node agent MBean was not found");
System.exit(-1);
}
}
catch (MalformedObjectNameException e)
{
System.out.println(e);
System.exit(-1);
}
catch (ConnectorException e)
{
System.out.println(e);
System.exit(-1);
}catch (Exception e){
e.printStackTrace();
System.exit(-1);
}
System.out.println("Found NodeAgent MBean for node " + nodeName);
}

It seems my problem was with adminClient.invoke method I wasn't passing parameters correctly. It got fixed after having correct parameters. I hope this helps if someone is having same problem.

How to identify the URL of an Java web application from within?

My Java web application contains a startup servlet. Its init() method is invoked, when the web application server (Tomcat) is started. Within this method I need the URL of my web application. Since there is no HttpServletRequest, how to get this information?

You can't. Because there is no "URL of an Java web application" as seen "from within". A servlet is not tied to an URL, that is done from the outside. (Perhaps you have a Apache server that connects to a Tomcat - Tomcat can't know about it)
It makes sense to ask a HttpServletRequest for its url, because we are speaking of the information of a event (the URL that was actually used to generate this request), it does not make sense to ask for a configuration URL.

A workaround could be to perform the initialization lazy when the first request arrives. You can implement a filter that do that once, e.g. by storing a boolean flag in a static variable and synchronizing access to the flag correctly. But it implies a little overhead because each subsequent request will go through the filter which then bypass the initialization. It was just a thought.

There is nothing in the servlet API that provides this information, plus any given resource may be bound to multiple URL's.
What you CAN do, is to inspect the servlet context when you receive an actual request and see what URL was used.

Here is how it works for me and probably for most configurations:
public static String getWebappUrl(ServletConfig servletConfig, boolean ssl) {
String protocol = ssl ? "https" : "http";
String host = getHostName();
String context = servletConfig.getServletContext().getServletContextName();
return protocol + "://" + host + "/" + context;
}
public static String getHostName() {
String[] hostnames = getHostNames();
if (hostnames.length == 0) return "localhost";
if (hostnames.length == 1) return hostnames[0];
for (int i = 0; i < hostnames.length; i++) {
if (!"localhost".equals(hostnames[i])) return hostnames[i];
}
return hostnames[0];
}
public static String[] getHostNames() {
String localhostName;
try {
localhostName = InetAddress.getLocalHost().getHostName();
} catch (UnknownHostException ex) {
return new String[] {"localhost"};
}
InetAddress ia[];
try {
ia = InetAddress.getAllByName(localhostName);
} catch (UnknownHostException ex) {
return new String[] {localhostName};
}
String[] sa = new String[ia.length];
for (int i = 0; i < ia.length; i++) {
sa[i] = ia[i].getHostName();
}
return sa;
}