I am working with a Java application that uses SolrJ to index documents to a Solr server.
In my local test environment, I run a local Solr instance on a Tomcat server on my windows xp box. When I run the Java app from a different Windows box, the indexing completes successfully and the Solr log files look normal.
However, running the same Java application deployed on linux webserver communicating to another linux webserver running Solr, I receive "read timed out" messages after every solr update command:
Jul 14, 2011 3:12:31 AM org.apache.solr.core.SolrCore execute INFO: []
webapp=/solr path=/update params={wt=javabin&version=1} status=400
QTime=20020 Jul 14, 2011 3:12:51 AM
org.apache.solr.update.processor.LogUpdateProcessor finish INFO: {} 0
20021 Jul 14, 2011 3:12:51 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException:
java.net.SocketTimeoutException: Read timed out at
org.apache.solr.handler.XMLLoader.load(XMLLoader.java:72) at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
at...
Caused by:
javax.xml.stream.XMLStreamException: java.net.SocketTimeoutException:
Read timed out
Any idea why this might be happening? My suspicion is that something is closing these connections after they are initiated (e.g. web filtering software, firewall...), but the network admins at my workplace say that no traffic is being blocked.
Is it getting timedout only with updates or even with querying?
Check the server settings on the linux server machine whether it has very less timeout value.
Related
I am trying to do distributed load testing on my network. I have two computers running WSL2 and the firewall is turned off on both of them. I am using OpenJDK version 1.8.0_292 on both WSLs. The Jmeter version on both systems is 5.4.2. I have server.rmi.ssl.disable=true in both Jmeter.properties files. I can access the web application when I type 192.168.1.6:4080 in my browser's URL on my computer running on 192.168.1.18.
The jmeter.properties file for the computer running the web application has remote_hosts=192.168.1.18
The jmeter.properties file for the computer that will be the slave machine has server.rmi.localport=4000
One of my Computers is running the web application and will also be running the client (i.e. ./jmeter). In the Jmeter test plan, the HTTP Request Defaults have the server name 192.168.1.6 and the port as 4080. My other computer has the IPv4 address of 192.168.1.18.
I start the slave machine first by entering ./jmeter-server -n -Djava.rmi.server.hostname=192.168.1.18 -X and the following is outputted Created remote object: UnicastServerRef2 [liveRef: [endpoint:[192.168.1.18:4000](local),objID:[-5f7e0f00:17ab805fd23:-7fff, -5246535002214415146]]]
Now I start the client by entering './jmeter -n -t ~/test_plan.jmx -l ~/result.jtl -X -r '. On the slave machine I get
Starting the test on host 192.168.1.18 # Sat Jul 17 22:02:16 PDT 2021 (1626584536943)
Finished the test on host 192.168.1.18 # Sat Jul 17 22:03:25 PDT 2021 (1626584605136)
And on the client machine I get
Creating summariser <summary>
Created the tree successfully using /home/jasperhale/test_plan.jmx
Configuring remote engine: 192.168.1.18
Starting distributed test with remote engines: [192.168.1.18] # Sat Jul 17 22:10:44 PDT 2021 (1626585044116)
Remote engines have been started:[192.168.1.18]
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
The problem is that after the slave finishes, the client does not stop and report results. When I inspect the jmeter-server.log file I get this error message
2021-07-17 22:56:22,545 ERROR o.a.j.s.RemoteListenerWrapper: testStarted(host) on 192.168.1.18
java.rmi.ConnectException: Connection refused to host: 192.168.1.6; nested exception is:
2021-07-17 22:11:50,420 INFO o.a.j.s.DataStrippingSampleSender: Test Ended on 192.168.1.18
2021-07-17 22:11:50,420 INFO o.a.j.s.BatchSampleSender: Test Ended on 192.168.1.18
2021-07-17 22:11:52,481 ERROR o.a.j.s.BatchSampleSender: testEnded(host)
java.rmi.ConnectException: Connection refused to host: 192.168.1.6; nested exception is:
the error message is way bigger but the main issue is the connection refused
the slave machine is making requests from the test_plan. Also mode=Standard in both properties files. The report files I am trying to collect are the aggregate report, view results tree, and summary report. I've been stuck on this for a while, any help is greatly appreciated.
WSL uses a different IPv4 Address and I was getting those addresses by entering ipconfig into a windows command prompt. I tried it on windows not WSL and it worked.
Try explicitly setting client.rmi.localport property on both master and slave machines to some free port which is not used by any other process like:
client.rmi.localport=1234
Double check that you can reach this host and port from the slave using telnet or netcat or whatever command is there
Once you will be able to reach out from the slave to the master and from the master to the slave your distributed setup should start working normally
More information:
Remote hosts and RMI configuration
JMeter Distributed Testing with Docker
Also having JMeter and the system under test running on the same host is not something you should be normally doing, JMeter can be very resource intensive and you may run into a race condition when both JMeter and the system under test will be struggling for the operating system resources (CPU, RAM, etc.) so test results might be inaccurate
Im trying to run jmeter test in distributed mode, following to official documentation:
https://jmeter.apache.org/usermanual/remote-test.html
client - windows 7
server - ubuntu 20
But I have next error:
Creating summariser <summary>
Created the tree successfully using D:\r.jmx
Configuring remote engine: xxx.xxx.xxx.xxx
Starting distributed test with remote engines: [xxx.xxx.xxx.xxx] # Tue Apr 27 23
:29:27 EEST 2021 (1619555367546)
Error in rconfigure() method java.rmi.MarshalException: error marshalling argume
nts; nested exception is:
java.net.SocketException: Connection reset by peer: socket write error
Remote engines have been started:[]
The following remote engines have not started:[xxx.xxx.xxx.xxx]
Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 44
45
How I can fix it or how to determine, what exactly I should fix?
Check the log files on both master and slave, most probably they contain the cause of the error or at least a cloue
Looking into the output my expectation is that there is something preventing network communication between the machines, make sure that:
machines can "see" each other, i.e. they are on the same subnet and you reach from master to slave and vice versa using ping/telnet commands
the following ports are open in the operating systems firewalls:
1099 or if you changed server_port property value this new port
the port you define as server.rmi.localport
the port(s) you define as client.rmi.localport
You can check out JMeter Distributed Testing with Docker article which provides comprehensive explanation of the network setup for the distributed more of running a JMeter test
I'm running a kubernetes cluster in which I am deploying a "cloud native hazelcast" following the instructions on the kubernetes-hazelcast github page. Once I have a number of hazelcast instances running, I try to connect a java client to one of the instances but for some reason the connection fails.
Some background
Using a kubernetes external endpoint I can connect to hazelcast from outside the kubernetes cluster. When I do a REST call with curl kubernetes-master:32469/hazelcast/rest/cluster, I get a correct response from hazelcast with it's cluster information. So I know my endpoint works.
The hazelcast-kubernetes deployment uses the hazelcast-kubernetes-bootstrapper which allows some configuration by setting environment variables with the replication controller, but I'm using all defaults. So my group and password are "someGroup" and "someSecret".
The java client
My Java client code is really straightforward:
ClientConfig clientConfig = new ClientConfig();
clientConfig.getNetworkConfig().setConnectionAttemptLimit(0);
clientConfig.getNetworkConfig().setConnectionTimeout(10000);
clientConfig.getNetworkConfig().setConnectionAttemptPeriod(2000);
clientConfig.getNetworkConfig().addAddress("kubernetes-master:32469");
clientConfig.getGroupConfig().setName("someGroup");
clientConfig.getGroupConfig().setPassword("someSecret")
HazelcastInstance client = HazelcastClient.newHazelcastClient(clientConfig);
When start my client this is the log output of the hazelcast container
2016-07-05 12:54:38.143 INFO 5 --- [thread-Acceptor] com.hazelcast.nio.tcp.SocketAcceptor : [172.16.15.4]:5701 [someGroup] [3.5.2] Accepting socket connection from /172.16.29.0:54333
2016-07-05 12:54:38.143 INFO 5 --- [ cached4] c.h.nio.tcp.TcpIpConnectionManager : [172.16.15.4]:5701 [someGroup] [3.5.2] Established socket connection between /172.16.15.4:5701
2016-07-05 12:54:38.157 INFO 5 --- [.IO.thread-in-1] c.h.nio.tcp.SocketClientMessageReader : [172.16.15.4]:5701 [someGroup] [3.5.2] Unknown client type: <
And the console output of the client
jul 05, 2016 2:54:37 PM com.hazelcast.core.LifecycleService
INFO: HazelcastClient[hz.client_0_someGroup][3.6.2] is STARTING
jul 05, 2016 2:54:38 PM com.hazelcast.core.LifecycleService
INFO: HazelcastClient[hz.client_0_someGroup][3.6.2] is STARTED
jul 05, 2016 2:54:48 PM com.hazelcast.client.spi.impl.ClusterListenerSupport
WARNING: Unable to get alive cluster connection, try in 0 ms later, attempt 1 of 2147483647.
jul 05, 2016 2:54:58 PM com.hazelcast.client.spi.impl.ClusterListenerSupport
WARNING: Unable to get alive cluster connection, try in 0 ms later, attempt 2 of 2147483647.
jul 05, 2016 2:55:08 PM com.hazelcast.client.spi.impl.ClusterListenerSupport
etc...
The client just keeps trying to connect but no connection is ever established.
What am I missing?
So why won't my client connect to the hazelcast instance? Is it some configuration part I'm missing?
Not sure about the official kubernetes support, however Hazelcast has a kubernetes discovery plugin (based on the new discovery spi) that works on both, client and nodes: https://github.com/noctarius/hazelcast-kubernetes-discovery
Looking at the console logs, you have different Hazelcast versions between Node and Client? Can you either update both to be 3.6.4 i.e., the latest or just change the cluster to be 3.6.2 to match with client. 3.6.x has many configuration changes and many bug fixes as well.
I have exception on tomcat console while using embedded Elastic Search instance. I have configured the embedded instance as a node client cluster, starts off with application runs on tomcat. I've got everything working fine for this cluster, however I'm getting following exception while starting off the instance. I also get same exception when I start another node or shut off existing node for the same cluster.
Apr 07, 2015 4:13:28 PM org.elasticsearch.discovery.zen.ping.multicast
WARNING: [Base] failed to read requesting data from /10.4.1.94:54328
java.io.IOException: Expected handle header, got [15]
at org.elasticsearch.common.io.stream.HandlesStreamInput.readString(Hand
lesStreamInput.java:65)
at org.elasticsearch.cluster.ClusterName.readFrom(ClusterName.java:64)
at org.elasticsearch.cluster.ClusterName.readClusterName(ClusterName.jav
a:58)
at org.elasticsearch.discovery.zen.ping.multicast.MulticastZenPing$Recei
ver.run(MulticastZenPing.java:402)
at java.lang.Thread.run(Thread.java:745)
Apr 07, 2015 4:13:30 PM org.elasticsearch.discovery.zen.ping.multicast
WARNING: [Base] failed to read requesting data from /10.4.1.94:54328
java.io.IOException: Expected handle header, got [15]
at org.elasticsearch.common.io.stream.HandlesStreamInput.readString(Hand
lesStreamInput.java:65)
at org.elasticsearch.cluster.ClusterName.readFrom(ClusterName.java:64)
at org.elasticsearch.cluster.ClusterName.readClusterName(ClusterName.jav
a:58)
at org.elasticsearch.discovery.zen.ping.multicast.MulticastZenPing$Recei
ver.run(MulticastZenPing.java:402)
at java.lang.Thread.run(Thread.java:745)
Apr 07, 2015 4:13:32 PM org.elasticsearch.discovery.zen.ping.multicast
WARNING: [Base] failed to read requesting data from /10.4.1.94:54328
java.io.IOException: Expected handle header, got [15]
at org.elasticsearch.common.io.stream.HandlesStreamInput.readString(Hand
lesStreamInput.java:65)
at org.elasticsearch.cluster.ClusterName.readFrom(ClusterName.java:64)
at org.elasticsearch.cluster.ClusterName.readClusterName(ClusterName.jav
a:58)
at org.elasticsearch.discovery.zen.ping.multicast.MulticastZenPing$Recei
ver.run(MulticastZenPing.java:402)
at java.lang.Thread.run(Thread.java:745)
From the exception it looks like a handshaking problem with other cluster nodes, despite this issue cluster remains healthy and happy to serve payload. I'm using same ElasticSearch version (1.4.4) for both java client and external installations. So answer for this question is not valid anymore (ElasticSearch - failed to read requesting data). Also note I've checked this with isolated node client (java main program) and I don't get to see this exception there.
I am trying to download the file http://repo.maven.apache.org/maven2/org/sonatype/aether/aether-api/1.13.1/aether-api-1.13.1.jar as part of a maven build by my ci server but I get the error:
Downloading: http://repo.maven.apache.org/maven2/org/sonatype/aether/aether-api/1.13.1/aether-api-1.13.1.jar
01-Nov-2012 08:44:26 Nov 01, 2012 8:44:26 AM org.apache.maven.wagon.providers.http.httpclient.impl.client.DefaultRequestDirector tryExecute
01-Nov-2012 08:44:26 INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset
01-Nov-2012 08:44:26 Nov 01, 2012 8:44:26 AM org.apache.maven.wagon.providers.http.httpclient.impl.client.DefaultRequestDirector tryExecute
01-Nov-2012 08:44:26 INFO: Retrying request
After locating this error in the logs I tried to download this artefact by hand with wget which didn't work also. Further investigation revealed that downloading form another server form this provider (different ip/same ip-range) is not possible either.
downloading this file to servers form other providers was successful at the same time.
I was able to ping repo.maven.apache.org so the server was reachable.
Is it possible that the ip-address of my ci-server is blocked for download?
Do I have to move my ci-server to a different provider?
(atm my ci-server is hosted at jiffybox/domainfactory, if that helps answering the question)
There are currently serious problems with maven's CDN provider, see
the support forum at
https://getsatisfaction.com/sonatype/topics/unable_to_dowload_maven_jetty_plugin_version_6_1_26_from_central
current issue reports at
https://issues.sonatype.org/browse/MVNCENTRAL-257
https://issues.sonatype.org/browse/MVNCENTRAL-259
https://issues.sonatype.org/browse/MVNCENTRAL-260
So obviously they are working on it.
The only workaround for me was to download through a VPN tunnel.