I am using HttpClient to connect to some third party url.Its working fine.but some times its giving java.net.UnknownHostException and also connection Reset Excetion. Please help to avoid the exception.
Below is my code snippet.
PostMethod post = null;
try
{
for(int i =0 ; i < 72 ; i++)
{
String url ="http://www.accuweather.com/en/in/amritsar/205593//hourly-weather-forecast/205593?hour=0";
post = new PostMethod(url);
post.setRequestHeader("Content-type", "application/x-www-form-urlencoded;charset=" + "UTF-8");
post.setRequestHeader("Accept-Charset", "UTF-8");
HttpClient httpclient = new HttpClient();
httpclient.getParams().setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 120000);
httpclient.getParams().setParameter(CoreConnectionPNames.SO_TIMEOUT, 120000);
result = httpclient.executeMethod(post);
}
}
catch(Exception e){}
finaly
{
post.releaseConnection();
}
Belowe is the exception trace:
java.net.UnknownHostException: www.accuweather.com
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory$1.doit(ControllerThreadSocketFactory.java:91)
at org.apache.commons.httpclient.protocol.ControllerThreadSocketFactory$SocketTask.run(ControllerThreadSocketFactory.java:158)
at java.lang.Thread.run(Unknown Source)
More than likely, you just have a flaky Internet connection. I'm guessing this is on a mobile phone, in which case flaky Internet connections are a common reality, and you'll just have to deal with it in some way.
Exactly what is the best way to deal with it depends on your program and context, of course. Since you state you run this regularly, one way to deal with it could be to simply ignore an error (catch the exception and just silently abort the attempt if it happens) and then wait for the next regular run. Another option could be to repeat the attempt a set number of times. I'm sure there are other conceivable ways as well.
Related
I have run into an issue developing a HTTP client with the use of BouncyCastle libraries.
Target versions (but the error is also reproducible in Java 1.8.0_91 with the same version of BouncyCastle.)
JRE 1.6.0_45-b06
BouncyCastle jdk15to18 167 (bcprov-jdk15to18-167.jar, bcpkix-jdk15to18-167.jar, bctls-jdk15to18-167.jar)
HTTPClient code:
String strURL = "https://www.<WEBSITE>.com";
// CODE to set default TrustStore, KeyStore to be used
// setup BC as SecurityProvider and SSLSocketFactoryProvider
/*
Security.insertProviderAt(new BouncyCastleProvider(), 1);
Security.insertProviderAt(new BouncyCastleJsseProvider(), 2);
Security.setProperty("ssl.KeyManagerFactory.algorithm", "PKIX");
Security.setProperty("ssl.TrustManagerFactory.algorithm", "PKIX");
Security.setProperty("ssl.SocketFactory.provider", "org.bouncycastle.jsse.provider.SSLSocketFactoryImpl");
System.setProperty("jdk.tls.trustNameService", "true");
*/
URL url = new URL( strURL );
HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setDoOutput(true);
conn.setRequestMethod("GET");
InputStream is = null;
if (conn.getResponseCode() == HttpURLConnection.HTTP_OK) {
System.out.println("OK");
is = conn.getInputStream();
} else if (conn.getResponseCode() == HttpURLConnection.HTTP_INTERNAL_ERROR) {
System.out.println("ERROR");
is = conn.getErrorStream();
} else {
System.out.println( conn.getResponseCode() );
System.out.println( conn.getResponseMessage() );
}
if (is != null) {
System.out.println( readFullyAsString(is, "UTF-8") );
}
conn.disconnect();
Accessing data on a website with TLS 1.2 without client_auth works fine (there are some issues with specific sites that return handshake_failure(40), but luckily not in our case). But when client_auth is required the code fails with the following (a little cryptic) error:
nov. 27, 2020 5:23:10 PM org.bouncycastle.jsse.provider.ProvTlsClient notifyAlertRaised
WARNING: Client raised fatal(2) internal_error(80) alert: Failed to read record
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.bouncycastle.tls.RecordStream$Record.fillTo(RecordStream.java:429)
at org.bouncycastle.tls.RecordStream$Record.readHeader(RecordStream.java:468)
at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:201)
at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
at org.bouncycastle.jsse.provider.ProvSSLSocketDirect$AppDataInput.read(ProvSSLSocketDirect.java:603)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at HttpsClient.main(HttpsClient.java:109)
nov. 27, 2020 5:23:10 PM org.bouncycastle.jsse.provider.ProvTlsClient notifyAlertRaised
WARNING: Client raised fatal(2) internal_error(80) alert: Failed to read record
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.bouncycastle.tls.RecordStream$Record.fillTo(RecordStream.java:429)
at org.bouncycastle.tls.RecordStream$Record.readHeader(RecordStream.java:468)
at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:201)
at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
at org.bouncycastle.jsse.provider.ProvSSLSocketDirect$AppDataInput.read(ProvSSLSocketDirect.java:603)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at HttpsClient.main(HttpsClient.java:109)
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at org.bouncycastle.tls.RecordStream$Record.fillTo(RecordStream.java:429)
at org.bouncycastle.tls.RecordStream$Record.readHeader(RecordStream.java:468)
at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:201)
at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
at org.bouncycastle.jsse.provider.ProvSSLSocketDirect$AppDataInput.read(ProvSSLSocketDirect.java:603)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.net.HttpURLConnection.getResponseCode(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(Unknown Source)
at HttpsClient.main(HttpsClient.java:109)
As expected the code runs just fine in Java 1.8 without BC as Security Provider ... but in Java 1.6 BC is needed for TLS 1.2 support.
I looked at the packets with Wireshark, but I don't have enough knowledge to analyze what's happening (failing) with the SSL handshake operation with the server.
What am I missing here? Or it's just a known limitation of BC?
Thanks for any suggestions.
After extensive research and testing (we set up our own dev environment that replicated the endpoint, we were trying to connect to with client cert) we have found a solution (posted below) that works for us in our dev environment, but couldn't be configured on the endpoint. That's why we went to plan B - TLS termination proxy and it worked like a charm in our case, when we didn't have control of the server environment.
Solution for IIS
In our case the web service requiring client auth was running on Microsoft IIS. As stated "Unfortunately our TLS libraries do not support renegotiation (and we do not plan to add it, although there are corresponding TLS 1.3 features that we will)." in this open issue (https://github.com/bcgit/bc-java/issues/593). After reading the IIS documentation and some testing we successfully configured the environment in a way that TLS sessions could be established with BC TLS lib:
Step 1:
Create an IIS site with default settings
Run IISCrypto with the "Best practices" template:
Step 2:
Enable SSLAlwaysNegoClientCert
Save the following text to a file called "Enable_SSL_Renegotiate.js"
var vdirObj=GetObject("IIS://localhost/W3svc/1");
// replace 1 on this line with the number of the web site you wish to configure
WScript.Echo("Value of SSLAlwaysNegoClientCert Before: " + vdirObj.SSLAlwaysNegoClientCert);
vdirObj.Put("SSLAlwaysNegoClientCert", true);
vdirObj.SetInfo();
WScript.Echo("Value of SSLAlwaysNegoClientCert After: " + vdirObj.SSLAlwaysNegoClientCert);
Run the following command from an elevated / administrator command prompt:
cscript.exe enable_ssl_renegotiate.js
I posted the same answer on BouncyCastle Github - #847
I am writing following program to download the URL using Apache Common-IO and I am getting ReadTimeOut exception,
Exception
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
at sun.security.ssl.AppInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
at java.net.URL.openStream(Unknown Source)
at org.apache.commons.io.FileUtils.copyURLToFile(FileUtils.java:1456)
at com.touseef.stock.FileDownload.main(FileDownload.java:23)
Program
String urlStr = "https://www.nseindia.com/";
File file = new File("C:\\User\\WorkSpace\\Output.txt");
URL url;
try {
url = new URL(urlStr);
FileUtils.copyURLToFile(url, file);
System.out.println("Successfully Completed.");
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
Other site are able to download. Please suggest.
Using commons-io-2.6 jar.
It seems like this site is protected by some web gateway (DOS protection service like Akamai?). Clients seem to be fingerprinted by TLS connection and the HTTP request (headers) and only valid web browsers can connect to the site.
The following code uses Apache commons http client 4.5 and works at least at the moment:
String urlStr = "https://www.nseindia.com/";
File file = new File("C:\\User\\WorkSpace\\Output.txt");
String userAgent = "-";
CloseableHttpClient httpclient = HttpClients.custom().setUserAgent(userAgent).build();
HttpGet httpget = new HttpGet(urlStr);
httpget.addHeader("Accept-Language", "en-US");
httpget.addHeader("Cookie", "");
System.out.println("Executing request " + httpget.getRequestLine());
try (CloseableHttpResponse response = httpclient.execute(httpget)) {
System.out.println("----------------------------------------");
System.out.println(response.getStatusLine());
String body = EntityUtils.toString(response.getEntity());
System.out.println(body);
Files.writeString(file.toPath(), body);
}
A request that e.g works from within Firefox does not work from Java (because the TLS connection with protocols and ciphers is different). I tried a few combinations using Apache commons http client. but is also fails (even though the same request works from Fiddler).
Hence using this web site from within Java is extremely difficult and even the code above works at the moment, the protection system can be adapted at any time so that it won't work again.
I would assume that such a site provides an API dedicated for program usage. Contact them and ask, that is the only advice I can give to you.
This question already has answers here:
InetAddress.getLocalHost() throws UnknownHostException
(9 answers)
Closed 6 years ago.
To develope a client applications, I need to read and modify worksheets and data in Google Sheets. I am facing with below issue:
SpreadsheetService service = new SpreadsheetService("sheet1");
String sheetUrl = "https://spreadsheets.google.com/feeds/list/1XP3KQTboWCArbvH99XYEGsVOldc97NqFzKD MiEepXRA/default/public/values"; //Sheet url
URL url = new URL(sheetUrl); // Get Feed of Spreadsheet url
ListFeed lf = service.getFeed(url, ListFeed.class); //Error statement
Issue:
java.net.UnknownHostException: spreadsheets.google.com at
java.net.AbstractPlainSocketImpl.connect(Unknown Source) at
java.net.PlainSocketImpl.connect(Unknown Source) at
java.net.SocksSocketImpl.connect(Unknown Source) at
java.net.Socket.connect(Unknown Source) at
sun.security.ssl.SSLSocketImpl.connect(Unknown Source) at
sun.security.ssl.BaseSSLSocketImpl.connect(Unknown Source) at
sun.net.NetworkClient.doConnect(Unknown Source) at
sun.net.www.http.HttpClient.openServer(Unknown Source) at
sun.net.www.http.HttpClient.openServer(Unknown Source) at
sun.net.www.protocol.https.HttpsClient.(Unknown Source) at
sun.net.www.protocol.https.HttpsClient.New(Unknown Source) at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(Unknown
Source) at
sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown
Source) at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown
Source) at
sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown
Source) at
com.google.gdata.client.http.HttpGDataRequest.execute(HttpGDataRequest.java:503)
at
com.google.gdata.client.http.GoogleGDataRequest.execute(GoogleGDataRequest.java:535)
at com.google.gdata.client.Service.getFeed(Service.java:1135) at
com.google.gdata.client.Service.getFeed(Service.java:998) at
com.google.gdata.client.GoogleService.getFeed(GoogleService.java:631)
at com.google.gdata.client.Service.getFeed(Service.java:1017) at
test_project.test_demo.main(test_demo.java:35)
As stated here, UnknownHostException means that your device was not able to resolve the host name. It emerges when you are trying to connect to a remote host using its host name, but the IP address of that host cannot be resolved. From this blog, you should check the input of Socket (or any other method that throws an UnknownHostException), and validate that it is the intended one. If you are not sure whether you have the correct host name, you can launch a terminal and use the nslookup command to see if your DNS server can resolve the host name to an IP address successfully.
If that doesn’t work, you should check if the host name you have is correct and then try to refresh your DNS cache. If that doesn’t work either, try to use a different DNS server, eg Google Public DNS is a very good alternative.
You can also check on this link. Hope this helps!
I'm running into a java.net.SocketException (Permission denied: connect) when sending a lot of requests to a server. I've tried the -Djava.net.preferIPv4Stack=true option as mentioned in other threads.
This issue only occurs after a lot of connections have already been made. The following code can be used to reproduce the issue:
public class Example {
public static void main(String[] args) {
if(args.length == 1) {
System.out.println(args[0]);
for(int i = 0; i < Integer.MAX_VALUE; i++) {
requestURL(args[0]);
}
}
}
public static void requestURL(String targetUrl) {
URL url = new URL(targetUrl);
HttpURLConnection httpCon = (HttpURLConnection)url.openConnection();
httpCon.setDoInput(true);
BufferedReader rd = new BufferedReader(new InputStreamReader(httpCon.getInputStream()));
//handle response here
rd.close();
httpCon.disconnect();
}
}
The code will successfully send the requests until the Exception occurs. This has been verified in the server's log files.
Please run the sample code only against a server that you are permitted to, as it generates quite a bit of traffic / load on the server.
So the question now is: How to avoid that Exception? Or any workarounds? And what is the actually issue here?
EDIT
added httpCon.disconnect(). (doesn't solve the problem)
EDIT #2 (StackTrace)
java.net.SocketException: Permission denied: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
at sun.net.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.protocol.http.HttpURLConnection.connect(Unknown Source)
at Example.requestUrl(Example.java)
at Example.main(Example.java)
When the issue occurs no further requests are sent to the server (no log entries on the server and no SYN packets (sniffed with tcpdump)).
EDIT #3
I've forgotten to run the application with the -Djava.net.preferIPv4Stack=true argument.
It hasn't thrown an exception yet when using both, the aforementioned argument and the httpCon.disconnect() method.
Try to use httpCon.disconnect() to be sure to realease the socket after you finish to read the response.
I need sent some requests to server side and get reponse, sometimes when I call specific method to run the flollowing common code, I get one error in line(addToCookieJar(connection);), any idea how this get happened?
URL url = new URL(providerURL);
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setRequestMethod("POST");
connection.setDoInput(true);
connection.setDoOutput(true);
connection.setUseCaches(false);
connection.setRequestProperty("Content-Type", "application/octet-stream");
// We understand gzip encoding
connection.addRequestProperty("Accept-Encoding", "gzip");
if (cookie != null && cookieHandler != null) {
connection.setRequestProperty("Cookie", cookie);
}
if (cookieHandler == null) {
addFromCookieJar(connection);
}
// Send the request
ObjectOutputStream oos = new ObjectOutputStream(connection.getOutputStream());
oos.writeObject(remote.getName());
oos.writeObject(m.getName()); // method name
oos.writeObject(m.getParameterTypes()); // formal parameters
oos.writeObject(args); // actual parameters
oos.flush();
oos.close();
if (cookieHandler == null) {
cookieJar.put(new URI(providerURL), connection.getHeaderFields());
}
Exception:
java.lang.reflect.UndeclaredThrowableException
at $Proxy0.updateDocument(Unknown Source)
at com.agst.ui.gantt.GanttPanel.doUpdateDocument(GanttPanel.java:1931)
at com.agst.ui.gantt.GanttPanel.save(GanttPanel.java:1419)
at com.agst.ui.gantt.GanttPanel$4.run(GanttPanel.java:1673)
at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: Connection reset
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$6.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at sun.net.www.protocol.http.HttpURLConnection.getChainedException(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at com.agst.rmi.RemoteCallHandler.call(RemoteCallHandler.java:196)
at com.agst.rmi.RemoteCallHandler.invoke(RemoteCallHandler.java:142)
... 5 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getHeaderFields(Unknown Source)
at com.agst.rmi.RemoteCallHandler.addToCookieJar(RemoteCallHandler.java:529)
at com.agst.rmi.RemoteCallHandler.call(RemoteCallHandler.java:192)
... 6 more
This error indicates that the remote side has closed the connection while your side was still trying to read from it. You should check if
there is a problem on the server (check it's logs) or
you are trying to read more data than the server supplies
What does the addToCookieJar method do? Passing in the connection object might be a problem to this method since you close the OutputStream obtained from the connection. Are you by any chance trying to retrieve and operate on the output stream object in the addToCookieJar method?