Develop Reference

Issue in calling Webservice using URLConnection

I have a url as following
String url = "http://host.com/connect/v1.3/serviceProducts?q=\"lookupName\" LIKE 'A9051%'"
This url points to a restful webservice. Whenever I try to hitting this url using HttpURLConnection but it always returns be 301 Moved Permanently response.
I have tried encoding the url. But still it failed to work. Below is the code of my attempt to encode it.
try {
url = URLEncoder.encode(url, "UTF-8");
} catch (UnsupportedEncodingException ex) {
java.util.logging.Logger.getLogger(DbInteraction.class.getName()).log(Level.SEVERE, null, ex);
}
url = url.replace("%3A", ":").replace("%2F", "/").replace("%3F", "?").replace("%3D", "=").replace("%25", "%");
After the encoding my url looks like this.
http://host.com/connect/v1.3/serviceProducts?q=%22lookupName%22+LIKE+%27A9051%%27
With this encoded url as well I'm still getting 301 Status.
I also tried with the below url format
http://host.com/connect/v1.3/serviceProducts?q=%22lookupName%22%20LIKE%20%27A9051%%27
I don't understand where am I going wrong. Please suggest.
I've tried testing the service in Postman and it works as expected there.
PS: This is a dummy url with modified host details.
UPDATE:
Here are some more ways I tried hitting the url:
http%3A%2F%2Fhost.com%2Fconnect%2Fv1.3%2FserviceProducts%3Fq%3D%22lookupName%22LIKE%27A9051%25%27
http://host.com/connect/v1.3/serviceProducts%3Fq%3D%22lookupName%22+LIKE+%27A9051%25%27
http://host.com/connect/v1.3/serviceProducts?q%3D%22lookupName%22+LIKE+%27A9051%25%27
http://host.com/connect/v1.3/serviceProducts?q=%22lookupName%22+LIKE+%27A9051%25%27

You should not do any replacements after you did URLEncoder.encode(url, "UTF-8"); because this is actually the encoded string that you need to use.

How to authenticate spring boot endpoint programmatically?

In Spring Boot application, I enable one endpoint i.e. metrics endpoint. At same moment I don't want to make it public so I configured it with the following setting:
security.user.name=admin
security.user.password=ENC(l2y+PuJeGIOMshbv+ddZgK8lOe2TRdt9YIuMwB5g5Ws=)
security.basic.enabled=false
management.context-path=/manager
management.port=8082
management.address=127.0.0.2
management.security.enabled=false
management.security.roles=SUPERUSER
management.ssl.enabled=true
management.ssl.key-store=file:keystore.jks
management.ssl.key-password=ENC(l2y+PuJeGIOMshbv+ddZgK8lOe2TRdt9YIuMwB5g5Ws=)
endpoints.metrics.id=metrics
endpoints.metrics.sensitive=true
endpoints.metrics.enabled=true
Basically, if someone trying to access https://127.0.0.2:8082/manager/metrics URI throw any browser then he/she needs to supply a username (security.user.name=admin) and password (security.user.password=ENC(l2y+PuJeGIOMshbv+ddZgK8lOe2TRdt9YIuMwB5g5Ws=)) in a popup.
Now I have java client which is running on the same machine(in future it may run in a remote location) but with a different host and port i.e. 127.0.0.1:8081 trying to access the above URI programmatically, but unable to do so and end up with response code 401.
401 is the response code for UNAUTHORISED access which is obvious. My query is is it possible to supply username and password programmatically to access the above URI i.e. https://127.0.0.2:8082/manager/metrics? or firewall is the only way to secure it?.
My java client code:
System.setProperty("javax.net.ssl.trustStore","D://SpringBoot/SpringWebAngularJS/truststore.ts");
System.setProperty("javax.net.ssl.trustStorePassword","p#ssw0rd");
try {
HttpsURLConnection.setDefaultHostnameVerifier(new HostnameVerifier(){
public boolean verify(String hostname,SSLSession sslSession) {
return hostname.equals("127.0.0.2");
}
});
URL obj = new URL("https://127.0.0.2:8082/manager/metrics");
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
con.setRequestProperty( "Content-Type", "application/json" );
con.setRequestProperty("Accept", "application/json");
con.setRequestMethod("GET");
con.setRequestProperty("User-Agent", "Mozilla/5.0");
int responseCode = con.getResponseCode();
System.err.println("GET Response Code :: " + responseCode); //Response code 401
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

Based on the described behavior of the browser, i assume that the endpoint is secured by basic authentication.
Basic authentication expects an "Authorization" Header containing the username and password in the following form encoded in base64: "username:password"
If you use java 8 you can use the base64 encoder provided in the java util package as follows:
import java.util.Base64;
con.setRequestProperty("Authorization", Base64.getEncoder().encode("yourUsername:yourPassword".getBytes());
Just to provide a little further information:
This is the exact same thing your browser does. It sends the request and gets a 401 Response containing a header WWW-Authenticate: Basic. So the browser knows that the authentication method is basic auth and asks you to provide your username and password which then will be encoded by base64 and added to the authorization header when the browser performs the same request a second time.

how to check https using HttpURLConnection

I am trying to connect to a url with HttpUrlConnection. the host which user enter can be running on http:// or https://. when i am connecting it throws an exception as EOFException.
Is there any way that i identify that url is running on https through some error code or something ??
Following code i am using for this purpose.
HttpURLConnection.setFollowRedirects(true);
con = (HttpURLConnection) new URL(url[0]).openConnection();
con.setRequestMethod("POST");
con.setConnectTimeout(20000);
I m using the abode code what if the url is not valid like i type wwww.gooooooodldldle.com
which is not valid url. I am getting Java.net.SocketTimeout exception here

The URL cannot change after the connection has been created if you don't reacreate it again. I would do it like this to know the protocol.
URL url;
try {
url = new URL(url[0]);
if(url.getProtocol().equalsIgnoreCase(HTTPS){
Lod.i(TAG, "Is HTTPS connection");
} else {
Log.i(TAG, "Is HTTP connection");
}
} catch (EOFException eofEx) {
eofEx.printStackTrace();
}
After update:
You can check the url string with a Regular Expresion and then if it's correct try the connection. Or put all the logic inside a try catch and show a toast or dialog if the exception is rised.
Hope it helps.

If you won't mind, try to use android-async-http library instead

Cannot upload file to a server

I'm using a function called UploadFFGS and this is its content:
URL url = new URL("http://linkedme.com/filebet.txt");
URLConnection ucn = url.openConnection();
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection = (HttpURLConnection) url.openConnection();
FileInputStream is = new FileInputStream("filebet.txt"); //before I download the same file because I must edit it and upload the new version
OutputStream ostream = connection.getOutputStream();
PrintWriter pwriter = new PrintWriter(ostream);
pwriter.print(jTextArea1.getText());
pwriter.close();
This program never uploads the file filebet I have on my desktop to my link (http://linkedme.com/filebet.txt). Any ideas? I call it in this way:
try {
UploadFFGS();
}
catch (MalformedURLException ex) {
Logger.getLogger(xGrep.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(xGrep.class.getName()).log(Level.SEVERE, null, ex);
}
Also, NetBeans gives me this error: "java.net.ProtocolException: cannot write to a URLConnection if doOutput=false - call setDoOutput(true)".

Your approach won't work because your API endpoint (most likely) is a regular file rather than an interpreted script. The endpoint must provide a API by means of which you upload a file (POST/PUT etc).

I have a different solution. Maybe this will be useful for someone.
Just have a look at your advanced proxy settings in your web browser.
System engineers in our company had changed the proxy settings but I was not aware of it.
This error cost me 3 work-days. I got this doOutput error while writing a ftp upload project in my company. I tried everything like adding conn.setDoOutput(true) or 'fifty shades' of similar solutions but non of them saved me.
But, after I changed my proxy settings to correct ones, the error dissapeared and now I am able to upload my files through ftp using urlConnection in java.
I used the code in the link below to make an upload process, and did not add anything except host, port, user and password.
http://www.ajaxapp.com/2009/02/21/a-simple-java-ftp-connection-file-download-and-upload/

Getting "java.net.ProtocolException: Server redirected too many times" Error

I'm making a simple URL request with code like this:
URL url = new URL(webpage);
URLConnection urlConnection = url.openConnection();
InputStream is = urlConnection.getInputStream();
But on that last line, I'm getting the "redirected too many times error". If my "webpage" var is, say, google.com then it works fine, but when I try to use my servlet's URL then it fails. It seems I can adjust the number of times it follows the redirects (default is 20) with this:
System.setProperty("http.maxRedirects", "100");
But when I crank it up to, say, 100 it definitely takes longer to throw the error so I know it is trying. However, the URL to my servlet works fine in (any) browser and using the "persist" option in firebug it seems to only be redirecting once.
A bit more info on my servlet ... it is running in tomcat and fronted by apache using 'mod-proxy-ajp'. Also of note, it is using form authentication so any URL you enter should redirect you to the login page. As I said, this works correctly in all browsers, but for some reason the redirect isn't working with the URLConnection in Java 6.
Thanks for reading ... ideas?

It's apparently redirecting in an infinite loop because you don't maintain the user session. The session is usually backed by a cookie. You need to create a CookieManager before you use URLConnection.
// First set the default cookie manager.
CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));
// All the following subsequent URLConnections will use the same cookie manager.
URLConnection connection = new URL(url).openConnection();
// ...
connection = new URL(url).openConnection();
// ...
connection = new URL(url).openConnection();
// ...
See also:
Using java.net.URLConnection to fire and handle HTTP requests

Duse, I have add this lines:
java.net.CookieManager cm = new java.net.CookieManager();
java.net.CookieHandler.setDefault(cm);
See this example:
java.net.CookieManager cm = new java.net.CookieManager();
java.net.CookieHandler.setDefault(cm);
String buf="";
dk = new DAKABrowser(input.getText());
try {
URL url = new URL(dk.toURL(input.getText()));
DataInputStream dis = new DataInputStream(url.openStream());
String inputLine;
while ((inputLine = dis.readLine()) != null) {
buf+=inputLine;
output.append(inputLine+"\n");
}
dis.close();
}
catch (MalformedURLException me) {
System.out.println("MalformedURLException: " + me);
}
catch (IOException ioe) {
System.out.println("IOException: " + ioe);
}
titulo.setText(dk.getTitle(buf));

I was using Jenkins on Tomcat6 on a unix environment and got this bug. For some reason, upgrading to Java7 solved it. I'd be interested to know exactly why that fixed it.

I had faced the same problem and it took considerable amount of time to understand the problem.
So to summarize the problem was in mismatch of headers.
Consider below being my Resource
#GET
#Path("booksMasterData")
#Produces(Array(core.MediaType.APPLICATION_JSON))
def booksMasterData(#QueryParam("stockStatus") stockStatus : String): Response = {
// some logic here to get the books and send it back
}
And here is client code, which was trying to connect to my above resource
ClientResponse clientResponse = restClient.resource("http://localhost:8080/booksService").path("rest").path("catalogue").path("booksMasterData").accept("application/boks-master-data+json").get(ClientResponse.class);
And the error was coming on exactly above line.
What was the problem?
My Resource was using
"application/json"
in
#Produces annotation
and my client was using
accept("application/boks-master-data+json")
and this was the problem.
It took me long to find out this as the error was no where related. Break through was when I tried to access my resource in postman with
Accept-> "application/json" header
it worked fine, however with
Accept-> "application/boks-master-data+json" header
it doesnt.
And again, even Postman was not giving me proper error. The error was too generic. Please see the below image for reference.