Is it possible to use the java URL.openStream() method to read the file into an input stream when the URL is a query string rather than a direct link to a file? E.g. the code I have is:
URL myURL = new URL("http://www.test.com/myFile.doc");
InputStream is = myURL.openStream();
This works fine for a direct file link. But what if the URL was http://www.test.com?file=myFile.doc ? Would I still be able to obtain the file stream from the server response?
Thanks!
Generally YES, it will work.
But note that URL.openStream() method doesn't follow redirects and not so agile with specifying some additional HTTP behaviours: request type, headers, etc.
I'd recommend to use Apache HTTP Client instead:
final CloseableHttpClient httpclient = HttpClients.createDefault();
final HttpGet request = new HttpGet("http://any-url");
try (CloseableHttpResponse response = httpclient.execute(request)) {
final int status = response.getStatusLine().getStatusCode();
if (status == 200) {
final InputStream is = response.getEntity().getContent();
} else {
throw new IOException("Got " + status + " from server!");
}
}
finally {
request.reset();
}
The URL class works for any url, including:
new URL("http://www.example.com/");
new URL("file://C/windows/system32/cmd.exe");
new URL("ftp://user:password#example.com/filename;type=i");
Its up to the application to do something with the data, for example download the data, or treat it as plain text.
Related
I can't figure out how to send a file via POST request to https://0x0.st in java
My code:
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addTextBody("key", key);
builder.addTextBody("client_id", client_id );
builder.addTextBody("direction_id", direction_id);
ContentType fileContentType = ContentType.create("image/jpeg");
String fileName = file.getName();
builder.addBinaryBody("client_files", file, fileContentType, fileName);
HttpEntity entity = builder.build();
Try this:
public static String uploadFile(String path, ContentType contentType) throws IOException {
File file = new File(path);
URI serverURL = URI.create("https://0x0.st/");
try(CloseableHttpClient client = HttpClientBuilder.create().build()) {
MultipartEntityBuilder builder = MultipartEntityBuilder.create();
builder.addBinaryBody("file", file, contentType, file.getName());
HttpEntity requestEntity = builder.build();
HttpPost post = new HttpPost(serverURL);
post.setEntity(requestEntity);
try(CloseableHttpResponse response = client.execute(post)) {
HttpEntity responseEntity = response.getEntity();
int responseCode = response.getStatusLine().getStatusCode();
String responseString = EntityUtils.toString(responseEntity, "UTF-8");
if(responseCode == 200)
return responseString;
else throw new RuntimeException(responseCode + ": " + responseString);
}
}
}
The key for your upload must be file, url or shorten, otherwise you will get a 400 bad request response. If the request is successful, the provided code returns the URL for your uploaded file.
There are several Http clients available that you can try. The popular ones would be
Apache Http client
OK Http client.
I actually wrote my own Http client which is part of MgntUtils Open Source library written and maintained by me. The reason I wrote my own Http client is to provide a very simple option. It doesn't support all the features provided in other clients but is very simple in use, and it does support uploading and downloading binary information. Assuming from your code that key, client_id, and direction_id could be passed as request headers your code could be something like this
byte[] content = readFile() //Read file as bytes here
byte[] content = readFile() //Read file as bytes hereHttpClient client = new HttpClient();
client.setContentType("image/jpeg");
client.setRequestHeader("key", key);
client.setRequestHeader("client_id", client_id);
client.setRequestHeader("direction_id", direction_id);
String result = client.sendHttpRequest(" https://0x0.st", HttpMethod.POST, ByteBuffer.wrap(content));
System.out.println("Upload result: " + result); //If you expect any textual reply
System.out.println("Upload HTTP response" + client.getLastResponseCode() + " " + client.getLastResponseMessage());
Here is Javadoc for HttpClient class. The library could be obtained as Maven artifacts or from Github, including source code and Javadoc
I'm trying to write a simple test where I submit a request to http://localhost:12345/%, knowing that this is an illegal URI, because I want to assert that my HTTP Server's error-handling code behaves correctly. However, I am having a hard time forcing Java to do this.
If I try to create a Java 11 HttpRequest with URI.create("localhost:12345/%"), I get a URISyntaxException, which is correct and not helpful.
Similarly, using a ws-rs WebTarget:
ClientBuilder.newBuilder().build().target("http://localhost:12345").path("/%")
builds me a WebTarget pointing to /%25, which would normally be very helpful, but is not what I want in this particular situation.
Is there a way to test my error-handling behavior without resorting to low-level bytestream manipulation?
Another possibility is just to use plain Socket - it's easy enough if you know the protocol (especially if using the new text-block feature). This will allow you to misformat the request in any way you like. Reading the response and analysing the result is - of course - a bit more involved:
String request = """
GET %s HTTP/1.1\r
Host: localhost:%s\r
Connection: close\r
\r
""".formatted("/%", port);
try (Socket client = new Socket("localhost", port);
OutputStream os = client.getOutputStream();
InputStream in = client.getInputStream()) {
os.write(request.getBytes(StandardCharsets.US_ASCII));
os.flush();
// This is optimistic: the server should close the
// connection since we asked for it, and we're hoping
// that the response will be in ASCII for the headers
// and UTF-8 for the body - and that it won't use
// chunk encoding.
byte[] bytes = in.readAllBytes();
String response = new String(bytes, StandardCharsets.UTF_8);
System.out.println("response: " + response);
}
Noah's comment lead me down the right path; I was able to do this with the URL class:
#Test
public void testUriMalformed() throws Exception {
final URL url = new URL(TEST_URI + "/%");
final HttpURLConnection connection = (HttpURLConnection)url.openConnection();
final int code = connection.getResponseCode();
final String contentType = connection.getHeaderField("Content-Type");
final String entity = IOUtils.toString(connection.getErrorStream(), Charsets.UTF_8);
assertEquals(500, code);
assertEquals(MediaType.APPLICATION_JSON, contentType);
assertTrue(entity.contains("error_id"));
}
I have a problem with a WebService on Android. I am getting a 400 error but there is no information on the ErrorStream.
What I am trying to do is a POST request on a WCF Webservice using JSON.
I must add that I have includeExceptionDetailInFaults Enabled on my Service. The last time I got a 400 error, it was because I hadn't defined the RequestProperty. Now I don't get any error in the stream.
Here is the code:
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
try {
// In my last error I had not included these lines. Maybe they are still wrong?
urlConnection.setRequestProperty("Content-Type", "application/json");
urlConnection.setRequestProperty("Accept", "application/json");
urlConnection.setRequestMethod("POST");
urlConnection.setDoOutput(true);
urlConnection.setChunkedStreamingMode(0);
OutputStream out = new BufferedOutputStream(urlConnection.getOutputStream());
OutputStreamWriter outputStreamWriter = new OutputStreamWriter(out);
outputStreamWriter.write(jsonObject.toString(), 0, jsonObject.length());
outputStreamWriter.flush();
//outputStreamWriter.close();
int code = urlConnection.getResponseCode();
System.out.println(code);
if(code == 400) {
BufferedInputStream errorStream = new BufferedInputStream(urlConnection.getErrorStream());
InputStreamReader errorStreamReader = new InputStreamReader(errorStream);
BufferedReader bufferedReader = new BufferedReader(errorStreamReader);
StringBuilder builder = new StringBuilder();
String aux = "";
while ((aux = bufferedReader.readLine()) != null) {
builder.append(aux);
}
String output = builder.toString(); // The output is empty.
System.out.print(output);
}
Check Retrofit library from Square it's more easy and thin for GET/POST request and especially for REST. I suggest you to try it. It will make your life easy.
You can use different JSON parsers, error handlers, etc. Very flexible.
POST request definition using retrofit it's simple like this:
An object can be specified for use as an HTTP request body with the #Body annotation.
#POST("/users/new")
void createUser(#Body User user, Callback<User> cb);
Methods can also be declared to send form-encoded and multipart data.
Form-encoded data is sent when #FormUrlEncoded is present on the method. Each key-value pair is annotated with #Field containing the name and the object providing the value.
#FormUrlEncoded
#POST("/user/edit")
User updateUser(#Field("first_name") String first, #Field("last_name") String last);
After you define method inside your Java interface like shown above instantiate it:
RestAdapter restAdapter = new RestAdapter.Builder()
.setEndpoint("https://api.soundcloud.com")
.build();
MyInterface service = restAdapter.create(MyInterface.class);
And then you can call your method synchronously or asynchronously (in case you pass Callback instance).
service.myapi(requestBody);
See Retrofit documentation (http://square.github.io/retrofit/javadoc/index.html) and samples on GitHub for more details.
A 400 error might be occuring (and usually occurs in my case) because of incorrect URL or bad JSON format in post. please check those two
Using an HttpPost object will make your job a lot easier in my opinion
HttpPost post = new HttpPost(url);
if(payload != null){
try {
StringEntity entity = new StringEntity(payload,HTTP.UTF_8);
entity.setContentType(contentType);
post.setEntity(entity);
} catch (UnsupportedEncodingException e) {
LOG.d(TAG, "post err url : " + url);
LOG.e(TAG, "post err url" , e);
throw new Exception(1, e);
}
}
HttpResponse response=executeRequest(owner, post);
I receive a post request from client. This request contains some json data which I want to part on the server side. I have created the server using httpcore. HttpRequestHandler is used for handling the request. Here is the code I thought would work
HttpEntity entity = ((HttpEntityEnclosingRequest)request).getEntity();
InputStream inputStream = entity.getContent();
String str = inputStream.toString();
System.out.println("Post contents: " + str);*/
But I cant seem to find a way to get the body of the request using the HttpRequest object. How can I extract the body from the request object ? Thanks
You should use EntityUtils and it's toString method:
String str = EntityUtils.toString(entity);
getContent returnes stream and you need to read all data from it manually using e.g. BufferedReader. But EntityUtils does it for you.
You can't use toString on stream, because it returns string representation of the object itself not it's data.
One more thing: AFAIK GET requests can't contain body so it seems you get POST request from client.
... and for MultipartEntity use this:
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try {
entity.writeTo(baos);
} catch (IOException e) {
e.printStackTrace();
}
String text = new String(baos.toByteArray());
I am trying to send a post request to a url using HttpURLConnection (for using cUrl in java).
The content of the request is xml and at the end point, the application processes the xml and stores a record to the database and then sends back a response in form of xml string. The app is hosted on apache-tomcat locally.
When I execute this code from the terminal, a row gets added to the db as expected. But an exception is thrown as follows while getting the InputStream from the connection
java.io.FileNotFoundException: http://localhost:8080/myapp/service/generate
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1401)
at org.kodeplay.helloworld.HttpCurl.main(HttpCurl.java:30)
Here is the code
public class HttpCurl {
public static void main(String [] args) {
HttpURLConnection con;
try {
con = (HttpURLConnection) new URL("http://localhost:8080/myapp/service/generate").openConnection();
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setDoInput(true);
File xmlFile = new File("test.xml");
String xml = ReadWriteTextFile.getContents(xmlFile);
con.getOutputStream().write(xml.getBytes("UTF-8"));
InputStream response = con.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(response));
for (String line ; (line = reader.readLine()) != null;) {
System.out.println(line);
}
reader.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Its confusing because the exception is traced to the line InputStream response = con.getInputStream(); and there doesn't seem to be any file involved for a FileNotFoundException.
When I try to open a connection to an xml file directly, it doesn't throw this exception.
The service app uses spring framework and Jaxb2Marshaller to create the response xml.
The class ReadWriteTextFile is taken from here
Thanks.
Edit:
Well it saves the data in the DB and sends back a 404 response status code at the same time.
I also tried doing a curl using php and print out the CURLINFO_HTTP_CODE which turns out to be 200.
Any ideas on how do I go about debugging this ? Both service and client are on the local server.
Resolved:
I could solve the problem after referring to an answer on SO itself.
It seems HttpURLConnection always returns 404 response when connecting to a url with a non standard port.
Adding these lines solved it
con.setRequestProperty("User-Agent","Mozilla/5.0 ( compatible ) ");
con.setRequestProperty("Accept","*/*");
I don't know about your Spring/JAXB combination, but the average REST webservice won't return a response body on POST/PUT, just a response status. You'd like to determine it instead of the body.
Replace
InputStream response = con.getInputStream();
by
int status = con.getResponseCode();
All available status codes and their meaning are available in the HTTP spec, as linked before. The webservice itself should also come along with some documentation which overviews all status codes supported by the webservice and their special meaning, if any.
If the status starts with 4nn or 5nn, you'd like to use getErrorStream() instead to read the response body which may contain the error details.
InputStream error = con.getErrorStream();
FileNotFound is just an unfortunate exception used to indicate that the web server returned a 404.
To anyone with this problem in the future, the reason is because the status code was a 404 (or in my case was a 500). It appears the InpuStream function will throw an error when the status code is not 200.
In my case I control my own server and was returning a 500 status code to indicate an error occurred. Despite me also sending a body with a string message detailing the error, the inputstream threw an error regardless of the body being completely readable.
If you control your server I suppose this can be handled by sending yourself a 200 status code and then handling whatever the string error response was.
For anybody else stumbling over this, the same happened to me while trying to send a SOAP request header to a SOAP service. The issue was a wrong order in the code, I requested the input stream first before sending the XML body. In the code snipped below, the line InputStream in = conn.getInputStream(); came immediately after ByteArrayOutputStream out = new ByteArrayOutputStream(); which is the incorrect order of things.
ByteArrayOutputStream out = new ByteArrayOutputStream();
// send SOAP request as part of HTTP body
byte[] data = request.getHttpBody().getBytes("UTF-8");
conn.getOutputStream().write(data);
if (conn.getResponseCode() != HttpURLConnection.HTTP_OK) {
Log.d(TAG, "http response code is " + conn.getResponseCode());
return null;
}
InputStream in = conn.getInputStream();
FileNotFound in this case was an unfortunate way to encode HTTP response code 400.
FileNotFound in this case means you got a 404 from your server - could it be that the server does not like "POST" requests?
FileNotFound in this case means you got a 404 from your server
You Have to Set the Request Content-Type Header Parameter
Set “content-type” request header to “application/json” to send the request content in JSON form.
This parameter has to be set to send the request body in JSON format.
Failing to do so, the server returns HTTP status code “400-bad request”.
con.setRequestProperty("Content-Type", "application/json; utf-8");
Full Script ->
public class SendDeviceDetails extends AsyncTask<String, Void, String> {
#Override
protected String doInBackground(String... params) {
String data = "";
String url = "";
HttpURLConnection con = null;
try {
// From the above URL object,
// we can invoke the openConnection method to get the HttpURLConnection object.
// We can't instantiate HttpURLConnection directly, as it's an abstract class:
con = (HttpURLConnection)new URL(url).openConnection();
//To send a POST request, we'll have to set the request method property to POST:
con.setRequestMethod("POST");
// Set the Request Content-Type Header Parameter
// Set “content-type” request header to “application/json” to send the request content in JSON form.
// This parameter has to be set to send the request body in JSON format.
//Failing to do so, the server returns HTTP status code “400-bad request”.
con.setRequestProperty("Content-Type", "application/json; utf-8");
//Set Response Format Type
//Set the “Accept” request header to “application/json” to read the response in the desired format:
con.setRequestProperty("Accept", "application/json");
//To send request content, let's enable the URLConnection object's doOutput property to true.
//Otherwise, we'll not be able to write content to the connection output stream:
con.setDoOutput(true);
//JSON String need to be constructed for the specific resource.
//We may construct complex JSON using any third-party JSON libraries such as jackson or org.json
String jsonInputString = params[0];
try(OutputStream os = con.getOutputStream()){
byte[] input = jsonInputString.getBytes("utf-8");
os.write(input, 0, input.length);
}
int code = con.getResponseCode();
System.out.println(code);
//Get the input stream to read the response content.
// Remember to use try-with-resources to close the response stream automatically.
try(BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream(), "utf-8"))){
StringBuilder response = new StringBuilder();
String responseLine = null;
while ((responseLine = br.readLine()) != null) {
response.append(responseLine.trim());
}
System.out.println(response.toString());
}
} catch (Exception e) {
e.printStackTrace();
} finally {
if (con != null) {
con.disconnect();
}
}
return data;
}
#Override
protected void onPostExecute(String result) {
super.onPostExecute(result);
Log.e("TAG", result); // this is expecting a response code to be sent from your server upon receiving the POST data
}
and call it
new SendDeviceDetails().execute("");
you can find more details in this tutorial
https://www.baeldung.com/httpurlconnection-post
The solution:
just change localhost for the IP of your PC
if you want to know this: Windows+r > cmd > ipconfig
example: http://192.168.0.107/directory/service/program.php?action=sendSomething
just replace 192.168.0.107 for your own IP (don't try 127.0.0.1 because it's same as localhost)
Please change
con = (HttpURLConnection) new URL("http://localhost:8080/myapp/service/generate").openConnection();
To
con = (HttpURLConnection) new URL("http://YOUR_IP:8080/myapp/service/generate").openConnection();