I am trying to connect to an endpoint using a post method however I keep getting the following error:
java.net.MalformedURLException: unknown protocol: localhost
at java.base/java.net.URL.<init>(URL.java:634)
at java.base/java.net.URL.<init>(URL.java:523)
at java.base/java.net.URL.<init>(URL.java:470)
at endpointtest.endpoint(endpointtest.java:23)
at main.main(endpoint.java:66)
I would expect my code to return the response based on the post request however that is not the case. Below is my code:
public class endpointtest {
public String endpoint(String urlStr, String username) {
final StringBuilder response = new StringBuilder();
try {
//creating the connection
URL url = new URL(urlStr);
HttpClient client = HttpClient.newHttpClient();
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "Value");
connection.connect();
//builds the post body, adds parameters
final DataOutputStream out = new DataOutputStream(connection.getOutputStream());
//out.writeBytes(toJSON(globalId));
out.flush();
out.close();
//Reading the response
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String inputline;
while ((inputline = in.readLine()) != null) {
response.append(inputline);
}
in.close();
connection.getResponseCode();
connection.disconnect();
} catch (final Exception ex) {
ex.printStackTrace();
System.out.println(" error ");
}
return response.toString();
}
}
class main {
public static void main(String[] args){
endpointtest ep = new endpointtest();
ep.endpoint("localhost:8080/endpoint","123");
}
}
Why is this error occuring? Forgive me if there are basic errors, I am new to web dev
you have not included protocol(http/https) in your URL string.
Change it to ep.endpoint("http://localhost:8080/endpoint", "123");
and it should work.
I need to create a hastebin paste in java, but I don't know how.
I tried this
public static String paste(String content) throws MalformedURLException, IOException {
URL url = new URL("https://hasteb.in/documents");
URLConnection con = url.openConnection();
HttpURLConnection http = (HttpURLConnection) con;
http.addRequestProperty("data", content);
http.setRequestMethod("POST");
http.setDoOutput(true);
InputStream in = new BufferedInputStream(http.getInputStream());
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder entirePage = new StringBuilder();
String inputLine;
while ((inputLine = reader.readLine()) != null) {
entirePage.append(inputLine);
}
reader.close();
if (!(entirePage.toString().contains("\"key\":\""))) {
return "UNKNOWN";
}
return "https://hasteb.in/"+entirePage.toString().split("\"key\":\"")[1].split("\",")[0];
}
But it does not work.
Error is Caused by: java.lang.IllegalArgumentException: Illegal character(s) in message header value
Any help?
To do a POST request using HttpURLConnection you need to write the request body using the provided OutputStream functionality rather than setting a HTTP header named "data":
try (OutputStream out = http.getOutputStream()) {
out.write(content.getBytes());
out.flush();
}
This needs to happen BEFORE you start reading from the InputStream (and I would suggest using try-with-resources for that as well).
I am trying to do a simple HTTP Get and POST request in java to my Eclipse Kura gateway but i dont know how to authenticate using username and password. I tried using the url syntax http://user:pw#ipaddress:port/ but i still get HTTP error code 401.
import java.io.*;
import java.net.*;
public class HTTP {
public static String getHTML() throws Exception {
StringBuilder result = new StringBuilder();
String urlToRead = "http://user:pw#ipaddress:port";
URL url = new URL(urlToRead);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestMethod("GET");
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
result.append(line);
}
rd.close();
return result.toString();
}
public static void main(String[] args) throws Exception {
System.out.println(getHTML());
}
}
I believe that you are not providing the credentials in the desired way. This very similar question already has an accepted answer, in which James Van Huis suggests using java.net.Authenticator for setting authentication data prior to opening any connections.
In my java application I used a Httpsurlconnection to post some string data to the server. When I test this code on android, it works perfectly. However, in a java application it does not work. Client java application is as follows:
public static void main(String[] args) {
disableSslVerification();
new HttpsClient().testIt();
}
private void testIt() {
String https_url = "https://XXX.XX.XXX.XXX:XXXX/XXXXX/TestServlet";
URL url;
try {
url = new URL(https_url);
HttpsURLConnection con = (HttpsURLConnection) url.openConnection();
print_content(con, "test");
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
private void print_content(HttpsURLConnection connection, String data) {
if (connection != null) {
try {
connection.setConnectTimeout(6000);
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type",
"application/x-www-form-urlencoded");
Charset cSet = Charset.forName("UTF-8");
byte bytes[] = data.getBytes(cSet);
connection.setRequestProperty("Content-Length", ""
+ Integer.toString(bytes.length));
connection.setRequestProperty("Content-Language", "tr");
connection.setRequestProperty("Accept-Charset", "UTF-8");
connection.setUseCaches(false);
connection.setDoInput(true);
connection.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(
connection.getOutputStream());
wr.write(bytes);
wr.flush();
wr.close();
InputStream is = connection.getInputStream();
BufferedReader rd = new BufferedReader(new InputStreamReader(is, cSet));
String line;
StringBuilder response = new StringBuilder();
while ((line = rd.readLine()) != null) {
response.append(line);
response.append(System.getProperty("line.separator"));
}
rd.close();
System.out.println(response.toString());
} catch (Exception e) {
System.out.println(e.getMessage());
} finally {
if (connection != null) {
connection.disconnect();
}
}
}
}
And the servlet is as follows:
#Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=UTF-8");
PrintWriter out = response.getWriter();
String s = getHTML(request);
try {
out.print("received data:");
out.print(s);
} finally {
out.close();
}
}
private String getHTML(HttpServletRequest request) throws IOException {
int n = request.getContentLength();
if (n < 1) {
return "";
}
byte bytes[] = new byte[n];
request.getInputStream().read(bytes);
return new String(bytes, "UTF-8");
}
When I run this application, servlet's response is:
received data:tâââ
Always only the first character is correctly send to the servlet. The same code works perfect on android. Can anyone help me please? Thanks...
I can't see an obvious problem with your code that would cause this.
Can anyone help me please?
I suggest that you take a methodical approach to investigating the problem. Use a packet sniffer to check what is actually being sent over the wire. Check that the actual headers in the request and response are correct. Check that the request and response bodies are really properly encoded UTF-8 ...
What you find in your investigation / evidence gathering will help you figure out where the problem (or problems) are occurring ... and that will allow you to home in on the part(s) of your code that is/are responsible.
request.getInputStream().read(bytes);
You might need to do this read in a loop. At the very least, check how many bytes have been read. The array appears to be empty except for the first char.
Reads some number of bytes from the input stream and stores them into
the buffer array b. The number of bytes actually read is returned as
an integer. This method blocks until input data is available, end of
file is detected, or an exception is thrown.
I'm trying to find Java's equivalent to Groovy's:
String content = "http://www.google.com".toURL().getText();
I want to read content from a URL into string. I don't want to pollute my code with buffered streams and loops for such a simple task. I looked into apache's HttpClient but I also don't see a one or two line implementation.
Now that some time has passed since the original answer was accepted, there's a better approach:
String out = new Scanner(new URL("http://www.google.com").openStream(), "UTF-8").useDelimiter("\\A").next();
If you want a slightly fuller implementation, which is not a single line, do this:
public static String readStringFromURL(String requestURL) throws IOException
{
try (Scanner scanner = new Scanner(new URL(requestURL).openStream(),
StandardCharsets.UTF_8.toString()))
{
scanner.useDelimiter("\\A");
return scanner.hasNext() ? scanner.next() : "";
}
}
This answer refers to an older version of Java. You may want to look at ccleve's answer.
Here is the traditional way to do this:
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static String getText(String url) throws Exception {
URL website = new URL(url);
URLConnection connection = website.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(
connection.getInputStream()));
StringBuilder response = new StringBuilder();
String inputLine;
while ((inputLine = in.readLine()) != null)
response.append(inputLine);
in.close();
return response.toString();
}
public static void main(String[] args) throws Exception {
String content = URLConnectionReader.getText(args[0]);
System.out.println(content);
}
}
As #extraneon has suggested, ioutils allows you to do this in a very eloquent way that's still in the Java spirit:
InputStream in = new URL( "http://jakarta.apache.org" ).openStream();
try {
System.out.println( IOUtils.toString( in ) );
} finally {
IOUtils.closeQuietly(in);
}
Or just use Apache Commons IOUtils.toString(URL url), or the variant that also accepts an encoding parameter.
There's an even better way as of Java 9:
URL u = new URL("http://www.example.com/");
try (InputStream in = u.openStream()) {
return new String(in.readAllBytes(), StandardCharsets.UTF_8);
}
Like the original groovy example, this assumes that the content is UTF-8 encoded. (If you need something more clever than that, you need to create a URLConnection and use it to figure out the encoding.)
Now that more time has passed, here's a way to do it in Java 8:
URLConnection conn = url.openConnection();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8))) {
pageText = reader.lines().collect(Collectors.joining("\n"));
}
Additional example using Guava:
URL xmlData = ...
String data = Resources.toString(xmlData, Charsets.UTF_8);
Java 11+:
URI uri = URI.create("http://www.google.com");
HttpRequest request = HttpRequest.newBuilder(uri).build();
String content = HttpClient.newHttpClient().send(request, BodyHandlers.ofString()).body();
If you have the input stream (see Joe's answer) also consider ioutils.toString( inputstream ).
http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toString(java.io.InputStream)
The following works with Java 7/8, secure urls, and shows how to add a cookie to your request as well. Note this is mostly a direct copy of this other great answer on this page, but added the cookie example, and clarification in that it works with secure urls as well ;-)
If you need to connect to a server with an invalid certificate or self signed certificate, this will throw security errors unless you import the certificate. If you need this functionality, you could consider the approach detailed in this answer to this related question on StackOverflow.
Example
String result = getUrlAsString("https://www.google.com");
System.out.println(result);
outputs
<!doctype html><html itemscope="" .... etc
Code
import java.net.URL;
import java.net.URLConnection;
import java.io.BufferedReader;
import java.io.InputStreamReader;
public static String getUrlAsString(String url)
{
try
{
URL urlObj = new URL(url);
URLConnection con = urlObj.openConnection();
con.setDoOutput(true); // we want the response
con.setRequestProperty("Cookie", "myCookie=test123");
con.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuilder response = new StringBuilder();
String inputLine;
String newLine = System.getProperty("line.separator");
while ((inputLine = in.readLine()) != null)
{
response.append(inputLine + newLine);
}
in.close();
return response.toString();
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}
Here's Jeanne's lovely answer, but wrapped in a tidy function for muppets like me:
private static String getUrl(String aUrl) throws MalformedURLException, IOException
{
String urlData = "";
URL urlObj = new URL(aUrl);
URLConnection conn = urlObj.openConnection();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8)))
{
urlData = reader.lines().collect(Collectors.joining("\n"));
}
return urlData;
}
URL to String in pure Java
Example call to get payload from http get call
String str = getStringFromUrl("YourUrl");
Implementation
You can use the method described in this answer, on How to read URL to an InputStream and combine it with this answer on How to read InputStream to String.
The outcome will be something like
public String getStringFromUrl(URL url) throws IOException {
return inputStreamToString(urlToInputStream(url,null));
}
public String inputStreamToString(InputStream inputStream) throws IOException {
try(ByteArrayOutputStream result = new ByteArrayOutputStream()) {
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
return result.toString(UTF_8);
}
}
private InputStream urlToInputStream(URL url, Map<String, String> args) {
HttpURLConnection con = null;
InputStream inputStream = null;
try {
con = (HttpURLConnection) url.openConnection();
con.setConnectTimeout(15000);
con.setReadTimeout(15000);
if (args != null) {
for (Entry<String, String> e : args.entrySet()) {
con.setRequestProperty(e.getKey(), e.getValue());
}
}
con.connect();
int responseCode = con.getResponseCode();
/* By default the connection will follow redirects. The following
* block is only entered if the implementation of HttpURLConnection
* does not perform the redirect. The exact behavior depends to
* the actual implementation (e.g. sun.net).
* !!! Attention: This block allows the connection to
* switch protocols (e.g. HTTP to HTTPS), which is <b>not</b>
* default behavior. See: https://stackoverflow.com/questions/1884230
* for more info!!!
*/
if (responseCode < 400 && responseCode > 299) {
String redirectUrl = con.getHeaderField("Location");
try {
URL newUrl = new URL(redirectUrl);
return urlToInputStream(newUrl, args);
} catch (MalformedURLException e) {
URL newUrl = new URL(url.getProtocol() + "://" + url.getHost() + redirectUrl);
return urlToInputStream(newUrl, args);
}
}
/*!!!!!*/
inputStream = con.getInputStream();
return inputStream;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
Pros
It is pure java
It can be easily enhanced by adding different headers as a map (instead of passing a null object, like the example above does), authentication, etc.
Handling of protocol switches is supported