Develop Reference

How to create a hastebin with java

I need to create a hastebin paste in java, but I don't know how.
I tried this
public static String paste(String content) throws MalformedURLException, IOException {
URL url = new URL("https://hasteb.in/documents");
URLConnection con = url.openConnection();
HttpURLConnection http = (HttpURLConnection) con;
http.addRequestProperty("data", content);
http.setRequestMethod("POST");
http.setDoOutput(true);
InputStream in = new BufferedInputStream(http.getInputStream());
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder entirePage = new StringBuilder();
String inputLine;
while ((inputLine = reader.readLine()) != null) {
entirePage.append(inputLine);
}
reader.close();
if (!(entirePage.toString().contains("\"key\":\""))) {
return "UNKNOWN";
}
return "https://hasteb.in/"+entirePage.toString().split("\"key\":\"")[1].split("\",")[0];
}
But it does not work.
Error is Caused by: java.lang.IllegalArgumentException: Illegal character(s) in message header value
Any help?

To do a POST request using HttpURLConnection you need to write the request body using the provided OutputStream functionality rather than setting a HTTP header named "data":
try (OutputStream out = http.getOutputStream()) {
out.write(content.getBytes());
out.flush();
}
This needs to happen BEFORE you start reading from the InputStream (and I would suggest using try-with-resources for that as well).

Querying Data from SUGARCRM

I have SugarCRM trail account. I can able to get Authenticate and get the AccessToken by the following url.
https://xxxxxxx.trial.sugarcrm.eu/rest/v10/oauth2/token
Method : POST
POST Data : postData: { "grant_type":"password", "client_id":"sugar", "client_secret":"", "username":"admin", "password":"Admin123", "platform":"base" }
Code I used to get the AccessToken
public static String getAccessToken() throws JSONException {
HttpURLConnection connection = null;
JSONObject requestBody = new JSONObject();
requestBody.put("grant_type", "password");
requestBody.put("client_id", CLIENT_ID);
requestBody.put("client_secret", CLIENT_SECRET);
requestBody.put("username", USERNAME);
requestBody.put("password", PASSWORD);
requestBody.put("platform", "base");
try {
URL url = new URL(HOST_URL + AUTH_URL);
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setRequestProperty("Content-Type", "application/json");
connection.setUseCaches(false);
connection.setDoOutput(true);
connection.connect();
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(connection.getOutputStream()));
out.write(requestBody.toString());
out.close();
int responseCode = connection.getResponseCode();
BufferedReader in = new BufferedReader(
new InputStreamReader(connection.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
JSONObject jObject = new JSONObject(response.toString());
if(!jObject.has("access_token")){
return null;
}
String accessToken = jObject.getString("access_token");
return accessToken;
} catch (Exception e) {
e.printStackTrace();
return null;
}
}
Now I have retrive Leads from CRM using rest API I can not able to find the appropriate method and Url to do the thing.
I can see the list rest of API's from /help but I cant understand what should be my module name and what I have to :record and how do I pass my access token for authentication.
Can anyone please help me?

The module name is simply the module you which to fetch records from, so in your case you'll want to do a GET request to rest/v10/Leads for a list of Leads. If you want to fetch a specific Lead you replace :record with the id of a Lead - for example: GET rest/v10/Leads/LEAD-ID-HERE
SugarCRM's documentation has a lot of relevant information that might not be included in /help plus working examples.
http://support.sugarcrm.com/Documentation/Sugar_Developer/Sugar_Developer_Guide_7.8/Integration/Web_Services/v10/Endpoints/module_GET/
http://support.sugarcrm.com/Documentation/Sugar_Developer/Sugar_Developer_Guide_7.8/Integration/Web_Services/v10/Examples/PHP/How_to_Fetch_Related_Records/

You need to include your retrieved token into an OAuth-Token header for subsequent requests, and then just use the module name as the endpoint i.e. in your case: "rest/v10/Leads" and call the GET method to retrieve them. Try something akin to this:
String token = getAccessToken();
HttpURLConnection connection = null;
try {
URL url = new URL(HOST_URL + "/rest/v10/Leads");
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Content-Type", "application/json");
connection.setRequestProperty("OAuth-Token", token);
connection.setUseCaches(false);
connection.setDoOutput(true);
connection.connect();
int responseCode = connection.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
JSONObject jObject = new JSONObject(response.toString());
System.out.println(jObject);
} catch (Exception e) {
e.printStackTrace();
}
In the case you want to filter it down to specific id's to cut down on the amount of returned data, you can specify it after the module name i.e. "rest/v10/Leads/{Id}"

Java HTTPS Client [duplicate]

I'm trying to find Java's equivalent to Groovy's:
String content = "http://www.google.com".toURL().getText();
I want to read content from a URL into string. I don't want to pollute my code with buffered streams and loops for such a simple task. I looked into apache's HttpClient but I also don't see a one or two line implementation.

Now that some time has passed since the original answer was accepted, there's a better approach:
String out = new Scanner(new URL("http://www.google.com").openStream(), "UTF-8").useDelimiter("\\A").next();
If you want a slightly fuller implementation, which is not a single line, do this:
public static String readStringFromURL(String requestURL) throws IOException
{
try (Scanner scanner = new Scanner(new URL(requestURL).openStream(),
StandardCharsets.UTF_8.toString()))
{
scanner.useDelimiter("\\A");
return scanner.hasNext() ? scanner.next() : "";
}
}

This answer refers to an older version of Java. You may want to look at ccleve's answer.
Here is the traditional way to do this:
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static String getText(String url) throws Exception {
URL website = new URL(url);
URLConnection connection = website.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(
connection.getInputStream()));
StringBuilder response = new StringBuilder();
String inputLine;
while ((inputLine = in.readLine()) != null)
response.append(inputLine);
in.close();
return response.toString();
}
public static void main(String[] args) throws Exception {
String content = URLConnectionReader.getText(args[0]);
System.out.println(content);
}
}
As #extraneon has suggested, ioutils allows you to do this in a very eloquent way that's still in the Java spirit:
InputStream in = new URL( "http://jakarta.apache.org" ).openStream();
try {
System.out.println( IOUtils.toString( in ) );
} finally {
IOUtils.closeQuietly(in);
}

Or just use Apache Commons IOUtils.toString(URL url), or the variant that also accepts an encoding parameter.

There's an even better way as of Java 9:
URL u = new URL("http://www.example.com/");
try (InputStream in = u.openStream()) {
return new String(in.readAllBytes(), StandardCharsets.UTF_8);
}
Like the original groovy example, this assumes that the content is UTF-8 encoded. (If you need something more clever than that, you need to create a URLConnection and use it to figure out the encoding.)

Now that more time has passed, here's a way to do it in Java 8:
URLConnection conn = url.openConnection();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8))) {
pageText = reader.lines().collect(Collectors.joining("\n"));
}

Additional example using Guava:
URL xmlData = ...
String data = Resources.toString(xmlData, Charsets.UTF_8);

Java 11+:
URI uri = URI.create("http://www.google.com");
HttpRequest request = HttpRequest.newBuilder(uri).build();
String content = HttpClient.newHttpClient().send(request, BodyHandlers.ofString()).body();

If you have the input stream (see Joe's answer) also consider ioutils.toString( inputstream ).
http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toString(java.io.InputStream)

The following works with Java 7/8, secure urls, and shows how to add a cookie to your request as well. Note this is mostly a direct copy of this other great answer on this page, but added the cookie example, and clarification in that it works with secure urls as well ;-)
If you need to connect to a server with an invalid certificate or self signed certificate, this will throw security errors unless you import the certificate. If you need this functionality, you could consider the approach detailed in this answer to this related question on StackOverflow.
Example
String result = getUrlAsString("https://www.google.com");
System.out.println(result);
outputs
<!doctype html><html itemscope="" .... etc
Code
import java.net.URL;
import java.net.URLConnection;
import java.io.BufferedReader;
import java.io.InputStreamReader;
public static String getUrlAsString(String url)
{
try
{
URL urlObj = new URL(url);
URLConnection con = urlObj.openConnection();
con.setDoOutput(true); // we want the response
con.setRequestProperty("Cookie", "myCookie=test123");
con.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
StringBuilder response = new StringBuilder();
String inputLine;
String newLine = System.getProperty("line.separator");
while ((inputLine = in.readLine()) != null)
{
response.append(inputLine + newLine);
}
in.close();
return response.toString();
}
catch (Exception e)
{
throw new RuntimeException(e);
}
}

Here's Jeanne's lovely answer, but wrapped in a tidy function for muppets like me:
private static String getUrl(String aUrl) throws MalformedURLException, IOException
{
String urlData = "";
URL urlObj = new URL(aUrl);
URLConnection conn = urlObj.openConnection();
try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream(), StandardCharsets.UTF_8)))
{
urlData = reader.lines().collect(Collectors.joining("\n"));
}
return urlData;
}

URL to String in pure Java
Example call to get payload from http get call
String str = getStringFromUrl("YourUrl");
Implementation
You can use the method described in this answer, on How to read URL to an InputStream and combine it with this answer on How to read InputStream to String.
The outcome will be something like
public String getStringFromUrl(URL url) throws IOException {
return inputStreamToString(urlToInputStream(url,null));
}
public String inputStreamToString(InputStream inputStream) throws IOException {
try(ByteArrayOutputStream result = new ByteArrayOutputStream()) {
byte[] buffer = new byte[1024];
int length;
while ((length = inputStream.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
return result.toString(UTF_8);
}
}
private InputStream urlToInputStream(URL url, Map<String, String> args) {
HttpURLConnection con = null;
InputStream inputStream = null;
try {
con = (HttpURLConnection) url.openConnection();
con.setConnectTimeout(15000);
con.setReadTimeout(15000);
if (args != null) {
for (Entry<String, String> e : args.entrySet()) {
con.setRequestProperty(e.getKey(), e.getValue());
}
}
con.connect();
int responseCode = con.getResponseCode();
/* By default the connection will follow redirects. The following
* block is only entered if the implementation of HttpURLConnection
* does not perform the redirect. The exact behavior depends to
* the actual implementation (e.g. sun.net).
* !!! Attention: This block allows the connection to
* switch protocols (e.g. HTTP to HTTPS), which is <b>not</b>
* default behavior. See: https://stackoverflow.com/questions/1884230
* for more info!!!
*/
if (responseCode < 400 && responseCode > 299) {
String redirectUrl = con.getHeaderField("Location");
try {
URL newUrl = new URL(redirectUrl);
return urlToInputStream(newUrl, args);
} catch (MalformedURLException e) {
URL newUrl = new URL(url.getProtocol() + "://" + url.getHost() + redirectUrl);
return urlToInputStream(newUrl, args);
}
}
/*!!!!!*/
inputStream = con.getInputStream();
return inputStream;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
Pros
It is pure java
It can be easily enhanced by adding different headers as a map (instead of passing a null object, like the example above does), authentication, etc.
Handling of protocol switches is supported

Why do I keep getting java.net.ConnectException in my Android application?

I have recently refactored some of my networking code in my android app to use the Google recommended HTTPUrlConnection. Previously, I was using Apache's HTTPClient. I'm not sure if either of those things are relevant.
The other thing I have recently done to my networking code is use AsyncTask for the calls. Previously I was just doing work on the main thread (obviously bad) and so my application would appear to hang when fetching data. The thing is, since switching to AsyncTask, I have pretty regularly experienced timeout errors.
Why do I get this timeout error?
java.net.ConnectException: failed to connect to <url> (port 80): connect failed: ETIMEDOUT (connection timed out)
Here is my AsyncTask that makes the network call.
private class PostToPHP extends AsyncTask<PostToPHP, Void, String>
{
private String functionName;
private ArrayList<NameValuePair> postKeyValuePairs;
public PostToPHP(String function, ArrayList<NameValuePair> keyValuePairs)
{
functionName= function;
postKeyValuePairs = keyValuePairs;
}
#Override
protected void onPreExecute()
{
progressDialog = ProgressDialog.show(BaseActivity.getInstance(), "Loading", "Please wait...", true, false);
}
#Override
protected String doInBackground(PostToPHP... params)
{
ArrayList<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>();
nameValuePairs.add(new BasicNameValuePair(FUNCTION_NAME, functionName));
for (int i = 0; i < postKeyValuePairs.size(); i++) {
nameValuePairs.add(postKeyValuePairs.get(i));
}
try {
URL url = new URL("www.myurl");
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setDoOutput(true);
urlConnection.setRequestProperty("Accept-Charset", iso-8859-1);
OutputStream os = urlConnection.getOutputStream();
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, iso-8859-1));
writer.write(getEncodedPostParameters(nameValuePairs, iso-8859-1));
writer.close();
os.close();
InputStream is = urlConnection.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(is, iso-8859-1));
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
urlConnection.disconnect();
return sb.toString();
}
catch (UnknownHostException e) {
// handle it
return null;
}
catch (Exception e) {
// this is where I'm always getting a timeout exception
return null;
}
}
#Override
protected void onPostExecute(String result)
{
progressDialog.dismiss();
}
}
EDIT: I thought this only occurred during one particular network call but I've now experienced it at various places.
EDIT 2: Looking at this again I found this and I think that's more of my problem. This didn't exist before I implemented Android AsyncTask for the network calls. I think that AsyncTask is somehow screwing up with threading and causing the timeouts.

You could explicitly set the timeout for the HTTPUrlConnection:
urlConnection.setConnectionTimeout(aLongerTimeout)

I'm not very satisfied with this answer, but it seems to be working. I think there must be an issue with my code, or a bug in HTTPUrlConnection of some sort. I know Google said it had bugs in Froyo and lower, but I'm seeing oddities in 2.3.4 and 4.2 which I'm testing with. Essentially I replaced my code above with Apache's code and I'm not seeing the timeouts.
private static String post(ArrayList<NameValuePair> params){
HttpClient httpclient = new DefaultHttpClient();
HttpPost httpreq = new HttpPost("www.url.com");
httpreq.setEntity(new UrlEncodedFormEntity(params));
HttpResponse response = httpclient.execute(httpreq);
HttpEntity entity = response.getEntity();
InputStream is = entity.getContent();
BufferedReader reader = new BufferedReader(new InputStreamReader(is, NetworkConstants.HTTP_ACCEPTED_CHARSET), 8);
StringBuilder sb = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
is.close();
return sb.toString();
}

Using HTTP Post between 2 Java servlets

I have setup a java servlet which accepts parameters from the URL and have it working properly:
public class GetThem extends HttpServlet {
public void doGet(HttpServletRequest request, HttpServletResponse response)
throws IOException, ServletException
{
try {
double lat=Double.parseDouble(request.getParameter("lat"));
double lon=Double.parseDouble(request.getParameter("lon"));
response.setContentType("text/html");
PrintWriter out = response.getWriter();
out.println(lat + " and " + lon);
} catch (Exception e) {
e.printStackTrace();
}
}
}
So visiting this link:
http://www.example.com:8080/HttpPost/HttpPost?lat=1&lon=2 would output:
"1.0 and 2.0"
I'm currently calling it from another java program using this code:
try{
URL objectGet = new URL("http://www.example.com:8080/HttpPost/HttpPost?lat=" + Double.toString(dg.getLatDouble()) + "&lon=" + Double.toString(dg.getLonDouble()));
URLConnection yc = objectGet.openConnection();
BufferedReader in = new BufferedReader(
new InputStreamReader(
yc.getInputStream()));
in = new BufferedReader(
new InputStreamReader(
yc.getInputStream()));
...
Now I want to change it so that I'm not using the URL parameters to pass this data to the server. I want to send much larger messages to this server. I am aware that I need to use http post rather than http get to achieve this but am not sure how to do it.
Do I need to change anything on the server side which is receiving data? What do I need to do on the client side which is posting this data?
Any help would be greatly appreciate thanks.
Ideally I'd like to send this data in JSON format.

Below there's sample from the first link found by "java HTTP POST example" in google.
try {
// Construct data
StringBuilder dataBuilder = new StringBuilder();
dataBuilder.append(URLEncoder.encode("key1", "UTF-8")).append('=').append(URLEncoder.encode("value1", "UTF-8")).
append(URLEncoder.encode("key2", "UTF-8")).append('=').append(URLEncoder.encode("value2", "UTF-8"));
// Send data
URL url = new URL("http://hostname:80/cgi");
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(dataBuilder.toString());
wr.flush();
// Get the response
BufferedReader rd = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
// Process line...
}
wr.close();
rd.close();
} catch (Exception e) {
}

I think you should be using HTTPClient instead of handling connections and streams. Check http://hc.apache.org/httpclient-3.x/tutorial.html