Develop Reference

POST parameter not present

I'm a bit new to using API's and I've been trying to use a few for a while now but for some reason I always received a connect error with every implementation I used. I finally found a working POST here but I keep getting {"success":false,"error":"Required POST parameter 'value1' not present."}
Here's the code that I have:
package org.apache.http.examples;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.DefaultHttpClient;
public class APINew7 {
public static void main(String[] args) {
try {
DefaultHttpClient httpClient = new DefaultHttpClient();
HttpPost postRequest = new HttpPost(
"https://website.com");
StringEntity input = new StringEntity("{\"value1\":\"123\",\"value2\":\"456\"}");
input.setContentType("application/json");
postRequest.setEntity(input);
HttpResponse response = httpClient.execute(postRequest);
BufferedReader br = new BufferedReader(new InputStreamReader((response.getEntity().getContent())));
String output;
System.out.println("Output from Server .... \n");
while ((output = br.readLine()) != null) {
System.out.println(output);
}
httpClient.getConnectionManager().shutdown();
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
I'm fairly sure the problem lies here however I'm not sure what's wrong with it:
StringEntity input = new StringEntity("{\"value1\":\"123\",\"value2\":\"456\"}");
I've seen a few different methods of passing the data through but this implementation is the only one that's successfully run. So to sum it up, this does actually run but the parameters are not being spotted.
I appreciate any help, thank you.
Edit:
I may have found what the problem is, but I don't really know a way around it. One of the parameters being sent through is a semi colon. I have to send this through, no way around it. How would I go about sending a semi colon through? Example: "abc;123"

Well I managed to figure it out. Sending a semicolon with the method I used wasn't working but for whoever might have the same problem, using LinkedHashMap to create an object and just using variable.put("parameter", "value") works out.
Thanks for the suggestion #cody123

Redirect Client in Java-Run Server

I am creating a Java HTTP server that checks to make sure a client is not banned before redirecting to the main server. I have already created everything for the server that is needed, I just don't know how to redirect to another port that is running the main server. Here is my code:
package netlyaccesscontrol;
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;
public class AllowedCheck {
public static void main(String[] args) {
String line = null;
try {
FileReader reader = new FileReader("Banned.txt");
BufferedReader buffer = new BufferedReader(reader);
ServerSocket s = new ServerSocket(80);
Socket c = s.accept();
String clientIP = c.getInetAddress().toString();
while ((line = buffer.readLine()) != null) {
if (clientIP == line) {
s.close();
} else {
// redirect to main server here
}
}
} catch (FileNotFoundException ex) {
System.out.println("The banned IP address file does not exist.");
} catch (IOException ex) {
ex.printStackTrace();
}
}
}

The redirection that you are thinking of is something supported by HTTP and the browsers. There's a specific HTTP response code that tells the caller to redirect and a way to specify it.
Raw sockets are a low-level network protocol that is not going to support redirection as you expect. The most you might be able to do is have this program be a proxy and, upon success, push all incoming data/outgoing responses to/from the ultimate server. But what you have here is by no means going to cut it.

How Do I write a client using axis2 to send a serialized xml object to a web service?

I'm having a conceptual problem preventing me from solving a trivial problem. I need to send an object to a web service. I have an endpoint, and I have code that can serialize the object, so I can create an org.jdom.Document or a byte[] object containing the serialized object.
I can also create a client snippet that uses axis2 to invoke the web service.
Finally I have tried sending a manually created message to the web service (it has no WSDL ;( )
AND I have used Charles to see what is going out (the request).
What I don't know how to do is convert the byte[] or org.jdom.Document object to an OMElement object. Evidently the serviceClient.sendReceive(elem) takes an OMElement parameter.
Here is what I tried so far (I removed the OMElement that I sent out once I was convinced it was going out):
package testAxis2Client01;
import java.util.Map;
import javax.xml.stream.XMLStreamException;
import org.apache.axiom.om.OMAbstractFactory;
import org.apache.axiom.om.OMElement;
import org.apache.axiom.om.OMFactory;
import org.apache.axiom.om.OMNamespace;
import org.apache.axiom.soap.SOAPEnvelope;
import org.apache.axis2.addressing.EndpointReference;
import org.apache.axis2.client.Options;
import org.apache.axis2.client.ServiceClient;
import org.apache.axis2.transport.http.HTTPConstants;
public class testAxis2Client01 {
private static final int MXMOCONNECTIONTIMEOUT = 2;//don't really know what this should be.
/**
* #param args
*/
public static void main(String[] args) {
try {
callAxisWS();
} catch (XMLStreamException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
}
public static void callAxisWS() throws XMLStreamException, Exception {
//Axis2 client code to call a WS
OMElement response=null;
try{
OMFactory factory = OMAbstractFactory.getSOAP11Factory();
SOAPEnvelope theEnvelope = OMAbstractFactory.getSOAP12Factory().getDefaultEnvelope();
theEnvelope.declareNamespace("http://www.w3.org/2001/XMLSchema","xsd");
theEnvelope.declareNamespace("http://www.w3.org/2001/XMLSchema-instance","xsi");
ServiceClient serviceClient = new ServiceClient();
Options options = serviceClient.getOptions();
options.setProperty(HTTPConstants.AUTO_RELEASE_CONNECTION, true); // Another API to release connection.
options.setTimeOutInMilliSeconds(10000); // Setting the connection timeout.
EndpointReference targetEPR = new EndpointReference(theUrl);
options.setTo(targetEPR);
options.setAction("processDocument");
serviceClient.setOptions(options);
//response = serviceClient.sendReceive(myOMElement);
response = serviceClient.sendReceive(elem)
if (response != null) {
System.out.println("SUCCESS!!");
System.out.println(response.toStringWithConsume());
}
}catch(Exception af){
af.printStackTrace();
System.out.println(af.getMessage());
}
}
}

The point of using axis2 is that it takes care of everything. You only have to provide a wsdl file and it will generate client stubs.
If you do not have an original wsdl, you can still make one yourself.
The best way for you is to create the wsdl file manually, generate the client stub and call the stub directly.

Android Application that connects to remote server

I am trying to build an Android App on Platform 2.2 Froyo. The app is supposed to connect to a remote server, fetch data from it and display to user in a different language.
So my question - What technologies I need to learn so that I can
build the above app.
Note - I have already installed the Android platform and have built simple apps like Hello, world. I know Java. Also I am using Eclipse.
Thank you for your answers. No rude comment please...
//--------------Code for connecting to web using HTTP protocol-----------------//
package in.androidbook.Networking;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import android.app.Activity;
import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.os.AsyncTask;
import android.os.Bundle;
import android.widget.ImageView;
import android.widget.Toast;
public class MainActivity extends Activity
{
ImageView img;
/* This is for making asynchronous calls to ensure that connection to server will not return until data is received */
private class BackgroundTask extends AsyncTask<String, Void, Bitmap>
{
protected Bitmap doInBackground(String...url)
{
Bitmap bitmap = DownloadImage(url[0]);
return bitmap;
}
protected void onPostExecute(Bitmap bitmap)
{
ImageView img = (ImageView) findViewById(R.id.img);
img.setImageBitmap(bitmap);
}
}
// Code for making HTTP connection
private InputStream OpenHttpConnection(String urlString) throws IOException
{
InputStream in = null;
int response = -1;
URL url = new URL(urlString);//We take an object of class URL
URLConnection conn = url.openConnection(); //Create a connection object and open the connection
if(!(conn instanceof HttpURLConnection)) throw new IOException("Not an Http connection");
try
{
HttpURLConnection httpConn = (HttpURLConnection) conn; //httpConn object is assigned the value of conn. Typecasting is done to avoid conflict.
httpConn.setAllowUserInteraction(false);
httpConn.setInstanceFollowRedirects(true);
httpConn.setRequestMethod("GET");
httpConn.connect();
response = httpConn.getResponseCode();
if(response == HttpURLConnection.HTTP_OK)
in = httpConn.getInputStream();
}
catch (Exception ex)
{
throw new IOException("Error connecting");
}
return in;
}
//------------------------------------------ OpenHttpConnection method completed----------------------------------------------------------//
//----------------------------------------------------------------------------------------------------------------------------------------------------------------//
//-------------------------------Method to download an image--------------------------------------------------------------------------------------//
private Bitmap DownloadImage(String URL)
{
Bitmap bitmap = null;
InputStream in = null;
try
{
in = OpenHttpConnection(URL);
bitmap = BitmapFactory.decodeStream(in);
in.close();
}
catch(IOException e1)
{
Toast.makeText(this, e1.getLocalizedMessage(), Toast.LENGTH_LONG).show();
//Toast displays a short msg to user. this refers to current object.
e1.printStackTrace();
}
return bitmap;
}
/** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
Bitmap bitmap = DownloadImage("http://i.zdnet.com/blogs/3-29-androids.jpg");
img = (ImageView) findViewById(R.id.img);
img.setImageBitmap(bitmap);
}
}

UPDATED (based on comment) : As we are talking about the client side, I confirm my answer. If the site is not yours, the first thing you need to do if check how/if it allows for some kind of communication via API, and in what kind of format (XML and JSON being the most used). With this information, it should be pretty easy. Take a look of the Android example using the Google Map or Twitter, you should find some.
Well, it depends what do you mean exactly : are you asking for the skills needed on the client side (the app) - in the idea that the server is already built, or will be by someone else, or the skills needed for the server ?
In the former case, I would advice to communicate with REST API and JSON. Take a look at apache HTTPGet, HTTPClient and HTTPResponse class and org.json (all are included in Android). If you want to test with them, use some public APIs (so you do not have to worry about the server), such as Google Map API (which is simple and free to use under some limits).
In the latter case, I'm with ColWinters : if you know java, use it there also, with Tomcat as a server and a basic Servlet. You'll find examples aplenty on the Internet.

Look up these technologies,
Apache Tomcat - Java Server Pages (server processing)
MySQL (storage of data)
and thats it. Also make sure to do the request in a seperate thread like an Async Task from an activity.

If you know java, I would suggest servlet as service hosted on server which reads data from mysql or anydatabase and contructs as json. in your android app make http using in buit httpclient to remote servlet and parse json response. So. Servlets,Httpclient,Json covers most if your app is native to phone app.

Using Java to pull data from a webpage?

I'm attempting to make my first program in Java. The goal is to write a program that browses to a website and downloads a file for me. However, I don't know how to use Java to interact with the internet. Can anyone tell me what topics to look up/read about or recommend some good resources?

The simplest solution (without depending on any third-party library or platform) is to create a URL instance pointing to the web page / link you want to download, and read the content using streams.
For example:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
public class DownloadPage {
public static void main(String[] args) throws IOException {
// Make a URL to the web page
URL url = new URL("http://stackoverflow.com/questions/6159118/using-java-to-pull-data-from-a-webpage");
// Get the input stream through URL Connection
URLConnection con = url.openConnection();
InputStream is = con.getInputStream();
// Once you have the Input Stream, it's just plain old Java IO stuff.
// For this case, since you are interested in getting plain-text web page
// I'll use a reader and output the text content to System.out.
// For binary content, it's better to directly read the bytes from stream and write
// to the target file.
try(BufferedReader br = new BufferedReader(new InputStreamReader(is))) {
String line = null;
// read each line and write to System.out
while ((line = br.readLine()) != null) {
System.out.println(line);
}
}
}
}
Hope this helps.

The Basics
Look at these to build a solution more or less from scratch:
Start from the basics: The Java Tutorial's chapter on Networking, including Working With URLs
Make things easier for yourself: Apache HttpComponents (including HttpClient)
The Easily Glued-Up and Stitched-Up Stuff
You always have the option of calling external tools from Java using the exec() and similar methods. For instance, you could use wget, or cURL.
The Hardcore Stuff
Then if you want to go into more fully-fledged stuff, thankfully the need for automated web-testing as given us very practical tools for this. Look at:
HtmlUnit (powerful and simple)
Selenium, Selenium-RC
WebDriver/Selenium2 (still in the works)
JBehave with JBehave Web
Some other libs are purposefully written with web-scraping in mind:
JSoup
Jaunt
Some Workarounds
Java is a language, but also a platform, with many other languages running on it. Some of which integrate great syntactic sugar or libraries to easily build scrapers.
Check out:
Groovy (and its XmlSlurper)
or Scala (with great XML support as presented here and here)
If you know of a great library for Ruby (JRuby, with an article on scraping with JRuby and HtmlUnit) or Python (Jython) or you prefer these languages, then give their JVM ports a chance.
Some Supplements
Some other similar questions:
Scrape data from HTML using Java
Options for HTML Scraping

Here's my solution using URL and try with resources phrase to catch the exceptions.
/**
* Created by mona on 5/27/16.
*/
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
public class ReadFromWeb {
public static void readFromWeb(String webURL) throws IOException {
URL url = new URL(webURL);
InputStream is = url.openStream();
try( BufferedReader br = new BufferedReader(new InputStreamReader(is))) {
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
}
}
catch (MalformedURLException e) {
e.printStackTrace();
throw new MalformedURLException("URL is malformed!!");
}
catch (IOException e) {
e.printStackTrace();
throw new IOException();
}
}
public static void main(String[] args) throws IOException {
String url = "https://madison.craigslist.org/search/sub";
readFromWeb(url);
}
}
You could additionally save it to file based on your needs or parse it using XML or HTML libraries.

Since Java 11 the most convenient way it to use java.net.http.HttpClient from the standard library.
Example:
HttpClient client = HttpClient.newBuilder()
.version(Version.HTTP_1_1)
.followRedirects(Redirect.NORMAL)
.connectTimeout(Duration.ofSeconds(20))
.proxy(ProxySelector.of(new InetSocketAddress("proxy.example.com", 80)))
.authenticator(Authenticator.getDefault())
.build();
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create("httpss://foo.com/"))
.timeout(Duration.ofMinutes(2))
.GET()
.build();
HttpResponse<String> response = client.send(request, BodyHandlers.ofString());
System.out.println(response.statusCode());
System.out.println(response.body());

I use the following code for my API:
try {
URL url = new URL("https://stackoverflow.com/questions/6159118/using-java-to-pull-data-from-a-webpage");
InputStream content = url.openStream();
int c;
while ((c = content.read())!=-1) System.out.print((char) c);
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException ie) {
ie.printStackTrace();
}
You can catch the characters and convert them to string.