Develop Reference

Encoding url in java avoiding special characters

I need to pass the encoded url but it needs to avoid special characters as well. So how do I encode it? None of the answers on stackoverflow worked for me. Can any one help? I want to do it in java
public String tweet_quote_func(Map attrs, FilterParam param) {
String url = param.getShorturi();
String text = attrs.get("display");
if (url != null && text != null) {
try {
text = StringEscapeUtils.unescapeHtml(text); // for double quotes
String encodedurl = "https://twitter.com/intent/tweet?url="+URLEncoder.encode(url, "UTF-8");
encodedurl = encodedurl + "&text=" + StringUtil.escapeUrl(text);
return "<span class=\"tweet_quote\"> " + text.trim() + "<span></span></span>";
} catch (UnsupportedEncodingException e) {
System.err.println(e);
return null;
}
} else
return null;
}

You should use URLEncoder for all URL query argument names and their values. Not StringUtils.escapeURL(), whatever that is.
I don't know what you think is different about 'the special characters ".", "-", "*", and "_"', but URLEncoder is defined to do the right thing.
For the URL paths themselves you should use new URI(null, path, null).toASCIIString().

Multithreading (Stateless Classes)

Apologies for the long code post but am wondering if someone can help with a multithreading question (I am quite new to multi-threading). I am trying to design a facade class to a RESTFUL web services API that can be shared with multiple threads. I am using HttpURLConnection to do the connection and Google GSON to convert to and from JSON data.
The below class is what I have so far. In this example it has one public method to make an API call (authenticateCustomer()) and the private methods are used to facilitate the API call (i.e to build the POST data string, make a POST request etc).
I make one instance of this class and share it with 1000 threads. The threads call the authenticateCustomer() method. Most of the threads work but there is some threads that get a null pointer exception which is because I haven't implemented any synchronization. If I make the authenticateCustomer() method 'synchronized' it works. The problem is this results in poor concurrency (say, for example, the POST request suddenly takes a long time to complete, this will then hold up all the other threads).
Now to my question. Is the below class not stateless and therefore thread-safe? The very few fields that are in the class are declared final and assigned in the constructor. All of the methods use local variables. The Gson object is stateless (according to their web site) and created as a local variable in the API method anyway.
public final class QuizSyncAPIFacade
{
// API Connection Details
private final String m_apiDomain;
private final String m_apiContentType;
private final int m_bufferSize;
// Constructors
public QuizSyncAPIFacade()
{
m_apiDomain = "http://*****************************";
m_apiContentType = ".json";
m_bufferSize = 8192; // 8k
}
private String readInputStream(InputStream stream) throws IOException
{
// Create a buffer for the input stream
byte[] buffer = new byte[m_bufferSize];
int readCount;
StringBuilder builder = new StringBuilder();
while ((readCount = stream.read(buffer)) > -1) {
builder.append(new String(buffer, 0, readCount));
}
return builder.toString();
}
private String buildPostData(HashMap<String,String> postData) throws UnsupportedEncodingException
{
String data = "";
for (Map.Entry<String,String> entry : postData.entrySet())
{
data += (URLEncoder.encode(entry.getKey(), "UTF-8") + "=" + URLEncoder.encode(entry.getValue(), "UTF-8") + "&");
}
// Trim the last character (a trailing ampersand)
int length = data.length();
if (length > 0) {
data = data.substring(0, (length - 1));
}
return data;
}
private String buildJSONError(String message, String name, String at)
{
String error = "{\"errors\":[{\"message\":\"" + message + "\",\"name\":\"" + name + "\",\"at\":\"" + at + "\"}]}";
return error;
}
private String callPost(String url, HashMap<String,String> postData) throws IOException
{
// Set up the URL for the API call
URL apiUrl = new URL(url);
// Build the post data
String data = buildPostData(postData);
// Call the API action
HttpURLConnection conn;
try {
conn = (HttpURLConnection)apiUrl.openConnection();
} catch (IOException e) {
throw new IOException(buildJSONError("Failed to open a connection.", "CONNECTION_FAILURE", ""));
}
// Set connection parameters for posting data
conn.setRequestMethod("POST");
conn.setUseCaches(false);
conn.setDoInput(true);
conn.setDoOutput(true);
// Write post data
try {
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(data);
wr.flush();
wr.close();
} catch (IOException e) {
throw new IOException(buildJSONError("Failed to post data in output stream (Connection OK?).", "POST_DATA_FAILURE", ""));
}
// Read the response from the server
InputStream is;
try {
is = conn.getInputStream();
} catch (IOException e) {
InputStream errStr = conn.getErrorStream();
if (errStr != null)
{
String errResponse = readInputStream(errStr);
throw new IOException(errResponse);
}
else
{
throw new IOException(buildJSONError("Failed to read error stream (Connection OK?).", "ERROR_STREAM_FAILURE", ""));
}
}
// Read and return response from the server
return readInputStream(is);
}
/* -------------------------------------
*
* Synchronous API calls
*
------------------------------------- */
public APIResponse<CustomerAuthentication> authenticateCustomer(HashMap<String,String> postData)
{
// Set the URL for this API call
String apiURL = m_apiDomain + "/customer/authenticate" + m_apiContentType;
Gson jsonConv = new Gson();
String apiResponse = "";
try
{
// Call the API action
apiResponse = callPost(apiURL, postData);
// Convert JSON response to the required object type
CustomerAuthentication customerAuth = jsonConv.fromJson(apiResponse, CustomerAuthentication.class);
// Build and return the API response object
APIResponse<CustomerAuthentication> result = new APIResponse<CustomerAuthentication>(true, customerAuth, null);
return result;
}
catch (IOException e)
{
// Build and return the API response object for a failure with error list
APIErrorList errorList = jsonConv.fromJson(e.getMessage(), APIErrorList.class);
APIResponse<CustomerAuthentication> result = new APIResponse<CustomerAuthentication>(false, null, errorList);
return result;
}
}
}

If you are getting an error it could be because you are overloading the authentication service (something which doesn't happen if you do this one at a time) Perhaps it returning a error like 500, 503 or 504 which you could be ignoring and getting nothing you expect back, you return null http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
I would use less threads assuming you don't have 1000 cpus, its possible having this many threads will be slower rather than more efficeint.
I would also check that your service is returning correctly every time and investigate why you get a null value.
If your service can only handle say 20 requests at once, you can try using a Semaphore as a last resort. This can be using to limit the numebr of concurrent requests.

Any stateless class is inherently threadsafe, provided that the objects it accesses are either private to the thread, or threadsafe themselves.

How do you access posted form data in a servlet?

If I have a servlet running JVM1.4.2, and it is receiving a POST request with form data fields. I use req.getParameterNames() to get, what I would expect, all the query string and form data. However, all I ever get are the querystring parameters.
Literature I am reading from various sources says that getParameterNames() and getParameterValues(String) should be the way to get all query string and posted form data sent by the browser for JDK 1.4. Here is the method I use to extract all the parameters, which I expect would include posted form data :
public Map getParameterMap(HttpServletRequest req) {
Map params= new HashMap();
String name = null;
System.out.println("<< Getting Parameter Map.>>");
Enumeration enumParams = req.getParameterNames();
for (; enumParams.hasMoreElements(); ) {
// Get the name of the request parameter
name = (String)enumParams.nextElement();
// Get the value of the request parameters
// If the request parameter can appear more than once
// in the query string, get all values
String[] values = req.getParameterValues(name);
params.put(name, values);
String sValues = "";
for(int i=0;i<values.length;i++){
if(0<i) {
sValues+=",";
}
sValues +=values[i];
}
System.out.println("Param " + name + ": " + sValues);
}
System.out.println("<< END >>");
return params;
}
This question also agrees with my expectations, but the servlet is not picking up the form data. Obviously I am missing something....
Update: The post data is very straight forward and is not a Multipart form or rich media. Just plain'ol text submitted via an AJAX POST that looks like this in post body
c1=Value%20A&c2=Value%20B&c3=Value%20C

I managed to identify the problem. Because there is so much chatter from JDK 1.5+ and talk of getParameterMaps() method for 1.5, info on how 1.4 handles form post data was scarce and ambiguous. (Please post a comment if you find something that is specific for 1.4).
Pre-1.5 you have to manually get the form data via getInputStream, and then parse it out. I found this method, (posted below), from the java sun site that does a nice job using a Hashtable. I had to make a minor mod for deprecated methods. But seems to work quite robustly, "out of the box", so you should able to just cut-n-paste. I know it's "old tech" but I thought it worthwhile for those who may be in the same situation as me who are stuck on solving (what seems to be) straight forward problems.
public Hashtable parsePostData(int length, ServletInputStream instream) {
String valArray[] = null;
int inputLen, offset;
byte[] postedBytes = null;
boolean dataRemaining=true;
String postedBody;
Hashtable ht = new Hashtable();
//Vector paramOrder = new Vector(10);
StringBuffer sb = new StringBuffer();
if (length <=0) {
return null;
}
postedBytes = new byte[length];
try {
offset = 0;
while(dataRemaining) {
inputLen = instream.read (postedBytes, offset, length - offset);
if (inputLen <= 0) {
throw new IOException ("read error");
}
offset += inputLen;
if((length-offset) ==0) {
dataRemaining=false;
}
}
} catch (IOException e) {
System.out.println("Exception ="+e);
return null;
}
postedBody = new String (postedBytes);
StringTokenizer st = new StringTokenizer(postedBody, "&");
String key=null;
String val=null;
while (st.hasMoreTokens()) {
String pair = (String)st.nextToken();
int pos = pair.indexOf('=');
if (pos == -1) {
throw new IllegalArgumentException();
}
try {
key = URLDecoder.decode(pair.substring(0, pos),"UTF8");
val = java.net.URLDecoder.decode(pair.substring(pos+1,pair.length()),"UTF8");
} catch (Exception e) {
throw new IllegalArgumentException();
}
if (ht.containsKey(key)) {
String oldVals[] = (String []) ht.get(key);
valArray = new String[oldVals.length + 1];
for (int i = 0; i < oldVals.length; i++) {
valArray[i] = oldVals[i];
}
valArray[oldVals.length] = val;
} else {
valArray = new String[1];
valArray[0] = val;
}
ht.put(key, valArray);
String sValues = "";
for(int i=0;i<valArray.length;i++) {
if (0<i) {
sValues+=",";
}
sValues = valArray[i];
}
System.out.println("Form data field " + key + ":" +sValues);
//paramOrder.addElement(key);
}
return ht;
}

That's true. The getParameterNames(), getParameterValues(), and getParameter() methods are the way to access form data unless it's a multipart form, in which case you'll have to use something like Commons Fileupload to parse the multipart request before all the parameters are accessible to you.
Edit: You're probably not encoding the POST data properly in your AJAX call. POST data must carry a Content-Type of application/x-www-form-urlencoded or else multipart/form-data. If you're sending it as something else, it doesn't qualify as a request parameter, and I expect you'd see the behavior you're describing. The solution you've engineered essentially consists of setting up custom parsing of custom content.

JAX-RS #PathParam How to pass a String with slashes, hyphens & equals too

I am new to JAX-RS and I am trying to use Jersey to build a simple RESTful Webservice.
I have 2 questions. Please clarify these:
I am trying to have my simple webservice like this URL http://localhost:8080/SampleJersey/rest/inchi/InChIName
The InChIName is a string like this InChI=1S/C9H8O4/c1-6(10)13-8-5-3-2-4-7(8)9(11)12/h2- 5H,1H3,(H,11,12). How do I pass this as a #PathParam, I mean a normal String is working fine but here there are slashes,hyphens, and commas. How do I make it to ignore these. I tried putting it in quotes, but that didnt work. How should I do this?
I need to pass that InChI to another webservice and that returns an XML as an output and I want to display that XML output as my Webservice's output. If I have #Produces("application/xml") will it work?
This is my code:
#Path("/inchi")
public class InChIto3D {
#GET
#Path("{inchiname}")
#Produces("application/xml")
public String get3DCoordinates(#PathParam("inchiname")String inchiName) {
String ne="";
try{
URL eutilsurl = new URL(
"http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?"
+ "db=pccompound&term=%22"+inchiName+"%22[inchi]");
BufferedReader in = new BufferedReader(
new InputStreamReader(eutilsurl.openStream()));
String inputline;
while ((inputline=in.readLine())!=null)
ne=ne+inputline;
}catch (MalformedURLException e1) {
}catch (IOException e2){
}
return ne;
}
}

This is how you enable slashes in path params:
#Path("{inchiname : .+}")
public String get3DCoordinates(#PathParam("inchiname")String inchiName)

Tomcat does not Accept %2F in URLs: http://tomcat.apache.org/security-6.html. You can turn off this behavior.

The following should work:
#GET
#Path("{inchiname : (.+)?}")
#Produces("application/xml")
public String get3DCoordinates(#PathParam("inchiname")String inchiName) {
(This is sort of mentioned in another answer and comment, I'm just explicitly putting it in a separate answer to make it clear)

The parameters should be URL encoded. You can use java.net.URLEncoder for this.
String encodedParam = URLEncoder.encode(unencodedParam, "UTF-8");
The / will then be translated to %2F.

I got this to work using #QueryParam() rather than #PathParam.
#Path("/inchi")
public class InChIto3D {
#GET
//#Path("{inchiname}")
#Produces("application/xml")
public String get3DCoordinates(#QueryParam("inchiname") String inchiName) {
String ne="";
try{
URL eutilsurl = new URL(
"http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?"
+ "db=pccompound&term=%22"+inchiName+"%22[inchi]");
BufferedReader in = new BufferedReader(
new InputStreamReader(eutilsurl.openStream()));
String inputline;
while ((inputline=in.readLine())!=null)
ne=ne+inputline;
}catch (MalformedURLException e1) {
}catch (IOException e2){
}
return ne;
}
}
Thus the URL structure would be like this http://myrestservice.com/rest/inchi?inchiname=InChIhere
With #PathParam I read in the API that it will not accept slashes. I am wondering if I can use any Regular Expression in #Path just to ignore all slashes in the string that would be entered in quotes" ".

How to deal with the URISyntaxException

I got this error message :
java.net.URISyntaxException: Illegal character in query at index 31: http://finance.yahoo.com/q/h?s=^IXIC
My_Url = http://finance.yahoo.com/q/h?s=^IXIC
When I copied it into a browser address field, it showed the correct page, it's a valid URL, but I can't parse it with this: new URI(My_Url)
I tried : My_Url=My_Url.replace("^","\\^"), but
It won't be the url I need
It doesn't work either
How to handle this ?
Frank

You need to encode the URI to replace illegal characters with legal encoded characters. If you first make a URL (so you don't have to do the parsing yourself) and then make a URI using the five-argument constructor, then the constructor will do the encoding for you.
import java.net.*;
public class Test {
public static void main(String[] args) {
String myURL = "http://finance.yahoo.com/q/h?s=^IXIC";
try {
URL url = new URL(myURL);
String nullFragment = null;
URI uri = new URI(url.getProtocol(), url.getHost(), url.getPath(), url.getQuery(), nullFragment);
System.out.println("URI " + uri.toString() + " is OK");
} catch (MalformedURLException e) {
System.out.println("URL " + myURL + " is a malformed URL");
} catch (URISyntaxException e) {
System.out.println("URI " + myURL + " is a malformed URL");
}
}
}

Use % encoding for the ^ character, viz. http://finance.yahoo.com/q/h?s=%5EIXIC

You have to encode your parameters.
Something like this will do:
import java.net.*;
import java.io.*;
public class EncodeParameter {
public static void main( String [] args ) throws URISyntaxException ,
UnsupportedEncodingException {
String myQuery = "^IXIC";
URI uri = new URI( String.format(
"http://finance.yahoo.com/q/h?s=%s",
URLEncoder.encode( myQuery , "UTF8" ) ) );
System.out.println( uri );
}
}
http://java.sun.com/javase/6/docs/api/java/net/URLEncoder.html

Rather than encoding the URL beforehand you can do the following
String link = "http://example.com";
URL url = null;
URI uri = null;
try {
url = new URL(link);
} catch(MalformedURLException e) {
e.printStackTrace();
}
try{
uri = new URI(url.toString())
} catch(URISyntaxException e {
try {
uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(),
url.getPort(), url.getPath(), url.getQuery(),
url.getRef());
} catch(URISyntaxException e1 {
e1.printStackTrace();
}
}
try {
url = uri.toURL()
} catch(MalfomedURLException e) {
e.printStackTrace();
}
String encodedLink = url.toString();

A general solution requires parsing the URL into a RFC 2396 compliant URI (note that this is an old version of the URI standard, which java.net.URI uses).
I have written a Java URL parsing library that makes this possible: galimatias. With this library, you can achieve your desired behaviour with this code:
String urlString = //...
URLParsingSettings settings = URLParsingSettings.create()
.withStandard(URLParsingSettings.Standard.RFC_2396);
URL url = URL.parse(settings, urlString);
Note that galimatias is in a very early stage and some features are experimental, but it is already quite solid for this use case.

A space is encoded to %20 in URLs, and to + in forms submitted data (content type application/x-www-form-urlencoded). You need the former.
Using Guava:
dependencies {
compile 'com.google.guava:guava:28.1-jre'
}
You can use UrlEscapers:
String encodedString = UrlEscapers.urlFragmentEscaper().escape(inputString);
Don't use String.replace, this would only encode the space. Use a library instead.

Coudn't imagine nothing better for
http://server.ru:8080/template/get?type=mail&format=html&key=ecm_task_assignment&label=Согласовать с контрагентом&descr=Описание&objectid=2231
that:
public static boolean checkForExternal(String str) {
int length = str.length();
for (int i = 0; i < length; i++) {
if (str.charAt(i) > 0x7F) {
return true;
}
}
return false;
}
private static final Pattern COLON = Pattern.compile("%3A", Pattern.LITERAL);
private static final Pattern SLASH = Pattern.compile("%2F", Pattern.LITERAL);
private static final Pattern QUEST_MARK = Pattern.compile("%3F", Pattern.LITERAL);
private static final Pattern EQUAL = Pattern.compile("%3D", Pattern.LITERAL);
private static final Pattern AMP = Pattern.compile("%26", Pattern.LITERAL);
public static String encodeUrl(String url) {
if (checkForExternal(url)) {
try {
String value = URLEncoder.encode(url, "UTF-8");
value = COLON.matcher(value).replaceAll(":");
value = SLASH.matcher(value).replaceAll("/");
value = QUEST_MARK.matcher(value).replaceAll("?");
value = EQUAL.matcher(value).replaceAll("=");
return AMP.matcher(value).replaceAll("&");
} catch (UnsupportedEncodingException e) {
throw LOGGER.getIllegalStateException(e);
}
} else {
return url;
}
}

I had this exception in the case of a test for checking some actual accessed URLs by users.
And the URLs are sometime contains an illegal-character and hang by this error.
So I make a function to encode only the characters in the URL string like this.
String encodeIllegalChar(String uriStr,String enc)
throws URISyntaxException,UnsupportedEncodingException {
String _uriStr = uriStr;
int retryCount = 17;
while(true){
try{
new URI(_uriStr);
break;
}catch(URISyntaxException e){
String reason = e.getReason();
if(reason == null ||
!(
reason.contains("in path") ||
reason.contains("in query") ||
reason.contains("in fragment")
)
){
throw e;
}
if(0 > retryCount--){
throw e;
}
String input = e.getInput();
int idx = e.getIndex();
String illChar = String.valueOf(input.charAt(idx));
_uriStr = input.replace(illChar,URLEncoder.encode(illChar,enc));
}
}
return _uriStr;
}
test:
String q = "\\'|&`^\"<>)(}{][";
String url = "http://test.com/?q=" + q + "#" + q;
String eic = encodeIllegalChar(url,'UTF-8');
System.out.println(String.format(" original:%s",url));
System.out.println(String.format(" encoded:%s",eic));
System.out.println(String.format(" uri-obj:%s",new URI(eic)));
System.out.println(String.format("re-decoded:%s",URLDecoder.decode(eic)));

If you're using RestangularV2 to post to a spring controller in java you can get this exception if you use RestangularV2.one() instead of RestangularV2.all()

Replace spaces in URL with + like If url contains dimension1=Incontinence Liners then replace it with dimension1=Incontinence+Liners.