Develop Reference

Java How to Normalise a URL and Remove Fragment

How to normalise a URL in Java to remove the fragment. I.e. from https://www.website.com#something to https://www.website.com
This is possible with the URL.Normalize code, although in this specific use case I've only got a full absolute URL which needs to remain intact.
I'd like to be able to modify this code slightly to remove the fragment from the URL;
//The website below is just an example. In reality, this URL is unknown and could be anything. Both with and without a fragment depending on the use case
URL absUrl = new URL("https://www.website.com#something");
My thoughts so far is that this is only going to be possible by breaking down the URL into the Protocol + Domain + Path then joining it all back together which does appear to work, but there must be a more elegant way of doing this.

Fragment removal is fairly simple using the conversion methods toURI and toURL. So to convert a URL to a URI:
URL url = /*what have you*/ …
URI u = url.toURI();
To remove any fragment from the URI:
if( u.getFragment() != null ) { // Remake with same parts, less the fragment:
u = new URI( u.getScheme(), u.getSchemeSpecificPart(), /*fragment*/null ); }
In reconstructing a URI from its parts like that, it’s important to use the decoded getters (as shown), not the corresponding raw ones. For authority on this usage, see e.g. the Identity section of the API.
To convert the result back to a URL:
url = u.toURL();

Fragments do not exist as a separate entity in Java URLs. But you can convert a URL into a URI and back to remove a fragment. I did it like this:
URL url;
...
if (url.toString().contains("#")) {
URI uri = null;
try {
uri = new URI(url.getProtocol(), url.getHost(), url.getPath(), null);
String file = "";
if (uri.getPath() != null) {
file += uri.getPath();
}
if (uri.getQuery() != null) {
file += uri.getQuery();
}
url = new URL(uri.getScheme(), uri.getHost(), uri.getPort(), file);
} catch (URISyntaxException e) {
...
} catch (MalformedURLException e) {
...
}
}

How to know whether a string path is Web URL or a File based

I have a text field to acquire location information (String type) from User. It could be file directory based (e.g. C:\directory) or Web url (e.g. http://localhost:8008/resouces). The system will read some predetermined metadata file from the location.
Given the input string, how can I detect the nature of the path location whether it is a file based or Web URL effectively.
So far I have tried.
URL url = new URL(location); // will get MalformedURLException if it is a file based.
url.getProtocol().equalsIgnoreCase("http");
File file = new File(location); // will not hit exception if it is a url.
file.exist(); // return false if it is a url.
I am still struggling to find a best way to tackle both scenarios. :-(
Basically I would not prefer to explicitly check the path using the prefix such as http:// or https://
Is there an elegant and proper way of doing this?

You can check if the location starts with http:// or https://:
String s = location.trim().toLowerCase();
boolean isWeb = s.startsWith("http://") || s.startsWith("https://");
Or you can use the URI class instead of URL, URI does not throw MalformedURLException like the URL class:
URI u = new URI(location);
boolean isWeb = "http".equalsIgnoreCase(u.getScheme())
|| "https".equalsIgnoreCase(u.getScheme())
Although new URI() may also throw URISyntaxException if you use backslash in location for example. Best way would be to either use prefix check (my first suggestion) or create a URL and catch MalformedURLException which if thrown you'll know it cannot be a valid web url.

If you're open to the use of a try/catch scenario being "elegant", here is a way that is more specific:
try {
processURL(new URL(location));
}
catch (MalformedURLException ex){
File file = new File(location);
if (file.exists()) {
processFile(file);
}
else {
throw new PersonalException("Can't find the file");
}
}
This way, you're getting the automatic URL syntax checking and, that failing, the check for file existence.

you can try:
static public boolean isValidURL(String urlStr) {
try {
URI uri = new URI(urlStr);
return uri.getScheme().equals("http") || uri.getScheme().equals("https");
}
catch (Exception e) {
return false;
}
}
note that this will return false for any other reason that invalidates the url, ofor a non http/https url: a malformed url is not necessarily an actual file name, and a good file name can be referring to a non exisiting one, so use it in conjunction with you file existence check.

public boolean urlIsFile(String input) {
if (input.startsWith("file:")) return true;
try { return new File(input).exists(); } catch (Exception e) {return false;}
}
This is the best method because it is hassle free, and will always return true if you have a file reference. For instance, other solutions don't and cannot cover the plethora of protocol schemes available such as ftp, sftp, scp, or any future protocol implementations. So this one is the one for all uses and purposes; with the caveat of the file must exist, if it doesn't begin with the file protocol.
if you look at the logic of the function by it's name, you should understand that, returning false for a non existent direct path lookup is not a bug, that is the fact.

How to get Camera RAW metadata info programmatically Android

I have an app, which sets the hardware parameters of the Camera programmatically.
However, as I've been told, and have come to observe, not all chipsets support all parameters.
For example, the Nexus 4 (Qualcomm) has sharpness, and sharpness-max parameters, the Galaxy Note II 3g doesn't have any.
Hence, when I set sharpness parameter, the Nexus responds well, but the Galaxy force closes:
java.lang.RuntimeException: setParameters failed
at android.hardware.Camera.native_setParameters(Native Method)
at android.hardware.Camera.setParameters(Camera.java:1452)
My question is, how can I get the RAW info programmatically? I need to get the parameters, their values, and whether they exist or not.
I wish to get the RAW-Metadata parameters, as like this: database

Alright, thought this would be a fun bit of practice. So, Android does not give a public API into this information. Why? I have no idea. Looks like you can do a Camera.Parameters#get(String) to check for any particular parameter that you're interested in, but lets say you're greedy and want the whole list to yourself. In that case, we can dive in using Reflection, but be aware that there is a strong possibility that this will not work on all versions of Android or may break in future versions. With that said, here's how you do it:
private static Map<String, String> getFullCameraParameters (Camera cam) {
Map<String, String> result = new HashMap<String, String>(64);
final String TAG = "CameraParametersRetrieval";
try {
Class camClass = cam.getClass();
//Internally, Android goes into native code to retrieve this String of values
Method getNativeParams = camClass.getDeclaredMethod("native_getParameters");
getNativeParams.setAccessible(true);
//Boom. Here's the raw String from the hardware
String rawParamsStr = (String) getNativeParams.invoke(cam);
//But let's do better. Here's what Android uses to parse the
//String into a usable Map -- a simple ';' StringSplitter, followed
//by splitting on '='
//
//Taken from Camera.Parameters unflatten() method
TextUtils.StringSplitter splitter = new TextUtils.SimpleStringSplitter(';');
splitter.setString(rawParamsStr);
for (String kv : splitter) {
int pos = kv.indexOf('=');
if (pos == -1) {
continue;
}
String k = kv.substring(0, pos);
String v = kv.substring(pos + 1);
result.put(k, v);
}
//And voila, you have a map of ALL supported parameters
return result;
} catch (NoSuchMethodException ex) {
Log.e(TAG, ex.toString());
} catch (IllegalAccessException ex) {
Log.e(TAG, ex.toString());
} catch (InvocationTargetException ex) {
Log.e(TAG, ex.toString());
}
//If there was any error, just return an empty Map
Log.e(TAG, "Unable to retrieve parameters from Camera.");
return result;
}

how to check protocol present in url or not?

how to check protocol is present in URL , if not present need to append it.
is there any class to achieve this in java?
eg: String URL = www.google.com
need to get http://www.google.com

Just use String.startsWith("http://") to check this.
public String ensure_has_protocol(final String a_url)
{
if (!a_url.startsWith("http://"))
{
return "http://" + a_url;
}
return a_url;
}
EDIT:
An alternative would use a java.net.URL instance, whose constructor would throw an java.net.MalformedURLException if the URL did not contain a (legal) protocol (or was invalid for any other reason):
public URL make_url(final String a_url) throws MalformedURLException
{
try
{
return new URL(a_url);
}
catch (final MalformedURLException e)
{
}
return new URL("http://" + a_url);
}
You can use URL.toString() to obtain string representation of the URL. This is an improvement on the startsWith() approach as it guarantees that return URL is valid.

Let's say you have String url = www.google.com. String class methods would be enough for the goal of checking protocol identifiers. For example, url.startsWith("https://") would check whether a specific string is starting with the given protocol name.
However, are these controls enough for validation?
I think they aren't enough. First of all, you should define a list of valid protocol identifiers, e.g. a String array like {"http", "ftp", "https", ...}. Then you can parse your input String with regex ("://") and test your URL header whether it belongs to the list of valid protocol identifiers. And domain name validation methods are beyond this question, you can/should handle it with different techniques as well.

Just for completeness, I would do something like the following:
import com.google.common.base.Strings;
private static boolean isUrlHttps(String url){
if(Strings.isNullOrEmpty(url))
return false;
return url.toLowerCase().startsWith("https://");
}

Java - Convert String to valid URI object

I am trying to get a java.net.URI object from a String. The string has some characters which will need to be replaced by their percentage escape sequences. But when I use URLEncoder to encode the String with UTF-8 encoding, even the / are replaced with their escape sequences.
How can I get a valid encoded URL from a String object?
http://www.google.com?q=a b gives http%3A%2F%2www.google.com... whereas I want the output to be http://www.google.com?q=a%20b
Can someone please tell me how to achieve this.
I am trying to do this in an Android app. So I have access to a limited number of libraries.

You might try: org.apache.commons.httpclient.util.URIUtil.encodeQuery in Apache commons-httpclient project
Like this (see URIUtil):
URIUtil.encodeQuery("http://www.google.com?q=a b")
will become:
http://www.google.com?q=a%20b
You can of course do it yourself, but URI parsing can get pretty messy...

Android has always had the Uri class as part of the SDK:
http://developer.android.com/reference/android/net/Uri.html
You can simply do something like:
String requestURL = String.format("http://www.example.com/?a=%s&b=%s", Uri.encode("foo bar"), Uri.encode("100% fubar'd"));

I'm going to add one suggestion here aimed at Android users. You can do this which avoids having to get any external libraries. Also, all the search/replace characters solutions suggested in some of the answers above are perilous and should be avoided.
Give this a try:
String urlStr = "http://abc.dev.domain.com/0007AC/ads/800x480 15sec h.264.mp4";
URL url = new URL(urlStr);
URI uri = new URI(url.getProtocol(), url.getUserInfo(), url.getHost(), url.getPort(), url.getPath(), url.getQuery(), url.getRef());
url = uri.toURL();
You can see that in this particular URL, I need to have those spaces encoded so that I can use it for a request.
This takes advantage of a couple features available to you in Android classes. First, the URL class can break a url into its proper components so there is no need for you to do any string search/replace work. Secondly, this approach takes advantage of the URI class feature of properly escaping components when you construct a URI via components rather than from a single string.
The beauty of this approach is that you can take any valid url string and have it work without needing any special knowledge of it yourself.

Even if this is an old post with an already accepted answer, I post my alternative answer because it works well for the present issue and it seems nobody mentioned this method.
With the java.net.URI library:
URI uri = URI.create(URLString);
And if you want a URL-formatted string corresponding to it:
String validURLString = uri.toASCIIString();
Unlike many other methods (e.g. java.net.URLEncoder) this one replaces only unsafe ASCII characters (like ç, é...).
In the above example, if URLString is the following String:
"http://www.domain.com/façon+word"
the resulting validURLString will be:
"http://www.domain.com/fa%C3%A7on+word"
which is a well-formatted URL.

If you don't like libraries, how about this?
Note that you should not use this function on the whole URL, instead you should use this on the components...e.g. just the "a b" component, as you build up the URL - otherwise the computer won't know what characters are supposed to have a special meaning and which ones are supposed to have a literal meaning.
/** Converts a string into something you can safely insert into a URL. */
public static String encodeURIcomponent(String s)
{
StringBuilder o = new StringBuilder();
for (char ch : s.toCharArray()) {
if (isUnsafe(ch)) {
o.append('%');
o.append(toHex(ch / 16));
o.append(toHex(ch % 16));
}
else o.append(ch);
}
return o.toString();
}
private static char toHex(int ch)
{
return (char)(ch < 10 ? '0' + ch : 'A' + ch - 10);
}
private static boolean isUnsafe(char ch)
{
if (ch > 128 || ch < 0)
return true;
return " %$&+,/:;=?#<>#%".indexOf(ch) >= 0;
}

You can use the multi-argument constructors of the URI class. From the URI javadoc:
The multi-argument constructors quote illegal characters as required by the components in which they appear. The percent character ('%') is always quoted by these constructors. Any other characters are preserved.
So if you use
URI uri = new URI("http", "www.google.com?q=a b");
Then you get http:www.google.com?q=a%20b which isn't quite right, but it's a little closer.
If you know that your string will not have URL fragments (e.g. http://example.com/page#anchor), then you can use the following code to get what you want:
String s = "http://www.google.com?q=a b";
String[] parts = s.split(":",2);
URI uri = new URI(parts[0], parts[1], null);
To be safe, you should scan the string for # characters, but this should get you started.

I had similar problems for one of my projects to create a URI object from a string. I couldn't find any clean solution either. Here's what I came up with :
public static URI encodeURL(String url) throws MalformedURLException, URISyntaxException
{
URI uriFormatted = null;
URL urlLink = new URL(url);
uriFormatted = new URI("http", urlLink.getHost(), urlLink.getPath(), urlLink.getQuery(), urlLink.getRef());
return uriFormatted;
}
You can use the following URI constructor instead to specify a port if needed:
URI uri = new URI(scheme, userInfo, host, port, path, query, fragment);

Well I tried using
String converted = URLDecoder.decode("toconvert","UTF-8");
I hope this is what you were actually looking for?

The java.net blog had a class the other day that might have done what you want (but it is down right now so I cannot check).
This code here could probably be modified to do what you want:
http://svn.apache.org/repos/asf/incubator/shindig/trunk/java/common/src/main/java/org/apache/shindig/common/uri/UriBuilder.java
Here is the one I was thinking of from java.net: https://urlencodedquerystring.dev.java.net/

Or perhaps you could use this class:
http://developer.android.com/reference/java/net/URLEncoder.html
Which is present in Android since API level 1.
Annoyingly however, it treats spaces specially (replacing them with + instead of %20). To get round this we simply use this fragment:
URLEncoder.encode(value, "UTF-8").replace("+", "%20");

I ended up using the httpclient-4.3.6:
import org.apache.http.client.utils.URIBuilder;
public static void main (String [] args) {
URIBuilder uri = new URIBuilder();
uri.setScheme("http")
.setHost("www.example.com")
.setPath("/somepage.php")
.setParameter("username", "Hello Günter")
.setParameter("p1", "parameter 1");
System.out.println(uri.toString());
}
Output will be:
http://www.example.com/somepage.php?username=Hello+G%C3%BCnter&p1=paramter+1