Detecting Questionmarks
I need to implement a protocol where the user may provide a question mark appended to an URL in oder to retrieve additional information about the provided resource. I am aware that the questionmark after the URL resource indicates query parameters. Unfortunately this protocol does demand question marks at the end of the URL as an indicator.
Implementation
I implemented the service using Jersey.
#GET
#Path("{/service/")
#Produces(MediaType.TEXT_PLAIN)
public String resolve(#PathParam("uri") String fqn,
#PathParam("ark") String arkLabel,
#Context UriInfo ui, #Context HttpServletRequest hsr) {
// here i need to test of the url ends with ?
if (url.endsWith("?")) {
// to something
}
}
The Problem
All the methods provided from UriInfo and HttpServletRequest i found stripe away the last questionmark, if it is not followed by a query parameter. How can I get the raw URL, including a question mark at the end?
Solution
Based on the answer from #cy3er I could solve the problem. The following snipped does the trick. The method getQueryString() returns null if the URL did not contain any query, i.e. if there was no ? attached to the URL. Then I can assume if the string is empty, that there was only one ?, because the query is empty. In the third caswe I can check if the only query parameter passed is one ?, which corresponds to two ?? in the URL.
if (hsr.getQueryString() == null) {
this.logger.info("Normal link");
} else if (hsr.getQueryString().equals("")) {
this.logger.info("one ?");
} else if (hsr.getQueryString().equals("?")) {
this.logger.info("Two ??");
} else {
this.logger.info("None of the above");
}
Use the getQueryString method of the HttpServletRequest, if it's null there wasn't any questionmark.
See documentation
I believe the definition and implementation of Java's URI.resolve method is incompatible with RFC 3986 section 5.2.2. I understand that the Java API defines how that method works, and if it were changed now it would break existing apps, but my question is this: Can anyone confirm my understanding that this method is incompatible with RFC 3986?
I'm using the example from this question: java.net.URI resolve against only query string, which I will copy here:
I'm trying to build URI's using the JDK java.net.URI.
I want to append to an absolute URI object, a query (in String). In example:
URI base = new URI("http://example.com/something/more/long");
String queryString = "query=http://local:282/rand&action=aaaa";
URI query = new URI(null, null, null, queryString, null);
URI result = base.resolve(query);
Theory (or what I think) is that resolve should return:
http://example.com/something/more/long?query=http://local:282/rand&action=aaaa
But what I got is:
http://example.com/something/more/?query=http://local:282/rand&action=aaaa
My understanding of RFC 3986 section 5.2.2 is that if the path of the relative URI is empty, then the entire path of the base URI is to be used:
if (R.path == "") then
T.path = Base.path;
if defined(R.query) then
T.query = R.query;
else
T.query = Base.query;
endif;
and only if a path is specified is the relative path to be merged against the base path:
else
if (R.path starts-with "/") then
T.path = remove_dot_segments(R.path);
else
T.path = merge(Base.path, R.path);
T.path = remove_dot_segments(T.path);
endif;
T.query = R.query;
endif;
but the Java implementation always does the merge, even if the path is empty:
String cp = (child.path == null) ? "" : child.path;
if ((cp.length() > 0) && (cp.charAt(0) == '/')) {
// 5.2 (5): Child path is absolute
ru.path = child.path;
} else {
// 5.2 (6): Resolve relative path
ru.path = resolvePath(base.path, cp, base.isAbsolute());
}
If my reading is correct, to get this behaviour from the RFC pseudocode, you could put a dot as the path in the relative URI, before the query string, which from my experience using relative URIs as links in web pages is what I would expect:
transform(Base="http://example.com/something/more/long", R=".?query")
=> T="http://example.com/something/more/?query"
But I would expect, in a web page, that a link on the page "http://example.com/something/more/long" to "?query" would go to "http://example.com/something/more/long?query", not "http://example.com/something/more/?query" - in other words, consistent with the RFC, but not with the Java implementation.
Is my reading of the RFC correct, and the Java method inconsistent with it, or am I missing something?
Yes, I agree that the URI.resolve(URI) method is incompatible with RFC 3986. The original question, on its own, presents a fantastic amount of research that contributes to this conclusion. First, let's clear up any confusion.
As Raedwald explained (in a now deleted answer), there is a distinction between base paths that end or do not end with /:
fizz relative to /foo/bar is: /foo/fizz
fizz relative to /foo/bar/ is: /foo/bar/fizz
While correct, it's not a complete answer because the original question is not asking about a path (i.e. "fizz", above). Instead, the question is concerned with the separate query component of the relative URI reference. The URI class constructor used in the example code accepts five distinct String arguments, and all but the queryString argument were passed as null. (Note that Java accepts a null String as the path parameter and this logically results in an "empty" path component because "the path component is never undefined" though it "may be empty (zero length)".) This will be important later.
In an earlier comment, Sajan Chandran pointed out that the java.net.URI class is documented to implement RFC 2396 and not the subject of the question, RFC 3986. The former was obsoleted by the latter in 2005. That the URI class Javadoc does not mention the newer RFC could be interpreted as more evidence of its incompatibility. Let's pile on some more:
JDK-6791060 suggests this class "should be updated for RFC 3986". A comment there warns that "RFC3986 is not completely backwards
compatible with 2396". It was closed in 2018 as a duplicate of JDK-8019345 (still open and unresolved as of October, 2022, with no notable activity since 2013).
Previous attempts were made to update parts of the URI class to be compliant with RFC 3986, such as JDK-6348622, but were then rolled back for breaking backwards compatibility. (Also see this discussion on the JDK mailing list.)
Although the path "merge" logic sounds similar, as noted by SubOptimal, the pseudocode specified in the newer RFC does not match the actual implementation. In the pseudocode, when the relative URI's path is empty, then the resulting target path is copied as-is from the base URI. The pseudocode's "merge" logic is not executed under those conditions. Contrary to that specification, Java's URI implementation trims the base path after the last / character, as observed in the question.
There are alternatives to the URI class, if you want RFC 3986 behavior. Java EE 6 through EE 8 implementations provide javax.ws.rs.core.UriBuilder, which (in Jersey 1.18) seems to behave as you expected (see below). It at least claims awareness of the RFC as far as encoding different URI components is concerned. With the switch from JavaEE to JakartaEE 9 (circa 2020), this class moved to jakarta.ws.rs.core.UriBuilder.
Outside of J2EE, Spring 3.0 introduced UriUtils, specifically documented for "encoding and decoding based on RFC 3986". Spring 3.1 deprecated some of that functionality and introduced the UriComponentsBuilder, but it does not document adherence to any specific RFC, unfortunately.
Test program, demonstrating different behaviors:
import java.net.*;
import java.util.*;
import java.util.function.*;
import javax.ws.rs.core.UriBuilder; // using Jersey 1.18
public class StackOverflow22203111 {
private URI withResolveURI(URI base, String targetQuery) {
URI reference = queryOnlyURI(targetQuery);
return base.resolve(reference);
}
private URI withUriBuilderReplaceQuery(URI base, String targetQuery) {
UriBuilder builder = UriBuilder.fromUri(base);
return builder.replaceQuery(targetQuery).build();
}
private URI withUriBuilderMergeURI(URI base, String targetQuery) {
URI reference = queryOnlyURI(targetQuery);
UriBuilder builder = UriBuilder.fromUri(base);
return builder.uri(reference).build();
}
public static void main(String... args) throws Exception {
final URI base = new URI("http://example.com/something/more/long");
final String queryString = "query=http://local:282/rand&action=aaaa";
final String expected =
"http://example.com/something/more/long?query=http://local:282/rand&action=aaaa";
StackOverflow22203111 test = new StackOverflow22203111();
Map<String, BiFunction<URI, String, URI>> strategies = new LinkedHashMap<>();
strategies.put("URI.resolve(URI)", test::withResolveURI);
strategies.put("UriBuilder.replaceQuery(String)", test::withUriBuilderReplaceQuery);
strategies.put("UriBuilder.uri(URI)", test::withUriBuilderMergeURI);
strategies.forEach((name, method) -> {
System.out.println(name);
URI result = method.apply(base, queryString);
if (expected.equals(result.toString())) {
System.out.println(" MATCHES: " + result);
}
else {
System.out.println(" EXPECTED: " + expected);
System.out.println(" but WAS: " + result);
}
});
}
private URI queryOnlyURI(String queryString)
{
try {
String scheme = null;
String authority = null;
String path = null;
String fragment = null;
return new URI(scheme, authority, path, queryString, fragment);
}
catch (URISyntaxException syntaxError) {
throw new IllegalStateException("unexpected", syntaxError);
}
}
}
Outputs:
URI.resolve(URI)
EXPECTED: http://example.com/something/more/long?query=http://local:282/rand&action=aaaa
but WAS: http://example.com/something/more/?query=http://local:282/rand&action=aaaa
UriBuilder.replaceQuery(String)
MATCHES: http://example.com/something/more/long?query=http://local:282/rand&action=aaaa
UriBuilder.uri(URI)
MATCHES: http://example.com/something/more/long?query=http://local:282/rand&action=aaaa
If you want better1 behavior from URI.resolve() and do not want to include another large dependency2 in your program, then I found the following code to work well within my requirements:
public URI resolve(URI base, URI relative) {
if (Strings.isNullOrEmpty(base.getPath()))
base = new URI(base.getScheme(), base.getAuthority(), "/",
base.getQuery(), base.getFragment());
if (Strings.isNullOrEmpty(uri.getPath()))
uri = new URI(uri.getScheme(), uri.getAuthority(), base.getPath(),
uri.getQuery(), uri.getFragment());
return base.resolve(uri);
}
The only non-JDK thing there is Strings from Guava, for readability - replace with your own 1-line-method if you don't have Guava.
Footnotes:
I cannot claim that the simple code sample here is RFC3986 compliant.
Such as Spring, javax.ws or - as mentioned in this answer - Apache HTTPClient.
for me there is no discrepancy. With the Java behaviour.
in RFC2396 5.2.6a
All but the last segment of the base URI's path component is copied to the buffer. In other words, any characters after the last (right-most) slash character, if any, are excluded.
in RFC3986 5.2.3
return a string consisting of the reference's path component appended to all but the last segment of the base URI's path (i.e., excluding any characters after the right-most /" in the base URI path, or excluding the entire base URI path if it does not contain any "/" characters).
The solution proposed by #Guss is a good enough work around, but unfortunately, there is a Guava dependency and some minor errors in it.
This is a refactor of his solution removing the Guava dependency and the errors. I use it in replacement of URI.resolve() and place it in a helper class called URIUtils of mine, together with other methods that would be part of an extended URI class if it was not final.
public static URI resolve(URI base, URI uri) throws URISyntaxException {
if (base.getPath() == null || base.getPath().isEmpty())
base = new URI(base.getScheme(), base.getAuthority(), "/", base.getQuery(), base.getFragment());
if (uri.getPath() == null || uri.getPath().isEmpty())
uri = new URI(uri.getScheme(), uri.getAuthority(), base.getPath(), uri.getQuery(), uri.getFragment());
return base.resolve(uri);
}
It is easy to check it works around URI.resolve() just by comparing their outputs for some common pitfalls:
public static void main(String[] args) throws URISyntaxException {
URI host = new URI("https://www.test.com");
URI uri = new URI("mypage.html");
System.out.println(host.resolve(uri));
System.out.println(URIUtils.resolve(host, uri));
System.out.println();
uri = new URI("./mypage.html");
System.out.println(host.resolve(uri));
System.out.println(URIUtils.resolve(host, uri));
System.out.println();
uri = new URI("#");
System.out.println(host.resolve(uri));
System.out.println(URIUtils.resolve(host, uri));
System.out.println();
uri = new URI("#second_block");
System.out.println(host.resolve(uri));
System.out.println(URIUtils.resolve(host, uri));
System.out.println();
}
https://www.test.commypage.html
https://www.test.com/mypage.html
https://www.test.commypage.html
https://www.test.com/mypage.html
https://www.test.com#
https://www.test.com/#
I'm new to developing web services in Java (previously I've done them in PHP and Ruby). I'm writing a resource that is of the following format:
<URL>/myService/<domain>/<app_name>/<system_name>
As you can see, I've got a three-level resource identifier, and I'm trying to figure out the best way to parse it. The application I'm adding this new service to doesn't make use of Jersey or any RESTful frameworks like that. Instead, it's just extending HttpServlet.
Currently they're following an algorithm like this:
Call request.getPathInfo()
Replace the "/" characters in the path info with "." characters
Use String.substring methods to extract individual pieces of information for this resource from the pathInfo string.
This doesn't seem very elegant to me, and I'm looking for a better way. I know that using the javax.ws.rs package makes this very easy (using #Path and #PathParam annotations), but using Jersey is probably not an option.
Using only the base HttpServletRequest object and standard Java libraries, is there a better way to parse this information than the method described above?
How about jersey UriTemplate?
import com.sun.jersey.api.uri.UriTemplate;
...
String path = "/foos/foo/bars/bar";
Map<String, String> map = new HashMap<String, String>();
UriTemplate template = new UriTemplate("/foos/{foo}/bars/{bar}");
if( template.match(path, map) ) {
System.out.println("Matched, " + map);
} else {
System.out.println("Not matched, " + map);
}
I've recently solved this issue in one of my applications. My URLs look like this.
/categories/{category}/subcategories/{subcategory}
My problem was that I wanted to map each url pattern with a Java class, so that I could call upon the correct class to render the data.
My application uses Netty, but the URL resolver doesn't use any third party libraries.
What this allows me to do is to parse the URL that is coming in from the browser, generate a map that has key-value pairs (in this case category, and subcategory), as well as instantiate the correct handler for each unique URL pattern. All in all only about 150 lines of Java code for the parsing, the application setup and the definition of the unique URL patterns.
You can view the code for the resolver in GitHub: https://github.com/joachimhs/Contentice/blob/master/Contentice.api/src/main/java/no/haagensoftware/contentice/util/URLResolver.java
UrlResolver.getValueForUrl will return a URLData with the information that you require about your URL:
https://github.com/joachimhs/Contentice/blob/master/Contentice.api/src/main/java/no/haagensoftware/contentice/data/URLData.java
Once this is setup, I can associate URLs with Netty Handlers:
this.urlResolver.addUrlPattern("/categories", CategoriesHandler.class);
this.urlResolver.addUrlPattern("/categories/{category}", CategoryHandler.class);
this.urlResolver.addUrlPattern("/categories/{category}/subcategories", SubCategoriesHandler.class);
this.urlResolver.addUrlPattern("/categories/{category}/subcategories/{subcategory}", SubCategoryHandler.class);
Inside my Handlers I can simply get the parameter map:
String category = null;
logger.info("parameterMap: " + getParameterMap());
if (getParameterMap() != null) {
category = getParameterMap().get("category");
}
I hope that helps :)
I had the same problem as you and, as I didn't find any suitable library, I decided to write URL-RESTify. You may use it or just take a look to write your own solution, it's a small project.
Jersey's UriTemplate mentioned in other answers is good, but it's a big library and it also includes many other dependency libraries.
Tiny solution with no dependency:
https://github.com/xitrum-framework/jauter
I believe first you need to create a framework for storing the REST method and class+method mappings in a property file or in memory data structue. Then write a top level servlet accepting all of your REST request. Depending on the URL starting from your context, you can try to fetch the mapping from your property file/in memory data structure to find out which class and which of its method need to be called. Then making use of reflection you can call the desired method. Take the method response and marshal it into the desired content-type format and send back to the servlet response output stream.
Implemented it myself (check the main method for example), just in case if you would want a custom implementation:
import lombok.AllArgsConstructor;
import lombok.NonNull;
import java.util.*;
public class Template {
final List<TemplateElement> templateElements = new ArrayList<>();
public static void main(String[] args) {
final Template template = new Template("/hello/{who}");
final Map<String, String> attributes = template.parse("/hello/world").get();
System.out.println(attributes.get("who")); // world
}
public Template(#NonNull final String template) {
validate(template);
final String[] pathElements = template.split("/");
for (final String element : pathElements) {
if (isAttribute(element)) {
final String elementName = element.substring(1, element.length() - 1); // exclude { and }
templateElements.add(new TemplateElement(ElementType.ATTRIBUTE, elementName));
} else {
templateElements.add(new TemplateElement(ElementType.FIXED, element));
}
}
}
public Optional<Map<String, String>> parse(#NonNull final String path) {
validate(path);
final String[] pathElements = path.split("/");
if (pathElements.length != templateElements.size()) return Optional.empty();
final Map<String, String> attributes = new HashMap<>();
// ignore the 0th element, it'll always be empty
for (int i = 1; i < templateElements.size(); i++) {
final String element = pathElements[i];
final TemplateElement templateElement = templateElements.get(i);
switch (templateElement.type) {
case FIXED:
if (!element.equals(templateElement.name)) return Optional.empty();
break;
case ATTRIBUTE:
attributes.put(templateElement.name, element);
break;
}
}
return Optional.of(attributes);
}
private void validate(#NonNull final String path) {
if (!path.startsWith("/"))
throw new RuntimeException("A template must start with /"); // a template must start with /
}
private boolean isAttribute(#NonNull final String str) {
return str.startsWith("{") && str.endsWith("}");
}
#AllArgsConstructor
class TemplateElement {
final ElementType type;
final String name;
}
enum ElementType {
FIXED, ATTRIBUTE
}
}
Please point out mistakes if any. Thanks.
This may be a duplicate question but I was not able to find a solution. As a result, I'm posting my own one.
My URL looks like this "/customer/www.bakeryx.com" where www.bakeryx.com is the URL dynamic part and maps to "/customer/:domain".
I was hoping that when I call ctx.request().getQueryString("domain") I would get the www.bakeryxcom. Otherwise, I get a null response and there is no way to get this value from the action.
Please find bellow my work around for this task. I had to get the ROUTE_PATTERN from the context args.
public class DomainVerifierAction extends Action<DomainVerifierFilter> {
#Override
public Result call(Http.Context ctx) throws Throwable {
//how to get the domain here??
//work around is to get the route_pattern
String routePatternPlay = (String) ctx.args.get("ROUTE_PATTERN");
String path = ctx.request().path();
//added logic to extract domain from the PATH using ROUTE_PATTERN.
}
}
Question: Is there any solution for this problem?
I think the problem you are having is that the getQueryString method you are using is looking for the "?" operator in the URL, as in a traditional GET request (e.g. ?id=1). Instead, try passing the domain as a parameter in the controller method. For example:
In your routes file:
GET /customer/:domain controllers.Application.function(domain: String)
Then in your Play controller (assuming Play Framework 2.x):
public static Result function(String domain){
//Do something with the passed domain string here
return ok(...);
}
I am looking for some nice solution. I've got a couple of textfields on my page and I am sending these via Ajax using jQuery serialize method. This serialized string is parsed in my java method to hashmap with key = 'nameOfTextfield' nad value = 'valueInTextfield'
For example, I've got this String stdSel=value1&stdNamText=value2&stdRevText=value3 and everything works fine.
String[] sForm = serializedForm.split("&");
Map<String, String> fForm = new HashMap<String, String>();
for (String part : sForm) {
String key = null;
String value = null;
try {
key = part.split("=")[0];
value = part.split("=",2)[1];
fForm.put(key, value);
//if textfield is empty
} catch(IndexOutOfBoundsException e) {
fForm.put(key, "");
}
}
But this method will break down when ampersand in some textfield appears, for example this stdSel=value1&stdNamText=value2&stdRevText=val&&ue3. My thought was that I'll replace ampersand as separator in searialized string for some other character or maybe more characters. Is it possible and good idea or is there any better way?
Regards
Ondrej
Ampersands are escaped by the serialize function, so they don't break the URL.
What you need to unescape a field you got from an URL is
value = URLDecoder.decode(value,"UTF-8");
But, as was pointed by... Pointy, if you're using a web framework and not using only vanilla java.net java you probably don't have to do this.