I want to write a code in java that takes a url identify whether it is tiny url or not. if yes then it will identify the url is malicious or not. if not malicious print the url...
Please can any body help me....
You can use HttpClient to detect whether the URL is redirected to another location. After that it's a simple case of:
if (!isMalicious(redirectTargetURL))
{
System.out.println(redirectTargetURL);
}
The isMalicious(...) implementation is left as an excercise for the reader.
If you trust google to implement isMalicious(...) then they have done so with their Safe Browsing API.
So 2 main things you want:
Identify if it's a tinyurl
Identify if the URL is malicious
The answer to part 1 is easy. Just check if the URL belongs to the domain 'tinyurl.com'. Should be straightforward to either test raw URL string, or the host part returned by the getHost() method of a java.net.URL object.
Part 2 is more difficult to code up from scratch...
First you will need your code to figure out where the tinyurl redirects to.
The next bit really depends on how you want to define 'malicious'. Detecting deceptive URLs will require a bit of work (e.g. finding the difference between something like www.stackoverflow.com and www.stack0verf10w.com), or comparing the target URL with a malicous URL list (there's sites that publish them). There's also checking for multiple redirects, popups, and the list of criteria could go on and on.
Related
I just realized that my base64 encoded Header "Authentication" can't be read
with request.getHeader("Authentication").
I found this post about that it's a security Feature in URLConnection
getRequestProperty("Authorization") always returns null
, i don't know why but it seems to be true for request.getHeader as well.
How can i still get this Header if l don't want to Switch to other libraries?
I was searching through https://fossies.org/dox/apache-tomcat-6.0.45-src/catalina_2connector_2Request_8java_source.html#l01947 and found a section where restricted headers will be used if Globals.IS_SECURITY_ENABLED is set.
Since I'm working on a reverse Proxy and only Need to pass requests/Responses through I did simply set "System.setSecurityManager(null);" and for my case it might be a valid solution but if you want to use authentication there is no reason to use this Workaround.
My bad, it does work with https now.
The accepted solution did not work for me – may have something to do with different runtime environments.
However, i've managed to come up with a working snippet to access the underlying MessageHeader-collection via reflection and extract the "Authorization"-header value.
Since I found it very difficult to explain my problem in the heading I'm going to explain it a little further:
I want to /I'm writing a JAX-RS web service (Jersey/Servlet3.0) and the corresponding JS library for a geographic use case. The input of the web service are two lists (source & target points) of geographic points (latitude, longitude) and each point having a list of parameters. Since there is basically no limit on the number of points I don't know how to combine the URL length limit and the unlimitedness of the parameter list.
Here are the restrictions again:
Easy to share URL (so POST probably wont quite cut it?) for social media sharing and of course easy debugging
An example configuration can be seen here please note that there can be nested sets of parameters (point 1 has parameters of it's own)
Needs to able to be integrated in external website (with bookmarkable url)
Not all of the parameters are mandatory, what is the best way to deal with defaults/missing values?
What I thought of so far is:
create a boatload of parameters
jsonify the configuration and send it to server via url parameter
But I don't really like these options. Am I missing something?
Sorry for this rather vague question.
Daniel
Ok to your points
for easy sharing why not just implement a tinyURl or bit.ly style sharing system - obviously you can't have both an easy to pass-in url (a URL that makes it easy to give the server detailed information) that is easy to share (human friendly and short) - but you could very easily save the results (or inputs and calculate each time) to a database and link that to a tinyURl.
as an aside POST will be the only way to handle this due to the amount of data.
Just pass as JSON - easy to nest paramaeters that way
Don't quite get this part - for an external site to use this they could post the data and you return the answers - or using point 1 method of 'tinyurl / bit.ly style system it could call this in an iframe?
You would deal with missing parameters / defaults at server side - create a function for each parameter - if parameter is expected then throw error - if parameter has a default include this in your function and if parameter is not included then don't run the function.
Hope that makes sense?
There's a case that the user change the parameters send to servlet through URL,
is there's any way to restrict user not to change paramters,
if not, how can I manage all parameters send to servlets? in a case they are many, is it reasonable to check each one in turn??
You can't restrict the user from sending you anything.
It is the server-side where you can add restrictions.
Usually you get only the parameters you need, so additional parameters should not bother you.
You definitely should check parameters send to your servlet. Thats basically what you do anyways since thats the way clients (such as webpages) communicate with your application.
The simplest way is to hash the parameters with some hidden secret, and pass that back with the URL, then compare the hash to the URL parameters to make sure they match.
Another way is to not use individual parameters, but encrypt them in to a encoded bunch of characters and the whole thing is decrypted on return.
The hash is easier to implement if you don't care that the user sees the actual parameters.
You cannot avoid someone from typing in the URL, but what you do in your servlet is filter the input recieved from the URL, with some java code.
Example:
Just found an interesting link where a Servlet Filter is used to filter out XSS attacks(As you see, there is no such code that avoids someone to type certain characters in the URL, or similar): Link
Simply put, you cannot stop the users from changing the parameters.
You must do input validation on all parameter values. If you have a variable that contains sensitive information, you do not put that on the URL. A really bad example: http://mydomain.com/myservlet?isAdmin=1. Information such as that needs to go into a session since that is stored on the server and out of the user's reach.
I have an high performance application which deals with URLs. For every URL it needs to retrieve the appropriate settings from a predefined pool. Every settings object is associated with a URL pattern which indicates which URLs should use these settings. The matching rules are as follows:
"google.com" match pattern should match all URLs pointing to the google domain (thus, maps.google.com and www.google.com/match are matched).
"*.google.com" should match all URLs pointing to a subdomain of google.com (thus, maps.google.com matches, but google.com and www.google.com don't).
"maps.google.com" should match all URLs pointing to this specific subdomain.
Apart from the above rules, every match rule can contain a path, which means that the path part of the URL should start with the match rule path. So: "*.google.com/maps" matches "maps.google.com/maps" but not "maps.google.com/advanced".
As you can see the rules above are overlapping. In the case two rules exist which match the same URL the most specific should apply. The list above is ranked from least specific to most specific.
This seems to be such a standard problem that I was hoping to use a ready made library rather than program my self. Google reveals a couple of options but without a clear way to choose between them. What would you recommend as a good library for this task?
Thanks,
Boaz
I don't think you need a specific library to solve this; the standard Java API has all that you need to write the code without too much work.
Take a look at java.util.regex.Pattern and work out the regular expressions you need to match each of your rules. You might also want to use java.net.URL to parse out the different fields from the URL.
You already said you have a priority scheme to handle scenarios where multiple patterns match the URL, so that should be the last piece for this puzzle.
It looks like a pretty straight-forward task.
I got the answer for If I disabled the cookies then using URL ReDirect I can pass the JSESSIONID but my URL is already very long as I use the GET method it has constraint. Then how
should I use my sessions.I want my application to be very security intensive.
This is one of the question asked to my friend in GOOGLE interview.
Apart from using one-letter parameter names (e.g. ?a=value1&b=value2&c=value3 or using RESTFul-like URL's (i.e. just the pathinfo, no query parameters, e.g. /value1/value2/value3, which is accessible by HttpServletRequest#getPathInfo() in the servlet) instead of ?name1=value1&name2=value2&name3=value3, you can also consider to Gzip and Base64-encode the query string so that it becomes shorter. Both JavaScript and Java are capable of (de)compressing and (d)e(n)coding it. You can eventually format the query string in JSON before compressing/encoding, it will be shorter in case of arrays/collections/maps.
That said, are you sure that the request URL's are often that unfriendly long (assuming that it's over 255 characters)? Why would you need to pass that much information in? Are they supposed to maintain the client state? If so, you shouldn't use the URL for this, but the HttpSession instance in the server side which is already associated with the jsessionid cooke. Use HttpSession#setAttribute() to store some information in session and use HttpSession#getAttribute() to retrieve it.
As far as I understand, your main problem with JSESSIONID in the URL is the total length.
Perhaps you should have a closer look at why the length of the URLs are too long in the first place. Since you allready have a session, it is not unlikely you can move some GET parameters to the session. There are also lots of different way to make shorter URLs for pages (a la mod_rewrite).
With regards to security, JSESSIONID is just as vunerable with HTTP GET as HTTP POST. The base64 encoding HTTP POST does is not a security measure at all. The best way to gain a bit more security is to encrypt the transport channel through TLS/SSL, in effect enable HTTPS. This will make sure that eavesdropping (or man in the middle attacks) will not have access to the plain text.
If you want your application to be security intensive why are you using GET. Use POST. This will also reduce the URL length.
As such, as per the HTTP protocol there is no max length limit to URL length. Most of the time its the browser that puts in the max length limit. Try different browsers
You should put forward the above points to the interviewer. They might be more interested in your ability to assess the system as a whole and identify any fundamental flaws.
If the URL is too long then you have to store that data somewhere else. Most sites would put the session ID in a cookie.