I am attempting to connect to URLs in Java to see if they are valid and I am wondering if I need to connect to HTTPS(port 443?) or if connecting to just HTTP(port 80) will be enough.
Does connecting to HTTP for an HTTPS website work? Is there anything with firewalls I should watch out for that wouldn't allow me to do this?
Thanks.
If you want to check that URLs are "valid" I think you want to know if they respond with a 200 status code to a GET request.
You'll need to check http and https separately if you want to know if they both work. They're two different protocols, and severs handle them differently. Some servers mirror the same content over both protocols, but many of them redirect the HTTP -> HTTPS etc.
Also not every server supports SSL connections, therefore HTTPS might not be available.
Since you rephrased your question I'll update my answer accoring to that.
To stay with your example:
Checking for URLS on port 80 is totally independent from checking urls on port 443. Maybe port 80 leads to the same content as port 443. Maybe port 80 leads to the end-user content, while port 443 leads to the admin-login.
Maybe apache operates on port 80 while nginx operates on port 443.
So to get the all of the content, you need to scan both ports. Additionally be prepared to find sometimes two different types of content, that don't have anything to do with each other. Admittedly this will happen rarely but it can happen.
Regarding firewalls:
If a web-service is intended to be public, firewalls will happily allow you to connect to the service. If a web-service is intended to be private and you can connect to it nonetheless, the firewall admin made a mistake :)
HTH
Related
I have a Spring application inside a tomcat 8 container, this application has both local (intranet) and remote (internet) service. I would like to serve local services with simple HTTP and remote with HTTPS, is it possibile editing tomcat configuration and without filter requests inside the application?
I should distinguish local from remote requests by its ip address.
You shouldn't need to. Your local network should be protected by a firewall, and you simply configure the firewall to only allow the secure port through.
Local traffic from the intranet doesn't go through the firewall, so it can access the HTTP port (80, 8080, ...).
External traffic comes in through the firewall, and it will block the HTTP port and allow the HTTPS port (443, 8443, ...).
Often with HTTPS, you don't even let Tomcat handle that, but instead put IIS (Windows) or Apache (Linux) in front of it. In that case you only have an AJP connector on localhost, and nobody can talk directly to Tomcat. The frontend web server will then do the required filtering and SSL/TLS handshake.
If you have anything that's worth using https for, I'd opt to go https all the way: Otherwise you'll sooner or later have information leaks because you've missed some crucial part of configuration. HTTPS is no black magic anymore, performance impact is low if exists it at all.
In fact, the typical usecase that I see described is exactly opposite to yours: Intranet usage is typically more protected than internet access (which is thought to be anonymous, but it depends on the nature of the site). However, an Intranet is typically authenticated (more so than the internet side, typically) and I'd expect it to be quite important to protect the authentication. The only mixed-mode solution (http/https) that I could come up with for this situation is: Use HSTS as soon as a user logs in, don't bother otherwise.
You're asking for the opposite of what I typically see - but my actually preferred solution (in all cases) is: Force https everywhere, use HSTS. And don't worry any more. Easier to maintain, Easier to setup and hard to get wrong.
I'm developing a Spring application on my Windows 7 machine and running it on localhost:8080.
Is there any chance that if I do this in a public location like Starbucks that it is possible for my app to be visible to others?
(I'm using Windows Firewall and Microsoft Security Essentials)
Be sure that it is binding to localhost:8080 (i.e., 127.0.0.1), not all interfaces (which is probably the default). Server programs usually have an option to say which IP addresses to bind. For example, in Apache, you can provide an IP address to the Listen directive in addition to a port number. If you only bind to 127.0.0.1, the port will not be open for people scanning your external IP address.
Short answer: No.
Long answer: Someone might access your app like this:
If you visit a malicious website (if anyone at Starbucks is poisoning the DNS cache or spoofing DNS responses then you may visit a malicious website at www.google.com not knowing about it) then the attacker may send you to his domain with a DNS server (which may in fact be on his laptop) replying to the same query with very short TTL once with his IP, then with 127.0.0.1, then his IP again, etc. That way when you are sent to (http://www.example.com:8080) it is resolved as the attacker's IP and you get the website which starts an AJAX request to the same domain and the same port (so the same origin policy is satisfied in every browser) but thanks to the short TTL you don't have the domain entry in your cache any more, so you ask again and this time you get the answer 127.0.0.1 which incidentally is your own loopback interface which you assume to be invisible from the outside - and it is in fact invisible from the outside but perfectly visible from your browser. Your browser will happily connect to your app at 127.0.0.1:8080 and proxy the request to the attacker with another AJAX connection or any other side channel. Voilà, your app is now connected to the world!
Things like this happen in the wild so be careful. Sometimes people even access admin interfaces in routers behind NAT and firewalls that way. Quite frankly I'm pretty scared by all of the responses telling you that it is impossible to access anything running on localhost. Be careful what you do.
Is there any chance that if I do this in a public location like Starbucks that it is possible for my app to be visible to others?
Of course! People can look over your shoulder.
what Jeremiah said. use netstat -an to check what addresses it bound to.
and never trust Windows firewall, or any other software to which you don't have the source.
If the port is open it can be. If you program it to only accept connections from localhost while developing then even if they can see it, they wont be able to connect to it.
I have a database application (or search engine) which is called Solr.
I connect to it via port 8983.
I do this from PHP code, so I add and remove records from it via php.
On my server I have a firewall.
I have set this firewall to only allow connections to and from this port (8983) from the ip address of my own server. In other words, only allow servers IP to access this port.
Is that safe? Or am I thinking all wrong here? Will others be able to "simulate" my ip address and act as the server?
This is because otherwise others may add/remove records as they want from their own IP addresses...
Thanks
It might be a good idea to also block all outgoing traffic from port 8983 on the server to anywhere but your own server's IP address. This, in addition to dropping any packet to that port not from your server, will doubly ensure that, even if someone is somehow able to modify the daemon listening on port 8983 on the server, allowing it to mirror traffic to another host, it would be dropped before it leaves your computer.
Yes, you are safe as long as no one gains control of your local server.
You can also cause Solr to bind to the "localhost" or "127.0.0.1" adapter as opposed to "0.0.0.0", which would have a similar effect. It never hurts to layer the firewall above that just in case the configuration is messed up.
You would not be safe if you are worried of tampering from the same network. There are many situations where the real threats are from inside the network, not from some script kiddie a continent away.
I agree with theatrus to use only localhost.
If you are deployed on multiple hosts there are several ways to create a secure tunnel, e.g
ssh -l 8983:localhost:8983 solr.server
this will create a secure tunnel. (Although it takes non trivial CPU when the bandwidth is high). There are also other solutions.
An additional advantage is that for a developer you can use a sample solr server locally and your code in your IDE, and it will just work with the same config as in production. The less that needs to be changed when deploying, the better.
This is safe. The ip address used in a TCP connection because of the three way handshake. This is a good firewall rule-set, but you should always test your rulesets with nmap.
What you do have to worry about is running an open proxy server on the server that is executing the PHP.
I have a web-service endpoint and a http connector on port X.
At some point this endpoint needs to switch to https, but on the same port!
(I know this is not the normal way of doing things, but this is what my clients expect from an old server they are using...)
Is there a way to do it in tomcat?
This is not possible with Tomcat.The HTTPS connector will accept SSL connection only.
We have such a proxy developed in house. It's not that hard to do. You just need to check the first incoming packet. Looking for the pattern of SSL handshake. We only look for CLIENT_HELLO. Once you figure out the protocol, you can forward the request accordingly.
This is really ugly. You shouldn't do it if all possible. We have to do it because the legacy clients do this and it's impossible to upgrade them all.
There is such a thing as HTTPS upgrade, whereby a plaintext HTTP connection is upgraded to HTTP by mutual agreement after it has been formed. Is that what you mean? If so, Tomcat doesn't seem to support it out of the box, and neither does Java out of the box either. You can probably write yourself a Tomcat Connector that will do it; on the client end you have a more interesting problem ;-)
But I would ask why? Ports aren't so expensive that you can't use two.
You don't need to run the HTTP & HTTPS on same port, Configure the Tomcat to redirect requests to HTTPS in server.xml file.
well I wonder why they are NOT usually on the same port! wouldn't that be easier?
the reason is probably that related Java APIS (javax.net.ssl) don't allow that; you must have different server sockets. are there any alternative SSL impls for Java? I'm not aware of any.
This might be one of those "huh, why?" questions, but I figured it would be worth the try.
How would one, from a server-side application, use the clients IP address as the applications IP address to another website? The basic idea is that any work the server side application does, is seen as the client itself doing the work, and not the servers static IP.
I am not sure if changing HTTP headers would work, but I could be wrong. Is there any documentation out there on this?
Thanks,
Kyle
Utterly, utterly impossible. You won't even be able to open a TCP connection because the other website's server will try to handshake with the client, and fail.
An IP address isn't just any old ID, it's the actually address that servers will send any response to. Spoofing it basically only makes sense if you can fit your request into a single IP packet (which rules out TCP and thus HTTP) and are not interested in the response. Even then it can fail because your ISP's routers may have anti-spoofing rules that drop packets with "outside" IP addresses originating from "inside" networks.
Why on earth would a legitimate application want to spoof its IP address?
Changing HTTP headers might cut it, but most likely it won't. Depends on how naive the other server is.
It sounds like you're trying to do something the wrong way, can you give a bit more information as to what exactly the use-case is?
If there's no processing to be done in between, you can do port forwarding on your server's IP firewall, so the client connects to your server but ends up talking to the other server.
If there's more involvement of your server, then the correct thing to do would be to pass the client's IP to the other server as part of the URL (if it's a web app) or elsewhere in the data (if not) so the receiving server can know and correctly log the process without any need for fakery. Of course this would also call for a change in the other app.
Again assuming we're talking about HTTP, another idea that came to my mind would be to redirect your client to the other server. As long as all necessary data is in the URI, you could advise the client's browser to connect to the other server with a URI of your own creation that could carry whatever extra value your server's processing adds to the request.
Decades ago, the designer of internet asked, "how can we prevent Kyle Rozendo from doing such a devious thing?"
If the client is cooperating, you can install some software on client machine, and do the work from there. For example, a signed java applet on your page. [kidding]If the client is not cooperating, install some trojan virus[/kidding]