I'm trying to use AWS as a scalable analytics tool. I'm using apache zeppelin as an interactive shell to a Spark cluster and trying to plot using wisp. This is causing a problem as the plotting approach in wisp is to start a web app based on what I think is a jetty server. This works well on my local machine but on AWS it does not work as it picks up the private IP address rather than the public one.
Within wisp, it uses java.net.InetAddress.getLocalHost.getCanonicalHostName
to retrieve the IP address of the machine. This always returns the private FQDN address. How can I make the java function return the public IP address or FQDN AWS provides without hardcoding something in wisp every time I spin up a cluster and rebuilding?
I have tried changing /etc/hosts and /etc/hostname but both have no effect. I don't really know where java.net.InetAddress.getLocalHost.getCanonicalHostName
is getting it's address from.
Any help or advice greatly appreciated.
Dean
I'm an idiot. It was a port issue. It appears that despite displaying the private IP, it's still available on the public IP.
Turn off the internal firewall with sudo ufw disable (on an ubuntu vm anyway, look up the alternative command for your flavour of vm).
Next, go to the AWS EC2 Management page for your account and click on the security group of your instance.
On the security group page, edit the inbound rules and add a custom tcp rule.
Under ports, put in '1 - 65536' and under source put in '0.0.0.0/0'. Please note, that what you have just done is open all ports on your instance to all incoming traffic from everywhere. You have just put down a welcome mat to the internet. I do not recommend doing this long term and it is extremely stupid. It worked for my proof of principle set up though and that is all I was aiming to do. I'll now be looking at ways to make it secure.
Thanks to everyone for their restraint in not mocking me for being a moron.
Related
I am trying to route my traffic with custom IP routes within a VPN. I have already had a similar issue with a windows VPN client and I was able to just write a batch file that runs when launching the client and it just runs route add commands.
The problem is it seems that this is impossible to do for Android. I am aware that there is the adb approach, but it gives me permissions errors when attempting to custom add routes. Upon further research it seemed in order to change the routing rules for the IP tables I would have to root my device, this is not an option in my case.
I have looked at the VPN service on the Android Developer page, but I didn't see any way to start adding custom routes to a VPN. I also saw no libraries readily available that provide a method for additionally routing traffic. Perhaps I am missing a piece of the puzzle.
I am still learning about Android Development and this is an issue that I would like to have some input on as I will not be able to run my application through the default VPN without adding my custom rules, and I do not have the ability to change the VPN subnets around to get rid of this issue at the moment.
I had a similar problem.
Android SDK has addRoute API to add a route policy through VPN.
But there is no remove route API in the public document.
Here is the addRoute documents:
Add a network route to the VPN interface. Both IPv4 and IPv6 routes are supported. Adding a route implicitly allows traffic from that address family (i.e., IPv4 or IPv6) to be routed over the VPN.
I basically need to create a test server, something that I can use locally to perform tasks for my application.
So my android app is basically a search engine, however the search queries will be made on the server and the results then fetched from the server on the mobile device as JSON objects.
Is there any way to replicate this functionality locally? I would just like to set up a server, include the Spring framework, the Google Custom Search API and develop it all in Java, then ship it to a live server directly when it's finished. (Most presumably a free tier Amazon AWS server, either EC2 or Lambda, not sure what the difference is).
Not quite sure where or how to get started so any help is appreciated, thank you!
p.s. I may try dabbling in some machine learning at a later date, but I don't know if this will affect anything, this will most likely be after I've got a live server up and running.
EDIT: Sorry, so how would I fetch data from a local server from my mobile device? Do I need special software? Do I need to configure a local server? How can my mobile device fetch data from a second project when its being run on the first?
My app will fetch data from a server, so how can I create a dummy server, for it to fetch data from? Will I need to create a fully functioning online server, or can this just be done through creating a local server of some sort and then simply plugging my android device in and running the code? I have absolutely no idea how to test it without putting it online.
You can run a local installation of Tomcat server on your own machine.
Access the server by visiting "http://localhost:8080/" (or) http://127.0.0.1:8080". You can basically listening to your own local machine / host's 8080 port. Sometimes, it could be port 80 instead of 8080.
Please explore on how to host a Spring or Java application on a local server.
A good starting point would be to learn J2EE or Enterprise Java. You can configure Databases from your local installation.
If you opt for PHP as your language on server side, there is a pre-defined package called WAMP (for Windows), LAMP (for Linux), MAMP (for MAC) and XAMPP.
The above package are nothing but a package which contains a Server, Database and a Server side language usually PHP, if needed Python or Perl.
Please explain in which step you are getting issues so that anyone can help you exactly.
I made a game with sockets that works fine on my machine, and now I would like to run the server on a host so that anyone can join in at any time.
All I'm finding are services for web applications, mine is just a swing application that can run by itself.
What kind of server can I use (instead of my own machine)? Would I need to change the code or is it possible to just find a host that runs my server application as is?
Also, note that for now I kept it simple, there is no data storage.
Take a look at Amazon EC2 cloud. Essentially what you're looking to do is acquire a public IP address (an be done from the Amazon EC2 admin console). You'll be installing java and then opening a port in your security settings to allow traffic to whatever port your game server is running on.
EC2 is free for low volumes of usage for the first year (new accounts only) and I've had fairly good luck with them in terms of downtime and affordability.
What method could I use to login to a Cisco or Juniper routers? I know I can use telnet to make a connection to the router itself but I am not aware of an API or anything that allows me to login non-interactively. So how could I do this?
Are there any libraries I can use to achieve this?
I have seen people use scripts that implement things like expect to know when to send the username and the password. But that is for shell scripting. How can I do this in java? I would prefer to keep strictly in java too.
The goal of this is allow a java program login into a router so I can query the router for interface states and execute commands on the router.
Any ideas?
You need to use some Java library for doing the same. http://www.pitman.co.za/projects/jssh/ is a good Java SSH client. http://commons.apache.org/proper/commons-net/ has a good telnet implementation
As far as extracting information, you should be able to do it by using SNMP to query the router. Pretty much everything is available over SNMP. Now, as far as command execution, there are SNMP set commands used to "write" information to a router, but you will need to look into that more carefully as I doubt the full command set can be replaced with SNMP set commands.
For Cisco, find out what the IOS image filename is for your device, then you can get the MIB file here: http://tools.cisco.com/Support/SNMP/do/MIBSupport.do?local=en&step=3. Review the MIB file and you will see what is available via SNMP for that device.
You should be able to do something similar for Juniper on their site.
Now, I don't know much about Java, but I do know that there are SNMP libraries available for it (like http://www.snmp4j.org/ ).
With that you will not need to worry about logging into the router, or about any interactive stuff.
Note that, in the case of Cisco, it's common to connect using a serial port on a PC directly into the service port on the router. This connection uses a programming protocol called IOS.
The easiest way to utilize this is with a programming language called expect. This name describes the language in that you expect to get a prompt from the port and then you respond. Sending that message (your response), you expect to get another response, to which you respond, ad completum - my term ;)
Note that this cable is proprietary to Cisco, I'm sure there are pin-outs available though.
Finally, it would be possible to emulate this program on java, via a serial port library.
Consider this setup:
So there's a JMX Server and we want it to be reachable from different JMX Clients, but the problem is that these different JMX clients knows HostX by different names/addresses. For the sake of the example lets say that HostA knows HostX by the name of hostx but HostB knows HostX by the name of hostxshadow.
I'm looking for recommendations on how to solve this problem under the following scenarios:
Where I have control over the source code of the JMX Clients and JMX Server.
Where I have no control over the source code, i.e. I can set System Properties on the JVM, but that's it.
The basic problem to be solved is that in the RMI world the server-side decides how a client should connect (right?). So setting e.g. java.rmi.server.hostname on HostX isn't going to help. As I see it I can only make either HostA happy or HostB happy, but not both at the same time.
Is the conclusion really that any RMI Server can only work if all clients would use the same name to access it?
Your conclusion is correct. It's a design flaw in RMI. There is an underlying assumption that there is a single 'most public' hostname or IP address by which the server host can be contacted by everybody.
Your statement about the host embedding its own name or address in the stub is also correct.
RMI is one possible transport of JMX. Another solution is JMXMP and switching to it is very simple, check at http://meteatamel.wordpress.com/2012/02/13/jmx-rmi-vs-jmxmp/ or
http://blog.markfeeney.com/2010/10/jmx-through-ssh-tunnel.html.
The basic problem to be solved is that in the RMI world the server-side decides how a client should connect (right?). So setting e.g. java.rmi.server.hostname on HostX isn't going to help. As I see it I can only make either HostA happy or HostB happy, but not both at the same time.
This is not going to help you but for others, the trick to do is to set the java.rmi.server.hostname system property to a name that means something on HostA which is inside of the firewall but another thing to HostB which is outside.
For example, in EC2 land we set the hostname property to be the EC2 node's "public hostname" which we get by doing a:
wget -q -O - http://169.254.169.254/latest/meta-data/public-hostname
This resolves to be some like:
ec2-54-1-2-3.compute-1.amazonaws.com
On the box itself and any others that are inside of EC2 this resolves to be 10.1.2.3 while on the outside world it resolves to the externally facing IP 54.1.2.3. Amazon does this DNS work by default. You may need to adjust the /etc/hosts file as necessary to get the names to be correct.
For more details on EC2 and JMX, see: How to connect to Java instances running on EC2 using JMX