There is a Linux VM with Hadoop installed and running.
And there is Java app running in Eclipse that retrieve data from HDFS.
If I am copying file(s) to or from HDFS inside the VM everything works fine.
But when i am running the app from my Windows physical machine I am getting the next exception:
WARN hdfs.DFSClient: Failed to connect to /127.0.0.1:50010 for block, add to
deadNodes and continue. java.net.ConnectException: Connection refused: no further
information. Could not obtain BP-*** from any node: java.io.IOException:
No live nodes contain current block. Will get new block locations from namenode and retry
I can only retrieve list of files from HDFS.
Seems that when retrieve data from data node it is connecting to my Windows localhost.
Because when I made a tunnel in putty from my localhost to VM everything was fine.
Here is my Java code:
Configuration config = new Configuration();
config.set("fs.defaultFS", "hdfs://ip:port/");
config.set("mapred.job.tracker", "hdfs://ip:port");
FileSystem dfs = FileSystem.get(new URI("hdfs://ip:port/"), config, "user");
dfs.copyToLocalFile(false, new Path("/tmp/sample.txt"),newPath("D://sample.txt"), true);
How can it be fixed?
Thanks.
P.S. This error occurs when I am using QuickStart VM from Cloudera.
Your DataNode is advertising its address to the NameNode as 127.0.0.1. You need to re-configure your Pseudo distributed cluster such that the nodes use externally available addresses (hostnames or IP addresses) when opening socket services.
I imagine if you run a netstat -atn on your VM, you'll see the Hadoop ports bound to 127.0.0.1 rather than 0.0.0.0 - this means they will only accept internal connections.
You need to look at your VM's /etc/hosts configuration file and ensure hostname doesn't have an entry resolving to 127.0.0.1.
Whenever you start a VM, it gets its own I.P. Something like 192.x.x.x or 172.x.x.x.
Using 127.0.0.1 for HDFS wont help when you are executing from your windows box, because this is mapped to local i.p. So, if you are using 127.0.0.1 from your windows machine, it will think that your HDFS is running on windows machine. This is why your connection is failing.
Find the i.p that is associated with your VM. Here is a link to get that if you are using Hyper-V. http://windowsitpro.com/hyper-v/quickly-view-all-ip-addresses-hyper-v-vms
Once you get the VMs I.P, use it in the application.
You need to change the ip. First go to linux VM and in its terminal find the IP address of your VM.
Command to see the ip address in linux VM is below
ifconfig
Then in your code change the ip address to the IP thats shown in your linux VM.
Related
I want to connect to HBase container on remote server that I connect to with ssh through VPN. Let's say it's 10.0.0.10.In /etc/hosts i placed:
10.0.0.10 hbaseaddr
In my java code I use hbase-client:
config.set("hbase.zookeeper.quorum", "hbaseaddr");
config.set("hbase.zookeeper.property.clientPort", "2181");
I get following error:
Can not resolve 791995b8a2df, please check your network
What is 791995b8a2df? Also, surprisingly, when I have a VPN turned off it just stays idle and does nothing, so it really is connecting to 10.0.0.10, then why do I have this error?
I read that it might be the issue with /etc/hosts. But I have /etc/hosts inside a local machine, /etc/hosts inside machine on 10.0.0.10 and /etc/hosts inside container with HBase.
What can I do to make it work?
Thanks in advance
I recently started using Apache Kafka and was successful sending data from producer to consumer within Windows 10 and my Linux Mint 19.3 (VM) environments. Now I want to stream data from my VM to my Windows 10 environment.
Both environments are able to ping each other.
When I try to run my consumer on Windows 10, I get an "Error connecting to node VM-VirtualBox:9092'" followed by a "java.net.UnknownHostException: VM-VirtualBox". Strangely, Windows 10 is able to SEE the topics listed on the VM environment; and I am able to create and delete topics from Windows. But I just cannot connect.
Could someone point out what I'm doing wrong?
My server.properties include lines:
broker.id=0
Kafka Broker Advanced Configuration
Snippet (Safety Valve)
############################# Socket Server Settings #############################
# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.
# FORMAT:
# listeners = listener_name://host_name:port
# EXAMPLE:
# listeners = PLAINTEXT://your.host.name:9092
#port=9092
#advertised.host.name=localhost
listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners = PLAINTEXT://VM-VirtualBox:9092
#advertised.listeners = PLAINTEXT://Windows10-7950:9092
domain:9092
I've also posted this on Reddit: https://www.reddit.com/r/apachekafka/comments/emqbms/virtualbox_streaming_data_from_vm_to_host_pc/
If Kafka is running on the host, you need to advertise the hosts hostname, or some address that the VM can resolve (it can be an IP address, too)
listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners = PLAINTEXT://EXTERNAL-HOST:9092
Otherwise, you need to edit the hosts file in the OS such that the DNS name is recognized
The reason you can list and create topics is because you're probably using Zookeeper, not the bootstrap address
I'm trying to connect JVisualVM, running on my local machine, to a remote machine which is running a WildFly server (version 8.1.0, to be specific.)
I didn't configure the WildFly server myself, and I don't know who did, but I do know that I can log in as an administrative user from my local machine by pointing my browser at:
https://[ip address of the remote machine]:9443/console
Note that it's https, not ordinary http, and that the port for that has been set to 9443 (I think the default is 8080 or 9990 or something... IDK, I saw a lot of port numbers online. I have been explicitly told that http was disabled for this WildFly server).
I can SSH into the remote machine. I can navigate to the bin directory for WildFly and run jboss-client.sh. I have to connect on port 9999 (I think the default is 9990 for that?)
I copied the jboss-client.jar (under bin/client) to my local machine and ran JVisualVM from the command line like this:
.\jvisualvm.exe -cp:a C:\[path to]\jboss-client.jar
It launches fine. File > Add Remote Host: Then I entered the IP. OK. I right clicked on it under Remote in the tree and picked Add JMX Connection. I entered
service:jmx:http-remoting-jmx://[ip]:9999
I checked off that I wanted to use the security credentials and entered the username and password. Checked off to save the security credentials. Left "Do not require SSL Connection" unchecked. Hit OK. It immediately spat out the message
Cannot connect to admin#service:jmx:http-remoting-jmx://[ip]:9999 using service:jmx:http-remoting-jmx://[ip]:9999
I also tried the port 9443, 9990, and 8080 instead. None of those worked. I tried https instead of http in the protocol name. That also didn't work.
What am I missing? How is it that I can access the console, and connect with jboss-client.sh, but I can't use JVisualVM? Is there some log I can use somewhere to see what's wrong? Maybe someone can point out a configuration I've missed somewhere?
Not sure if it's important or not, but my local machine is running Windows 10 with JDK8 installed. The WildFly server is using Java 6 on CentOS 6.3.
You need to add the jboss-client.jar (or jboss-cli-client.jar) to the class path for JVisualVM. The library can be found in the bin/client directory of the WildFly install.
I used the following command to add the library to the class path.
jvisualvm --cp:a ~/servers/wildfly-10.0.0.Final/bin/client/jboss-client.jar
Then I used service:jmx:remote+http://[ip]:[port] and was able to connect.
I don't know if someone else is also (still) having the same issue (Wildfly10 on a remote machine where management console is available at 9443 with HTTPS). The following worked for me.
For ssh connections:
Starting jvisualvm with jboss-client.jar
jvisualvm --cp:a #JBOSS_HOME/bin/client/jboss-client.jar
Using the following connection string:
service:jmx:remote+https://remote-server:9443
NOTE: I used here remote+https
Provide username and password
Hope this helps.
you missed run jstatd command in remote host ,
this little program is RMI server that possible connection from client to remote host though you using jmx connection it used jmxrmi protocol for that connection .
so first in remote host create file name as security.policy with this contain :
grant codebase "file:${java.home}/../lib/tools.jar" {
permission java.security.AllPermission;
};
off course you must in file section for linux put explicit path and then of creation this file put it in bin directory of jdk.home
then you should run this command on remote host
$JAVA_HOME/bin/jstatd -J-Djava.security.policy=path of /security.policy -J-Djava.rmi.server.hostname=remote ip address -J-Djava.net.preferIPv4Stack=true
then you could connect to server off course with correct settings.
Include jboss-cli-client.jar and jboss-client.jar under \lib\visualvm\platform\lib and restart jvisualvm to pickup new jars.
When I connect to GridGain node using JConsole or VisualVM, I get IOException.
How to overcome this?
Check if the hostname correctly resolves to the host address.
Run the command:
hostname -i
If it reports 127.0.0.1, JConsole/VisualVM would not be able to connect through JMX to the JVM running on that Linux machine. To fix this issue, edit /etc/hosts such that the hostname resolves to the host address.
You can also add system property to GridGain: -Djava.rmi.server.hostname=<hostname>
I'm on a vista machine. I've started tomcat 5.5.27 with these options:
CATALINA_OPTS="-Dcom.sun.management.jmxremote.port=9003 \
-Dcom.sun.management.jmxremote.ssl=false \
-Dcom.sun.management.jmxremote.authenticate=false"
When I connect via jconsole and added the following service url
service:jmx:rmi:///jndi/rmi://localhost:9003/jmxrmi
it would not connect. Any ideas ?
Ok, I orignally supposed the URL given by op was wrong but it turns out no. So I can't answer.
Still, here are the basics:
For a simple connection through jconsole.
If you know that the JMX Server you want to connect to has the RMI registry port at 9003 for example, connect using
localhost:9003
instead.
Otherwise, here's what I found out from the ground up:
Suppose you have the JMX Server (alias 'JMX Agent' alias 'the JVM you want to connect to') running on 'TARGET MACHINE' with the RMI registry port at 'RMI REGISTRY PORT' and the JMX RMI server port at 'JMX RMI SERVER PORT'.
Note:
The RMI registry tells JMX clients where to find the JMX RMI server port; information can be obtained under key jmxrmi.
The RMI registry port is generally known as it is set through system properties at JVM startup.
The JMX RMI server port is generally not known as the JVM chooses it at random (if no other precautions are taken).
The following URI will lead to success (tested)
service:jmx:rmi://<TARGET_MACHINE>:<JMX_RMI_SERVER_PORT>/jndi/rmi://<TARGET_MACHINE>:<RMI_REGISTRY_PORT>/jmxrmi
This looks nasty. Let's cut it apart.
This URI is an RFC2609 "Service Location Protocol URL" (well, it's really an URI, right?)
It is composed of:
service - a constant
jmx:rmi - the service type composed of: abstract type jmx and URL scheme rmi
the rest - the sap (service access protocol specification)
sap is decomposed into:
//<TARGET_MACHINE>:<JMX_RMI_SERVER_PORT> - ipsite
/jndi/rmi://<TARGET_MACHINE>:<RMI_REGISTRY_PORT>/jmxrmi - URL part
A well-informed JMX client connects to the "ipsite" to do JMX-over-RMI exchanges; but what of the JMX client that doesn't KNOW that port? Patience...
URL part is decomposed into:
/jndi/ - This seems to tell the JMX client that it can get lookup information at the location that follows
rmi://<TARGET_MACHINE>:<RMI_REGISTRY_PORT>/jmxrmi - Yep, we get information about the JMX RMI Server at the RMI registry, under the lookup key jmxrmi
This is somewhat cart-before-horse, as one has to contact the RMI registry given by the latter part of the SLP URL first.
After scratching head, intuitively, let's try:
service:jmx:rmi://<TARGET_MACHINE>/jndi/rmi://<TARGET_MACHINE>:<RMI_REGISTRY_PORT>/jmxrmi
Yes, that works! The JMX RMI server port is nicely obtained from the registry. On second thoughts, the target machine should also be obtained from the registry, thus:
service:jmx:rmi:///jndi/rmi://<TARGET_MACHINE>:<RMI_REGISTRY_PORT>/jmxrmi
Even better, that works, too!
References:
1 download.oracle.com/javase/6/docs/api/javax/management/remote/rmi/package-summary.html
2 download.oracle.com/javase/6/docs/api/javax/management/remote/JMXServiceURL.html
3 mx4j.sourceforge.net/docs/ch03s04.html
4 download.oracle.com/javase/6/docs/technotes/guides/management/agent.html#gdevg
5 http://www.rfc-editor.org/rfc/rfc2609.txt
On Ubuntu 10.04, using OpenJDK 6 and Tomcat 6.0.29, I was unable to activate JMX for a local jconsole session, no matter how many com.sun.management.jmxremote.* options I passed to java with CATALINA_OPTS. The problem was the -Djava.io.tmpdir setting, which defaults to $CATALINA_BASE/temp. I simply had to set:
CATALINA_TMPDIR="/tmp"
at the beginning of bin/catalina.sh and I was able to connect locally with jconsole, jmap, jps etc. There was no need for any com.sun.management.jmxremote.* settings at all.
Are the processes run under the same user?
You can also check by running jps and jconsole (both in the JDK_HOME/bin directory)
This is also needed for OS X 10.7 aka Lion.
I answered a similar question here:java.rmi.ConnectException: Connection refused to host: 127.0.1.1;
I found many of the Q&A on this topic, not nothing was helping me - that's because my issue was more basic ( what can I say I am not a networking guru :) ). My ip address in /etc/hosts was incorrect. What I had tried included the following for CATALINA_OPTS:
CATALINA_OPTS="$CATALINA_OPTS -Djava.awt.headless=true -Xmx128M -server
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=7091
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Djava.rmi.server.hostname=A.B.C.D" #howeverI put the wrong ip here!
export CATALINA_OPTS
My problem was that I had changed my ip address many months ago, but never updated my /etc/hosts file. it seems that by default the jconsole uses the hostname -i ip address in some fashion even though I was viewing local processes. The best solution was to simply change the /etc/hosts file.
The other solution which can work is to get your correct ip address from /sbin/ifconfig and use that ip address when specifying the ip address in, for example, a catalina.sh script:
-Djava.rmi.server.hostname=A.B.C.D