Develop Reference

Hive JDBC connection problems

I am trying to connect to Hive2 server via JDBC with kerberos authentication. After numerous attempts to make it work, I can't get it to work with the Cloudera driver.
If someone can help me to solve the problem, I can greatly appreciate it.
I have this method:
private Connection establishConnection() {
final String driverPropertyClassName = "driver";
final String urlProperty = "url";
Properties hiveProperties = config.getMatchingProperties("hive.jdbc");
String driverClassName = (String) hiveProperties.remove(driverPropertyClassName);
String url = (String) hiveProperties.remove(urlProperty);
Configuration hadoopConfig = new Configuration();
hadoopConfig.set("hadoop.security.authentication", "Kerberos");
String p = config.getProperty("hadoop.core.site.path");
Path path = new Path(p);
hadoopConfig.addResource(path);
UserGroupInformation.setConfiguration(hadoopConfig);
Connection conn = null;
if (driverClassName != null) {
try {
UserGroupInformation.loginUserFromKeytab(config.getProperty("login.user"), config.getProperty("keytab.file"));
Driver driver = (Driver) Class.forName(driverClassName).newInstance();
DriverManager.registerDriver(driver);
conn = DriverManager.getConnection(url, hiveProperties);
} catch (Throwable e) {
LOG.error("Failed to establish Hive connection", e);
}
}
return conn;
}
URL for the server, that I am getting from the properties in the format described in Cloudera documentation
I am getting an exception:
2018-05-05 18:26:49 ERROR HiveReader:147 - Failed to establish Hive connection
java.sql.SQLException: [Cloudera][HiveJDBCDriver](500164) Error initialized or created transport for authentication: Peer indicated failure: Unsupported mechanism type PLAIN.
at com.cloudera.hiveserver2.hivecommon.api.HiveServer2ClientFactory.createTransport(Unknown Source)
at com.cloudera.hiveserver2.hivecommon.api.ZooKeeperEnabledExtendedHS2Factory.createClient(Unknown Source)
...
I thought, that it is missing AuthMech attribute and added AuthMech=1 to the URL. Now I am getting:
java.sql.SQLNonTransientConnectionException: [Cloudera][JDBC](10100) Connection Refused: [Cloudera][JDBC](11640) Required Connection Key(s): KrbHostFQDN, KrbServiceName; [Cloudera][JDBC](11480) Optional Connection Key(s): AsyncExecPollInterval, AutomaticColumnRename, CatalogSchemaSwitch, DecimalColumnScale, DefaultStringColumnLength, DelegationToken, DelegationUID, krbAuthType, KrbRealm, PreparedMetaLimitZero, RowsFetchedPerBlock, SocketTimeOut, ssl, StripCatalogName, transportMode, UseCustomTypeCoercionMap, UseNativeQuery, zk
at com.cloudera.hiveserver2.exceptions.ExceptionConverter.toSQLException(Unknown Source)
at com.cloudera.hiveserver2.jdbc.common.BaseConnectionFactory.checkResponseMap(Unknown Source)
...
But KrbHostFQDN is already specified in the principal property as required in the documentation.
Am I missing something or is this documentation wrong?

Below is the one of the similar kind of problem statement in Impala (just JDBC engine changes others are same) that is resolved by setting "KrbHostFQDN" related properties in JDBC connection string itself.
Try to use the URL below. Hopefully works for u.
String jdbcConnStr = "jdbc:impala://myserver.mycompany.corp:21050/default;SSL=1;AuthMech=1;KrbHostFQDN=myserver.mycompany.corp;KrbRealm=MYCOMPANY.CORP;KrbServiceName=impala"
I suppose that if you are not using SSL=1 but only Kerberos, you just drop that part from the connection string and don't worry about setting up SSL certificates in the java key store, which is yet another hassle.
However in order to get Kerberos to work properly we did the following:
Install MIT Kerberos 4.0.1, which is a kerberos ticket manager. (This is for Windows)
This ticket manager asks you for authentication every time you initiate a connection, creates a ticket and stores it in a kerberos_ticket.dat binary file, whose location can be configured somehow but I do not recall exactly how.
Finally, before launching your JAVA app you have to set an environment variable KRB5CCNAME=C:/path/to/kerberos_ticket.dat. In your java app, you can check that the variable was correctly set by doing System.out.println( "KRB5CCNAME = " + System.getenv( "KRB5CCNAME" ) ). If you are working with eclipse or other IDE you might even have to close the IDE,set up the environment variable and start the IDE again.
NOTE: this last bit is very important, I have observed that if this variable is not properly set up, the connection wont be established...
In Linux, instead MIT Kerberos 4.0.1, there is a program called kinit which does the same thing, although without a graphical interface, which is even more convenient for automation.

I wanted to put it in the comment but it was too long for the comment, therefore I am placing it here:
I tried your suggestion and got another exception:
java.sql.SQLException: [Cloudera]HiveJDBCDriver Error
creating login context using ticket cache: Unable to obtain Principal
Name for authentication .
May be my problem is, that I do not have environment variable KRB5CCNAME set.
I, honestly, never heard about it before.
What is supposed to be in that ticket file.
I do have, however, following line in my main method:
System.setProperty("java.security.krb5.conf", "path/to/krb5.conf");
Which is supposed to be used by
UserGroupInformation.loginUserFromKeytab(config.getProperty("login.user"), config.getProperty("keytab.file"));
to obtain the kerberos ticket.

To solve this issue update Java Cryptography Extension for the Java version that you use in your system.
Here's the link when you can download JCE for Java 1.7
Uncompress and overwrite those files in $JDK_HOME/jre/lib/security
Restart your computer.

sometimes I unable to connect to connect to socket using php java bridge

we are using java bridge from PHP application to connect to java application.
But at times, connection get failed to connect. Below is the fucntion used to connect. I am not getting what the issue is. Sometimes it works and sometimes it fails to connect. Any persistent connection issues or count is set?
function open() {
$errno = null;
$errstr = null;
$socket = JAVA_PERSISTENT_SERVLET_CONNECTIONS ?
pfsockopen("{$this->ssl}{$this->host}", $this->port, $errno, $errstr, 20) :
fsockopen("{$this->ssl}{$this->host}", $this->port, $errno, $errstr, 20);
if (!$socket)
throw new java_ConnectException("Could not connect to the J2EE server {$this->ssl}{$this->host}:{$this->port}. Please start it. Or define('JAVA_HOSTS', 8080); define('JAVA_SERVLET', false); before including 'Java.inc' and try again. Error message: $errstr ($errno)\n");
stream_set_timeout($socket, -1);
return $socket;
}
function java_HttpHandler($protocol, $ssl, $host, $port) {
parent::java_SimpleHttpHandler($protocol, $ssl, $host, $port);
try {
$this->socket = $this->open();
} catch (Exception $e) {
$cogLink = "http://xxxx.xx.com/products/sup_products.asp?prod_id=81174";
echo "eeeeerrrr";
}
}

Any persistent connection issues or count is set?
Most likely the back-end ran out of handles. Please note that, in order to obtain a persistent connection to a java "continuation", you'll have to send a servlet engine a POST request first. After that you can pfsockopen() to the java "continuation" the servlet engine has returned and use protocol request
(F p="A"/)
to recycle- and
(F p="F"/)
to put the "continuation" back to the pool.
I have checked the Java.inc code and it doesn't contain the code you have posted. Therefore I assume that you have written custom code to emulate Java.inc.
If so, please make sure to no exceed the bridge thread pool size (20 by default).

Connection to derby database refused

I am learning Java and trying to put together a simple App containing a few jTables connected to database that can be updated etc. To do this I have created a database with a few tables through Netbeans which I understand (and wish) to be embedded in the final distributable app.
I am following the Programming Knowledge tutorials on Youtube to create most of the GUI. Everting is OK as long as I open the Services tab on Netbeans and manually right-click my database(testDB) and click Start Server. Then when I run the following code I get a successful connection:
private void jButton1ActionPerformed(java.awt.event.ActionEvent evt) {
try{
//Register JDBC driver
Class.forName("org.apache.derby.jdbc.ClientDriver");
//Open a connection
String DB_URL = "jdbc:derby://localhost:1527/testDB";
String u_name = jTextField1.getText();
String p_word = jPasswordField1.getText();
conn = DriverManager.getConnection("jdbc:derby://localhost:1527/testDB",u_name,p_word);
JOptionPane.showMessageDialog(null,"Details Correct - Connection established");
Close_me();
Open_Table_GUI(u_name,p_word);
}
catch(ClassNotFoundException | SQLException e){
System.out.println(e);
}
}
However if I run that code WITHOUT manually clicking on start server I get the following:
java.sql.SQLNonTransientConnectionException: java.net.ConnectException : Error connecting to server localhost on port 1,527 with message Connection refused.
I have read the apache documentation and due to my level of inexperience I am not getting anywhere.
I have also checked out the answers to similar connection questions on here but again I can't seem to relate the issue in a way that works.
The ultimate goal for me is to have an app I can distribute to run on windows machines that will have the database/tables all included individually editable etc. I would hope to eventually create a database that resides on a shared drive and each individual can connect to automatically - but that's way down the line for now.
My request here is that someone can help me understand what I need to change in my code so that the "Start Server" is automatically done.
Thanks in advance for nay response.

OK guys so I found code that seems to work for me from a question asked on here by LMS
private void setup(){
try{
Class.forName("org.apache.derby.jdbc.EmbeddedDriver");
connection=DriverManager.getConnection("jdbc:derby:sheet;create=true","test","1234");
statement=connection.createStatement();
}
catch(ClassNotFoundException cnf){
System.out.println("class error");
}
catch(SQLException se){
System.out.println(se);
}
}
If anyone cares to link me to a reason why this in fact works that would be great, I'm assuming the Embedded driver doesn't go through the port and just looks within the project??
Anyway initially this seems to be solved!!

Please check your Server setting i.e.
-username and pwd in your program
and check that did you start the Derby server and is it listening at port 1527?

Java URLConnection Error

This error is from a PC connected to a comcast modem. Is port blocking an issue when a person is trying to make an outbound TCP channel with a Java URL . . . the relevant code is :
URL u = new URL("mailto:someguy#gmail.com"); // make a mailto: url
URLConnection c = u.openConnection(); // make its urlconnection
c.connect(); // connect to mail host
catch (Exception e) { // handle any exceptions, print error message
System.err.println(e);
}
The console returns the message :
connect. Timeout = -1
java.net.UnknownHostException: mailhost

MailToURLConnection is NOT in Java 6 or 7 or any earlier version. It is part of the OpenJDK implementation of J2SE specifications. Download and install OpenJDK and use that as your Java JDK and the problem will probably go away. I have checked, OpenJDK 6 has the MailToURLConnection class.

This exception java.net.UnknownHostException: mailhost is thrown to indicate that the IP address of a host could not be determined.
I reckon you might be trying to send a mail.

I cannot find any documentation but it seems URL.openConnection() above creates a sun.net.www.protocol.mailto.MailToURLConnection. Looking at the code here it creates an SmtpClient.
This creates a connection to System.getProperty("mail.host") or to "localhost" or to "mailhost". I guess the "mail.host" is not defined for you and localhost is not accepting the connection so the dns error for mailhost.
Where did find information to use "mailto:" anyway? That same place should also have the rest of usage documentation. If you were just trying something or ran into it accidentally then I would advice not to use it.

A problem connecting to a MySQL DB using JDBC

Here's how I'm trying to connect:
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
} catch (Exception e) {
throw new DbConnectionException();
}
try {
connection = DriverManager.getConnection(url,username,password);
} catch (SQLException e) {
e.printStackTrace();
throw new DbConnectionException();
}
I'm 100% sure that the url, username, password strings are correct. I've already connected successfully using an external tool (MySQL query browser).
This is the error I receive:
com.mysql.jdbc.CommunicationsException:
Communications link failure due to
underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException MESSAGE:
java.net.ConnectException: Connection
refused
...

Possibly a url issue. If your code is pointing to MySQL localhost, try changing localhost to 127.0.0.1 on your url.
E.g.:
jdbc:mysql://localhost:3306/MY_DB
to
jdbc:mysql://127.0.0.1:3306/MY_DB
And see if this works.

did you run the mysql browser from the same machine where the code is running? What I am getting at is the permissions in mysql can be host-specific, and depending on how you set them up you might not be able to connect from the machine where the code is running.
Also, you might want to double check the url, name, pword again, perhaps with log statements or a debugger to make sure there are no typos, trailing whitespaces, etc...

Double check the format of your url. It should start with "jdbc:mysql:". Make sure you are using a current version for the driver as well.

Check that you can connect to the database from the mysql admin tool, that will drive out whether your mysql is running and that the port is open.

In my case the problem was that I was using a connection from emulator to localhost.
If you use emulator to localhost don't use localhost value in connection String but use 10.0.2.2 instead:
jdbc:mysql://10.0.2.2:3306/MY_DB
Hope this helps.