I am trying to connect to HiveServer2 via JDBC drivers from R using RJDBC package. I have seen a broad explanation on passing additional arguments to dbConnect wrapper for various drivers(What arguments can I pass to dbConnect?), but there appear that situation with JDBCDriver is a bit tricker than for other drivers. I can connect to HiveServer2 under this specific URL adress url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000, "/loghost;auth=noSasl") . The correspoding code works and enables me to write statements on Hive from R
library(RJDBC)
dbConnect(drv = JDBC(driverClass = "org.apache.hive.jdbc.HiveDriver",
classPath = c("/opt/hive/lib/hive-jdbc-1.0.0-standalone.jar",
"/usr/share/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar",
"/usr/share/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar"),
identifier.quote = "`"), # to juz niekoniecznie jest potrzebne
url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000, "/loghost;auth=noSasl"),
username = "mkosinski") -> conn
I am wondering if there is a way to pass arguments such as database name (loghost) or a no_authentication_mode (auth=noSasl) to ... in dbConnect such that I could only specify standard URL address (url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000)) and somehow pass the rest of parametrs like this
library(RJDBC)
dbConnect(drv = JDBC(driverClass = "org.apache.hive.jdbc.HiveDriver",
classPath = c("/opt/hive/lib/hive-jdbc-1.0.0-standalone.jar",
"/usr/share/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar",
"/usr/share/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar"),
identifier.quote = "`"), # to juz niekoniecznie jest potrzebne
url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000),
username = "mkosinski", dbname = "loghost", auth = "noSasl") -> conn
But the second approach doesn't look to work, despite the various combinations of names and values of additional arguments I try.
Does anyone know how to pass additional arguments to DBI::dbConnect through ... parameter for JDBCDriver?
According to the author's answer: https://github.com/s-u/RJDBC/issues/31#issuecomment-173934951
Simply anything - all that dbConnect does is to collect whatever you
pass (including ...) and collect it all into a property dictionary
(java.util.Properties) that is passed to the driver's connect()
method. So any named argument you pass is included. So the only
special argument is url which is passed directly, everything else is
included in the properties. How that gets interpreted is out of
RJDBC's hands - it's entirely up to the driver.
there you can use the full url
library(RJDBC)
drv <- JDBC("org.postgresql.Driver","C:/R/postgresql-9.4.1211.jar")
con <- dbConnect(drv, url="jdbc:postgresql://host:port/dbname", user="<user name>", password="<password>")
Related
My calls to Oracle via jruby aren't closing their db connections.
Here is the code from the webpage making the call:
<%
require 'jdbc_ssl_connection'
# Database settings
url = "jdbc:oracle:thin:#(DESCRIPTION=(ADDRESS=(PROTOCOL=TCPS)(HOST=**REMOVED**)(PORT=**REMOVED**))(CONNECT_DATA=(SERVICE_NAME=**REMOVED**)))"
output = ""
select_stmt, rest, select_sql = nil
begin
conn = OracleConnection.create(url)
# Display connection using the to_s method of OracleConnection
select_sql = "select FIELD from SCHEMA.TABLE WHERE FIELD='"+#subject["file-name"].first+"'"
select_stmt = conn.create_statement
rset = select_stmt.execute_query select_sql
while (rset.next)
output = output + rset.getString(1)
end
rescue
error = "Error:", $!, "\n"
ensure
if (!select_stmt.nil?)
select_stmt.close
end
if (!rset.nil?)
rset.close
end
if (!conn.nil?)
conn.close_connection
end
end
%>
Here is the class that interacts with the driver.
# jdbc_ssl_connection.rb
require 'java'
java_import 'oracle.jdbc.OracleDriver'
java_import 'java.sql.DriverManager'
java_import 'java.util.Properties'
class OracleConnection
#conn = nil
def initialize (url)
#url = url
properties = java.util.Properties.new
properties['user'] = 'REMOVED'
properties['password'] = 'REMOVED'
# Load driver class
oradriver = OracleDriver.new
DriverManager.registerDriver oradriver
#conn = DriverManager.get_connection url, properties
#conn.auto_commit = false
end
# Add getters and setters for all attributes we wish to expose
attr_reader :url, :connection
def close_connection()
#conn.close() unless #conn
end
def prepare_call(call)
#conn.prepare_call call
end
def create_statement()
#conn.create_statement
end
def prepare_statement(sql)
#conn.prepare_statement sql
end
def commit()
#conn.commit
end
def self.create(url)
conn = new(url)
end
def to_s
"OracleConnection [url=#{#url}]"
end
alias_method :to_string, :to_s
end
The code works and is pretty simple. I ran a test and I have about 100 open sessions on the db. For some reason the call to close the connection isn't stopping the session. Any ideas what might be wrong?
def close_connection()
#conn.close() unless #conn
end
because of the conditional, you really wanted: #conn.close if #conn
I have written applications in Java which connect to the database by taking the username and password for the database which is set in the environment variables. So the database username and password are stored as clear text in a file on the server.
Here is the code which creates the connection string:
if (dataBase.equals("oracle"))
{
url = "jdbc:oracle:thin:#";
url = url + getParameter("Database IP", "setup.ini"); // ip
url = url + ":" + getParameter("Database Port", "setup.ini"); //port
String envVarValue = System.getenv("DBNAME");
// Environment variable has value DBNAME =
String a[] = envVarValue.split("#");
url = url + ":" + getParameter("SID", "setup.ini");
String b[] = a[0].split("/");
userName = b[0]; //User name
password = b[1]; //Password
}
Now I want my application to be able to connect to the database by the Unix user who would be running this application on the server without the need to get the password for the database user in the code.
Please tell me if this is possible and how if yes?
Refer this link on how to configure a user account for OS authentication in Oracle: http://docs.oracle.com/cd/B28359_01/java.111/b31224/clntsec.htm#CIHCBCBC
You can read the current OS user running your java program by calling System.getProperty("user.name"). When creating connection through JDBC, you may have to append this value to the prefix OS_AUTHENT_PREFIX you created as shown in the link above.
Spaces not changing to underscored when sent from Java-->PHP-->SQL
Java code:
String urlString = "http://www.mysite.com/auth/verifyuser.php?name="+name.toLowerCase().replace(" ","_");
PHP code:
$name = mysql_real_escape_string($_GET['name']);
$name = str_replace(' ', '_', $name);
$query = "select * from authinfo where name LIKE '$name'";
mysql_query($query);
$num = mysql_affected_rows();
if ($num > 0) {
echo '1';
} else {
echo '0';
}
when I implement a test log on the SQL database, it somehow still seems to show up with spaces instead of underscores(even though I replace it in Java and PHP) and the PHP file returns '0' rather than '1'. I've heard the issue might be whitespaces? It seems to happen to only certain users, mostly mac users.
If your php file is returning a 0, that means your query is not getting executed. Where are you establishing a connection with the database before executing the query?
Remark: where name = '$name'
mysql_affected_rows concerns INSERT, UPDATE and DELETE.
$r = mysql_query($query);
$num = mysql_num_rows($r);
It's unsafe to pass raw name into URL without encoding it.
String urlString = "http://www.example.com/auth/verifyuser.php?name=" + URLEncoder.encode(name.toLowerCase(), "UTF-8");
In PHP you can obtain data:
$name = urldecode($_GET['name']);
I'm doing a recursive url harvest.. when I find an link in the source that doesn't start with "http" then I append it to the current url. Problem is when I run into a dynamic site the link without an http is usually a new parameter for the current url. For example if the current url is something like http://www.somewebapp.com/default.aspx?pageid=4088 and in the source for that page there is a link which is default.aspx?pageid=2111. In this case I need do some string manipulation; this is where I need help.
pseudocode:
if part of the link found is a contains a substring of the current url
save the substring
save the unique part of the link found
replace whatever is after the substring in the current url with the unique saved part
What would this look like in java? Any ideas for doing this differently? Thanks.
As per comment, here's what I've tried:
if (!matched.startsWith("http")) {
String[] splitted = url.toString().split("/");
java.lang.String endOfURL = splitted[splitted.length-1];
boolean b = false;
while (!b && endOfURL.length() > 5) { // f.bar shortest val
endOfURL = endOfURL.substring(0, endOfURL.length()-2);
if (matched.contains(endOfURL)) {
matched = matched.substring(endOfURL.length()-1);
matched = url.toString().substring(url.toString().length() - matched.length()) + matched;
b = true;
}
}
it's not working well..
I think you are doing this the wrong way. Java has two classes URL and URI which are capable of parsing URL/URL strings much more accurately than a "string bashing" solution. For example the URL constructor URL(URL, String) will create a new URL object in the context of an existing one, without you needing to worry whether the String is an absolute URL or a relative one. You would use it something like this:
URL currentPageUrl = ...
String linkUrlString = ...
// (Exception handling not included ...)
URL linkUrl = new URL(currentPageUrl, linkUrlString);
I need to create TQL queries to query out sets of data from the UCMDB.
I am having 2 problems:
1) How can I find relationships which exists between CIs ( i do not have administrative privileges so need to do it in code somehow)
I need this to get required data.
2) I have created the following query: But I keep getting the IP property value as null.
I checked that IP has an attribute called ip_address.
Code:
import com.hp.ucmdb.api.types.TopologyRelation;
public class Main {
public static void main(String[] args)throws Exception {
final String HOST_NAME = "192.168.159.132";
final int PORT = 8080;
UcmdbServiceProvider provider = UcmdbServiceFactory.getServiceProvider(HOST_NAME, PORT);
final String USERNAME = "username";
final String PASSWORD = "password";
Credentials credentials = provider.createCredentials(USERNAME, PASSWORD);
ClientContext clientContext = provider.createClientContext("Test");
UcmdbService ucmdbService = provider.connect(credentials, clientContext);
TopologyQueryService queryService = ucmdbService.getTopologyQueryService();
Topology topology = queryService.executeNamedQuery("Host IP");
Collection<TopologyCI> hosts = topology.getAllCIs();
for (TopologyCI host : hosts) {
for (TopologyRelation relation : host.getOutgoingRelations()) {
System.out.print("Host " + host.getPropertyValue("display_label"));
System.out.println (" has IP " + relation.getEnd2CI().getPropertyValue("ip_address"));
}
}
}
In the above query output: I get the host names with IP = null
I have a sample query in JYthon which I am unable to figure out: Its for the above code only.
Attaching it for anyone who can understand it.
import sys
UCMDB_API="c:/ucmdb/api/ucmdb-api.jar"
sys.path.append(UCMDB_API)
from com.hp.ucmdb.api import *
# 0) Connection settings
HOST_NAME="192.168.159.132"
PORT=8080
USERNAME="username"
PASSWORD="password"
# 1) Get a Service Provider from the UcmdbServiceFactory
provider = UcmdbServiceFactory.getServiceProvider(HOST_NAME, PORT)
# 2) Setup credentials to log in
credentials = provider.createCredentials(USERNAME, PASSWORD)
# 3) Create a client context
clientContext = provider.createClientContext("TESTING")
# 4) Connect and retrieve a UcmdbService object
ucmdbService = provider.connect(credentials, clientContext)
# 5) Get the TopologyQueryService from the UcmdbService
queryService = ucmdbService.getTopologyQueryService()
# ======= Everything After this is specific to the query =======
# 6) Execute a Named Query and get the Topology
topology = queryService.executeNamedQuery('Host IP')
# 7) Get the hosts
hosts = topology.getAllCIs()
# 8) Print the hosts and IPs
host_ip = {}
for host in hosts:
host_name = host.getPropertyValue("display_label")
if host_name in host_ip.keys():
ips = host_ip[host_name]
else:
ips = {}
host_ip[host_name] = ips
for relation in host.getOutgoingRelations():
ip_address = relation.getEnd2CI().getPropertyValue("display_label")
if ip_address in ips.keys():
pass
else:
ips[ip_address] = ''
print "%s , %s" % (host_name, ip_address)
Please help.
I am unable to understand how to go about this further.
Thank you.
The easiest fix would be use the display_label property from the IP address CI instead of the ip_address property. The Jython reference code uses display_label for its logic.
I'd be a little concerned about using display_label since the display_label formatting logic could be changed to no display the IP address for IP CIs. Getting data directly from the ip_address property is a better choice and should work if the TQL is defined to return that data. Check the Host IP TQL and ensure that it's configured to return ip_address for IP CIs.