I have a small program wrapping Oracles SQLLoader utility. The database address parameter SQLLoader takes seems to be a TNS style address like so:
username/password#schemaname
My program also uses JDBC to access the database and perform post load diagnostics. Unfortunately at the moment, you need to supply the address as well like so:
jdbc:oracle:thin:username/password#172.17.125.131:1521:EE
Is there anyway to convert to and from these? I.e. if given a JDBC url, convert to a TNS address SQLLoader will take, and vice versa.
Your SQL*Loader connection is using a TNS alias (so schemaname is not quite right generically, but may happen to be true for you). Converting from that to something like the JDBC string would involve parsing the data out of the tnsnames.ora file, which is doable but not trivial (though I'm sure someone with better awk-fu than me would disagree). If you look in that file you should see an entry with address values corresponding to the JDBC values.
Going the other way is much simpler as you can use 'easy connect' syntax in your SQL*Loader command in place of the TNS alias:
sqlldr username/password#172.17.125.131:1521/EE control=...
As you can see, this is almost trivial to convert to (or from) a JDBC address, as you just have to add (or remove) the jdbc:oracle:thin: part. Except, the final delimiter changes from / to :. How you change that will depend on your wrapper program - whether it's a Windows batch file, a *nix shell script, a C executable etc. The simplest way may be to obtain the elements separately and concatenate them differently, but depends on how you're calling it too.
You can create the required strings you need from the database once connected (and without been sysadmin).
Perhaps it seems foolish but to query the same parameters from any utility will take you very long time to program.
SELECT
USER
||'/******#'
|| sys_context('userenv','db_name') as simpleconnection
,'jdbc:oracle:thin:'
||USER
||'/******#'
|| sys_context('userenv', 'server_host') --host
|| ':'
|| sys_context('USERENV', 'SID') --port
|| ':'
|| sys_context('userenv','db_name') AS "Oracle Eight" --SID
,'jdbc:oracle:thin:'
||USER
||'/******#'
||sys_context('userenv', 'server_host') --host
||':'
||sys_context('USERENV', 'SID') --port
||'/'
|| sys_context('userenv','service_name') AS "host form"
,'jdbc:oracle:thin:'
||USER
||'/******#'
||sys_context('userenv', 'ip_address') --ip
||':'
||sys_context('USERENV', 'SID') --port
||'/'
|| sys_context('userenv','service_name') AS "IP form"
,'jdbc:oracle:thin:'
||USER
||'/******#(description=(address=(host='
||sys_context('userenv', 'server_host') --host
||')(protocol='
||sys_context('userenv', 'network_protocol') --protocol
||')(port='
||sys_context('USERENV', 'SID') --port
||'))(connect_data=(service_name='
|| sys_context('userenv','service_name') --service name
||')(server='
||sys_context('userenv','host') --name of the machine
||')))' AS "TNS form"
FROM dual d;
btw it's not considered a good practice to put login credentials into the connection string because a lack of readibility, you can add them as parameters.
Related
I am trying to remotely invoke an MBean via a commandline. Right now, I am able to list attributes and operations. For example, I can list all the attributes and operations for HotspotDiagnostic using this command:
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd 10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic
Which gives me this list of Attributes and Operations
Attributes:
DiagnosticOptions: DiagnosticOptions (type=[Ljavax.management.openmbean.CompositeData;)
ObjectName: ObjectName (type=javax.management.ObjectName)
Operations:
dumpHeap: dumpHeap
Parameters 2, return type=void
name=p0 type=java.lang.String p0
name=p1 type=boolean p1
getVMOption: getVMOption
Parameters 1, return type=javax.management.openmbean.CompositeData
name=p0 type=java.lang.String p0
setVMOption: setVMOption
Parameters 2, return type=void
name=p0 type=java.lang.String p0
name=p1 type=java.lang.String p1
But now lets say I want to invoke the dumpHeap operation which takes two parameters p0 and p1 of type string and boolean, respectively. How would I pass those arguments in?
I've tried these:
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic dumpHeap p0=aaa p1=true
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic dumpHeap aaa true
But I'm not sure what the syntax is, or even what I'm supposed to pass for the string parameter. This isn't for anything specific btw. Merely want to learn and understand more about how to leverage these operations from the command line. Any docs and assistance much appreciated.
EDIT: I'm naive. Oracle docs indicate the string param is an output file per this link. But still uncertain about how to pass the parameters into my command.
According to the cmdline-jmxclient documentation:
http://crawler.archive.org/cmdline-jmxclient/ you have to use comma-delimited parameters to pass to your operation.
So in your case if would be:
java -jar cmdline-jmxclient-0.10.3.jar admin:P#sSw0rd10.11.12.13:1111 com.sun.management:type=HotSpotDiagnostic dumpHeap test,true
Take note that there is an present bug in the cmdline jar file that doesn't take into account Java primitives(int, booelean, byte, etc.) and will throw a ClassNotFoundException because it can't find by the primitive name.
If you find yourself coming across this issue you can either apply the patch to the jar code that is documented here: https://webarchive.jira.com/browse/HER-1630. Or simply change the type field in the jmx endpoint code from it's primitive type to it's Wrapper object type (int -> Integer)
I want to parallelize my data writing process. I am writing a data frame to Oracle Database. This data has 4 million rows and 8 columns. It takes 6.5 hours without parallelizing.
When I try to go parallel, I get the error
Error in checkForRemoteErrors(val) :
7 nodes produced errors; first error: No running JVM detected. Maybe .jinit() would help.
I know this error. I can solve it when I work with single cluster. But I do not know how to tell other clusters the location of Java. Here is my code
Sys.setenv(JAVA_HOME='C:/Program Files/Java/jre1.8.0_181')
library(rJava)
library(RJDBC)
library(DBI)
library(compiler)
library(dplyr)
library(data.table)
jdbcDriver =JDBC("oracle.jdbc.OracleDriver",classPath="C:/Program Files/directory/ojdbc6.jar", identifier.quote = "\"")
jdbcConnection =dbConnect(jdbcDriver, "jdbc:oracle:thin:#//XXXXX", "YYYYY", "ZZZZZ")
By using Sys.setenv(JAVA_HOME='C:/Program Files/Java/jre1.8.0_181') I solve the same problem for single core. But when I go parallel
library(parallel)
no_cores <- detectCores() - 1
cl <- makeCluster(no_cores)
clusterExport(cl, varlist = list("jdbcConnection", "brand3.merge.u"))
clusterEvalQ(cl, .libPaths("C:/Users/onur.boyar/Documents/R/win-library/3.5"))
clusterEvalQ(cl, library(RJDBC))
clusterEvalQ(cl, library(rJava))
parLapply(cl, 1:length(brand3.merge.u$CELL_PH_NUM), function(x) dbSendUpdate(jdbcConnection, "INSERT INTO xxnvdw.an_cust_analytics VALUES(?,?,?,?,?,?,?,?)", brand3.merge.u[x, 1], brand3.merge.u[x,2], brand3.merge.u[x,3],brand3.merge.u[x,4],brand3.merge.u[x,5],brand3.merge.u[x,6],brand3.merge.u[x,7],brand3.merge.u[x,8]))
#brand3.merge.u is my data frame that I try to write.
I get the above error and I do not know how to set my Java location for other nodes.
I want to use parLapply since it is faster than foreach. Any help would be appreciated. Thanks!
JAVA_HOME environment variable
If the problem really is with the location of Java, you could set the environment variable in your .Renviron file. It is likely located in ~/.Renviron. Add a line to that file and this will be propagated to all R session that run via your user:
JAVA_HOME='C:/Program Files/Java/jre1.8.0_181'
Alternatively, you can just add that location to your PATH environment variable.
JVM Initialization via rJava
On the other hand the error message may point to just a JVM not being initialized, which you can solve with .jinit, a minimal example:
library(parallel)
cl <- makeCluster(detectCores())
parallel::parLapply(cl, 1:5, function(x) {
rJava::.jinit()
rJava::.jnew(class = "java/lang/Integer", x)$toString()
})
Working around Java use
This was not specifically asked, but you can also work around the need for Java dependency using ODBC drivers, which for Oracle should be accessible here:
con <- DBI::dbConnect(
odbc::odbc(),
Driver = "[your driver's name]",
...
)
I am trying to read events from the event log of two windows machines. One is Windows 10 Enterprise (works perfectly) and one is Small Business Server 2008. On the SBS2008 machine output is truncated at the 78th character but I can’t see why.
This is the powershell command that I am running:
get-eventlog Application -After (Get-Date).AddDays(-10) |
Where-Object {$_.EntryType -like 'Error' -or $_.EntryType -like
'Information'} | Format-Table -Property TimeGenerated, Index,
EventID, EntryType, Source, Message -autosize | Out-String -Width 4000
It performs fine in a powershell edit window on both machines, no truncation.
If I run it in a command window using powershell -file GetEvents.ps1 the output is truncated:
C:\BITS>powershell -file GetEvents.ps1
I cannot work out what is doing this. This is the Java code:
String command = "powershell.exe " + Cmd;
Process powerShellProcess = Runtime.getRuntime().exec(command);
powerShellProcess.getOutputStream().close();
String line;
BufferedReader stdout = new BufferedReader(new InputStreamReader(powerShellProcess.getInputStream()));
while ((line = stdout.readLine()) != null) {
if (line.length() > 0) {
System.out.println(line);
}
}
Can anyone suggest a better way to get the events out of the log or suggest how I read the output from the powershell script without it being broken up by carriage returns? This was quite disappointing as I had tested it extensively on my machine (the W10 one) only to find it fails on the SBS2008 customer machine!
I have checked the libraries and java versions used on the different machines and they are the same. It’s not the println statement because I do some parsing of the string within the final ‘While’ block and the incoming line is definitely truncated.
I have subsequently tried using Get-winevent but when I put a filterhashtable flag on the command it fails when run on SBS2008 saying 'the parameter is incorrect' (works fine on W10).
My ideal would be able to get all the events from a specific logfile since a given eventID (because I can store that) but it seems to be virtually impossible to get the same output across all windows operating systems in the same format. Any suggestions welcome.
The answer for SBS2008 is here; http://www.mcbsys.com/blog/2011/04/powershell-get-winevent-vs-get-eventlog/
SBS2008 cannot use a hashtable, must use filterxml. Unfortunately when I use filterxml on SBS2008 it does not return the error message, everything else, just no the message. This is using the prescribed method of cutting and pasting the XML query from Event Viewer (https://blogs.msdn.microsoft.com/powershell/2011/04/14/using-get-winevent-filterxml-to-process-windows-events/).
After more research I have come up with a script which does (sort of) what I want. It lacks the eventindex (which is a shame) but it consistently returns the events from the System & Application eventlogs:
$fx = '<QueryList>
<Query Id="0" Path="Application">
<Select Path="Application">*[System[(Level=1 or Level=2 or Level=3) and TimeCreated[timediff(#SystemTime) <= 43200000]]]</Select>
</Query>
</QueryList>'
function Set-Culture([System.Globalization.CultureInfo] $culture) { [System.Threading.Thread]::CurrentThread.CurrentUICulture = $culture ; [System.Threading.Thread]::CurrentThread.CurrentCulture = $culture } ; Set-Culture en-US ; get-winevent -FilterXml $fx | out-string -width 470
$fx = '<QueryList>
<Query Id="0" Path="System">
<Select Path="System">*[System[(Level=1 or Level=2 or Level=3) and TimeCreated[timediff(#SystemTime) <= 43200000]]]</Select>
</Query>
</QueryList>'
Set-Culture en-US ; get-winevent -FilterXml $fx | out-string -width 470
I hope that this is useful to someone else!
I am new to JDBC, and I found something strange when I use:
catch (SQLException e) {
System.out.println("Error STATE: " + e.getSQLState());
System.out.println("With the following message: " + e.getMessage() );
}
Sometimes the message is parsed, but sometimes not.
Like:
First:
Second:
One is parsed, while the other is not, but I can get the error message through googling the corresponding error code.
I don't know what's going on.. And I have tried googling it but with no similar question posted. Does it mean my java.sql.* library is incomplete?
All help would be appreciated.
It means your operating system settings don't support the symbols for the error message in the language being used. The ORA-01017 message is coming before the database applies your language setting so in in English, and more importantly in Western script. Once you've connected the Java locale is honoured.
For example, I can see both these from the same code run with java -Duser.language=zh -Duser.country=CN; the first has incorrect credentials supplied, the second is trying to create an existing table:
java.sql.SQLException: ORA-01017: invalid username/password; logon denied
java.sql.SQLSyntaxErrorException: ORA-00955: 名称已由现有对象使用
I'm seeing ten symbols, where you are seeing ten question marks. My operating system session (Linux in this case) has LANG=en_US.UTF-8. If I change that to something which has fewer symbols defined, e.g. export LANG="en_US.ASCII", I still see the first message but now I get the same as you for the second one:
java.sql.SQLException: ORA-01017: invalid username/password; logon denied
java.sql.SQLSyntaxErrorException: ORA-00955: ??????????
The Chinese symbols can now no longer be rendered by my operating system session.
So set your operating system locale to something that can represent the symbols of the language you're using, preferably UTF8. For example, if Java is running with a Chinese locale, you could do this to be consistent under Linux:
export LANG="zh_CN.UTF-8"
java -Duser.language=zh -Duser.country=CN
ORA-00955: 名称已由现有对象使用
Or change your Java locale to English-language if you want to see all the messages in English:
export LANG="en_CN.UTF-8"
java -Duser.language=en -Duser.country=CN -
ORA-00955: name is already used by an existing object
(Although Java should pick up the language from your locale by default anyway, so maybe don't supply the language or country explicitly in the java call at all; just setting LANG properly would then be enough)
I've been trying to query data from a PostgreSQL DB via R. I've tried skinning the cat with a few different packages (RODBC, RJDBC, DBI, RPostgres, etc), but I seem to keep getting driver errors. Oddly, I never have trouble using the same drivers/URL's and settings to connect to Postgres from SQLWorkbench/J.
I've tried using postgresql-9.2-1002.jdbc4.jar and postgresql-9.3-1100.jdbc41.jar, as well as the generic "PostgreSQL" driver in R. The two jar files are the (i) the driver I use all the time with SQLWorkbench/J and (ii) the slightly newer version of that same driver, respectively. Yet, when I try to use it...
drv_custom <- JDBC(driverClass = "org.postgresql.Driver", classPath="/Users/xxxx/postgresql-9.3-1100.jdbc41.jar")
I get an error:
Error in .jfindClass(as.character(driverClass)[1]) : class not found
OK, so next I try it with the generic driver:
drv_generic <- dbDriver("PostgreSQL")
and strangely, it doesn't want me to enter a username:
>con <- dbConnect(drv=drv_generic, "jdbc:postgresql://xxx.xxxxx.com", port=xxxx, uid="xxxx", password="xxxx")
>Error in postgresqlNewConnection(drv, ...) : unused argument (uid = "xxxx")
so I try it without user/uid:
con <- dbConnect(drv_generic, "jdbc:postgresql://padb-01.jiwiredev.com:5439", password="paraccel")
and get an error....
Error in postgresqlNewConnection(drv, ...) :
RS-DBI driver: (could not connect jdbc:postgresql://padb-01.xxx.com:5439#local on dbname "jdbc:postgresql://xxxx.xxxx.com:5439")
Apparently the syntax is wrong?
Then I circle back to trying the "custom" driver (either one of the .jar files from earlier) but without the driverClass specified.
drv_custom1 <- JDBC( classPath="/Users/xxxx/postgresql-9.2-1002.jdbc4.jar")
con <- dbConnect(drv=drv_custom1, "jdbc:postgresql://xxx.xxx.com", port=5439, uid="paraccel", pwd="paraccel")
and get this error:
Error in .jcall(drv#jdrv, "Ljava/sql/Connection;", "connect", as.character(url)[1], :
RcallMethod: attempt to call a method of a NULL object.
I tried it again with a slight alteration to the syntax:
con <- dbConnect(drv=drv_custom1, url="jdbc:postgresql://xxxx.xxxx.com", port=xxxx, uid="xxxx", pwd="xxxxx",dsn="xxxx")
and got the same error. I tried a number of other variations/approaches as well. I think part of my confusion comes from the fact that the documentation is handled in a very piecemeal way between packages like DBI and those that build upon it like RJDBC, so that when I look at documentation such as ?dbConnect many of the options I need to specify are not even mentioned, and I've been working based off of miscellaneous Google search results related to these packages/errors.
One thread I found suggested trying
.jaddClassPath( "xxxxx/postgresql-9.2-1002.jdbc4.jar" )
first, but that didn't seem to help.
I also tried using
x <- PostgreSQL(max.con = 16, fetch.default.rec = 500, force.reload = FALSE)
to no avail and I experimented with RODBC as the driver.
UPDATE:
I tried using an older version of the driver (jdbc3 instead of jdbc4), restarting R, and detaching all unnecessary packages.
I was able to load the driver
> drv_custom <- JDBC(driverClass = "org.postgresql.Driver", classPath="/xxxxx/xxxxx/postgresql-9.3-1100.jdbc3.jar")
but I still can't connect...
> con <- dbConnect(drv=drv_custom, "jdbc:postgresql://xxxxx.xxxxx.com", port=5439, uid="xxxxx", pwd="xxxxx")
Error in .verify.JDBC.result(jc, "Unable to connect JDBC to ", url) :
Unable to connect JDBC to jdbc:postgresql://xxxxx.xxxx.com
This works for me:
library(RJDBC)
drv <- JDBC("org.postgresql.Driver","C:/R/postgresql-9.4.1211.jar")
con <- dbConnect(drv, url="jdbc:postgresql://host:port/dbname", user="<user name>", password="<password>")
The trick was to include port and dbname in the url. For some reason, jdbc:postgresql does not seem to like reading those information from the dbConnect parameters.
If you are not sure what the dbname is, it is perhaps postgres.
If you are not sure what the port is, it is perhaps 5432.
So a typical call would look like:
con <- dbConnect(drv, url="jdbc:postgresql://10.10.10.10:5432/postgres", user="<user name>", password="<password>")
You can get the jar file from https://jdbc.postgresql.org/
It took some troubleshooting on IRC, but here's what had to happen:
I needed to clear the workspace, detach RODBC and RJDBC, then restart
Load RPostgreSQL
Use only the generic driver
Modify the syntax to
con <- dbConnect(drv=drv_generic, "xxx.xxx.com", port=vvvv, user="ffff", password="ffff", dbname="ggg")
Note: removing the jdbc:postgresql: part was key. Those would've been necessary with RJDBC, but RJDBC turned out to be an unnecessarily painful route.