I need to create TQL queries to query out sets of data from the UCMDB.
I am having 2 problems:
1) How can I find relationships which exists between CIs ( i do not have administrative privileges so need to do it in code somehow)
I need this to get required data.
2) I have created the following query: But I keep getting the IP property value as null.
I checked that IP has an attribute called ip_address.
Code:
import com.hp.ucmdb.api.types.TopologyRelation;
public class Main {
public static void main(String[] args)throws Exception {
final String HOST_NAME = "192.168.159.132";
final int PORT = 8080;
UcmdbServiceProvider provider = UcmdbServiceFactory.getServiceProvider(HOST_NAME, PORT);
final String USERNAME = "username";
final String PASSWORD = "password";
Credentials credentials = provider.createCredentials(USERNAME, PASSWORD);
ClientContext clientContext = provider.createClientContext("Test");
UcmdbService ucmdbService = provider.connect(credentials, clientContext);
TopologyQueryService queryService = ucmdbService.getTopologyQueryService();
Topology topology = queryService.executeNamedQuery("Host IP");
Collection<TopologyCI> hosts = topology.getAllCIs();
for (TopologyCI host : hosts) {
for (TopologyRelation relation : host.getOutgoingRelations()) {
System.out.print("Host " + host.getPropertyValue("display_label"));
System.out.println (" has IP " + relation.getEnd2CI().getPropertyValue("ip_address"));
}
}
}
In the above query output: I get the host names with IP = null
I have a sample query in JYthon which I am unable to figure out: Its for the above code only.
Attaching it for anyone who can understand it.
import sys
UCMDB_API="c:/ucmdb/api/ucmdb-api.jar"
sys.path.append(UCMDB_API)
from com.hp.ucmdb.api import *
# 0) Connection settings
HOST_NAME="192.168.159.132"
PORT=8080
USERNAME="username"
PASSWORD="password"
# 1) Get a Service Provider from the UcmdbServiceFactory
provider = UcmdbServiceFactory.getServiceProvider(HOST_NAME, PORT)
# 2) Setup credentials to log in
credentials = provider.createCredentials(USERNAME, PASSWORD)
# 3) Create a client context
clientContext = provider.createClientContext("TESTING")
# 4) Connect and retrieve a UcmdbService object
ucmdbService = provider.connect(credentials, clientContext)
# 5) Get the TopologyQueryService from the UcmdbService
queryService = ucmdbService.getTopologyQueryService()
# ======= Everything After this is specific to the query =======
# 6) Execute a Named Query and get the Topology
topology = queryService.executeNamedQuery('Host IP')
# 7) Get the hosts
hosts = topology.getAllCIs()
# 8) Print the hosts and IPs
host_ip = {}
for host in hosts:
host_name = host.getPropertyValue("display_label")
if host_name in host_ip.keys():
ips = host_ip[host_name]
else:
ips = {}
host_ip[host_name] = ips
for relation in host.getOutgoingRelations():
ip_address = relation.getEnd2CI().getPropertyValue("display_label")
if ip_address in ips.keys():
pass
else:
ips[ip_address] = ''
print "%s , %s" % (host_name, ip_address)
Please help.
I am unable to understand how to go about this further.
Thank you.
The easiest fix would be use the display_label property from the IP address CI instead of the ip_address property. The Jython reference code uses display_label for its logic.
I'd be a little concerned about using display_label since the display_label formatting logic could be changed to no display the IP address for IP CIs. Getting data directly from the ip_address property is a better choice and should work if the TQL is defined to return that data. Check the Host IP TQL and ensure that it's configured to return ip_address for IP CIs.
Related
I need to populate a database with the fields of the HTTP requests based on if the sender IP is valid or not.
For example if someone make a GET request on my IP with that:
/test/demo.php?name1=value1&name2=value2
How can I receive it so I can handle it and perform actions like:
Get the ip of the sender (And validate it - just confronting it with a list -)
Recognize the type of the Request
Extrapolate the fields (value1 and value2) and save them in variables
I'm using java.net.http package
You can't perform that using package java.net.http because you need to create an http server not an http client. To achieve that, you need HttpServer which is found in package com.sun.net.httpserver.
The first thing is to create a new instance of HttpServer :
final int port = 3000;//You can change with your port number you need
final HttpServer httpServer = HttpServer.create(new InetSocketAddress(port), 0);
System.out.println("Listening port : " + port);
Then, configure an http context. The first parameter accepts the route you defines and the second parameter accepts the http handler in which you can extract all your data that you need to store in your database.
httpServer.createContext("/test/demo.php", buildHttpHandler());//buildHttpHandler is to create
What contains the function buildHttpHandler()? :
Each time where route /test/demo.php is called, the content of arrow function is called. Here, we attempt only to create a simple page html and serves it to the http client. But before responding http client, we need to extract all data you need (ip, request type and parameters).
return httpExchange -> {
final String html = "<!DOCTYPE html>\n" +
"<html>\n" +
" <head>\n" +
" <title>Page</title>\n" +
" <meta charset='utf-8'/>\n" +
" </head>\n" +
" <body>Ok</body>\n" +
"</html>";
//Function to create
extractData(httpExchange);
httpExchange.getResponseHeaders().set("Content-Type", "text/html; charset=utf-8");
httpExchange.sendResponseHeaders(200, html.length());
final OutputStream outputStream = httpExchange.getResponseBody();
outputStream.write(html.getBytes(StandardCharsets.UTF_8));
outputStream.close();
};
What contains extractData() function?
In this function we will extract the data you need.
final String ip = getClientIp(httpExchange);
System.out.println("IP : " + ip);
final String requestType = httpExchange.getRequestMethod();
System.out.println("Request type : " + requestType);
final Map<String, String> parameters = extractParameters(httpExchange);
displayParameters(parameters);
Extracting ip client is more complicated because sometime client uses proxy that's why we create a dedicated function getClientIp() to extract the ip.
In this function, we attempt to extract firstly proxy ip. If not found, we extract standard ip from remote :
final String ip = getProxyIp(httpExchange);
return ip == null ? httpExchange.getRemoteAddress().getAddress().getHostAddress() : ip;
To extract proxy ip, we create another function getProxyIp(). It attempts to extract the ip provided from x-forwarded-for request header.
final List<String> ips = httpExchange.getRequestHeaders().get("x-forwarded-for");
return ips == null ? null : ips.get(ips.size() - 1);
I don't know what do you mean with extrapolating fields but you can store all data in a Map variable. The key will be the name of parameter and the value will be the value of parameter. But this is complicated also because we need to parse the string value from name1=value1&name2=value2. So, we create a new function extractParameters(). It contains :
final String query = httpExchange.getRequestURI().getQuery();
final Map<String, String> parameters = new HashMap<>();
if (query != null) {
final String[] firstParts = query.split("&");
for (final String firstPart : firstParts) {
final String[] secondParts = firstPart.split("=");
final String name = secondParts[0];
final String value = secondParts.length > 1 ? secondParts[1] : null;
parameters.put(name, value);
}
}
return parameters;
You notice maybe what is displayParameters() content. It just attempts to display, parameters retrieved from extractParameters().
for (final Map.Entry<String, String> parameter : parameters.entrySet()) {
System.out.println("Parameter key : " + parameter.getKey());
System.out.println("Parameter value : " + parameter.getValue());
}
And finally don't forget to start the http server :
httpServer.start();
You can check for a full code here https://gist.github.com/razafinarivohanania/24fe0986ea5868097404f2a758131823
When you test it, you can get something like :
IP : 127.0.0.1
Request type : GET
Parameter key : name2
Parameter value : value2
Parameter key : name1
Parameter value : value1
I try to understand which system variables has this value. I execute this code and get some_string.
InetAddress.getLocalHost().GetCanonicalHostName();
After this I print all my system env
System.getenv().forEach((k, v) -> System.out.println("K = " + k + "
V = " + v));
I find all variables which has some_string, and replace all value with
#ClassRule
public final static EnvironmentVariables ENVIRONMENT_VARIABLES = new EnvironmentVariables();
final String pcNameForEnv = "test-pc-name";
ENVIRONMENT_VARIABLES.set("USERDOMAIN", pcNameForEnv);
But my test failed
#Test
public void getMachineNameFromEnv() {
final String pcNameForEnv = "test-pc-name";
ENVIRONMENT_VARIABLES.set("USERDOMAIN", pcNameForEnv);
final String machineName = networkUtil.getPCNetworkName();
assertEquals(pcNameForEnv, machineName);
}
I finded that I need replace this ver:
"user.name",
"user.home",
"LOGONSERVER",
"COMPUTERNAME",
"USERDOMAIN_ROAMINGPROFILE",
"USERDOMAIN",
"COMPUTERNAME",
"MACHINENAME";
But it don't help me too.
You can't set an environment variable to make Java think you are on a different machine. From java.lang.InetAddress -
Returns the address of the local host. This is achieved by retrieving the name of the host from the system, then resolving that name into an InetAddress.
I am new to Kafka. Tried to implement consumer and producer classes to send and receive messages. Need to configure bootstrap.servers for both classes which is a list of broker's ip and port separated by ,. For example,
producerConfig.put("bootstrap.servers",
"172.16.20.207:9946,172.16.20.234:9125,172.16.20.36:9636");
Since the application will be running on the master node of a cluster, it should be able to retrieve the broker information from ZooKeeper just like the answer to Kafka: Get broker host from ZooKeeper.
public static void main(String[] args) throws Exception {
ZooKeeper zk = new ZooKeeper("localhost:2181", 10000, null);
List<String> ids = zk.getChildren("/brokers/ids", false);
for (String id : ids) {
String brokerInfo = new String(zk.getData("/brokers/ids/" + id, false, null));
System.out.println(id + ": " + brokerInfo);
}
}
However this brokerInfo is in Json format which looks like this:
{"jmx_port":-1,"timestamp":"1428512949385","host":"192.168.0.11","version":1,"port":9093}
In this same post, another one suggested the following way of getting connection string for each broker and join them together with comma.
for (String id : ids) {
String brokerInfoString = new String(zk.getData("/brokers/ids/" + id, false, null));
Broker broker = Broker.createBroker(Integer.valueOf(id), brokerInfoString);
if (broker != null) {
brokerList.add(broker.connectionString());
}
}
If this Broker class is from org.apache.kafka.common.requests.UpdateMetadataRequest.Broker, it does not have methods createBroker and connectionString.
Found another similar post Getting the list of Brokers Dynamically. But it did not say how to get the attribute from broker info such as host and port. I can probably write a parser for the json like string to extract them, but I suspect there is more Kafka native way to do that. Any suggestions?
EDIT: I realized the Broker class is from kafka.cluster.Broker. Still it does not have method connectionstring().
You could use ZkUtils to retrieve all the broker information in the cluster, as show below:
ZkUtils zk = ZkUtils.apply("zkHost:2181", 6000, 6000, true);
List<Broker> brokers = JavaConversions.seqAsJavaList(zk.getAllBrokersInCluster());
for (Broker broker : brokers) {
//assuming you do not enable security
System.out.println(broker.getBrokerEndPoint(SecurityProtocol.PLAINTEXT).host());
}
zk.close();
I am trying to connect to HiveServer2 via JDBC drivers from R using RJDBC package. I have seen a broad explanation on passing additional arguments to dbConnect wrapper for various drivers(What arguments can I pass to dbConnect?), but there appear that situation with JDBCDriver is a bit tricker than for other drivers. I can connect to HiveServer2 under this specific URL adress url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000, "/loghost;auth=noSasl") . The correspoding code works and enables me to write statements on Hive from R
library(RJDBC)
dbConnect(drv = JDBC(driverClass = "org.apache.hive.jdbc.HiveDriver",
classPath = c("/opt/hive/lib/hive-jdbc-1.0.0-standalone.jar",
"/usr/share/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar",
"/usr/share/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar"),
identifier.quote = "`"), # to juz niekoniecznie jest potrzebne
url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000, "/loghost;auth=noSasl"),
username = "mkosinski") -> conn
I am wondering if there is a way to pass arguments such as database name (loghost) or a no_authentication_mode (auth=noSasl) to ... in dbConnect such that I could only specify standard URL address (url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000)) and somehow pass the rest of parametrs like this
library(RJDBC)
dbConnect(drv = JDBC(driverClass = "org.apache.hive.jdbc.HiveDriver",
classPath = c("/opt/hive/lib/hive-jdbc-1.0.0-standalone.jar",
"/usr/share/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar",
"/usr/share/hadoop/share/hadoop/common/hadoop-common-2.4.1.jar"),
identifier.quote = "`"), # to juz niekoniecznie jest potrzebne
url = paste0("jdbc:hive2://", host = 'tools-1.hadoop.srv', ":", port = 10000),
username = "mkosinski", dbname = "loghost", auth = "noSasl") -> conn
But the second approach doesn't look to work, despite the various combinations of names and values of additional arguments I try.
Does anyone know how to pass additional arguments to DBI::dbConnect through ... parameter for JDBCDriver?
According to the author's answer: https://github.com/s-u/RJDBC/issues/31#issuecomment-173934951
Simply anything - all that dbConnect does is to collect whatever you
pass (including ...) and collect it all into a property dictionary
(java.util.Properties) that is passed to the driver's connect()
method. So any named argument you pass is included. So the only
special argument is url which is passed directly, everything else is
included in the properties. How that gets interpreted is out of
RJDBC's hands - it's entirely up to the driver.
there you can use the full url
library(RJDBC)
drv <- JDBC("org.postgresql.Driver","C:/R/postgresql-9.4.1211.jar")
con <- dbConnect(drv, url="jdbc:postgresql://host:port/dbname", user="<user name>", password="<password>")
I have written applications in Java which connect to the database by taking the username and password for the database which is set in the environment variables. So the database username and password are stored as clear text in a file on the server.
Here is the code which creates the connection string:
if (dataBase.equals("oracle"))
{
url = "jdbc:oracle:thin:#";
url = url + getParameter("Database IP", "setup.ini"); // ip
url = url + ":" + getParameter("Database Port", "setup.ini"); //port
String envVarValue = System.getenv("DBNAME");
// Environment variable has value DBNAME =
String a[] = envVarValue.split("#");
url = url + ":" + getParameter("SID", "setup.ini");
String b[] = a[0].split("/");
userName = b[0]; //User name
password = b[1]; //Password
}
Now I want my application to be able to connect to the database by the Unix user who would be running this application on the server without the need to get the password for the database user in the code.
Please tell me if this is possible and how if yes?
Refer this link on how to configure a user account for OS authentication in Oracle: http://docs.oracle.com/cd/B28359_01/java.111/b31224/clntsec.htm#CIHCBCBC
You can read the current OS user running your java program by calling System.getProperty("user.name"). When creating connection through JDBC, you may have to append this value to the prefix OS_AUTHENT_PREFIX you created as shown in the link above.