I'm trying to get the name of the columns from the metadata of a ResultSet but in Hive I get column names if the form of table_name.column_name and I'd like to have only column_name (similarly to what I get from MySQL for example). Is it possible ?
Yes it is possible.
There are two ways by which it can be done:
1.) Using Hive JDBC connection(which you are looking for).
2.) Using HiveMetastoreClient.
Here are code snippets for the above two approaches:
First Approach:
ResultSet resultSet = <custom_class_for_hive_jdbc_connector>.executeQuery("DESCRIBE <TABLE_NAME>");
ResultSetMetaData metaData = resultSet.getMetaData();
while (resultSet.next()) {
System.out.println(" Column names : "+resultSet.getString(metaData.getColumnName(1)));
}
In this approach we need to create a resultsetmetadata object and using this object we can get the details of the query output. In your case "describe table"
Second Approach:
HiveMetaStoreClient metastoreClient = null;
HiveConf hiveConf = new HiveConf();
metastoreClient = new HiveMetaStoreClient(hiveConf);
List<FieldSchema> list = metastoreClient.getFields(<database_name>,<table_name>);
for (int i = 0; i < list.size(); i++) {
System.out.println("Column names : "+list.get(i).getName());
}
metastoreClient.close();
In this approach we are not running any query, we are simply connecting to the HiveMetastore and from there we are getting the details of a given table under given database.
Hope it helps...!!!
To prevent the Hive table names from being prepended to the column names, use this property setting in the Hive-site.xml file.
<property>
<name>hive.resultset.use.unique.column.names</name>
<value>false</value>
</property>
beginning with Hive .13, a new Hive property called hive.resultset.use.unique.column.names was created. The default value of this property is TRUE.
If the default value is used, the Hive table name is prepended to all column names. This is a change in behavior from previous versions of Hive.
This is also very important if you are trying to use Spring Jdbctemplate with Beanpropertyrowmapper to do the camel case to underscore format conversion for you.
Related
I have a MySQL database on server #1 and I need to migrate it to server #2. The database is very large and methods like mysqldump or MySQL Workbench Migration didn't work for me, so I wanted to write my own Java application, that would perform the following steps:
get all table names from the source database schema
for each table, it would select a batch of records (let's say 10.000 at a time) and insert them in the corresponding table in destination database. The schema is already prepared with correctly defined tables.
repeat until there are no rows left for current table.
repeat for each table.
The problem is, AFAIK, when using JDBC, it is needed to iterate through ResultSet and specify all column types and names, like this:
while (resultset.next()) {
System.out.println(resultset.getString("Col 1"));
System.out.println(resultset.getString("Col 2"));
System.out.println(resultset.getString("Col 3"));
System.out.println(resultset.getString("Col n"));
}
I want to do this for all tables and all their columns without specifying their names and types manually. I can't just type manually all these columns, as I have 150 tables and each of them has like 10-50 columns.
Is there any general way how to do this? Maybe taking advantage of the fact that both source and destination schemas are the same (same tables with same column names/types and same foreign keys)?
You can use ResultSetMetaData with ResultSet to get the columnNames, columnCount and few other details.
ResultSet resultSet = ps.executeQuery();
ResultSetMetaData resultSetMetaData = resultSet.getMetaData();
while (resultSet.next()) {
for (int i = 1; i <= resultSetMetaData.getColumnCount(); i++) {
System.out.println(resultSet.getString(i));
}
}
if you need to fetch the type of Column also to make the decision you could use Switch conditions based on the type returned from resultSetMetaData.getColumnTypeName(i) for using something like
resultSet.getString()
resultSet.getInt()
resultSet.getBoolean()
resultSet.getDate()
Many more
On top of #koushlendra answer, you can also use these methods for additional informations(if needed)
resultSetMetaData.getTableName(int column) Returns the column’s table name.
resultSetMetaData.getSchemaName(int column) Returns the name of the schema of the column’s table.
Using Netbeans, I have my database and table set up, and have added my data manually, in which I am able to see within my application I am building, as intended.
I would like the user to add their own data in which will be appended to a new row on the table. However, I am having trouble trying to write code in order to do this.
Class.forName("org.apache.derby.jdbc.EmbeddedDriver").newInstance();
Connection conn = DriverManager.getConnection("jdbc:derby://localhost:1527/stockApplication");
Statement stat = conn.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);
String insertDerbyData = "INSERT INTO TIGER_INFO"
+ "(TIGER_ID, TIGER_LOCATION)"
+ "VALUES (123456, Store)";
stat.executeUpdate(insertDerbyData);
I cannot execute the above code as I'm returned with an error mentioning that 'STORE' is not in any table. 'STORE' is meant to be a value for my 'TIGER_LOCATION' column. What's going on here?
In theory, I have two columns, and I would like to add both values, '123456' and 'Store' into their respective columns. How do I go about correctly doing so?
If TIGER_LOCATION is a string/varchar column, and Store is a string literal, then the value must be enclosed in single quotes, as in most SQL-based databases:
INSERT INTO TIGER_INFO (TIGER_ID, TIGER_LOCATION) VALUES (123456, 'Store')
Strings should be between '...' you have to use :
VALUES (123456, 'Store')
//--------------^-----^
I've got an embedded Derby Database in my java application, and have multiple table's (that are created and deleted, so nothing is set in stone). I wanted to be able to return a list of names of all the tables currently in the database as I have to display the list in the application as well as get all the information from them.
Easiest way to do so? I don't need code just a method or methods. I'm a terrible google-fu user.
Currently my code works by grabbing a ResultSet from a specific table name entered, but it's only for testing purposes and I need to be able to display the full list of tables I have.
EDIT: My current workaround is actually different than posted. I simply have another table that holds all the table names created and updates when one is created/deleted. Obviously not the best approach but it works for me right now.
DatabaseMetaData metaData = connection.getMetaData();
ResultSet resultSet = metaData.getTables(null, "schenaName", "%" ,new String[] {"TABLE"} );
while (resultSet.next()) {
System.out.println(resultSet.getString(3));
}
Adding new answer:
Connection connection = getDBConnection();
DatabaseMetaData dbMetaData = connection.getMetaData();
//getting catalogs for mysql DB, if it is not working for your DB, try dbMetaData.getSchemas();
ResultSet catalogs = dbMetaData.getCatalogs();
while(catalogs.next()){
String catalogName = catalogs.getString(1);
//excluding table names from "mysql" schema from mysql DB.
if(!"mysql".equalsIgnoreCase(catalogName)){
ResultSet tables = dbMetaData.getTables(catalogName, null, null, null);
while(tables.next()){
System.out.println(catalogName + "::"+tables.getString(3));
}
}
}
Using metadata is the (somewhat) more portable solution. Note that you don't need the catalog stuff with Derby, as there are no catalogs. You can issue dmd.getTables(...) directly with null for the catalog. If all the tables you track are in a single schema, (and there aren't any other tables in that schema), getTables(null, "schemaName", null, null) should do the trick.
If need more fancy querying and you're not concerned about portability, you can check out
the dataBaseMetaData tool which gives you access to metadata as tables so that you can perform joins and other sophisticated queries on them.
Try this:
select tableName from sys.systables
You should get all the tables your system.
I'm writing a Java GUI application which uses a MySQL database. In this application the users can see a JTable with the rows of DB's table and modify the attributes of a selected row with a form. I'd like to update only the modified attributes of the row. I know I have to specify every SQL table's column in the String command if I use PreparedStatement and placeholders
String command = "UPDATE table SET attr0 = ?, attr1 = ? WHERE id = ?";
PreparedStatement statement = connection.prepareStatement(command);
but It's not what I'm looking for. Moreover, my DB's table has many columns, so I can't use different String commands for every combination of attributes.
Can someone help me? Thanks.
Unfortunately with straight JDBC, the best you can do is build the SET clause dynamically (see rough code sample below). JDBC can't handle optional parameters and will throw an Exception if not all parameters are bound before executing.
`
// 'columns' assumed a Map
// 'id' to be a String
List<String> setClauses = new ArrayList<String>();
for (String key : columns.keySet()) {
setClauses.add(String.format("%s=?", key));
}
// StringUtils is from Apache Commons Lang
// although it's pretty easy to build your own join routine.
String command = String.format("UPDATE table SET %s WHERE id=?"
, StringUtils.join(setClauses, ",")
);
PreparedStatement statement = connection.prepareStatement(command);
int p = 1;
for (String key : columns.keySet()) {
statement.setString(p++, columns.get(key));
}
statement.setString(p++, id);
`
JDBC also doesn't have named parameters either so this is why you have to do the incrementing. If you are able to do it, I would recommend investigating Hibernate (which allows to work with JavaBeans) or Spring JDBCTemplate (which does have named parameters).
While using Tomcat as the server and Derby as the database I did a look-up and executed the query like :
Context initContext = new InitialContext();
Context envContext = (Context)initContext.lookup("java:comp/env");
DataSource ds = (DataSource)envContext.lookup("jdbc/PollDatasource");
Connection connection = ds.getConnection();
// write the query that tells the current weight of actor
String currentWeightQuery = "SELECT " + justPolled + ",total FROM pollresult";
PreparedStatement currentWeight = connection.prepareStatement(currentWeightQuery);
ResultSet cwSet = currentWeight.executeQuery();
Now I am using Microsoft SQL Server 2005 and have to query the database from the java desktop application. What do I need to do query the sql server 2005 ? I have already loaded the sqlserver-jdbc driver and connected to the database but I don't know how to get the data from the database .
Now you need to iterate the ResultSet : Retrieving and Modifying Values from Result Sets.
The ResultSet is actually a wrapper around a cursor in SQL server.
In this case you will probably get only one result.
You need to retrieve the value form the query using one of the getter methods of java.sql.ResultSet, depending on the expected datatype of your query result. As a method parameter you use either the name of the column as a string (""), or the sequence number of the column in the query, staring at 1. (one !)
Try this after your last line of code above:
while (cwSet.next()) {
String string = cwSet.getString(1); // instead of the 1 you can also use the name of the column as a String
int i = cwSet.getInt("total"); //could also have used getInt(2)
// etc...
}