I have written a java program to insert an integer and a double in aerospike database. Here is the code snippet
import com.aerospike.client.AerospikeClient;
import com.aerospike.client.Bin;
import com.aerospike.client.Key;
import com.aerospike.client.policy.WritePolicy;
public class Test{
public static void main(String[] args){
try{
AerospikeClient client = new AerospikeClient("192.168.140.62", 3000);
for(int i=0;i<10000;i++){
try {
WritePolicy writePolicy = new WritePolicy();
Key key = new Key("test" , "myset", "mykey");
double randomDouble = Math.random();
Bin bin1 = new Bin("ID",i);
Bin bin2 = new Bin("Value",randomDouble);
client.put(writePolicy,key,bin1,bin2);
} catch (Exception e) {
e.printStackTrace();
}
}
client.close();
}
catch(Exception e){
e.printStackTrace();
}
}
}
The problem is the above code snippet is neither inserting any data nor throwing any errors. Please advise.
PS: I am new to aerospike
Related
Would anyone please explain why I am getting this error? Not just this one
Constructor 'StockMachine(java.lang.String)' is never used
but also one that says Method is never used. I am not entirely sure why this is happening. Here is a snippet of my code:
import java.io.*;
import java.util.*;
import java.io.FileInputStream;
public class StockMachine {
private StockPriceService [] stockPriceServices;
private String tickerSymbols[];
private final int NUMBEROFSERVICES = 3;
public StockMachine(String fileName) {
stockPriceServices = new StockPriceService[3];
stockPriceServices[0] = new UHStockService();
stockPriceServices[1] = new NLPService();
stockPriceServices[2] = new ExternalService();
try {
FileInputStream fis = new FileInputStream(fileName);
Scanner scan = new Scanner(fis);
this.tickerSymbols = new String[scan.nextInt()];
int i = 0;
while (scan.hasNext()) {
this.tickerSymbols[i] = scan.next();
i++;
}
}
catch (FileNotFoundException e) {
e.printStackTrace();
}
}
}
I have to implement a simple GUI to connect to MongoDB client using java driver.
This is a picture of my GUI:
As you can see, one can choose a database and then press OK to connect.
In MongoDBClient class there's a method (getDatabasesName) to detect all databases names that are present:
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.mongodb.MongoClient;
public class MongoDBClient {
private MongoClient client;
public MongoDBClient() {
try {
this.client = new MongoClient("127.0.0.1", 27017);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public MongoClient getClient() {
return client;
}
public ArrayList<String> getDatabasesName(){
ArrayList<String> dbsName = new ArrayList<>();
List<String> dbs = this.client.getDatabaseNames();
Iterator<String> iterator = dbs.iterator();
while(iterator.hasNext()){
dbsName.add(iterator.next());
}
return dbsName;
}
}
Then, in my main I create a new instance of MongoDBClient and I pass the arrayList of databases names to the Window1 class:
import java.util.ArrayList;
public class Main {
public static void main(String[] args) {
// TODO Auto-generated method stub
MongoDBClient mongoClient = new MongoDBClient();
ArrayList<String> dbName = mongoClient.getDatabasesName();
try {
Window1 w1 = new Window1(dbName);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
Finally, in my window one can choose a database from a group of buttons:
public void chooseDB(ArrayList<String> dbName) {
dbButton= new JRadioButton[dbName.size()];
ButtonGroup group = new ButtonGroup();
int x = 5;
for(int i = 0; i<dbName.size(); i++) {
dbButton[i] = new JRadioButton(dbName.get(i));
dbButton[i].setBounds(x, 28, 65, 15);
dbButton[i].setBackground(Color.WHITE);
this.panel.add(dbButton[i]);
x=x+100;
group.add(dbButton[i]);
}
}
After one presses OK, I want to create a new Instance of DB, but in main class not inside window1 class.
I want to write the equivalent of this sql request in Hive :
select * from information_schema.columns where table_schema='database_name'
How can I access hive's metastore and retrieve all the columns of all the tables stored in a specific database? I know that we can do
it by table via describe [table_name] but is there anyway to have all
the columns for all the tables in a database in the same request?
How can I access hive's metastore and retrieve all the columns of all
the tables stored in a specific database?
This is one way to connect HiveMetaStoreClient and you can use method getTableColumnsInformation will get columns.
In this class along with columns all the other information like partitions can be extracted. pls see example client and sample methods.
import org.apache.hadoop.hive.conf.HiveConf;
// test program
public class Test {
public static void main(String[] args){
HiveConf hiveConf = new HiveConf();
hiveConf.setIntVar(HiveConf.ConfVars.METASTORETHRIFTCONNECTIONRETRIES, 3);
hiveConf.setVar(HiveConf.ConfVars.METASTOREURIS, "thrift://host:port");
HiveMetaStoreConnector hiveMetaStoreConnector = new HiveMetaStoreConnector(hiveConf);
if(hiveMetaStoreConnector != null){
System.out.print(hiveMetaStoreConnector.getAllPartitionInfo("tablename"));
}
}
}
// define a class like this
import com.google.common.base.Joiner;
import com.google.common.collect.Lists;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.metastore.HiveMetaStoreClient;
import org.apache.hadoop.hive.metastore.api.FieldSchema;
import org.apache.hadoop.hive.metastore.api.MetaException;
import org.apache.hadoop.hive.metastore.api.Partition;
import org.apache.hadoop.hive.metastore.api.hive_metastoreConstants;
import org.apache.hadoop.hive.ql.metadata.Hive;
import org.apache.thrift.TException;
import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class HiveMetaStoreConnector {
private HiveConf hiveConf;
HiveMetaStoreClient hiveMetaStoreClient;
public HiveMetaStoreConnector(String msAddr, String msPort){
try {
hiveConf = new HiveConf();
hiveConf.setVar(HiveConf.ConfVars.METASTOREURIS, msAddr+":"+ msPort);
hiveMetaStoreClient = new HiveMetaStoreClient(hiveConf);
} catch (MetaException e) {
e.printStackTrace();
System.err.println("Constructor error");
System.err.println(e.toString());
System.exit(-100);
}
}
public HiveMetaStoreConnector(HiveConf hiveConf){
try {
this.hiveConf = hiveConf;
hiveMetaStoreClient = new HiveMetaStoreClient(hiveConf);
} catch (MetaException e) {
e.printStackTrace();
System.err.println("Constructor error");
System.err.println(e.toString());
System.exit(-100);
}
}
public String getAllPartitionInfo(String dbName){
List<String> res = Lists.newArrayList();
try {
List<String> tableList = hiveMetaStoreClient.getAllTables(dbName);
for(String tableName:tableList){
res.addAll(getTablePartitionInformation(dbName,tableName));
}
} catch (MetaException e) {
e.printStackTrace();
System.out.println("getAllTableStatistic error");
System.out.println(e.toString());
System.exit(-100);
}
return Joiner.on("\n").join(res);
}
public List<String> getTablePartitionInformation(String dbName, String tableName){
List<String> partitionsInfo = Lists.newArrayList();
try {
List<String> partitionNames = hiveMetaStoreClient.listPartitionNames(dbName,tableName, (short) 10000);
List<Partition> partitions = hiveMetaStoreClient.listPartitions(dbName,tableName, (short) 10000);
for(Partition partition:partitions){
StringBuffer sb = new StringBuffer();
sb.append(tableName);
sb.append("\t");
List<String> partitionValues = partition.getValues();
if(partitionValues.size()<4){
int size = partitionValues.size();
for(int j=0; j<4-size;j++){
partitionValues.add("null");
}
}
sb.append(Joiner.on("\t").join(partitionValues));
sb.append("\t");
DateTime createDate = new DateTime((long)partition.getCreateTime()*1000);
sb.append(createDate.toString("yyyy-MM-dd HH:mm:ss"));
partitionsInfo.add(sb.toString());
}
} catch (TException e) {
e.printStackTrace();
return Arrays.asList(new String[]{"error for request on" + tableName});
}
return partitionsInfo;
}
public String getAllTableStatistic(String dbName){
List<String> res = Lists.newArrayList();
try {
List<String> tableList = hiveMetaStoreClient.getAllTables(dbName);
for(String tableName:tableList){
res.addAll(getTableColumnsInformation(dbName,tableName));
}
} catch (MetaException e) {
e.printStackTrace();
System.out.println("getAllTableStatistic error");
System.out.println(e.toString());
System.exit(-100);
}
return Joiner.on("\n").join(res);
}
public List<String> getTableColumnsInformation(String dbName, String tableName){
try {
List<FieldSchema> fields = hiveMetaStoreClient.getFields(dbName, tableName);
List<String> infs = Lists.newArrayList();
int cnt = 0;
for(FieldSchema fs : fields){
StringBuffer sb = new StringBuffer();
sb.append(tableName);
sb.append("\t");
sb.append(cnt);
sb.append("\t");
cnt++;
sb.append(fs.getName());
sb.append("\t");
sb.append(fs.getType());
sb.append("\t");
sb.append(fs.getComment());
infs.add(sb.toString());
}
return infs;
} catch (TException e) {
e.printStackTrace();
System.out.println("getTableColumnsInformation error");
System.out.println(e.toString());
System.exit(-100);
return null;
}
}
}
If you want to have the ability to run such queries that return hive metadata, you can setup Hive metastore with MySQL, metadata used in Hive is stored in a specific account of MySQL.
You will have to create a user of MySQL for hive by doing CREATE USER 'hive'#'metastorehost' IDENTIFIED BY 'mypassword'.
Then you will find tables like COLUMNS_VS with the info you are looking for.
An example query to retrieve all columns in all tables could be: SELECT COLUMN_NAME, TBL_NAME FROM COLUMNS_V2 c JOIN TBLS a ON c.CD_ID=a.TBL_ID
Alternatively, you can access this information via REST calls to WebHCat see wiki for more info.
I'm having a few issues with an extra credit assignment for my Java class. The objective is to decrypt a file without the password. It is encrypted with the PBEWithSHA1AndDESede algorithm and the password is a dictionary word with no numbers or special characters.
The way I'm trying to solve this is by guessing the password over and over again until I get it right using the code below.
The problem I'm running into is that the extra_out.txt file is being output after the first cycle of the for loop, when I want it to only be output if the correct word is guessed.
So when it runs, I get the exception "Encryption Error" and then the extra_out.txt file is output (still encrypted) and then 9999 more "Encryption Errors."
Any helpful advice is greatly appreciated!
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Random;
import java.util.Scanner;
public class WordGuess {
public static void main(String[] args) {
ArrayList<String> words = new ArrayList();
Random numGen = new Random();
String curWord = "";
try {
File aFile = new File("english.txt");
Scanner reader = new Scanner(aFile);
while (reader.hasNext()) {
curWord = reader.next();
if (curWord.length() == 5) {
words.add(curWord);
}
}
}
catch (FileNotFoundException e) {
System.out.println("Error: " + e);
}
for(int i = 0; i < 10000; i++){
int rand = Math.abs(numGen.nextInt(words.size()));
File fileIn = new File("extracredit.enc");
File fileOut = new File("extra_out.txt");
String password = words.get(rand);
crackFile(fileIn, fileOut, password);
}
}
public static void crackFile(File input, File output, String password) {
try{
Crypt c = new Crypt(password);
byte[] bytes = FileIO.read(input);
FileIO.write(output, c.decrypt(bytes));
}
catch (IOException e) {
System.out.println("Could not read/write file");
}
catch (Exception e) {
System.out.println("Encryption error");
}
}
}
I can't seem to find sample code for constructing a Berkeley DB in Java and inserting records into it. Any samples? And I do not mean the Berkeley DB Java Edition either.
http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/BDB_Prog_Reference.pdf
Chapter 5
If you download db-5.0.21.NC.zip you will see plenty of samples.
Here is one that seems to do what you want
/*-
* See the file LICENSE for redistribution information.
*
* Copyright (c) 2004, 2010 Oracle and/or its affiliates. All rights reserved.
*
* $Id$
*/
// File: ExampleDatabaseLoad.java
package db.GettingStarted;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.List;
import java.util.StringTokenizer;
import java.util.Vector;
import com.sleepycat.bind.EntryBinding;
import com.sleepycat.bind.serial.SerialBinding;
import com.sleepycat.bind.tuple.TupleBinding;
import com.sleepycat.db.DatabaseEntry;
import com.sleepycat.db.DatabaseException;
public class ExampleDatabaseLoad {
private static String myDbsPath = "./";
private static File inventoryFile = new File("./inventory.txt");
private static File vendorsFile = new File("./vendors.txt");
// DatabaseEntries used for loading records
private static DatabaseEntry theKey = new DatabaseEntry();
private static DatabaseEntry theData = new DatabaseEntry();
// Encapsulates the databases.
private static MyDbs myDbs = new MyDbs();
private static void usage() {
System.out.println("ExampleDatabaseLoad [-h <database home>]");
System.out.println(" [-i <inventory file>] [-v <vendors file>]");
System.exit(-1);
}
public static void main(String args[]) {
ExampleDatabaseLoad edl = new ExampleDatabaseLoad();
try {
edl.run(args);
} catch (DatabaseException dbe) {
System.err.println("ExampleDatabaseLoad: " + dbe.toString());
dbe.printStackTrace();
} catch (Exception e) {
System.out.println("Exception: " + e.toString());
e.printStackTrace();
} finally {
myDbs.close();
}
System.out.println("All done.");
}
private void run(String args[])
throws DatabaseException {
// Parse the arguments list
parseArgs(args);
myDbs.setup(myDbsPath);
System.out.println("loading vendors db....");
loadVendorsDb();
System.out.println("loading inventory db....");
loadInventoryDb();
}
private void loadVendorsDb()
throws DatabaseException {
// loadFile opens a flat-text file that contains our data
// and loads it into a list for us to work with. The integer
// parameter represents the number of fields expected in the
// file.
List vendors = loadFile(vendorsFile, 8);
// Now load the data into the database. The vendor's name is the
// key, and the data is a Vendor class object.
// Need a serial binding for the data
EntryBinding dataBinding =
new SerialBinding(myDbs.getClassCatalog(), Vendor.class);
for (int i = 0; i < vendors.size(); i++) {
String[] sArray = (String[])vendors.get(i);
Vendor theVendor = new Vendor();
theVendor.setVendorName(sArray[0]);
theVendor.setAddress(sArray[1]);
theVendor.setCity(sArray[2]);
theVendor.setState(sArray[3]);
theVendor.setZipcode(sArray[4]);
theVendor.setBusinessPhoneNumber(sArray[5]);
theVendor.setRepName(sArray[6]);
theVendor.setRepPhoneNumber(sArray[7]);
// The key is the vendor's name.
// ASSUMES THE VENDOR'S NAME IS UNIQUE!
String vendorName = theVendor.getVendorName();
try {
theKey = new DatabaseEntry(vendorName.getBytes("UTF-8"));
} catch (IOException willNeverOccur) {}
// Convert the Vendor object to a DatabaseEntry object
// using our SerialBinding
dataBinding.objectToEntry(theVendor, theData);
// Put it in the database.
myDbs.getVendorDB().put(null, theKey, theData);
}
}
private void loadInventoryDb()
throws DatabaseException {
// loadFile opens a flat-text file that contains our data
// and loads it into a list for us to work with. The integer
// parameter represents the number of fields expected in the
// file.
List inventoryArray = loadFile(inventoryFile, 6);
// Now load the data into the database. The item's sku is the
// key, and the data is an Inventory class object.
// Need a tuple binding for the Inventory class.
TupleBinding inventoryBinding = new InventoryBinding();
for (int i = 0; i < inventoryArray.size(); i++) {
String[] sArray = (String[])inventoryArray.get(i);
String sku = sArray[1];
try {
theKey = new DatabaseEntry(sku.getBytes("UTF-8"));
} catch (IOException willNeverOccur) {}
Inventory theInventory = new Inventory();
theInventory.setItemName(sArray[0]);
theInventory.setSku(sArray[1]);
theInventory.setVendorPrice((new Float(sArray[2])).floatValue());
theInventory.setVendorInventory((new Integer(sArray[3])).intValue());
theInventory.setCategory(sArray[4]);
theInventory.setVendor(sArray[5]);
// Place the Vendor object on the DatabaseEntry object using our
// the tuple binding we implemented in InventoryBinding.java
inventoryBinding.objectToEntry(theInventory, theData);
// Put it in the database. Note that this causes our secondary database
// to be automatically updated for us.
myDbs.getInventoryDB().put(null, theKey, theData);
}
}
private static void parseArgs(String args[]) {
for(int i = 0; i < args.length; ++i) {
if (args[i].startsWith("-")) {
switch(args[i].charAt(1)) {
case 'h':
myDbsPath = new String(args[++i]);
break;
case 'i':
inventoryFile = new File(args[++i]);
break;
case 'v':
vendorsFile = new File(args[++i]);
break;
default:
usage();
}
}
}
}
private List loadFile(File theFile, int numFields) {
List records = new ArrayList();
try {
String theLine = null;
FileInputStream fis = new FileInputStream(theFile);
BufferedReader br = new BufferedReader(new InputStreamReader(fis));
while((theLine=br.readLine()) != null) {
String[] theLineArray = splitString(theLine, "#");
if (theLineArray.length != numFields) {
System.out.println("Malformed line found in " + theFile.getPath());
System.out.println("Line was: '" + theLine);
System.out.println("length found was: " + theLineArray.length);
System.exit(-1);
}
records.add(theLineArray);
}
fis.close();
} catch (FileNotFoundException e) {
System.err.println(theFile.getPath() + " does not exist.");
e.printStackTrace();
usage();
} catch (IOException e) {
System.err.println("IO Exception: " + e.toString());
e.printStackTrace();
System.exit(-1);
}
return records;
}
private static String[] splitString(String s, String delimiter) {
Vector resultVector = new Vector();
StringTokenizer tokenizer = new StringTokenizer(s, delimiter);
while (tokenizer.hasMoreTokens())
resultVector.add(tokenizer.nextToken());
String[] resultArray = new String[resultVector.size()];
resultVector.copyInto(resultArray);
return resultArray;
}
protected ExampleDatabaseLoad() {}
}
There are a number of good Getting Started Guides for Oracle Berkeley DB Java Edition. They are included in the documentation set. You'll find example code in the documentation. If you're new to Oracle Berkeley DB Java Edition that's the right place to start. There are other examples in the download package as well.
I'm the product manager for Oracle Berkeley DB, I hope this addressed your question. If not please let me know how else I can help you.