Develop Reference

How to deserialize objects after closing an opening program in Java 11

I have a program in which I am trying to implement saving and loading of obejcts, however I couldn't get the loading to work after the program closes, so effectively only saving and loading works while the program is open, but no data is ever loaded once the program starts. I assume this is something to do with overwiting. I created a test program to see if I could get it to work just using a simple Person class. I store my Peson objects inside an ArrayList and serialize it, then deserialize it. Currently I am storing all loaded Person objects in a JComboBox. I have looked online and could not find anything that will help. Also note I am aware that using serialization is not the best method of saving objects, but it's something suitable to use for my program.
My App Class:
import javax.swing.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.io.*;
import java.util.ArrayList;
public class App extends JFrame {
public static JComboBox<Person> peopleBox;
public App(){
try {
Person.peopleList = loadList();
}
catch(IOException | ClassNotFoundException e){
System.out.println(e.getMessage());
}
try {
saveList(Person.peopleList);
}catch (IOException e){
System.out.println(e.getMessage());
}
peopleBox = new JComboBox<>();
peopleBox.setModel(getComboBoxModel(Person.peopleList));
add(peopleBox);
pack();
setSize(600, 400);
setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
}
public DefaultComboBoxModel<Person> getComboBoxModel(ArrayList<Person> peopleList){
Person[] comboBoxModel = peopleList.toArray(new Person[0]);
return new DefaultComboBoxModel<>(comboBoxModel);
}
public static void saveList(ArrayList<Person> peopleList) throws IOException {
ObjectOutputStream objectOutputStream = new ObjectOutputStream(new FileOutputStream("test.bin"));
objectOutputStream.writeObject(peopleList);
}
public static ArrayList<Person> loadList() throws IOException, ClassNotFoundException {
ObjectInputStream objectInputStream = new ObjectInputStream(new FileInputStream("test.bin"));
Person.peopleList = (ArrayList<Person>) objectInputStream.readObject();
return Person.peopleList;
}
public static void main(String[] args){
// Person p = new Person("Sean", 22);
try {
saveList(Person.peopleList);
}catch (IOException e){
System.out.println(e.getMessage());
}
App app = new App();
app.pack();
app.setVisible(true);
}
}
Person Class
import java.io.Serializable;
import java.util.ArrayList;
public class Person implements Serializable {
public int age;
public String name;
public static ArrayList<Person> peopleList = new ArrayList<>();
public Person(String name, int age){
this.age = age;
this.name = name;
peopleList.add(this);
for(Person p : peopleList){
System.out.println(p.toString());
}
}
public Person(){
}
public String toString(){
return "Name : " + name + " Age: " + age;
}
}
I expect when I save the list to the "test.bin" file, close the program, then open it again that it will load the list and display the Objects I created before I closed the program. I appreciate any help, thanks.

You are saving an empty list before you load Person from the file.
I suggest this approach:
import javax.swing.*;
import java.io.*;
import java.util.ArrayList;
import java.util.List;
public class App extends JFrame {
public static JComboBox<Person> peopleBox;
public App() {
try {
loadList();
} catch (IOException | ClassNotFoundException e) {
System.out.println(e.getMessage());
}
try {
saveList(Person.peopleList);
} catch (IOException e) {
System.out.println(e.getMessage());
}
setSize(600, 400);
setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
}
public void updateData(){
peopleBox = new JComboBox<>();
peopleBox.setModel(getComboBoxModel(Person.peopleList));
add(peopleBox);
pack();
}
public DefaultComboBoxModel<Person> getComboBoxModel(ArrayList<Person> peopleList) {
Person[] comboBoxModel = peopleList.toArray(new Person[0]);
return new DefaultComboBoxModel<>(comboBoxModel);
}
public static void saveList(ArrayList<Person> peopleList) throws IOException {
ObjectOutputStream objectOutputStream = new ObjectOutputStream(new FileOutputStream("test.bin"));
objectOutputStream.writeObject(peopleList);
}
public static void loadList() throws IOException, ClassNotFoundException {
ObjectInputStream objectInputStream = new ObjectInputStream(new FileInputStream("test.bin"));
Person.peopleList.addAll((List<Person>) objectInputStream.readObject());
}
public static void main(String[] args) {
App app = new App();
Person p = new Person("Sean2", 24);
try {
saveList(Person.peopleList);
} catch (IOException e) {
System.out.println(e.getMessage());
}
app.updateData();
app.setVisible(true);
}
}

xe:beanNamePicker, cant get my values from a notes view into result set

I have set up a java class that I want to use for an xe:beanNamePicker. Somehow I am not able to add a created SimplePickerResult into the result set.
package se.myorg.myproject.app;
import java.io.IOException;
import java.util.List;
import java.util.Properties;
import java.util.TreeSet;
import se.sebank.namis.utils.Utils;
import lotus.domino.Database;
import lotus.domino.Document;
import lotus.domino.DocumentCollection;
import lotus.domino.NotesException;
import lotus.domino.View;
import com.ibm.xsp.complex.ValueBindingObjectImpl;
import com.ibm.xsp.extlib.component.picker.data.INamePickerData;
import com.ibm.xsp.extlib.component.picker.data.IPickerEntry;
import com.ibm.xsp.extlib.component.picker.data.IPickerOptions;
import com.ibm.xsp.extlib.component.picker.data.IPickerResult;
import com.ibm.xsp.extlib.component.picker.data.SimplePickerResult;
public class DirectoryNamePicker extends ValueBindingObjectImpl implements INamePickerData {
private Utils utils;
Properties props;
public DirectoryNamePicker(){
//constructor
utils = new Utils();
utils.printToConsole(this.getClass().getSimpleName().toString() + " - DirectoryNamePicker() // constructor");
try {
props = utils.getDataSourceProperties();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public String[] getSourceLabels () {
// TODO Auto-generated method stub
return null;
}
public boolean hasCapability (final int arg0) {
// TODO Auto-generated method stub
return false;
}
public List<IPickerEntry> loadEntries (final Object[] arg0, final String[] arg1) {
// TODO Auto-generated method stub
return null;
}
#SuppressWarnings("unchecked")
public IPickerResult readEntries (final IPickerOptions options) {
String startKey = options.getStartKey();
int count = options.getCount();
TreeSet<IPickerEntry> entries = new TreeSet<IPickerEntry>();
if (startKey != null) {
// User is performing a search
try {
entries = this.dirLookup(startKey, count);
} catch (NotesException e) {
System.err.println("Exception trying to perform directory lookup: " + e.getMessage());
e.printStackTrace();
}
}
return new SimplePickerResult((List<IPickerEntry>) entries, -1);
}
public TreeSet<IPickerEntry> dirLookup(final String search, final int limit) throws NotesException {
TreeSet<IPickerEntry> result = new TreeSet<IPickerEntry>();
String server = props.getProperty("server_notesname");
String filepath = props.getProperty("db_project_data");
Database db = utils.getSession().getDatabase(server, filepath);
View vw = db.getView("vw_all_todo_namespicker");
vw.setAutoUpdate(false);
DocumentCollection dc = vw.getAllDocumentsByKey(search, false);
int count = 0;
Document tmpdoc;
Document doc = dc.getFirstDocument();
while (doc != null && count < limit) {
String person = doc.getItemValueString("app_ProjMgrName");
IPickerEntry entry = new SimplePickerResult.Entry(person, person);
result.add(entry);
// result.add(entry does not seem to work
tmpdoc = dc.getNextDocument();
doc.recycle();
doc = tmpdoc;
count = count +1;
}
vw.setAutoUpdate(true);
return result;
}
}
Is there anyone that can tell me what I m doing wrong? I have choosen a treeset instead of an arraylist. this is because I go to a view with lots of multiple entries so I do not want duplicates and have it sorted by values.

You're casting TreeSet to (List) at the line:
return new SimplePickerResult((List<IPickerEntry>) entries, -1);
because the SimplePickerResult needs a List (it won't accept a Collection), but TreeSet does not implement List, so that cast will fail.
You'll probably have to change it back to an ArrayList.
To sort, try using java.util.Collections.sort(List list, Comparator c) with a custom comparator that compares the entry.getLabel() value, as SimplePickerResult.Entry doesn't have an in-built compare method.

Hive, how do I retrieve all the database's tables columns

I want to write the equivalent of this sql request in Hive :
select * from information_schema.columns where table_schema='database_name'
How can I access hive's metastore and retrieve all the columns of all the tables stored in a specific database? I know that we can do
it by table via describe [table_name] but is there anyway to have all
the columns for all the tables in a database in the same request?

How can I access hive's metastore and retrieve all the columns of all
the tables stored in a specific database?
This is one way to connect HiveMetaStoreClient and you can use method getTableColumnsInformation will get columns.
In this class along with columns all the other information like partitions can be extracted. pls see example client and sample methods.
import org.apache.hadoop.hive.conf.HiveConf;
// test program
public class Test {
public static void main(String[] args){
HiveConf hiveConf = new HiveConf();
hiveConf.setIntVar(HiveConf.ConfVars.METASTORETHRIFTCONNECTIONRETRIES, 3);
hiveConf.setVar(HiveConf.ConfVars.METASTOREURIS, "thrift://host:port");
HiveMetaStoreConnector hiveMetaStoreConnector = new HiveMetaStoreConnector(hiveConf);
if(hiveMetaStoreConnector != null){
System.out.print(hiveMetaStoreConnector.getAllPartitionInfo("tablename"));
}
}
}
// define a class like this
import com.google.common.base.Joiner;
import com.google.common.collect.Lists;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.metastore.HiveMetaStoreClient;
import org.apache.hadoop.hive.metastore.api.FieldSchema;
import org.apache.hadoop.hive.metastore.api.MetaException;
import org.apache.hadoop.hive.metastore.api.Partition;
import org.apache.hadoop.hive.metastore.api.hive_metastoreConstants;
import org.apache.hadoop.hive.ql.metadata.Hive;
import org.apache.thrift.TException;
import org.joda.time.DateTime;
import org.joda.time.format.DateTimeFormatter;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
public class HiveMetaStoreConnector {
private HiveConf hiveConf;
HiveMetaStoreClient hiveMetaStoreClient;
public HiveMetaStoreConnector(String msAddr, String msPort){
try {
hiveConf = new HiveConf();
hiveConf.setVar(HiveConf.ConfVars.METASTOREURIS, msAddr+":"+ msPort);
hiveMetaStoreClient = new HiveMetaStoreClient(hiveConf);
} catch (MetaException e) {
e.printStackTrace();
System.err.println("Constructor error");
System.err.println(e.toString());
System.exit(-100);
}
}
public HiveMetaStoreConnector(HiveConf hiveConf){
try {
this.hiveConf = hiveConf;
hiveMetaStoreClient = new HiveMetaStoreClient(hiveConf);
} catch (MetaException e) {
e.printStackTrace();
System.err.println("Constructor error");
System.err.println(e.toString());
System.exit(-100);
}
}
public String getAllPartitionInfo(String dbName){
List<String> res = Lists.newArrayList();
try {
List<String> tableList = hiveMetaStoreClient.getAllTables(dbName);
for(String tableName:tableList){
res.addAll(getTablePartitionInformation(dbName,tableName));
}
} catch (MetaException e) {
e.printStackTrace();
System.out.println("getAllTableStatistic error");
System.out.println(e.toString());
System.exit(-100);
}
return Joiner.on("\n").join(res);
}
public List<String> getTablePartitionInformation(String dbName, String tableName){
List<String> partitionsInfo = Lists.newArrayList();
try {
List<String> partitionNames = hiveMetaStoreClient.listPartitionNames(dbName,tableName, (short) 10000);
List<Partition> partitions = hiveMetaStoreClient.listPartitions(dbName,tableName, (short) 10000);
for(Partition partition:partitions){
StringBuffer sb = new StringBuffer();
sb.append(tableName);
sb.append("\t");
List<String> partitionValues = partition.getValues();
if(partitionValues.size()<4){
int size = partitionValues.size();
for(int j=0; j<4-size;j++){
partitionValues.add("null");
}
}
sb.append(Joiner.on("\t").join(partitionValues));
sb.append("\t");
DateTime createDate = new DateTime((long)partition.getCreateTime()*1000);
sb.append(createDate.toString("yyyy-MM-dd HH:mm:ss"));
partitionsInfo.add(sb.toString());
}
} catch (TException e) {
e.printStackTrace();
return Arrays.asList(new String[]{"error for request on" + tableName});
}
return partitionsInfo;
}
public String getAllTableStatistic(String dbName){
List<String> res = Lists.newArrayList();
try {
List<String> tableList = hiveMetaStoreClient.getAllTables(dbName);
for(String tableName:tableList){
res.addAll(getTableColumnsInformation(dbName,tableName));
}
} catch (MetaException e) {
e.printStackTrace();
System.out.println("getAllTableStatistic error");
System.out.println(e.toString());
System.exit(-100);
}
return Joiner.on("\n").join(res);
}
public List<String> getTableColumnsInformation(String dbName, String tableName){
try {
List<FieldSchema> fields = hiveMetaStoreClient.getFields(dbName, tableName);
List<String> infs = Lists.newArrayList();
int cnt = 0;
for(FieldSchema fs : fields){
StringBuffer sb = new StringBuffer();
sb.append(tableName);
sb.append("\t");
sb.append(cnt);
sb.append("\t");
cnt++;
sb.append(fs.getName());
sb.append("\t");
sb.append(fs.getType());
sb.append("\t");
sb.append(fs.getComment());
infs.add(sb.toString());
}
return infs;
} catch (TException e) {
e.printStackTrace();
System.out.println("getTableColumnsInformation error");
System.out.println(e.toString());
System.exit(-100);
return null;
}
}
}

If you want to have the ability to run such queries that return hive metadata, you can setup Hive metastore with MySQL, metadata used in Hive is stored in a specific account of MySQL.
You will have to create a user of MySQL for hive by doing CREATE USER 'hive'#'metastorehost' IDENTIFIED BY 'mypassword'.
Then you will find tables like COLUMNS_VS with the info you are looking for.
An example query to retrieve all columns in all tables could be: SELECT COLUMN_NAME, TBL_NAME FROM COLUMNS_V2 c JOIN TBLS a ON c.CD_ID=a.TBL_ID
Alternatively, you can access this information via REST calls to WebHCat see wiki for more info.

Instantiating a class in call method of T Callable Class

Basically I've got a little threading class used by ExecutorService and a fixed thread pool. Each thread instantiates my threading class and the call method is fired, works great!
However I really need to call another class (via instantiation or static means) to process & return some data within the call method, however when trying this I understandably get concurrent.ExecutionException, along with related methods.
I think it will be easier to paste all my code here, note its very rough
MainController
package com.multithreading.excutorservice;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
public class MainController {
private static List<String> urls;
public static void main(String[] args) {
populateList();
// futures to retrieve task results
List<Future<ArrayList>> futures = new ArrayList<Future<ArrayList>>();
// results
List<ArrayList> results = new ArrayList<ArrayList>();
// pool with 5 threads
ExecutorService exec = Executors.newFixedThreadPool(5);
// enqueue tasks
for(String url: urls) {
futures.add(exec.submit(new ThreadTask(url)));
}
// attempt to move ArrayLists within Future<ArrayList> into a normal ArrayList
for(Future<ArrayList> future: futures) {
try {
results.add(future.get());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
// for(ArrayList<String> s: results) {
// System.out.println(s);
// }
}
private static void populateList() {
urls = new ArrayList<String>();
urls.add("http://www.google.com");
urls.add("http://www.msn.co.uk");
urls.add("http://www.yahoo.co.uk");
urls.add("http://www.google.com");
urls.add("http://www.msn.co.uk");
urls.add("http://www.yahoo.co.uk");
}
}
ThreadTask
package com.multithreading.excutorservice;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
public class ThreadTask implements Callable<ArrayList> {
private String url;
HtmlParser parseHtml;
public ThreadTask(String url) {
this.url = url;
}
public ArrayList call() {
int counter = 0;
String html = null;
try {
URL myUrl = new URL(url);
BufferedReader reader = new BufferedReader(new InputStreamReader(myUrl.openStream()));
while ((html = reader.readLine()) != null) {
//counter += inputLine.length();
html += html;
}
}
catch (Exception ex) {
System.out.println(ex.toString());
}
ArrayList<String> storeLinks = new ArrayList<String>();
HtmlParser par = new HtmlParser();
storeLinks = par.returnNewUrls(html);
// for(String s: parseHtml) {
// System.out.println(s);
// }
//returns an ArrayList of URLS which is stored in a List<Future<ArrayList>> temporarily
return storeLinks;
}
}
HtmlParser
package com.multithreading.excutorservice;
import java.util.ArrayList;
import java.util.concurrent.Callable;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class HtmlParser {
private final String regex_links = "\\s*(?i)href\\s*=\\s*(\"([^\"]*\")|'[^']*'|([^'\">\\s]+))";
private ArrayList<String> extractedUrls;
public ArrayList<String> returnNewUrls (String data) {
extractedUrls = new ArrayList<String>();
Pattern p = Pattern.compile(regex_links);
Matcher m = p.matcher(data);
System.out.println("Test");
while (m.find()) {
System.out.println("Test");
extractedUrls.add(m.group(1));
}
return getLinks();
}
//returns the links
public ArrayList getLinks() {
return extractedUrls;
}
}

You're doing some pretty weird stuff here. Multiple threads are accessing the same static extractedUrls field, and each call to returnNewUrls creates a new field. In your returnNewUrls method, create a new ArrayList which is local to the method scope. Something along the lines of:
public static ArrayList<String> returnNewUrls(String data) {
ArrayList<String> urls = new ArrayList<String>();
addStuffToUrlsList();
return urls;
}
Another thing - not a bug, but you're doing unnecessary stuff - in the call method you don't need to create a new list if you're just assigning to a variable:
ArrayList<String> parseHtml = new ArrayList<String>();
parseHtml = HtmlParser.returnNewUrls(html);
This is better:
ArrayList<String> parseHtml = HtmlParser.returnNewUrls(html);

You have several concurrent tasks, but they use the same variable HtmlParser.extractedUrls? without any synchronization. Move this variable inside the returnNewUrls method.
BTW even without concurrency, using static variables is not encouraged, especially in a case like that, where it can easily be replaced with a local variable.

Convert Matlab output into ArrayList in Java

I'm doing sampling from log-normal distribution in Java using Matlab code (with javabuilder java).
Here is the code :
import demo2.*;
import com.mathworks.toolbox.javabuilder.*;
import java.util.*;
public class ht {
/**
* #param args
*/
public static void main(String[] args) {
// TODO Auto-generated method stub
Object[] x = null; //?? What does Object[] mean?//
ArrayList th = new ArrayList();
demo y = null;
try {
y = new demo(); //the class created by Matlab builder ja//
x=y.lognorma(1, 10); //function to sample the distribution//
} catch (MWException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
What does Object[] mean in this context and how to change Object[] x into normal ArrayList in Java?

In the end of "catch" block add
th = new ArrayList(x);
or
List<Object> res = Arrays.asList(x);