I have a table with < frozen < set < text > > > , column. I can't understand how I may read a column value from a row.Trying below but not able to get pass through.
else if (key.getType() == DataType.frozenSet(text))
{
str = row.getSet( , )
}
I tried the below , but i see nothing is printing out from that frozen set column.
public static void main(String[] args) throws FileNotFoundException {
long startTime = System.currentTimeMillis();
PrintWriter pw = new PrintWriter(new File("test.csv"));
String keyspace = "xxxxx";
String table = "xxxxxx";
String username = "xxxx";
String password = "xxxx";
String host = "xxxxxxx";
double count = 0;
Cluster.Builder clusterBuilder = Cluster.builder()
.addContactPoints(host)
.withCredentials(username, password);
Cluster cluster = clusterBuilder.build();
Session session = cluster.connect(keyspace);
Statement stmt = new SimpleStatement("SELECT * FROM " + table);
stmt.setFetchSize(2000);
ResultSet rs = session.execute(stmt);
Iterator<Row> iter = rs.iterator();
while ( !rs.isFullyFetched()) {
if (rs.getAvailableWithoutFetching() == 120 )
rs.fetchMoreResults();
Row row = iter.next();
if ( rs != null )
{
StringBuilder line = new StringBuilder();
for (Definition key : row.getColumnDefinitions().asList())
{
String val = myGetValue(key, row);
line.append("\"");
line.append(val);
line.append("\"");
line.append(',');
}
line.deleteCharAt(line.length()-1);
line.append('\n');
pw.write(line.toString());
System.out.println(line.toString());
++count;
}
}
pw.close();
session.close();
cluster.close();
System.out.println(count + "\t rows copied into csv");
long endTime = System.currentTimeMillis();
System.out.println("Took "+(endTime - startTime) + " ms");
}
public static String myGetValue(Definition key, Row row)
{
String str = "";
if (key != null)
{
String col = key.getName();
try
{
if (key.getType() == DataType.cdouble())
{
str = new Double(row.getDouble(col)).toString();
}
else if (key.getType() == DataType.cint())
{
str = new Integer(row.getInt(col)).toString();
}
else if (key.getType() == DataType.uuid())
{
str = row.getUUID(col).toString();
}
else if (key.getType() == DataType.cfloat())
{
str = new Float(row.getFloat(col)).toString();
}
else if (key.getType() == DataType.timestamp())
{
str = row.getDate(col).toString();
SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd HH:mm:ssZ");
str = fmt.format(row.getDate(col));
}
else if (key.getType().equals(DataType.set(DataType.text()) ))
{
Set<String> st1 = row.getSet(i, String.class);
str = st1.toString();
++i;
}
else
{
str = row.getString(col);
}
} catch (Exception e)
{
str = "";
}
}
return str;
}
}
schema:
create table view_page_details(
clickid uuid,
view text,
clickin int,
names frozen <set<text>>,
msg_view text,
number_of_times int,
primary key ((clickid, view, clickin), names);
cqlsh output for that specific 'names' column in that table:
| {').'{'2', 'STOP', 'williams', 'The', 'appts:', 'at', 'family''s', 'first', 'of', 'one', 'reminder', 'starts', 'to', 'your'}
| {'2'{'2', 'STOP', 'shane', 'The', 'appts:', 'at', 'family''s', 'first', 'of'}
| {'1', '2/1/15.'{'2', 'STOP', 'brete', 'The', 'appts:', 'at', 'family''s', 'first', 'of', 'one', 'reminder', 'starts', 'to', 'your'}
My program output for that specific 'names'column in that table:
,,
,,
,,
,,
From what i observed by debugging line by line , String val = myGetValue(key, row); gets the entire row value correctly from cluster, when returns value from that function its not giving out the specified column value. Not sure if something wrong inside the function logic for that specific column.
The first argument is index of the column in the ResultSet.
By accessing this way you need to get get columns by their index not by name or key.
Set<String> mySet = row.getSet(index, String.class);
If you don't have the index, then you will need keep a counter as you iterate the keys.
You can find out more from the javadoc
https://docs.datastax.com/en/latest-java-driver-api/com/datastax/driver/core/GettableByIndexData.html#getSet-int-java.lang.Class-
longer answer:
I'm really guessing what most of your code does, but with the given schema I can output the set using this code:
Row row = rs.one();
String str = "";
int i = 0;
Iterator<ColumnDefinitions.Definition> it = rs.getColumnDefinitions().iterator();
while (it.hasNext()) {
ColumnDefinitions.Definition key = it.next();
if (key.getType().equals(DataType.set(DataType.varchar()))) {
Set<String> st1 = row.getSet(i, String.class);
str = st1.toString();
System.out.println(str);
}
i++;
}
Related
I'm trying to migrate a big db from derby to HSQLDB in a Spring Boot service, like 1.5M regs in few tables of 10 columns. I'm checking with VisualVM; byte and char consume a lot of memory. But the biggest delta in time are in the derby classes.
Sometimes the error is thrown here, but other times thrown in other controllers. I don't want to touch all files to add my catchOutofMemory to restart.
Following is a version of my code, the block comment shows the resume of the process:
run(){//thread inside static function.
while(keepMigrating){
keepMigrating=Migrate();
}
}
private static boolean Migrate(JdbcTemplate derby,JdbcTemplate hsql){
int regs = 100000;
PreparedStatement statement = null;
ResultSet rs = null;
PreparedStatement statementHSQL = null;
try {
for (String table : tables) {//tables contains all tables to migrate
//check how many registers left asd asign to cant, if cant is 0 the empty is true.
PreparedStatement statementUpd[];
while (!empty) {
if (trys <= 0) throw new Exception("redo");
//check how many registers left asd asign to cant, if cant is 0 the empty is true and out of bucle and ready to check next table
/*
*Next process resume as:
*fetch data from derby that hasnt been migrated limited by cant
*create a batch to insert in hsql
*create a update for derby
*create a delete in case someting goes wrong
*excecute insert and update, if someting in batch fail delete the entry in migrate table
*reduce regs to get out of migrate method at some ponint.
*/
statement = derby.getDataSource().getConnection().prepareStatement(
MessageFormat.format(select_all_migrate_false_and_fetch_cant,table));
statementUpd = new PreparedStatement[cant];
ArrayList<String> deleteIds = new ArrayList<>();
StringBuilder columnNames = new StringBuilder();
StringBuilder updateSQL = new StringBuilder();
StringBuilder bindVariables = new StringBuilder();
try {
ResultSetMetaData meta = rs.getMetaData();
for (int i = 1; i <= meta.getColumnCount(); i++) {
if (!meta.getColumnName(i).equals("MIGRATED")) {
if (i > 1) {
columnNames.append(", ");
bindVariables.append(", ");
}
columnNames.append(meta.getColumnName(i));
bindVariables.append('?');
}
}
String sql = "INSERT INTO " + table.substring(4) + " ("
+ columnNames
+ ") VALUES ("
+ bindVariables
+ ")";
statementHSQL = hsql.getDataSource().getConnection().prepareStatement(sql);
HashMap<String, Object> data = new HashMap<>();
int row = 0;
int lastId = 0;
String columnName;
while (rs.next()) {
for (int i = 1; i <= meta.getColumnCount(); i++) {
columnName = meta.getColumnName(i);
Object o = rs.getObject(i);
statementHSQL.setObject(i, o);
if (columnName.equals(mainColumn))
deleteIds.add(String.valueOf(o));
if (!(meta.getColumnType(i) == 2004)) data.put(columnName, o);
if (columnName.equals(mainColumn)) id = rs.getObject(i);
}
int c = 1;
String update = MessageFormat.format("INSERT INTO {0}M ({1}M, MIGRATED) VALUES(?, TRUE)",
table.substring(4), mainColumn).replace("\"M", "M\"");//migrated state is saved in other table
lastId = Integer.valueOf(String.valueOf(id));
statementUpd[row] = derby.getDataSource().getConnection().prepareStatement(update);
statementUpd[row].setObject(1, rs.getObject(mainColumn));
updateSQL = new StringBuilder();
statementHSQL.addBatch();
row += 1;
}
/*
* Build delete query in case of inserted values in HSQLDB but not updated in DERBY
*/
StringBuilder builder = new StringBuilder();
builder.append("(");
int count = 1;
for (String s : deleteIds) {
if (count > 1) builder.append(", ");
builder.append("?");
count++;
}
builder.append(")");
String str = builder.toString();
String queryDelete = "DELETE FROM " + table.substring(4) + " WHERE " + mainColumn + " IN " + str;
PreparedStatement statementHSQLDel = hsql.getDataSource().getConnection().prepareStatement
(queryDelete);
int c = 1;
for (String s : deleteIds) {
statementHSQLDel.setObject(c, s);
c++;
}
boolean deletes = statementHSQLDel.execute();
statementHSQLDel.close();
try {
DatabaseUtils.close(statementHSQLDel);
} catch (Exception e) {
catchOutOfMemory(e);
}
int[] result = statementHSQL.executeBatch();
StringBuilder resultS = new StringBuilder();
int stCounter = 0;
int stCounterInsert = 0;
int stCounterUpdate = 0;
String notarydebug;
for (int i : result) {
int upd = 0;
try {
if (i == 1) upd = statementUpd[stCounter].executeUpdate();
} catch (Exception e) {
catchOutOfMemory(e);
}
stCounterInsert += i;
stCounterUpdate += upd;
resultS.append(",").append(String.valueOf(i)).append("-").append(String.valueOf(upd));
stCounter += 1;
}
statementHSQL.clearBatch();
try {
DatabaseUtils.close(statementHSQL);
} catch (Exception e) {
catchOutOfMemory(e);
}
} catch (SQLException se) {
catchOutOfMemory(se);//otherstuff
} catch (Exception e) {
catchOutOfMemory(e);
}
try {
DatabaseUtils.close(rs);
DatabaseUtils.close(statement);
} catch (Exception e) {
catchOutOfMemory(e);
}
regs=regs-cant;
}
}
}catch (Exception e) {
if (e.getMessage().equals("redo")) return true;//end the loop of regs maximun and get out of method.
}
return false;//end migration succesfully
}
private static int catchOutOfMemory(Throwable e) {
if (e == null) return 0;
if (e instanceof OutOfMemoryError) {
Application.restartBat();
return 1;
} else {
return catchOutOfMemory(e.getCause());
}
}
edit:
So I change as sugested inthe comment to accept a commit, something like this:
Connection hsqlCon;
PrepareStatement hsqlStm;
hsqlCon = JdbcHSQLDB.getDataSource().getConnection();
hsqlStm = hsqlCon.prepareStatement(sql);
hsqlStm.addBatch();
hsqlStm.execute();
hsqlStm.close();
hsqlCon.close();
but i got the same heap memory consumpsion:
The type of table in HSQLDB is not clear from the supplied code. You must use this statement once for each table, to make sure the table data is stored in the finename.data file:
SET TABLE tableName TYPE CACHED
The reported sequence of batch INSERT is not correct. Use this sequence:
Connection hsqlCon;
PrepareStatement hsqlStm;
hsqlCon = JdbcHSQLDB.getDataSource().getConnection();
hsqlStm = hsqlCon.prepareStatement(sql);
{ // repeat this block until all is finished
{ // repeat for 1000 rows
hsqlStm.addBatch();
}
hsqlStm.executeBatch(); // after every 1000 rows
}
hsqlStm.close();
hsqlCon.close();
I tried to make an exporting data to PDF but when I try to export it, the pdf can't show up like "no data found"
this code on bean
public JasperPrint exportTo() {
if(this.listReportMaster == null || this.listReportMaster.isEmpty()){
FacesMessage messageFailed = new FacesMessage(FacesMessage.SEVERITY_INFO,"Info","No data found");
RequestContext.getCurrentInstance().showMessageInDialog(messageFailed);
return null;
}
String path = FacesContext.getCurrentInstance().getExternalContext().getRealPath("/resources/report/PRPKReportPDF.jasper");
JRBeanCollectionDataSource beanCollectionDataSource = new JRBeanCollectionDataSource(this.listReportMaster);
try {
JasperPrint jasperPrint = JasperFillManager.fillReport(path, null, beanCollectionDataSource);
return jasperPrint;
} catch (JRException e) {
e.printStackTrace();
return null;
}
}
public void exportToPdf(ActionEvent actionEvent){
if(this.lazyMasterReportDataModel != null){
System.out.println("masuk exporttopdf");
String sql = ((LazyMasterReportDataModel) this.lazyMasterReportDataModel).getSqlReportPrint();
List<Object> listObject = ((LazyMasterReportDataModel) this.lazyMasterReportDataModel).getObjectSqlListReportPrint();
this.listReportMaster = reportMasterPRPKController.getPRPKForReport(sql, listObject);
JasperPrint jasperPrint = exportTo();
String fileName = "PRPKNew_Report".concat("_").concat(".pdf");
if(jasperPrint != null) reportMasterPRPKController.exportToPDF(fileName, jasperPrint);
else System.out.println("jasperprint null");
}else{
System.out.println("keluar exporttopdf");
FacesMessage messageFailed = new FacesMessage(FacesMessage.SEVERITY_INFO,"Info","No data found");
RequestContext.getCurrentInstance().showMessageInDialog(messageFailed);
}
}
every I try to export it, always show "no data found" which is the program run this code FacesMessage messageFailed = new FacesMessage(FacesMessage.SEVERITY_INFO,"Info","No data found"); and which meam the "this.lazyMasterReportDataModel" is null but when I check it again, there's nothing wrong on code, I don't know if it have a wrong code or deficiency code
this the lazy code
List<ReportMasterPRPK> listMasterPRPK = new ArrayList<>();
ReportMasterPRPKQuery reportMasterPRPKQuery = new ReportMasterPRPKQuery();
Page page = new Page();
String order = "GROUP a.prpk_number, a.prpk_type_id, a.created_date, a.pic_prpk_id, a.business_unit_id, a.pic_department_id, a.prpk_desc, a.prpk_request, a.prpk_background, a.prpk_analysis_benefit, a.priority_level_id, a.cost, b.prpk_type_name, c.business_unit, d.department_name, e.priority_name, f.user_name ORDER BY a.created_date ";
String columname = "";
String sql = "";
List<Object> objectSqlList = new ArrayList<>();
String sqlReport = "";
String sqlReportPrint = "";
List<Object> objectSqlListReport = new ArrayList<>();
List<Object> objectSqlListReportPrint = new ArrayList<>();
String flag;
public LazyMasterReportDataModel() {
}
public LazyMasterReportDataModel(String flag) { //ini
this.flag = flag;
}
public LazyMasterReportDataModel(String sqlReport, List<Object> objectSqlListReport) {
this.sqlReport = sqlReport;
this.objectSqlListReport = objectSqlListReport;
}
#Override
public List<ReportMasterPRPK> load(int first, int pageSize, String sortField, SortOrder sortOrder,
Map<String, Object> filters) {
page.setLimit(pageSize);
page.setOffset(first);
if(this.sqlReport != null){
this.sql = this.sqlReport;
this.objectSqlList = this.objectSqlListReport;
}else{
sql = "";
objectSqlList = new ArrayList<>();
//objectSqlList.clear();
}
if(flag != null){ //ini
if(flag.equals("no selected")){
sql = sql+" AND c.is_selected = 'n' ";
}
}
if (filters != null){
for(String key: filters.keySet()){
String filterColumnName = "";
for(Field field : ReportMasterPRPK.class.getDeclaredFields()){
if(field.getName().equals(key)) filterColumnName = field.getAnnotation(Column.class).value();
}
if(filters.get(key) instanceof String){
if("receivedDate".equals(key)){
if(((String)filters.get(key)).trim().length() > 20){
String startDate = "'" + filters.get(key).toString().substring(0, 10) + "'";
String endDate = "'" + filters.get(key).toString().substring(11, 21) + "'";
sql = sql + " AND " + filterColumnName + " BETWEEN " + startDate + " AND " + endDate+ " ";
}
}else{
if(((String) filters.get(key)).trim().length() > 0){
sql = sql+"AND "+filterColumnName+" ILIKE ? ";
String value = "%"+filters.get(key)+"%";
objectSqlList.add(value);
}
}
}else{
if(((String[]) filters.get(key)).length > 0){
sql = sql+" AND "+filterColumnName+" in ";
String value = "(";
for(String string : (String[]) filters.get(key)){
value = value+"'"+string+"',";
}
value = value.substring(0, value.length()-1)+") ";
sql = sql + value;
}
}
}
}
if(sortField != null){
for(Field field : ReportMasterPRPK.class.getDeclaredFields()){
if(field.getName().equals(sortField)) columname = field.getAnnotation(Column.class).value();
}
if(sortOrder.toString().equals("ASCENDING")) order = " ASC";
else order = " DESC";
sql = sql+" GROUP a.prpk_number, a.prpk_type_id, a.created_date, a.pic_prpk_id, a.business_unit_id, a.pic_department_id, a.prpk_desc, a.prpk_request, a.prpk_background, a.prpk_analysis_benefit, a.priority_level_id, a.cost, b.prpk_type_name, c.business_unit, d.department_name, e.priority_name, f.user_name ORDER BY "+columname+" "+order;
System.out.println("sql sort: "+sql+" : "+objectSqlList.size());
}else{
sql = sql + order;
}
sqlReportPrint = sql;
objectSqlListReportPrint = objectSqlList;
this.listMasterPRPK = reportMasterPRPKQuery.retrivePage(page, sql, objectSqlList.toArray());
int dataSize = reportMasterPRPKQuery.retrieveMaxRow(sql, objectSqlList.toArray());
this.setRowCount(dataSize);
//objectSqlList.clear();
if(this.sqlReport != null){
this.sql = this.sqlReport;
this.objectSqlList = this.objectSqlListReport;
}else{
sql = "";
objectSqlList.clear();
}
order = "GROUP a.prpk_number, a.prpk_type_id, a.created_date, a.pic_prpk_id, a.business_unit_id, a.pic_department_id, a.prpk_desc, a.prpk_request, a.prpk_background, a.prpk_analysis_benefit, a.priority_level_id, a.cost, b.prpk_type_name, c.business_unit, d.department_name, e.priority_name, f.user_name ORDER BY a.created_date ";
return listMasterPRPK;
}
public List<ReportMasterPRPK> calculateRownum(List<ReportMasterPRPK> listMasterPRPK, int first){
int i = 1;
for (ReportMasterPRPK masterPRPK : listMasterPRPK) {
masterPRPK.setRownum(first + i);
i++;
}
return listMasterPRPK;
}
public String getSqlReportPrint() {
return sqlReportPrint;
}
public void setSqlReportPrint(String sqlReportPrint) {
this.sqlReportPrint = sqlReportPrint;
}
public List<Object> getObjectSqlListReportPrint() {
return objectSqlListReportPrint;
}
public void setObjectSqlListReportPrint(List<Object> objectSqlListReportPrint) {
this.objectSqlListReportPrint = objectSqlListReportPrint;
}
sorry before, if my english is to bad, but I hope you understand about what I mean...
thanks before...
Hi I have an arraylist of strings, I want to show the content of the arraylist on JLabel separated by a space or comma. But it shows me only one String, the last one.
public void ShowMovie(int idMovie) throws SQLException, IOException {
int ID = idMovie;
String IDMOVIE = Integer.toString(ID);
IDMovieLabel.setText(IDMOVIE);
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException ex) {
Logger.getLogger(Cover.class.getName()).log(Level.SEVERE, null, ex);
}
con = DriverManager.getConnection("jdbc:mysql://localhost/whichmovie", "Asis", "dekrayat24");
String sql = "SELECT Title,Year,Country,recomendacion,Cover,Rating,NameDirec,Name FROM movie "
+ "Inner join direction on (movie.idMovie=direction.idMovie5)"
+ "Inner join director on (direction.idDirector=director.idDirector)"
+ "Inner join cast on (movie.idMovie=cast.idMovie4)"
+ "Inner join actor on (cast.idActor=actor.idActor)"
+ "where idMovie= '" + ID + "'";
st = con.prepareStatement(sql);
rs = st.executeQuery(sql);
while (rs.next()) {
String titulo = rs.getString(1);
int añoInt = rs.getInt(2);
String año = Integer.toString(añoInt);
byte[] imagedataCover = rs.getBytes("Country");
byte[] imagedataCover1 = rs.getBytes("Cover");
format = new ImageIcon(imagedataCover);
format2 = new ImageIcon(imagedataCover1);
TituloLabel.setText(titulo);
AñoLabel.setText(año);
CountryLabel.setIcon(format);
DirectorLabel.setText(rs.getString(7));
int Recomend = rs.getInt(4);
String Recom = Integer.toString(Recomend);
RecommendLabel.setText(Recom);
int Rating = rs.getInt(6);
String Rat = Integer.toString(Rating);
RatingLabel.setText(Rat);
starRater1.setSelection(Rating);
starRater1.setEnabled(false);
Image imgEscalada = format2.getImage().getScaledInstance(CoverLabel.getWidth(),
CoverLabel.getHeight(), Image.SCALE_SMOOTH);
Icon iconoEscalado = new ImageIcon(imgEscalada);
CoverLabel.setIcon(iconoEscalado);
ArrayList<String> actors = new ArrayList<>();
actors.add(rs.getString(8));
System.out.println(actors);// Here i can see i get 9 actors.
StringBuilder sb = new StringBuilder();
boolean first = true;
for (String s : actors) {
if (!first) {
sb.append(' ');
}
sb.append(s);
first = false;
}
CastLabel1.setText(sb.toString());
}
rs.close();
st.close();
con.close();
}
Any help ?
Edit:unfortunately no solution has helped me, maybe I'm doing something wrong in the method, I post the full method.
String text = "";
for(int i = 0; i < actors.size(); i++){
text = text + actors.get(i);
if(i < actors.size() - 2){
text = text + ", ";
}
}
CastLabel1.setText(text);
The problem is you are resetting the label for each step in the for loop, and not creating a cumulative result. See below:
StringBuilder buf = new StringBuilder();
for(int i = 0; i < actors.size(); i++){
buf.append(actors.get(i));
if(i < actors.size() -1){
buf.append(" ");
}
}
CastLabel1.setText(buf.toString())
You should build the string you want to show first then set it to the text of the label:
StringBuilder sb = new StringBuilder();
boolean first = true;
for (String s : actors) {
if (!first)
sb.append(' ');
sb.append(s);
first = false;
}
CastLabel1.setText(sb.toString());
What you're currently doing is changing the entire label text during each iteration, so the final text is that of the last element in the list.
I have a webservice where from the Client-side some parameters are passed to perform a query on the DB, the Server-Side is supposed to carry out the query and return the results.Since the result might be more than one row and i will have to use it on the client-side to show an output this what i did:
1.Perform the query
2.take each row of the result and put it in an array
3.convert the array to String and pass it to the client side(converted array to String, because it was simple)
BUT the problem is that it doesnt pass the the array-turned-string but only the value which was used to initialize the string, here is the code
String ris = "";
String q;
String beta = null;
String one="";
String errore = connetti();
try {
if (errore.equals("")) {
Statement st = conn.createStatement();
//ESECUZIONE QUERY
q = "SELECT DISTINCT nome FROM malattia WHERE eta='" + age + "' AND sesso='" + sexstr + "' AND etnia='" + etniastr + "' AND sintomi IN(" + tes + ")";
ResultSet rs = st.executeQuery(q);
if (!rs.last()) {
ris = "no";
}
//This is the part which i'm talking about
else {
//getRowCount is another class used to find out number of rows,I use it to declare an array which would contain the result of the query
int two=getRowCount(rs);
String[] alpha= new String[two];
//Loop through the resultstatement and put result from the column **nome** in the array **alpha**
while(rs.next()){
alpha[i]=rs.getString("nome");
i++;
}
//The value of ris which is empty, is returned
ris="";
//instead of this one, where i convert the array **alpha** to String
ris=arrayToString(alpha,",");
}
}
else {
ris = errore;
}
conn.close();
} catch (Exception e) {
ris = e.toString();
}
return ris;
}
//returns the number of rows of **ris**
public static int getRowCount(ResultSet set) throws SQLException
{
int rowCount;
int currentRow = set.getRow(); // Get current row
rowCount = set.last() ? set.getRow() : 0; // Determine number of rows
if (currentRow == 0) // If there was no current row
set.beforeFirst(); // We want next() to go to first row
else // If there WAS a current row
set.absolute(currentRow); // Restore it
return rowCount;
}
//converts the array to String
public String arrayToString(String[] array, String delimiter) {
StringBuilder arTostr = new StringBuilder();
if (array.length > 0) {
arTostr.append(array[0]);
for (int i=1; i<array.length; i++) {
arTostr.append(delimiter);
arTostr.append(array[i]);
}
}
return arTostr.toString();
Thanks alot in advance!
After conn.close() you return beta instead of ris. This may be the cause of the behavior you are experiencing. However, I am not sure because I can not properly see how you open and close the curly brackets.
I have a problem with my code. I need to do several operations on a log file with this structure:
190.12.1.100 2011-03-02 12:12 test.html
190.12.1.100 2011-03-03 13:18 data.html
128.33.100.1 2011-03-03 15:25 test.html
128.33.100.1 2011-03-04 18:30 info.html
I need to get the number of visits per month, number of visits per page and number of unique visitors based on the IP. That is not the question, I managed to get all three operations working. The problem is, only the first choice runs correctly while the other choices just return values of 0 afterwards, as if the file is empty, so i am guessing i made a mistake with the I/O somewhere. Here's the code:
import java.io.*;
import java.util.*;
public class WebServerAnalyzer {
private Map<String, Integer> hm1;
private Map<String, Integer> hm2;
private int[] months;
private Scanner input;
public WebServerAnalyzer() throws IOException {
hm1 = new HashMap<String, Integer>();
hm2 = new HashMap<String, Integer>();
months = new int[12];
for (int i = 0; i < 12; i++) {
months[i] = 0;
}
File file = new File("webserver.log");
try {
input = new Scanner(file);
} catch (FileNotFoundException fne) {
input = null;
}
}
public String nextLine() {
String line = null;
if (input != null && input.hasNextLine()) {
line = input.nextLine();
}
return line;
}
public int getMonth(String line) {
StringTokenizer tok = new StringTokenizer(line);
if (tok.countTokens() == 4) {
String ip = tok.nextToken();
String date = tok.nextToken();
String hour = tok.nextToken();
String page = tok.nextToken();
StringTokenizer dtok = new StringTokenizer(date, "-");
if (dtok.countTokens() == 3) {
String year = dtok.nextToken();
String month = dtok.nextToken();
String day = dtok.nextToken();
int m = Integer.parseInt(month);
return m;
}
}
return -1;
}
public String getIP(String line) {
StringTokenizer tok = new StringTokenizer(line);
if (tok.countTokens() == 4) {
String ip = tok.nextToken();
String date = tok.nextToken();
String hour = tok.nextToken();
String page = tok.nextToken();
StringTokenizer dtok = new StringTokenizer(date, "-");
return ip;
}
return null;
}
public String getPage(String line) {
StringTokenizer tok = new StringTokenizer(line);
if (tok.countTokens() == 4) {
String ip = tok.nextToken();
String date = tok.nextToken();
String hour = tok.nextToken();
String page = tok.nextToken();
StringTokenizer dtok = new StringTokenizer(date, "-");
return page;
}
return null;
}
public void visitsPerMonth() {
String line = null;
do {
line = nextLine();
if (line != null) {
int m = getMonth(line);
if (m != -1) {
months[m - 1]++;
}
}
} while (line != null);
// Print the result
String[] monthName = {"JAN ", "FEB ", "MAR ",
"APR ", "MAY ", "JUN ", "JUL ", "AUG ", "SEP ",
"OCT ", "NOV ", "DEC "};
for (int i = 0; i < 12; i++) {
System.out.println(monthName[i] + months[i]);
}
}
public int count() throws IOException {
InputStream is = new BufferedInputStream(new FileInputStream("webserver.log"));
try {
byte[] c = new byte[1024];
int count = 0;
int readChars = 0;
while ((readChars = is.read(c)) != -1) {
for (int i = 0; i < readChars; ++i) {
if (c[i] == '\n')
++count;
}
}
return count;
} finally {
is.close();
}
}
public void UniqueIP() throws IOException{
String line = null;
for (int x = 0; x <count(); x++){
line = nextLine();
if (line != null) {
if(hm1.containsKey(getIP(line)) == false) {
hm1.put(getIP(line), 1);
} else {
hm1.put(getIP(line), hm1.get(getIP(line)) +1 );
}
}
}
Set set = hm1.entrySet();
Iterator i = set.iterator();
System.out.println("\nNumber of unique visitors: " + hm1.size());
while(i.hasNext()) {
Map.Entry me = (Map.Entry)i.next();
System.out.print(me.getKey() + " - ");
System.out.println(me.getValue() + " visits");
}
}
public void pageVisits() throws IOException{
String line = null;
for (int x = 0; x <count(); x++){
line = nextLine();
if (line != null) {
if(hm2.containsKey(getPage(line)) == false)
hm2.put(getPage(line), 1);
else
hm2.put(getPage(line), hm2.get(getPage(line)) +1 );
}
}
Set set = hm2.entrySet();
Iterator i = set.iterator();
System.out.println("\nNumber of pages visited: " + hm2.size());
while(i.hasNext()) {
Map.Entry me = (Map.Entry)i.next();
System.out.print(me.getKey() + " - ");
System.out.println(me.getValue() + " visits");
}
}
Any help figuring out the problem would be much appreciated as I am quite stuck.
I didn't read the code thoroughly yet, but I guess you're not setting the read position back to the beginning of the file when you start a new operation. Thus nextLine() would return null.
You should create a new Scanner for each operation and close it afterwards. AFAIK scanner doesn't provide a method to go back to the first byte.
Currently I could also think of 3 alternatives:
Use a BufferedReader and call reset() for each new operation. This should cause the reader to go back to byte 0 provided you didn't call mark() somewhere.
Read the file contents once and iterate over the lines in memory, i.e. put all lines into a List<String> and then start at each line.
Read the file once, parse each line and construct an apropriate data structure that contains the data you need. For example, you could use a TreeMap<Date, Map<Page, Map<IPAdress, List<Visit>>>>, i.e. you'd store the visits per ip address per page for each date. You could then select the appropriate submaps by date, page and ip address.
The reset method of BufferedReader that Thomas recommended would only work if the file size is smaller than the buffer size or if you called mark with a large enough read ahead limit.
I would recommend reading throught the file once and to update your maps and month array for each line. BTW, you don't need a Scanner just to read lines, BufferedReader has a readLine method itself.
BufferedReader br = ...;
String line;
while (null != (line = br.readLine())) {
String ip = getIP(line);
String page = getPage(line);
int month = getMonth(line);
// update hashmaps and arrays
}