Develop Reference

Get All User timeline tweets using twitter4j and java

I have a problem if anyone can help,
I'm trying to get tweets done by a specific user, here's my code:
Paging pg = new Paging();
String userName = "Obama";
pg.setCount(200);
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey("");
cb.setOAuthConsumerSecret("");
cb.setOAuthAccessToken("");
cb.setOAuthAccessTokenSecret("");
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
int numberOfTweets = 1000000;
long lastID = Long.MAX_VALUE;
ArrayList<Status> tweets = new ArrayList<Status>();
while (tweets.size () < numberOfTweets) {
tweets.addAll(twitter.getUserTimeline(userName,pg));
//System.out.println("Gathered " + tweets.size() + " tweets");
for (Status t: tweets) {
System.out.println(t.getUser().getName() + ": " + t.getText()+ " " );
};
pg.setMaxId(lastID-1);
}
System.out.println(tweets.size());
}
The problem is that the result is only the same results, the algorithm takes only the first few tweets from the timeline and makes them X time, and the profile has a million of tweets.
Can someone tell me how can I solve this problem please?
Thanks

Here is a way to do :
ArrayList<Status> statuses = new ArrayList<>();
int pageno = 1;
while(true) {
try {
System.out.println("getting tweets");
int size = statuses.size(); // actual tweets count we got
Paging page = new Paging(pageno, 200);
statuses.addAll(twitter.getUserTimeline(screenName, page));
System.out.println("total got : " + statuses.size());
if (statuses.size() == size) { break; } // we did not get new tweets so we have done the job
pageno++;
sleep(1000); // 900 rqt / 15 mn <=> 1 rqt/s
}
catch (TwitterException e) {
System.out.println(e.getErrorMessage());
}
} // while(true)
And you need a sleep function to respect rate limit :
static void sleep(long ms) {
try { Thread.sleep(ms); }
catch(InterruptedException ex) { Thread.currentThread().interrupt(); }
}
Reference : https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html

100 records at a time to udf

I have to pass record to an UDF which calls an API but as we want to do it parallely,we are using spark and thats why UDF is being developed, the problem here is that that UDF needs to take only 100 records at a time not more than that, it can't handle more than 100 records parallely, so how to ensure that only 100 record pass to it in one go please note we don't want to use count() function on whole record.
I am attaching the UDF code here,it's a generic UDF which returns array of struct.moreover if we pass 100 records in batchsize variable each time then,if suppose there are 198 records then if as we dont want to use count() we will not be knowing that its last batchsize is going to be 98.so how to handle that thing.
Guys... I have a generic UDF in which call is made for an API but before calling it creates batch of 100 firstly then only call restapi.. So the argument UDF takes are x1:string, x2:string, batchsize:integer(currently the batchsize is 100)..so in UDF until and unless the batchsize is not 100 the call will not happen.. And for each record it will return null.
So till 99th record it will return. Null but at 100th record the call will happen
[So, now the problem part:as we are taking batchsize 100 and call will take place only at 100th record. So, in condition like if we have suppose 198 record in file then 100 record will get the output but, other 98 will only return null as they will not get processed..
So please help a way around, and UDF take one record at a time, but it keep on collecting till 100th record.. I hope this clears up
public class Standardize_Address extends GenericUDF {
private static final Logger logger = LoggerFactory.getLogger(Standardize_Address.class);
private int counter = 0;
Client client = null;
private Batch batch = new Batch();
public Standardize_Address() {
client = new ClientBuilder().withUrl("https://ss-staging-public.beringmedia.com/street-address").build();
}
// StringObjectInspector streeti;
PrimitiveObjectInspector streeti;
PrimitiveObjectInspector cityi;
PrimitiveObjectInspector zipi;
PrimitiveObjectInspector statei;
PrimitiveObjectInspector batchsizei;
private ArrayList ret;
#Override
public String getDisplayString(String[] argument) {
return "My display string";
}
#Override
public ObjectInspector initialize(ObjectInspector[] args) throws UDFArgumentException {
System.out.println("under initialize");
if (args[0] == null) {
throw new UDFArgumentTypeException(0, "NO Street is mentioned");
}
if (args[1] == null) {
throw new UDFArgumentTypeException(0, "No Zip is mentioned");
}
if (args[2] == null) {
throw new UDFArgumentTypeException(0, "No city is mentioned");
}
if (args[3] == null) {
throw new UDFArgumentTypeException(0, "No State is mentioned");
}
if (args[4] == null) {
throw new UDFArgumentTypeException(0, "No batch size is mentioned");
}
/// streeti =args[0];
streeti = (PrimitiveObjectInspector)args[0];
// this.streetvalue = (StringObjectInspector) streeti;
cityi = (PrimitiveObjectInspector)args[1];
zipi = (PrimitiveObjectInspector)args[2];
statei = (PrimitiveObjectInspector)args[3];
batchsizei = (PrimitiveObjectInspector)args[4];
ret = new ArrayList();
ArrayList structFieldNames = new ArrayList();
ArrayList structFieldObjectInspectors = new ArrayList();
structFieldNames.add("Street");
structFieldNames.add("city");
structFieldNames.add("zip");
structFieldNames.add("state");
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
StructObjectInspector si2 = ObjectInspectorFactory.getStandardStructObjectInspector(structFieldNames,
structFieldObjectInspectors);
ListObjectInspector li2;
li2 = ObjectInspectorFactory.getStandardListObjectInspector(si2);
return li2;
}
#Override
public Object evaluate(DeferredObject[] args) throws HiveException {
ret.clear();
System.out.println("under evaluate");
// String street1 = streetvalue.getPrimitiveJavaObject(args[0].get());
Object oin = args[4].get();
System.out.println("under typecasting");
int batchsize = (Integer) batchsizei.getPrimitiveJavaObject(oin);
System.out.println("batchsize");
Object oin1 = args[0].get();
String street1 = (String) streeti.getPrimitiveJavaObject(oin1);
Object oin2 = args[1].get();
String zip1 = (String) zipi.getPrimitiveJavaObject(oin2);
Object oin3 = args[2].get();
String city1 = (String) cityi.getPrimitiveJavaObject(oin3);
Object oin4 = args[3].get();
String state1 = (String) statei.getPrimitiveJavaObject(oin4);
logger.info("address passed, street=" + street1 + ",zip=" + zip1 + ",city=" + city1 + ",state=" + state1);
counter++;
try {
System.out.println("under try");
Lookup lookup = new Lookup();
lookup.setStreet(street1);
lookup.setCity(city1);
lookup.setState(state1);
lookup.setZipCode(zip1);
lookup.setMaxCandidates(1);
batch.add(lookup);
} catch (BatchFullException ex) {
logger.error(ex.getMessage(), ex);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
/* batch.add(lookup); */
if (counter == batchsize) {
System.out.println("under if");
try {
logger.info("batch input street " + batch.get(0).getStreet());
try {
client.send(batch);
} catch (Exception e) {
logger.error(e.getMessage(), e);
logger.warn("skipping current batch, continuing with the next batch");
batch.clear();
counter = 0;
return null;
}
Vector<Lookup> lookups = batch.getAllLookups();
for (int i = 0; i < batch.size(); i++) {
// ListObjectInspector candidates;
ArrayList<Candidate> candidates = lookups.get(i).getResult();
if (candidates.isEmpty()) {
logger.warn("Address " + i + " is invalid.\n");
continue;
}
logger.info("Address " + i + " is valid. (There is at least one candidate)");
for (Candidate candidate : candidates) {
final Components components = candidate.getComponents();
final Metadata metadata = candidate.getMetadata();
logger.info("\nCandidate " + candidate.getCandidateIndex() + ":");
logger.info("Delivery line 1: " + candidate.getDeliveryLine1());
logger.info("Last line: " + candidate.getLastLine());
logger.info("ZIP Code: " + components.getZipCode() + "-" + components.getPlus4Code());
logger.info("County: " + metadata.getCountyName());
logger.info("Latitude: " + metadata.getLatitude());
logger.info("Longitude: " + metadata.getLongitude());
}
Object[] e;
e = new Object[4];
e[0] = new Text(candidates.get(i).getComponents().getStreetName());
e[1] = new Text(candidates.get(i).getComponents().getCityName());
e[2] = new Text(candidates.get(i).getComponents().getZipCode());
e[3] = new Text(candidates.get(i).getComponents().getState());
ret.add(e);
}
counter = 0;
batch.clear();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
return ret;
} else {
return null;
}
}
}

IN OR multiple operator SAP Java

I am creating a query using JCO, SAP util for the following code for example:
public static void TEST() throws JCoException {
JCoDestination destination;
JCoRepository sapRepository;
destination = JCoDestinationManager.getDestination(ABAP_AS);
JCoDestinationManager.getDestination(ABAP_AS);
System.out.println("Attributes:");
System.out.println(destination.getAttributes());
System.out.println();
try {
JCoContext.begin(destination);
sapRepository = destination.getRepository();
if (sapRepository == null) {
System.out.println("Couldn't get repository!");
System.exit(0);
}
JCoFunctionTemplate functionTemplate = sapRepository.getFunctionTemplate("EM_GET_NUMBER_OF_ENTRIES");
JCoFunction function = functionTemplate.getFunction();
JCoTable itTable = function.getTableParameterList().getTable("IT_TABLES");
itTable.appendRow();
itTable.setValue("TABNAME", "USR02");
// JCoTable returnOptions_ = function.getTableParameterList().getTable("OPTIONS");
// returnOptions_.appendRow();
//// //returnOptions.setValue("TEXT", "MODDA GE '20140908' AND MODTI GT '000000'");
// returnOptions_.setValue("TEXT", "BNAME EQ 'USER'");
function.execute(destination);
System.out.println( function.getTableParameterList().getTable("IT_TABLES").getInt("TABROWS"));
JCoFunctionTemplate template2 = sapRepository.getFunctionTemplate("RFC_READ_TABLE");
System.out.println("Getting template");
JCoFunction function2 = template2.getFunction();
function2.getImportParameterList().setValue("QUERY_TABLE", "USR02");
function2.getImportParameterList().setValue("DELIMITER", ",");
function2.getImportParameterList().setValue( "ROWCOUNT",5);
function2.getImportParameterList().setValue( "ROWSKIPS",5);
System.out.println("Setting OPTIONS");
// Date date = new Date(1410152400000L);
SimpleDateFormat formatter = new SimpleDateFormat("yyyyMMddHHmmss");
// String dateString = formatter.format(date);
// String dt = dateString.substring(0, 8);
// String tm = dateString.substring(8);
// System.out.println("dt > " + dt + ", tm > " + tm);
JCoTable returnOptions = function2.getTableParameterList().getTable("OPTIONS");
returnOptions.appendRow();
//returnOptions.setValue("TEXT", "MODDA GE '20140908' AND MODTI GT '000000'");
returnOptions.setValue("TEXT", "BNAME LIKE 'S%'");
// returnOptions.appendRow();
// returnOptions.setValue("TEXT", "AND TYPE = 'DN'");
System.out.println("Setting FIELDS");
JCoTable returnFields = function2.getTableParameterList().getTable("FIELDS");
returnFields.appendRow();
returnFields.setValue("FIELDNAME", "BNAME");
returnFields.appendRow();
returnFields.setValue("FIELDNAME", "GLTGB");
returnFields.appendRow();
returnFields.setValue("FIELDNAME", "CLASS");
// returnFields.appendRow();
function2.execute(destination);
// JCoTable jcoTablef = function2.getTableParameterList().getTable("FIELDS");
JCoTable jcoTabled = function2.getTableParameterList().getTable("DATA");
int icodeOffSet = 0;
int icodeLength = 0;
int numRows = jcoTabled.getNumRows();
System.out.println("numRows > " + numRows);
for(int i=0; i<numRows; i++) {
jcoTabled.setRow(i);
System.out.println(jcoTabled.getRow());
String BNAME = "BNAE:" + jcoTabled.getString(0);
// String GLTGB = "GLTGB:" + jcoTabled.getString(2);
// String cls = "GLTGB:" + jcoTabled.getString(3);
System.out.println(BNAME + "..." );
}
} catch (Exception e) {
e.printStackTrace();
System.out.println("ERROR: " + e.getMessage());
} finally {
JCoContext.end(destination);
}
}
static void createDestinationDataFile(String destinationName, Properties connectProperties)
{
File destCfg = new File(destinationName+".jcoDestination");
try
{
FileOutputStream fos = new FileOutputStream(destCfg, false);
connectProperties.store(fos, "for tests only !");
fos.close();
}
catch (Exception e)
{
throw new RuntimeException("Unable to create the destination files", e);
}
}
The previous code worked well when I used the EQ operator.
However, when I used the IN operator:
BNAME IN ('USER1','USER','USER3')
or
BNAME EQ 'USER1' OR BNAME EQ 'USER' OR BNAME EQ 'USER3'
It throws an exception: Unexpected dynamic condition
Are there any limitations to the condition size? Since I have 22 field in the IN condition and each value has a size of 10?

You need to specify a valid OpenSQL condition, you need to observe the rules for dynamic conditions and you need to ensure that the condition is properly split into lines of 72 characters. My guess would be that the last bit might have been an issue if you're specifying 22 conditions...

How do you write to a file in java?

This is a code snippet that shows me trying to write to a file.
public void printContents() {
int i = 0;
try {
FileReader fl = new FileReader("Product List.txt");
Scanner scn = new Scanner(fl);
while (scn.hasNext()) {
String productName = scn.next();
double productPrice = scn.nextDouble();
int productAmount = scn.nextInt();
System.out.println(productName + " is " + productPrice + " pula. There are " + productAmount + " items left in stalk.");
productList[i] = new ReadingAndWritting(productName, productPrice, productAmount);
i = i + 1;
}
scn.close();
} catch (IOException exc) {
exc.printStackTrace();
} catch (Exception exc) {
exc.printStackTrace();
}
}
public void writeContents() {
try {
//FileOutputStream formater = new FileOutputStream("Product List.txt",true);
Formatter writer = new Formatter(new FileOutputStream("Product List.txt", false));
for (int i = 0; i < 2; ++i) {
writer.format(productList[i].name + "", (productList[i].price + 200.0 + ""), (productList[i].number - 1), "\n");
}
writer.close();
} catch (Exception exc) {
exc.printStackTrace();
}
}
The exception thrown while trying to run this code is:
java.util.NoSuchElementException at ReadingAndWritting.printContents(ReadingAndWritting.java:37).
I tried multiple things and only ended up with: "cokefruitgushersAlnassma" in the file. What I want is:
coke 7.95 10
fruitgushers 98.00 6
Alnassma 9.80 7

The Problem seems to be in
String productName = scn.next();
// Here:
double productPrice = scn.nextDouble();
// And here:
int productAmount = scn.nextInt();
After scn.next(), you don't check if scn.hasNext() before requesting the next element (double or int, respectively). So, either your file is not complete, or not in the exact structure you expect, or you just missed the two additional checks before trying to work with data which just isn't there.
Solution could be like:
while (scn.hasNext()) {
String productName = scn.next();
if ( scn.hasNext() ) {
double productPrice = scn.nextDouble();
if ( scn.hasNext() ) {
int productAmount = scn.nextInt();
// Do something with the three values read...
} else {
// Premature end of file(?)
}
} else {
// Premature end of file(?)
}
}

Android SQLite cursor returns 33 results, but debug only outputs 17

Hi folks I've got a strange cade. I'm trying to debug the SQLite DB in an app. If I do a query SELECT * from table I get 33 results, but if I iterate over the cursor it ends at result #17.
Here's my debug class (the method in question is public static void WriteToFile(Cursor cursor, String query , String tables, String uri)) :
package com.s2u.android.ps;
import java.io.File;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.List;
import android.content.Context;
import android.database.Cursor;
import android.database.sqlite.SQLiteDatabase;
import android.os.Environment;
import android.util.Log;
import com.s2u.android.ps.BO.App;
import com.s2u.android.ps.BO.AppMember;
import com.s2u.android.ps.datamodel.DatabaseManager;
import com.s2u.android.ps.networkApis.AndroidLog;
import com.s2u.android.ps.networkApis.AppConfig;
public class DebugToFile {
private static String TAG = "DebugToFile";
private static File path = new File(Environment.getExternalStorageDirectory() + "/ps_debug");
public static void WriteToFile(String lines , String tag)
{
WriteToFile(lines , tag , "txt");
}
public static void WriteToFile(String lines , String tag , String ext)
{
if (!Validate("WriteToFile(String lines)"))
return;
File file = new File(path, tag + "_" + GetDateStampTens() + "." + ext);
try
{
FileWriter fw = new FileWriter(file.getAbsolutePath() , true);
PrintWriter pw = new PrintWriter(fw);
pw.println(GetDateStamp() + " - " + lines);
pw.println();
pw.flush();
pw.close();
//Log.e(TAG, "WriteToFile(String lines) - " + file.toString());
}
catch (Exception e)
{
Log.e(TAG, "WriteToFile(String lines) failed!", e);
}
}
public static void WriteToFileAppMember(ArrayList<AppMember> appMembers , String tag)
{
if (!Validate("WriteToFile(AppMember)"))
return;
File file = new File(path, tag + "_" + GetDateStampTens() + ".csv");
try
{
FileWriter fw = new FileWriter(file.getAbsolutePath() , true);
PrintWriter pw = new PrintWriter(fw);
pw.println(GetDateStamp() + " - " + "AppMembers");
boolean doOnce = true;
for(com.s2u.android.ps.BO.AppMember appMember : appMembers)
{
if (doOnce)
{
doOnce = false;
pw.println(appMember.getCsvLabels());
}
pw.println(appMember.getCsvString());
}
pw.println();
pw.flush();
pw.close();
//Log.e(TAG, "WriteToFile(String lines) - " + file.toString());
}
catch (Exception e)
{
Log.e(TAG, "WriteToFile(String lines) failed!", e);
}
}
public static void WriteToFileAppMember(List<AppMember> appMembers , String tag)
{
if (!Validate("WriteToFile(AppMember)"))
return;
File file = new File(path, tag + "_" + GetDateStampTens() + ".csv");
try
{
FileWriter fw = new FileWriter(file.getAbsolutePath() , true);
PrintWriter pw = new PrintWriter(fw);
pw.println(GetDateStamp() + " - " + "AppMembers");
boolean doOnce = true;
for(com.s2u.android.ps.BO.AppMember appMember : appMembers)
{
if (doOnce)
{
doOnce = false;
pw.println(appMember.getCsvLabels());
}
pw.println(appMember.getCsvString());
}
pw.println();
pw.flush();
pw.close();
//Log.e(TAG, "WriteToFile(String lines) - " + file.toString());
}
catch (Exception e)
{
Log.e(TAG, "WriteToFile(String lines) failed!", e);
}
}
public static void WriteToFileApps(List<App> apps , String tag)
{
if (!Validate("WriteToFile(AppMember)"))
return;
File file = new File(path, tag + "_" + GetDateStampTens() + ".csv");
try
{
FileWriter fw = new FileWriter(file.getAbsolutePath() , true);
PrintWriter pw = new PrintWriter(fw);
pw.println(GetDateStamp() + " - " + "App objects");
boolean doOnce = true;
for(com.s2u.android.ps.BO.App app : apps)
{
if (doOnce)
{
doOnce = false;
pw.println(app.getCsvLabels());
}
pw.println(app.getCsvString());
}
pw.println();
pw.flush();
pw.close();
//Log.e(TAG, "WriteToFile(String lines) - " + file.toString());
}
catch (Exception e)
{
Log.e(TAG, "WriteToFile(String lines) failed!", e);
}
}
public static void WriteToFile(Cursor cursor, String query , String tables, String uri)
{
if (!Validate("WriteToFile(cursor)"))
return;
File file = new File(path, uri + "_" + GetDateStampTens() + ".csv");
try
{
FileWriter fw = new FileWriter(file.getAbsolutePath(), true);
PrintWriter pw = new PrintWriter(fw);
int resultCount = cursor.getCount();
pw.println("time: " + GetDateStamp());
pw.println("tables: " + tables);
pw.println("query: " + query);
pw.println("result count: " + Integer.toString(resultCount));
int row = 0;
String labels = "row,";
int startPosition = cursor.getPosition();
cursor.moveToPosition(-1);
while (cursor.moveToNext())
{
int colCount = cursor.getColumnCount();
row++;
if (row >= resultCount)
{
pw.println("Error! rows >= cursor count -- at row : " + Integer.toString(row) );
break;
}
StringBuilder line = new StringBuilder(512);
if (colCount <= 0)
pw.println("Empty row?");
for(int i = 0; i < colCount; i++)
{
if (row == 1)
{
labels += cursor.getColumnName(i) + "[" + GetCursorFieldTypeString(cursor, i) + "]";
if (i < colCount - 1)
labels += ",";
}
if (i == 0)
line.append(Integer.toString(row) + ",");
line.append(GetCursorString(cursor, i));
if (i < colCount - 1)
{
line.append(",");
}
}
if (row == 1)
pw.println(labels);
pw.println(line.toString());
cursor.moveToNext();
if (row > 100)
{
pw.println("max rows output - stopped at row: " + Integer.toString(row));
break;
}
}
pw.println("END");
pw.println();
pw.flush();
pw.close();
//Log.e(TAG, "WriteToFile(cursor) - " + file.toString());
cursor.moveToPosition(startPosition);
}
catch (Exception e)
{
Log.e(TAG, "WriteToFile(cursor) failed!", e);
}
}
private static boolean Validate(String methodName)
{
if (!AppConfig.isTestBuild())
{
Log.i(TAG, methodName + " - this is not a test build!");
return false;
}
if (!isExternalStorageWritable())
{
AndroidLog.e(TAG, methodName + " - external storage not accessible");
return false;
}
if (!path.exists())
{
path.mkdir();
if (!path.exists())
{
AndroidLog.e(TAG, methodName + " - directory doesn't exist and couldn't create it: " + path.toString());
return false;
}
}
return true;
}
private static String GetDateStamp()
{
Calendar c = Calendar.getInstance();
SimpleDateFormat df = new SimpleDateFormat("yyyyMMdd-kk:mm:ss.SSS");
String date = df.format(c.getTime());
return date;
}
private static String GetDateStampTens()
{
String date = GetDateStamp();
date = date.substring(0,date.length() - 1) + "0";
return date;
}
private static boolean isExternalStorageWritable() {
String state = Environment.getExternalStorageState();
if (Environment.MEDIA_MOUNTED.equals(state)) {
return true;
}
return false;
}
private static String GetCursorString(Cursor cursor, Integer i)
{
String result = "undefined";
switch(cursor.getType(i))
{
case Cursor.FIELD_TYPE_NULL:
result = "NULL";
break;
case Cursor.FIELD_TYPE_BLOB:
result = "BLOB length: " + Integer.toString(cursor.getBlob(i).length);
break;
case Cursor.FIELD_TYPE_FLOAT:
result = Float.toString(cursor.getFloat(i));
break;
case Cursor.FIELD_TYPE_INTEGER:
result = Integer.toString(cursor.getInt(i));
break;
case Cursor.FIELD_TYPE_STRING:
result = cursor.getString(i);
break;
default:
result = "undefined cursor value type(" + Integer.toString(cursor.getType(i)) + ") -- try getString: " + cursor.getString(i);
}
result.replace("", " ");
return result;
}
private static String GetCursorFieldTypeString(Cursor cursor, Integer i)
{
String result = "UNK";
switch(cursor.getType(i))
{
case Cursor.FIELD_TYPE_NULL:
result = "NULL";
break;
case Cursor.FIELD_TYPE_BLOB:
result = "BLOB";
break;
case Cursor.FIELD_TYPE_FLOAT:
result = "F";
break;
case Cursor.FIELD_TYPE_INTEGER:
result = "INT";
break;
case Cursor.FIELD_TYPE_STRING:
result = "STR";
break;
default:
result = "UNK(" + Integer.toString(cursor.getType(i)) + ") ";
}
return result;
}
public static String AppListTypeToString(int appListType)
{
if (appListType == 0)
return "kAppListMain";
else if (appListType == 1)
return "kAppListProfile";
else if (appListType == 2)
return "kAppListPromoted";
return "unknown list type int: " + Integer.toString(appListType);
}
public static void DumpDatabaseToFiles(DatabaseManager db)
{
SQLiteDatabase readableDb = db.getReadableDatabase();
DumpDatabaseToFiles(readableDb);
}
public static void DumpDatabaseToFiles(SQLiteDatabase db)
{
if (!Validate("DumpDatabaseToFiles"))
return;
Cursor c = db.rawQuery("SELECT name FROM sqlite_master WHERE type='table'", null);
if (c.getCount() <= 0)
{
WriteToFile("table name count: " + Integer.toString(c.getCount()) , "dbdump_err");
c.close();
return;
}
//AndroidLog.i(TAG , "DumpDB table count: " + Integer.toString(c.getCount()));
List<String> tableNames = new ArrayList<String>();
if (c.moveToFirst())
{
while(!c.isAfterLast())
{
tableNames.add(c.getString(c.getColumnIndex("name")));
c.moveToNext();
}
}
c.close();
for (int i = 0; i < tableNames.size(); i++)
{
String table = tableNames.get(i);
c = db.rawQuery("SELECT * FROM " + table + " LIMIT 100 ", null);
WriteToFile(c, "all" , table, table);
c.close();
}
}
}
The output csv file is:
tables: app - from AppDAO.bulkInsertApp
query: SELECT * FROM app
result count: 33
row,_id[INT],packageName[STR],appName[STR],iconUrl1[STR],iconUrl2[NULL],publisher[STR],publisherEmail[NULL],price[INT],currency[STR],version[STR],category[STR],releaseDate[NULL],updatedOn[NULL],hasTried[INT],promo_url[NULL],promoParam[NULL],promoValueKey[NULL]
1,8192,com.shared2you.android.powerslyde,Powerslyde,https://lh5.ggpht.com/1qigt9Zz7oh5kTFiIS9ukJljVTm7W-Ur34XzcaQhFjc9GlMzATJ-ATRwYB6gxQhscHEU=w300,NULL,Shared2you, Inc.,NULL,0,, 1.08 ,Lifestyle,NULL,NULL,1,NULL,NULL,NULL
2,8219,com.android.providers.downloads.ui,com.android.providers.downloads.ui,NULL,NULL,NULL,NULL,NULL,,NULL,NULL,NULL,NULL,1,NULL,NULL,NULL
3,8225,com.google.android.apps.maps,Maps,https://lh3.ggpht.com/JW-F0fkeBHpKyh8lDcyQ7CveTRynYGByVBH9hUqnJxw4x64ORhoFJISdOWhekULemw0=w300,NULL,Google Inc.,NULL,0,, Varies with devic,Travel & Local,NULL,NULL,1,NULL,NULL,NULL
4,8231,com.android.vending,com.android.vending,NULL,NULL,NULL,NULL,NULL,,NULL,NULL,NULL,NULL,1,NULL,NULL,NULL
5,8246,com.google.android.apps.magazines,Google Play Newsstand,https://lh5.ggpht.com/rowOPaiODov-bNG7rnD6awPZwLnOc7Vzab-29GpfvB6jfE8DhOR42owBqAmLUXj-W2sI=w300,NULL,Google Inc.,NULL,0,, 3.1.0 ,News & Magazines,NULL,NULL,1,NULL,NULL,NULL
6,8248,com.google.android.gm,Gmail,https://lh4.ggpht.com/Ebn-CW55BnkwG7ng5nuGpijVpJeabTa-uPijd4keKbHpedz29SvDj3EZkfr20ZZzznE=w300,NULL,Google Inc.,NULL,0,, Varies with devic,Communication,NULL,NULL,1,NULL,NULL,NULL
7,8250,com.google.android.music,Google Play Music,https://lh6.ggpht.com/5opWBg-m6yFcjWzJz1LlT05YIf2Alyiy9YtpQm1f6U42LXWmCvB54M1zEkV9-hCaoTc=w300,NULL,Google Inc.,NULL,0,, Varies with devic,Music & Audio,NULL,NULL,1,NULL,NULL,NULL
8,8253,com.google.android.videos,Google Play Movies & TV,https://lh5.ggpht.com/fFPQTALNNU4xflvbazvbwPL5o4X3a_CqYHUWIh4FXmfU78aSSuP1OMkGXhXouxXzWPov=w300,NULL,Google Inc.,NULL,0,, Varies with devic,Media & Video,NULL,NULL,1,NULL,NULL,NULL
9,8312,com.android.chrome,Chrome Browser - Google,https://lh6.ggpht.com/lum4KYB0TtgvR-8vRMUZ_JhRnMQ4YqBIR0yjspc4ETsM9iJ8-4YHZ0s0HO9i0ez_=w300,NULL,Google Inc.,NULL,0,, Varies with devic,Communication,NULL,NULL,1,NULL,NULL,NULL
10,8316,com.google.android.calendar,Google Calendar,https://lh5.ggpht.com/qgUPYBPSTb61cPrijI9YXV3BEy00t5bhoBugDpEXTdEsQEv9B9-j8_ZDs_ClQzPbskc=w300,NULL,Google Inc.,NULL,0,, 201308023 ,Productivity,NULL,NULL,1,NULL,NULL,NULL
11,8433,com.estrongs.android.pop,ES File Explorer File Manager,https://lh5.ggpht.com/P31CiAbF5UMC1wbJxv2sPT4tSLLqfqUZPp8N0ATEaA0ZeMxXv_NjVDiswVKjeUUSS2w=w300,NULL,ES APP Group,NULL,0,, Varies with devic,Productivity,NULL,NULL,1,NULL,NULL,NULL
12,8867,com.devhd.feedly,Feedly,https://lh4.ggpht.com/rkouDgWbT3WNztDRa5QvnN8SatDK3zeHHwOMHZbiu2Vlf3-9hLlmH89W9gJpGEtxo3U=w300,NULL,Feedly Team,NULL,0,, 18.1.2 ,News & Magazines,NULL,NULL,1,NULL,NULL,NULL
13,8917,com.google.android.email,com.google.android.email,NULL,NULL,NULL,NULL,NULL,,NULL,NULL,NULL,NULL,1,NULL,NULL,NULL
14,12113,com.google.android.play.games,Google Play Games,https://lh5.ggpht.com/tkg8ndU21RjzO5WSz7JRpYJ35P-oDTm0md2sNwvVoBtQ0kE_ORHhorrzQWcjVTevxP8_=w300,NULL,Google Inc.,NULL,0,, 1.1.04 ,Entertainment,NULL,NULL,1,NULL,NULL,NULL
15,87853,com.google.android.apps.docs.editors.sheets,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,NULL,NULL,NULL
16,87862,com.google.android.apps.photos,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,NULL,NULL,NULL
17,87867,com.umfersolutions.eatthiszombies,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1,NULL,NULL,NULL
END
Thanks!

You are advancing the cursor position two times, one in
while (cursor.moveToNext())
and the other one at the end of the loop in
pw.println(line.toString());
cursor.moveToNext();
Thats why you always will get half of the results, since at the end you move it one position, and then at then when checking the while condition it will advance again, so its reading position 0, then position 2, then 4...and so on...

Duplicate cursor.moveToNext() in the loop:
while (cursor.moveToNext())
{
...
pw.println(line.toString());
cursor.moveToNext();
...
}