I have a problem if anyone can help,
I'm trying to get tweets done by a specific user, here's my code:
Paging pg = new Paging();
String userName = "Obama";
pg.setCount(200);
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey("");
cb.setOAuthConsumerSecret("");
cb.setOAuthAccessToken("");
cb.setOAuthAccessTokenSecret("");
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
int numberOfTweets = 1000000;
long lastID = Long.MAX_VALUE;
ArrayList<Status> tweets = new ArrayList<Status>();
while (tweets.size () < numberOfTweets) {
tweets.addAll(twitter.getUserTimeline(userName,pg));
//System.out.println("Gathered " + tweets.size() + " tweets");
for (Status t: tweets) {
System.out.println(t.getUser().getName() + ": " + t.getText()+ " " );
};
pg.setMaxId(lastID-1);
}
System.out.println(tweets.size());
}
The problem is that the result is only the same results, the algorithm takes only the first few tweets from the timeline and makes them X time, and the profile has a million of tweets.
Can someone tell me how can I solve this problem please?
Thanks
Here is a way to do :
ArrayList<Status> statuses = new ArrayList<>();
int pageno = 1;
while(true) {
try {
System.out.println("getting tweets");
int size = statuses.size(); // actual tweets count we got
Paging page = new Paging(pageno, 200);
statuses.addAll(twitter.getUserTimeline(screenName, page));
System.out.println("total got : " + statuses.size());
if (statuses.size() == size) { break; } // we did not get new tweets so we have done the job
pageno++;
sleep(1000); // 900 rqt / 15 mn <=> 1 rqt/s
}
catch (TwitterException e) {
System.out.println(e.getErrorMessage());
}
} // while(true)
And you need a sleep function to respect rate limit :
static void sleep(long ms) {
try { Thread.sleep(ms); }
catch(InterruptedException ex) { Thread.currentThread().interrupt(); }
}
Reference : https://developer.twitter.com/en/docs/tweets/timelines/api-reference/get-statuses-user_timeline.html
Related
I am trying to build a plugin on cordova that can find the Bluetooth, USB and Network printer and print text, images, QR code, bar-codes... I have an issue in the network printer scanning, need some help with that. I have this code below that can search for the network printer connected to wifi. It works well with android 7 and 6 but in case of android 5, it is unable to return callback.This probably might be cause of the thread limit or something on android 5
scanWifi(ips, new OnIPScanningCallback() {
#Override
public void onScanningComplete(List<String> results) {
Log.d("TAG", "onScanningComplete: " + results.size()+" : "+results.toString());
Printers printerList =null;
for(String printerIps:results) {
String mac = getHardwareAddress(printerIps);
printerList = new Printers(printerIps, mac);
printers.put(printerIps);
list.add(printerList);
}
Log.d(TAGS,"The List of all wifi Printers : "+list);
}
});
isStart = false;
}
private static void scanWifi(final List<String> ips, final
OnIPScanningCallback callback) {
final Vector<String> results = new Vector<String>();
final int totalSize = ips.size();
final int splitSize = 10;
final int[] index = {0};
final long start = System.currentTimeMillis();
for (int i = 0; i < totalSize; i += splitSize) {
final List<String> child = new ArrayList<String>(ips.subList(i, Math.min(totalSize, i + splitSize)));
new Thread() {
#Override
public void run() {
for (String ip : child) {
Log.d("TAG", " scanning : " + index[0] + ", ip: " + ip);
boolean isPrinter = connect(ip);
if (isPrinter) {
results.add(ip);
}
if (index[0] == ips.size() - 1) {
long end = System.currentTimeMillis();
Log.d("TAG", "scanning time: " + (end - start) / 1000);
callback.onScanningComplete(results);
} else {
index[0]++;
}
}
}
}.start();
}
}
public interface OnIPScanningCallback {
void onScanningComplete(List<String> results);
}
I have also tried the async task and it works on all the versions of android but the problem is the process takes 170 to 193 secs which is way too long as in the above code it was able to do the same in 20 secs
scanWifi(ips, new PrintingPlugin.OnIPScanningCallback() {
#Override
public void onScanningComplete(List<String> results) {
Log.d(TAGS, "onScanningComplete: " + results.size() + " : " + results.toString());
Printers printerList;
for (String printerIps : results) {
String mac = getHardwareAddress(printerIps);
printerList = new Printers(printerIps, mac);
printers.put(printerIps);
list.add(printerList);
}
Log.d(TAGS, "The List of all wifi Printers : " + list);
}
});
isStart = false;
} catch (Exception e) {
Log.e(TAGS, "Error while scanning for wifi"+e.getMessage());
}
return printers;
}
private Integer index = 0;
void resetIndex() {
index = 0;
}
private void scanWifi(final List<String> ips, final PrintingPlugin.OnIPScanningCallback callback) {
Log.d(TAGS, " scanWifi" );
final Vector<String> results = new Vector<String>();
final int totalSize = ips.size();
final int splitSize = 10;
resetIndex();
final long start = System.currentTimeMillis();
for (int i = 0; i < totalSize; i += splitSize) {
final List<String> child = new ArrayList<String>(ips.subList(i, Math.min(totalSize, i + splitSize)));
executeTask(new AsyncTask() {
#Override
protected Object doInBackground(Object[] objects) {
synchronized (index) {
for (String ip : child) {
Log.d(TAGS, " scanning : " + index + ", ip: " + ip);
boolean isPrinter = connect(ip);
if (isPrinter) {
results.add(ip);
}
if (index == ips.size() - 1) {
long end = System.currentTimeMillis();
Log.d(TAGS, "scanning time: " + (end - start) / 1000);
callback.onScanningComplete(results);
} else {
index++;
}
}
}
return null;
}
});
}
public void executeTask(AsyncTask asyncTask) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
asyncTask.executeOnExecutor(AsyncTask.THREAD_POOL_EXECUTOR);
} else {
asyncTask.execute();
}
}
This is the message that I get when I run on android 5.
D/ConnectivityManager.CallbackHandler: CM callback handler got msg 524290
Any help to make this thing work in any way (the first code to work on android 5 or the second code to work faster and in an efficient way ) will be deeply appreciated. I have looked on the issues related to this but I don't want to use print services.
As the index value was the reason why the callback was not sent, so to solve it just sending the callback as soon as you discover the Network Printer and storing it in a var(JSONArray in my case ) will give you the list of the printers and its effective.
I try to save tweets with keywords, I know that free API gives only 7 days of the result, but it never gets any set of a timeline greater than few hours, sometimes it even gives me a range of an hour. I did set since() and until() to the searching query. The maximum number of the tweets I've got from a single run was less than 400. And can anyone tell me why it stopped automatically with such few results? Thanks.
public static void main(String[] args) throws TwitterException {
String KEY_word;
String Exclude;
String Since;
String Until;
String OPT_dir;
String time;
int x;
Propertyloader confg = new Propertyloader();
KEY_word = confg.getProperty("KEY_word");
Exclude = confg.getProperty("Exclude");
Since = confg.getProperty("Since");
Until = confg.getProperty("Until");
OPT_dir = confg.getProperty("OPT_dir");
Twitter twitter = new TwitterFactory().getInstance();
try {
time = new SimpleDateFormat("yyyyMMddHHmm'.txt'").format(new Date());
x = 1;
Query query = new Query(KEY_word + Exclude);
query.since(Since);
query.until(Until);
QueryResult result;
do {
result = twitter.search(query);
List<Status> tweets = result.getTweets();
for (Status tweet : tweets) {
try {
String filedir = OPT_dir + KEY_word + time;
writeStringToFile(filedir, x + ". " + "#" + tweet.getUser().getScreenName() + ", At: " + tweet.getCreatedAt() + ", Rt= " + tweet.getRetweetCount() + ", Text: " + tweet.getText());
x += 1;
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} while ((query = result.nextQuery()) != null);
System.exit(0);
} catch (TwitterException te) {
te.printStackTrace();
System.out.println("Failed to search tweets: " + te.getMessage());
System.exit(-1);
}
}
public static void writeStringToFile(String filePathAndName, String stringToBeWritten) throws IOException{
try
{
String filename= filePathAndName;
boolean append = true;
FileWriter fw = new FileWriter(filename,append);
fw.write(stringToBeWritten);//appends the string to the file
fw.write("\n" +"\n");
fw.close();
}
catch(IOException ioe)
{
System.err.println("IOException: " + ioe.getMessage());
}
}
You can get more tweets by using setMaxId. Here is an example :
long lowestTweetId = Long.MAX_VALUE;
x = 1;
Query query = new Query("stackoverflow");
query.since("2018-08-10");
query.until("2018-08-16");
query.setCount(100); //The number of tweets to return per page, up to a maximum of 100. Defaults to 15. https://developer.twitter.com/en/docs/tweets/search/api-reference/get-search-tweets.html
query.setResultType(Query.RECENT); // to get an order
int searchResultCount=100;
QueryResult result;
do {
result = twitter.search(query);
List<Status> tweets = result.getTweets();
for (Status tweet : tweets) {
try {
System.out.println( "#" + tweet.getUser().getScreenName() + ", At: " + tweet.getCreatedAt() );
x += 1;
if (tweet.getId() < lowestTweetId) {
lowestTweetId = tweet.getId();
query.setMaxId(lowestTweetId-1);
}
else {// each new maxid should be smaller than the other one so break here
//do whatever you want to handle it ex: break from two loops
}
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
} while (searchResultCount != 0 );
I have to pass record to an UDF which calls an API but as we want to do it parallely,we are using spark and thats why UDF is being developed, the problem here is that that UDF needs to take only 100 records at a time not more than that, it can't handle more than 100 records parallely, so how to ensure that only 100 record pass to it in one go please note we don't want to use count() function on whole record.
I am attaching the UDF code here,it's a generic UDF which returns array of struct.moreover if we pass 100 records in batchsize variable each time then,if suppose there are 198 records then if as we dont want to use count() we will not be knowing that its last batchsize is going to be 98.so how to handle that thing.
Guys... I have a generic UDF in which call is made for an API but before calling it creates batch of 100 firstly then only call restapi.. So the argument UDF takes are x1:string, x2:string, batchsize:integer(currently the batchsize is 100)..so in UDF until and unless the batchsize is not 100 the call will not happen.. And for each record it will return null.
So till 99th record it will return. Null but at 100th record the call will happen
[So, now the problem part:as we are taking batchsize 100 and call will take place only at 100th record. So, in condition like if we have suppose 198 record in file then 100 record will get the output but, other 98 will only return null as they will not get processed..
So please help a way around, and UDF take one record at a time, but it keep on collecting till 100th record.. I hope this clears up
public class Standardize_Address extends GenericUDF {
private static final Logger logger = LoggerFactory.getLogger(Standardize_Address.class);
private int counter = 0;
Client client = null;
private Batch batch = new Batch();
public Standardize_Address() {
client = new ClientBuilder().withUrl("https://ss-staging-public.beringmedia.com/street-address").build();
}
// StringObjectInspector streeti;
PrimitiveObjectInspector streeti;
PrimitiveObjectInspector cityi;
PrimitiveObjectInspector zipi;
PrimitiveObjectInspector statei;
PrimitiveObjectInspector batchsizei;
private ArrayList ret;
#Override
public String getDisplayString(String[] argument) {
return "My display string";
}
#Override
public ObjectInspector initialize(ObjectInspector[] args) throws UDFArgumentException {
System.out.println("under initialize");
if (args[0] == null) {
throw new UDFArgumentTypeException(0, "NO Street is mentioned");
}
if (args[1] == null) {
throw new UDFArgumentTypeException(0, "No Zip is mentioned");
}
if (args[2] == null) {
throw new UDFArgumentTypeException(0, "No city is mentioned");
}
if (args[3] == null) {
throw new UDFArgumentTypeException(0, "No State is mentioned");
}
if (args[4] == null) {
throw new UDFArgumentTypeException(0, "No batch size is mentioned");
}
/// streeti =args[0];
streeti = (PrimitiveObjectInspector)args[0];
// this.streetvalue = (StringObjectInspector) streeti;
cityi = (PrimitiveObjectInspector)args[1];
zipi = (PrimitiveObjectInspector)args[2];
statei = (PrimitiveObjectInspector)args[3];
batchsizei = (PrimitiveObjectInspector)args[4];
ret = new ArrayList();
ArrayList structFieldNames = new ArrayList();
ArrayList structFieldObjectInspectors = new ArrayList();
structFieldNames.add("Street");
structFieldNames.add("city");
structFieldNames.add("zip");
structFieldNames.add("state");
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
structFieldObjectInspectors.add(PrimitiveObjectInspectorFactory.writableStringObjectInspector);
StructObjectInspector si2 = ObjectInspectorFactory.getStandardStructObjectInspector(structFieldNames,
structFieldObjectInspectors);
ListObjectInspector li2;
li2 = ObjectInspectorFactory.getStandardListObjectInspector(si2);
return li2;
}
#Override
public Object evaluate(DeferredObject[] args) throws HiveException {
ret.clear();
System.out.println("under evaluate");
// String street1 = streetvalue.getPrimitiveJavaObject(args[0].get());
Object oin = args[4].get();
System.out.println("under typecasting");
int batchsize = (Integer) batchsizei.getPrimitiveJavaObject(oin);
System.out.println("batchsize");
Object oin1 = args[0].get();
String street1 = (String) streeti.getPrimitiveJavaObject(oin1);
Object oin2 = args[1].get();
String zip1 = (String) zipi.getPrimitiveJavaObject(oin2);
Object oin3 = args[2].get();
String city1 = (String) cityi.getPrimitiveJavaObject(oin3);
Object oin4 = args[3].get();
String state1 = (String) statei.getPrimitiveJavaObject(oin4);
logger.info("address passed, street=" + street1 + ",zip=" + zip1 + ",city=" + city1 + ",state=" + state1);
counter++;
try {
System.out.println("under try");
Lookup lookup = new Lookup();
lookup.setStreet(street1);
lookup.setCity(city1);
lookup.setState(state1);
lookup.setZipCode(zip1);
lookup.setMaxCandidates(1);
batch.add(lookup);
} catch (BatchFullException ex) {
logger.error(ex.getMessage(), ex);
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
/* batch.add(lookup); */
if (counter == batchsize) {
System.out.println("under if");
try {
logger.info("batch input street " + batch.get(0).getStreet());
try {
client.send(batch);
} catch (Exception e) {
logger.error(e.getMessage(), e);
logger.warn("skipping current batch, continuing with the next batch");
batch.clear();
counter = 0;
return null;
}
Vector<Lookup> lookups = batch.getAllLookups();
for (int i = 0; i < batch.size(); i++) {
// ListObjectInspector candidates;
ArrayList<Candidate> candidates = lookups.get(i).getResult();
if (candidates.isEmpty()) {
logger.warn("Address " + i + " is invalid.\n");
continue;
}
logger.info("Address " + i + " is valid. (There is at least one candidate)");
for (Candidate candidate : candidates) {
final Components components = candidate.getComponents();
final Metadata metadata = candidate.getMetadata();
logger.info("\nCandidate " + candidate.getCandidateIndex() + ":");
logger.info("Delivery line 1: " + candidate.getDeliveryLine1());
logger.info("Last line: " + candidate.getLastLine());
logger.info("ZIP Code: " + components.getZipCode() + "-" + components.getPlus4Code());
logger.info("County: " + metadata.getCountyName());
logger.info("Latitude: " + metadata.getLatitude());
logger.info("Longitude: " + metadata.getLongitude());
}
Object[] e;
e = new Object[4];
e[0] = new Text(candidates.get(i).getComponents().getStreetName());
e[1] = new Text(candidates.get(i).getComponents().getCityName());
e[2] = new Text(candidates.get(i).getComponents().getZipCode());
e[3] = new Text(candidates.get(i).getComponents().getState());
ret.add(e);
}
counter = 0;
batch.clear();
} catch (Exception e) {
logger.error(e.getMessage(), e);
}
return ret;
} else {
return null;
}
}
}
I am trying to add a scroll function to my code as currently it merely displays the first couple tweets, enough to fill the canvas but no more. How do I go about adding a scroll/click function so the canvas lists other tweets as well?
This is what I have so far with my Twitter API tokens omitted.
import java.util.Date;
import java.util.Arrays;
ArrayList<String> words = new ArrayList();
void setup() {
//Set the size of the stage, and the background to black.
size(750, 750);
background(0);
//Credentials
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey("x");
cb.setOAuthConsumerSecret("x");
cb.setOAuthAccessToken("x");
cb.setOAuthAccessTokenSecret("x");
TwitterFactory twitterFactory;
twitterFactory = new TwitterFactory(cb.build());
Twitter twitter = twitterFactory.getInstance();
Query query = new Query("StackOverFow");
query.count(100);
try {
QueryResult result = twitter.search(query);
ArrayList tweets = (ArrayList) result.getTweets();
for (int i = 0; i < tweets.size (); i++) {
Status t = (Status) tweets.get(i);
User u=(User) t.getUser();
String user=u.getName();
String msg = t.getText();
Date d = t.getCreatedAt();
text("User Name - " + user + " Time - " + d + " Message - " + msg,
20, 15+i*50, width-40, 50);
println("Tweet by " + user + " at " + d + ": " + msg);
//Break the tweet into words
String[] input = msg.split(" ");
println(input);
for (int j = 0; j < input.length; j++) {
//Put each word into the words ArrayList
words.add(input[j]);
}
};
}
catch (TwitterException te) {
println("Couldn't connect: " + te);
};
}
int j=0;
void mousePressed() {
saveData();
}
void saveData() {
String[] data = new String[words.size()];
words.toArray(data);
saveStrings("data/data.txt", data);
}
You already have a handler for mousePressed, adding click and scroll handling is the same concept: add mouseClicked for full click handling, and mouseWheel() for handling scroll events.
I am working on an app which will ping the multiple urls (servers) at regular intervals to ensure whether the server is running fine or it's going down. I have completed the coding part(java program) to achieve this.
Here is the code is :
public class PingServers {
public static boolean pingUrl(String address) {
try {
final URL url = new URL("http://" + address);
final HttpURLConnection urlConn = (HttpURLConnection) url.openConnection();
urlConn.setConnectTimeout(1000 * 100); // Timeout is in milliseconds
final long startTime = System.currentTimeMillis();
urlConn.connect();
final long endTime = System.currentTimeMillis();
if (urlConn.getResponseCode() == HttpURLConnection.HTTP_OK) {
System.out.println("Time (ms) : " + (endTime - startTime));
System.out.println("Ping to " + address + " was success");
return true;
}
} catch (final MalformedURLException e1) {
System.out.println("inside MalformedURLException");
e1.printStackTrace();
} catch (final Exception e) {
System.out.println("inside Exception");
e.printStackTrace();
System.out.println(address + " is either going down or is an unknownhost.");
}
return false;
}
public static void main(String args[]) throws InterruptedException {
//for periodic execution
for (int i = 0; i < 5; i++) {
System.out.println("Execution in Main Thread...." + i);
Date now = new Date(); // initialize date
System.out.println("Time is :" + now); // Display current time
String url1 = "www.google.com";
String url2 = "www.yahoo.com";
String url3 = "www.invalidurl.com";
pingUrl(url1);
pingUrl(url2);
pingUrl(url3);
Thread.sleep(10000);
if (i == 5) {
System.out.println("Application Terminates");
System.exit(0);
}
}
}
}
It's running well in cmd (console). But I need to form it as an App which will be running in taskbar automatically and as soon as the condition is matched (server goes down or unknownhost) it will show the pop-up.
Would anyone please tell me the best way to achieve this(Step by Step) from here onwards.
It would be highly appreciable.