I'm trying to get a list of recent statuses from each user on a persons list of followers. I've got the following to get the users...
IDs list = twitter.getFriendsIDs(0);
for(long ID : list.getIDs()){
twitter4j.User TW_user = twitter.showUser(ID);
}
All I can get from this is getStatus() which is their most recent status. getHomeTimeline() is also insufficient as I need a list of recent tweets from each user. Is there anyway I can achieve this using Twitter4J?
I was just trying to find this answer myself. I had decent success using the getUserTimeline method. Looks like you're trying to look up a list of friend IDs, so this method below should take the long[] and spit out all the user statuses. lookupUsers also accepts a String[] of screen names if you want to look users up that way instead.
public static void lookupUsers(long[] usersList) {
try {
Twitter twitter = new TwitterFactory().getInstance();
ResponseList<User> users = twitter.lookupUsers(usersList);
Paging paging = new Paging(1, 100);
List<Status> statuses;
for (User user : users) {
statuses = twitter.getUserTimeline(user.getScreenName(), paging);
System.out.println("\nUser: #" + user.getScreenName());
for (Status s : statuses) {
System.out.println(s.getText());
}
}
} catch (TwitterException e) {
e.printStackTrace();
}
}
Alex's answer is close, but will only get you 100 tweets per user. The following will get you all (or at least the API's max limit):
IDs list = twitter.getFriendsIDs(0);
for(long ID : list.getIDs()) {
Status[] tweets = getAllTweets(twitter, ID);
System.out.println(ID + ": " + tweets.length);
}
Status[] getAllTweets(Twitter twitter, long userId)
{
int pageno = 1;
List statuses = new ArrayList();
while (true)
{
try
{
int size = statuses.size();
Paging page = new Paging(pageno++, 100);
statuses.addAll(twitter.getUserTimeline(userId, page));
if (statuses.size() == size)
break;
}
catch (TwitterException e)
{
e.printStackTrace();
}
}
return (Status[]) statuses.toArray(new Status[0]);
}
Related
I have been wondering if there is a way to access all the twitter followers list.
We have tried using call to the REST API via twitter4j:
public List<User> getFriendList() {
List<User> friendList = null;
try {
friendList = mTwitter.getFollowersList(mTwitter.getId(), -1);
} catch (IllegalStateException e) {
e.printStackTrace();
} catch (TwitterException e) {
e.printStackTrace();
}
return friendList;
}
But it returns only a list of 20 followers.
I tried using the same call in loop, but it cause a rate limit exception - says we are not allowed to make too many requests in a small interval of time.
Do we have a way around this?
You should definitely use getFollowersIDs. As the documentation says, this returns an array (list) of IDs objects. Note that it causes the list to be broken into pages of around 5000 IDs at a time. To begin paging provide a value of -1 as the cursor. The response from the API will include a previous_cursor and next_cursor to allow paging back and forth.
The tricky part is to handle the cursor. If you can do this, then you will not have the problem of getting only 20 followers.
The first call to getFollowersIDs will need to be given a cursor of -1. For subsequent calls, you need to update the cursor value, by getting the next cursor, as done in the while part of the loop.
long cursor =-1L;
IDs ids;
do {
ids = twitter.getFollowersIDs(cursor);
for(long userID : ids.getIDs()){
friendList.add(userID);
}
} while((cursor = ids.getNextCursor())!=0 );
Here is a very good reference:
https://github.com/yusuke/twitter4j/blob/master/twitter4j-examples/src/main/java/twitter4j/examples/friendsandfollowers/GetFriendsIDs.java
Now, if the user has more than around 75000 followers, you will have to do some waiting (see Vishal's answer).
The first 15 calls will yield you around 75000 IDs. Then you will have to sleep for 15 minutes. Then make another 15 calls, and so on till you get all the followers. This can be done using a simple Thread.sleep(time_in_milliseconds) outside the for loop.
Just Change like this and try, this is working for me
try {
Log.i("act twitter...........", "ModifiedCustomTabBarActivity.class");
// final JSONArray twitterFriendsIDsJsonArray = new JSONArray();
IDs ids = mTwitter.mTwitter.getFriendsIDs(-1);// ids
// for (long id : ids.getIDs()) {
do {
for (long id : ids.getIDs()) {
String ID = "followers ID #" + id;
String[] firstname = ID.split("#");
String first_Name = firstname[0];
String Id = firstname[1];
Log.i("split...........", first_Name + Id);
String Name = mTwitter.mTwitter.showUser(id).getName();
String screenname = mTwitter.mTwitter.showUser(id).getScreenName();
// Log.i("id.......", "followers ID #" + id);
// Log.i("Name..", mTwitter.mTwitter.showUser(id).getName());
// Log.i("Screen_Name...", mTwitter.mTwitter.showUser(id).getScreenName());
// Log.i("image...", mTwitter.mTwitter.showUser(id).getProfileImageURL());
}
} while (ids.hasNext());
} catch (Exception e) {
e.printStackTrace();
}
Try This...
ConfigurationBuilder confbuilder = new ConfigurationBuilder();
confbuilder.setOAuthAccessToken(accessToken)
.setOAuthAccessTokenSecret(secretToken)
.setOAuthConsumerKey(TwitterOAuthActivity.CONSUMER_KEY)
.setOAuthConsumerSecret(TwitterOAuthActivity.CONSUMER_SECRET);
Twitter twitter = new TwitterFactory(confbuilder.build()).getInstance();
PagableResponseList<User> followersList;
ArrayList<String> list = new ArrayList<String>();
try
{
followersList = twitter.getFollowersList(screenName, cursor);
for (int i = 0; i < followersList.size(); i++)
{
User user = followersList.get(i);
String name = user.getName();
list.add(name);
System.out.println("Name" + i + ":" + name);
}
listView.setAdapter(new ArrayAdapter<String>(this, android.R.layout.simple_list_item_1 , list));
listView.setVisibility(View.VISIBLE);
friend_list.setVisibility(View.INVISIBLE);
post_feeds.setVisibility(View.INVISIBLE);
twit.setVisibility(View.INVISIBLE);
}
This is a tricky one.
You should specify whether you're using application or per user tokens and the number of users you're fetching followers_ids for.
You get just 15 calls per 15 minutes in case of an application token. You can fetch a maximum of 5000 followers_ids per call. That gives you a maximum of 75K followers_ids per 15 minutes.
If any of the users you're fetching followers_ids for has over 75K followers, you'll get the rate_limit error immediately. If you're fetching for more than 1 user, you'll need to build strong rate_limit handling in your code with sleeps and be very patient.
The same applies for friends_ids.
I've not had to deal with fetching more than 75K followers/friends for a given user but come to think of it, I don't know if it's even possible anymore.
I am trying to get my followers and following list via Twitter 4J.
I pass the configuration builder from the main method as a parameter to the method.
Here is the code:
try {
Twitter twitter = new TwitterFactory().getInstance();
long cursor = -1;
IDs ids;
System.out.println("Listing following ids.");
do {
if (0 < args.length) {
ids = twitter.getFriendsIDs(args[0], cursor);
} else {
ids = twitter.getFriendsIDs(cursor);
}
for (long id : ids.getIDs()) {
System.out.println(id);
}
} while ((cursor = ids.getNextCursor()) != 0);
System.exit(0);
} catch (TwitterException te) {
te.printStackTrace();
System.out.println("Failed to get friends' ids: " + te.getMessage());
System.exit(-1);
}
So far this only returns IDS but it does not return the screen name of the followers. Does anyone have any idea? I am working in Java and printing to the terminal.
Thanks.
String twitterScreenName = twitter.getScreenName();
PagableResponseList<User> statuse = twitter.getFollowersList(twitterScreenName, -1);
for (User follower : statuse) {
System.out.println(follower.getName());
}
I´m developing an analyzing program for Twitter Data.
I´m using mongoDB and at the moment. I try to write a Java program to get tweets from the Twitter API and put them in the database.
Getting the Tweets already works very well, but I have a problem when I want to put them in the database. As the Twitter API often returns just the same Tweets, I have to place some kind of index in the database.
First of all, I connect to the database and get the collection related to the search-term, or create this collection if this doesn´t exist.
public void connectdb(String keyword)
{
try {
// on constructor load initialize MongoDB and load collection
initMongoDB();
items = db.getCollection(keyword);
BasicDBObject index = new BasicDBObject("tweet_ID", 1);
items.ensureIndex(index);
} catch (MongoException ex) {
System.out.println("MongoException :" + ex.getMessage());
}
}
Then I get the tweets and put them in the database:
public void getTweetByQuery(boolean loadRecords, String keyword) {
if (cb != null) {
TwitterFactory tf = new TwitterFactory(cb.build());
Twitter twitter = tf.getInstance();
try {
Query query = new Query(keyword);
query.setCount(50);
QueryResult result;
result = twitter.search(query);
System.out.println("Getting Tweets...");
List<Status> tweets = result.getTweets();
for (Status tweet : tweets) {
BasicDBObject basicObj = new BasicDBObject();
basicObj.put("user_name", tweet.getUser().getScreenName());
basicObj.put("retweet_count", tweet.getRetweetCount());
basicObj.put("tweet_followers_count", tweet.getUser().getFollowersCount());
UserMentionEntity[] mentioned = tweet.getUserMentionEntities();
basicObj.put("tweet_mentioned_count", mentioned.length);
basicObj.put("tweet_ID", tweet.getId());
basicObj.put("tweet_text", tweet.getText());
if (mentioned.length > 0) {
// System.out.println("Mentioned length " + mentioned.length + " Mentioned: " + mentioned[0].getName());
}
try {
items.insert(basicObj);
} catch (Exception e) {
System.out.println("MongoDB Connection Error : " + e.getMessage());
loadMenu();
}
}
// Printing fetched records from DB.
if (loadRecords) {
getTweetsRecords();
}
} catch (TwitterException te) {
System.out.println("te.getErrorCode() " + te.getErrorCode());
System.out.println("te.getExceptionCode() " + te.getExceptionCode());
System.out.println("te.getStatusCode() " + te.getStatusCode());
if (te.getStatusCode() == 401) {
System.out.println("Twitter Error : \nAuthentication credentials (https://dev.twitter.com/pages/auth) were missing or incorrect.\nEnsure that you have set valid consumer key/secret, access token/secret, and the system clock is in sync.");
} else {
System.out.println("Twitter Error : " + te.getMessage());
}
loadMenu();
}
} else {
System.out.println("MongoDB is not Connected! Please check mongoDB intance running..");
}
}
But as I mentioned before, there are often the same tweets, and they have duplicates in the database.
I think the tweet_ID field is a good field for an index and should be unique in the collection.
Set the unique option on your index to have MongoDb enforce uniqueness:
items.ensureIndex(index, new BasicDBObject("unique", true));
Note that you'll need to manually drop the existing index and remove all duplicates or you won't be able to create the unique index.
This question is already answered but I would like to contribute a bit since MongoDB API 2.11 offers a method which receives unique option as a parameter:
public void ensureIndex(DBObject keys, String name, boolean unique)
A minor remind to someone who would like to store json documents on MongoDBNote is that uniqueness must be applied to a BasicObject key and not over values. For example:
BasicDBObject basicObj = new BasicDBObject();
basicObj.put("user_name", tweet.getUser().getScreenName());
basicObj.put("retweet_count", tweet.getRetweetCount());
basicObj.put("tweet_ID", tweet.getId());
basicObj.put("tweet_text", tweet.getText());
basicObj.put("a_json_text", "{"info_details":{"info_id":"1234"},"info_date":{"year":"2012"}, {"month":"12"}, {"day":"10"}}");
On this case, you can create unique index only to basic object keys:
BasicDBObject index = new BasicDBObject();
int directionOrder = 1;
index.put("tweet_ID", directionOrder);
boolean isUnique = true;
items.ensureIndex(index, "unique_tweet_ID", isUnique);
Any index regarding JSON value like "info_id" would not work since it´s not a BasicObject key.
Using indexes on MongDB is not as easy as it sounds. You may also check MongoDB docs for more details here Mongo Indexing Tutorials and Mongo Index Concepts. Direction order might be pretty important to understand once you need a composed index which is well explained here Why Direction order matter.
How can I obtain the friends list of a friend or follower using Twitter4J?
Using getFriendsId(), I'm only able to retrieve the friend's/follower's list of that current user which is authenticated. What I want is to obtain the list of friends of a follower or friend of the authenticated user.
This will show the name of your friend's followers.
User u1 = null ;
long cursor = -1;
IDs ids;
System.out.println("Listing followers's ids.");
do {
ids = twitter.getFollowersIDs("username", cursor);
for (long id : ids.getIDs()) {
System.out.println(id);
User user = twitter.showUser(id);
System.out.println(user.getName());
}
} while ((cursor = ids.getNextCursor()) != 0);
You only need to do this:
Twitter twitter = mTwitterApp.getTwitterInstance();
long cursor = -1;
List<User> users=twitter.getFriendsList(mTwitterApp.getUserID(), cursor);
Here users is a list users who are your friends(you are following them).
mTwitterApp.getUserID() is your login useris which is a long value.
long lCursor = -1;
IDs friendsIDs = twitter.getFriendsIDs(userID, lCursor);
System.out.println(twitter.showUser(userID).getName());
System.out.println("==========================");
do
{
for (long i : friendsIDs.getIDs())
{
System.out.println("follower ID #" + i);
System.out.println(twitter.showUser(i).getName());
}
}while(friendsIDs.hasNext());
This code works! (without exceeding rate limits). Referred twitter4j documentation and other answers on StackOverflow.
try {
// get friends
long cursor = -1;
PagableResponseList<User> pagableFollowings;
do {
pagableFollowings = twitter.getFriendsList(twitter.getId(), cursor);
for (User user : pagableFollowings) {
listFriends.add(user); // ArrayList<User>
}
} while ((cursor = pagableFollowings.getNextCursor()) != 0);
// get followers
cursor = -1;
PagableResponseList<User> pagableFollowers;
do {
pagableFollowers = twitter.getFollowersList(twitter.getId(), cursor);
for (User user : pagableFollowers) {
listFollowers.add(user); // ArrayList<User>
}
} while ((cursor = pagableFollowers.getNextCursor()) != 0);
} catch (TwitterException e) {
printError(e);
}
You can use
twitter.getFollowersIDs("username", cursor);
http://twitter4j.org/javadoc/twitter4j/api/FriendsFollowersResources.html#getFollowersIDs-java.lang.String-long-
which returns only 5000 user not all users. Also it is limited 15 times in 15 minutes.(https://dev.twitter.com/rest/reference/get/friends/ids)
Also, you can use,
twitter.getFollowersList("username", cursor);
http://twitter4j.org/javadoc/twitter4j/api/FriendsFollowersResources.html#getFollowersList-java.lang.String-long- which is also limited with 20 user. Also it is limited 15 times in 15 minutes for user auth, 30 times in 15 minutes for app auth (https://dev.twitter.com/rest/reference/get/friends/list)
For unlimited access, you can look at https://gnip.com/ or whitelisted user access of twitter.
PagableResponseList<User> friendlist= twitter.getFriendsList(user.getScreenName(), -1);
int sizeoffreindlist= friendlist.size();
for(int i=0;i<sizeoffreindlist;i++)
{
System.out.println(friendlist.get(i));
}
It will provide you a list of 20 friends as the default limit is 20
What about something like get Friends List? https://dev.twitter.com/docs/api/1.1/get/friends/list
According to the docs:
Returns a cursored collection of user objects for every user the
specified user is following (otherwise known as their "friends").
There is an interface for this in twitter4j.api, but I can't figure out how to use it:
PagableResponseList<User> getFriendsList(String screenName, long cursor) throws TwitterException;
I have used to search a tweets using search method with passing a keyword in twitter4j. This is my code
String query = "Cricket";
Query searchQuery = new Query(query);
try {
QueryResult queryResult = twitter.search(searchQuery);
} catch (TwitterException e) {
log.error("Unable to search query = {}", query, e);
}
When i test this code, its showing only top 20 tweets. But i need to search a tweets in recursive fashion. So, how can i search a tweets in real time using a recursive fashion?
public static void main(String[] args) throws TwitterException {
Twitter twitter = new TwitterFactory().getInstance();
Query query = new Query("**your query**");
query.setRpp(100); // here you show 100 tweets
QueryResult result = twitter.search(query);
for (Tweet tweet : result.getTweets()) {
System.out.println(tweet.getFromUser() + ":" + tweet.getText());
}
}
Why do you want to use recursive function?