At the moment my program has a Jpanel with about 15 different inputs. These inputs are created to one long string called "searchInput".
"searchInput" is then put into a query named which then gets metadata about the tweet result one at a time, such as createdAt, user and text. These results are printed to a textArea named tweetsResult.
createdAt, user and text are only some of the metadata behind each tweet, more being favouriteCount and retweetCount.
Because Twitter has a maximum amount of Tweets it can deliver to you using the API, I feel it would be best if I created a new class which uses searchInput, and then get the desired amount of Tweets with all of the metadata into an object. And then from the main class, I would only call the results with the specific meta data that I would like.
Is this the right way to do it? If so how would I create this new object, presumably using getters and setters?
I hope this is clear to you, and many thanks for the help!
List<Status> tweets = new ArrayList<Status>();
//Standard Twitter and Twitter4j authentication
ConfigurationBuilder builder = new ConfigurationBuilder();
builder.setOAuthConsumerKey(consumerKey);
builder.setOAuthConsumerSecret(consumerSecret);
Configuration configuration = builder.build();
TwitterFactory twitterFactory = new TwitterFactory(configuration);
Twitter twitter = twitterFactory.getInstance();
twitter.setOAuthAccessToken(new AccessToken(accessToken, accessTokenSecret));
//The query is split up into 5 sections
//All the text boxes are read in the seperate methods
String searchWordInput = getWordSearchQuery();
String searchPeopleInput = getPeopleSearchQuery();
String filtersSearchInput = getFiltersSearchQuery();
String dateAndLocationSearchInput = getDateAndLocationSearchQuery();
String radiusAndLocationSearchInput = getRadiusAndLocationSearchQuery();
String searchInput = searchWordInput + searchPeopleInput + filtersSearchInput + dateAndLocationSearchInput + radiusAndLocationSearchInput;
//One big search String called "searchInput" is taken to be sent to Twitter
//Increases maximum amount of tweets in one search
int wantedTweets = 100; //CHANGE THIS FOR DIFFERENT AMOUNT OF TWEETS
long lastSearchID = Long.MAX_VALUE;
int remainingTweets = wantedTweets;
Query query = new Query(searchInput);
tweetsResultInput.append(searchInput);
try {
while (remainingTweets > 0) {
remainingTweets = wantedTweets - tweets.size();
if (remainingTweets > 100) {
query.count(100);
} else {
query.count(remainingTweets);
}
QueryResult result = twitter.search(query);
tweets.addAll(result.getTweets());
Status s = tweets.get(tweets.size() - 1);
lastSearchID = s.getId();
query.setMaxId(lastSearchID);
remainingTweets = wantedTweets - tweets.size();
}
//Increases maximum amount of tweets in ones search
tweetsResult.append(searchInput);
tweetsResult.append("\n");
//For the query tweets, using "searchInput", get every tweet in the format; when Tweet was sent, who it was sent by, and the Tweets text and write to tweetsResults
for (Status tweet : tweets) {
tweetsResult.append(tweet.getCreatedAt() + ":\t#" + tweet.getUser().getScreenName() + " - " + tweet.getText());
tweetsResult.append("\n");
}
} catch (TwitterException te) {
System.out.println("Failed to search tweets: " + te.getMessage());
System.exit(-1);
}
Related
i am trying to fetch all the documents from a database without knowing the exact url's . I got one query
DocumentPage documents =docMgr.read();
while (documents.hasNext()) {
DocumentRecord document = documents.next();
System.out.println(document.getUri());
}
But i do not have specific urls , i want all the documents
The first step is to enable your uris lexicon on the database.
You could eval some XQuery and run cts:uris() (or server-side JS and run cts.uris()):
ServerEvaluationCall call = client.newServerEval()
.xquery("cts:uris()");
for ( EvalResult result : call.eval() ) {
String uri = result.getString();
System.out.println(uri);
}
Two drawbacks are: (1) you'd need a user with privileges and (2) there is no pagination.
If you have a small number of documents, you don't need pagination. But for a large number of documents pagination is recommended. Here's some code using the search API and pagination:
// do the next eight lines just once
String options =
"<options xmlns='http://marklogic.com/appservices/search'>" +
" <values name='uris'>" +
" <uri/>" +
" </values>" +
"</options>";
QueryOptionsManager optionsMgr = client.newServerConfigManager().newQueryOptionsManager();
optionsMgr.writeOptions("uriOptions", new StringHandle(options));
// run the following each time you need to list all uris
QueryManager queryMgr = client.newQueryManager();
long pageLength = 10000;
queryMgr.setPageLength(pageLength);
ValuesDefinition query = queryMgr.newValuesDefinition("uris", "uriOptions");
// the following "and" query just matches all documents
query.setQueryDefinition(new StructuredQueryBuilder().and());
int start = 1;
boolean hasMore = true;
Transaction transaction = client.openTransaction();
try {
while ( hasMore ) {
CountedDistinctValue[] uriValues =
queryMgr.values(query, new ValuesHandle(), start, transaction).getValues();
for (CountedDistinctValue uriValue : uriValues) {
String uri = uriValue.get("string", String.class);
//System.out.println(uri);
}
start += uriValues.length;
// this is the last page if uriValues is smaller than pageLength
hasMore = uriValues.length == pageLength;
}
} finally {
transaction.commit();
}
The transaction is only necessary if you need a guaranteed "snapshot" list isolated from adds/deletes happening concurrently with this process. Since it adds some overhead, feel free to remove it if you don't need such exactness.
find out the page length and in the queryMgr you can specify the starting point to access. Keep on increasing the starting point and loop through all the URL. I was able to fetch all URI. This could be not so good approach but works.
List<String> uriList = new ArrayList<>();
QueryManager queryMgr = client.newQueryManager();
StructuredQueryBuilder qb = new StructuredQueryBuilder();
StructuredQueryDefinition querydef = qb.and(qb.collection("xxxx"), qb.collection("whatever"), qb.collection("whatever"));//outputs 241152
SearchHandle results = queryMgr.search(querydef, new SearchHandle(), 10);
long pageLength = results.getPageLength();
long totalResults = results.getTotalResults();
System.out.println("Total Reuslts: " + totalResults);
long timesToLoop = totalResults / pageLength;
for (int i = 0; i < timesToLoop; i = (int) (i + pageLength)) {
System.out.println("Printing Results from: " + (i) + " to: " + (i + pageLength));
results = queryMgr.search(querydef, new SearchHandle(), i);
MatchDocumentSummary[] summaries = results.getMatchResults();//10 results because page length is 10
for (MatchDocumentSummary summary : summaries) {
// System.out.println("Extracted friom URI-> " + summary.getUri());
uriList.add(summary.getUri());
}
if (i >= 1000) {//number of URI to store/retreive. plus 10
break;
}
}
uriList= uriList.stream().distinct().collect(Collectors.toList());
return uriList;
I tried to get the twitter followers using the screen name. But i am able to get only my followers screen names where as i am expecting the followers of my followers. But i didn't found any supported method for this.
My code is as follows.
TwitterFactory factory = new TwitterFactory();
Twitter twitter = factory.getInstance();
twitter.setOAuthConsumer(consumerKey, consumerSecret);
AccessToken accessToken = new AccessToken(twitterToken, twitterSecret);
twitter.setOAuthAccessToken(accessToken);
String twitterScreenName = twitter.getScreenName();
IDs followerIDs = twitter.getFollowersIDs(twitterScreenName, -1);
long[] ids = followerIDs.getIDs();
for (long id : ids) {
twitter4j.User user = twitter.showUser(id);
//here i am trying to fetch the followers of each id
System.out.println("Name: " + user.getScreenName());
System.out.println("Location:" + user.getLocation());
}
Can anyone please help me in this.
You will need to do the nesting over here. You are just getting the list of current users followers. But you need to get the list of followers of your followers.
Sample code is as below:
TwitterFactory factory = new TwitterFactory();
Twitter twitter = factory.getInstance();
String twitterScreenName;
try {
twitterScreenName = twitter.getScreenName();
IDs followerIDs = twitter.getFollowersIDs(twitterScreenName, -1);
long[] ids = followerIDs.getIDs();
for (long id : ids) {
twitter4j.User user = twitter.showUser(id);
//here i am trying to fetch the followers of each id
String userScreenName = user.getScreenName();
System.out.println("Name: " + user.getScreenName());
System.out.println("Location:" + user.getLocation());
IDs followerIDsOfFollowers = twitter.getFollowersIDs(user.getScreenName(), -1);
long[]fofIDs = followerIDsOfFollowers.getIDs();
for(long subId : fofIDs) {
twitter4j.User user1 = twitter.showUser(subId);
System.out.println("Follower Master:" + userScreenName +" Follower of Follower Name: " + user1.getScreenName());
System.out.println("Location:" + user1.getLocation());
}
I am able to get the iterations under the project object. Now how do I get the iteration I need under that project and then drill down to the stories in that iteration using the JAVA toolkit?
https://sandbox.rallydev.com/slm/webservice/v2.0/project/7191194697/iterations
Given a project:
String projectRef = "/project/1234";
You may scope your requests as follows:
iterationRequest.setProject(projectRef);
or
storyRequest.setProject(projectRef);
If you scoped a story request to a project, then you may query on stories by traversing Iteration.Name if you know the iteration already:
storyRequest.setQueryFilter(new QueryFilter("Iteration.Name", "=", "my Iteration 1"));
Here is a more complex example that returns stories assigned to iterations that fall within the timbox of a specific release. If, for example, you have 4 iterations per release, this code will return stories assigned to all four iterations.
If you code against the sandbox, replace the value in the host variable accordingly.
public class FindIterationsByReleaseDateAndStories {
public static void main(String[] args) throws URISyntaxException, IOException {
String host = "https://rally1.rallydev.com";
String username = "user#co.com";
String password = "secret";
String projectRef = "/project/12352608219";
String applicationName = "Find Iterations by Release Dates and Stories";
RallyRestApi restApi = null;
try {
restApi = new RallyRestApi(
new URI(host),
username,
password);
restApi.setApplicationName(applicationName);
System.out.println(restApi.getWsapiVersion());
QueryRequest releaseRequest = new QueryRequest("Release");
releaseRequest.setFetch(new Fetch("ReleaseStartDate", "ReleaseDate"));
releaseRequest.setScopedDown(false);
releaseRequest.setScopedUp(false);
releaseRequest.setProject(projectRef);
releaseRequest.setQueryFilter(new QueryFilter("Name", "=", "r1"));
QueryResponse releaseQueryResponse = restApi.query(releaseRequest);
int numberOfReleasesInProject = releaseQueryResponse.getTotalResultCount();
System.out.println(numberOfReleasesInProject);
JsonObject releaseJsonObject = releaseQueryResponse.getResults().get(0).getAsJsonObject();
System.out.println(releaseJsonObject.get("ReleaseStartDate"));
System.out.println(releaseJsonObject.get("ReleaseDate"));
String rsd = releaseJsonObject.get("ReleaseStartDate").getAsString();
String rd = releaseJsonObject.get("ReleaseDate").getAsString();
QueryRequest iterationRequest = new QueryRequest("Iteration");
iterationRequest.setFetch(new Fetch("Name","StartDate","EndDate"));
iterationRequest.setScopedDown(false);
iterationRequest.setScopedUp(false);
iterationRequest.setProject(projectRef);
iterationRequest.setQueryFilter(new QueryFilter("StartDate", ">=", rsd).and(new QueryFilter("EndDate", "<=", rd)));
QueryResponse iterationQueryResponse = restApi.query(iterationRequest);
int numberOfIteraitons = iterationQueryResponse.getTotalResultCount();
System.out.println("numberOfIteraitons " + numberOfIteraitons);
if(numberOfIteraitons >0){
for (int i=0;i<numberOfIteraitons;i++){
JsonObject iterationJsonObject = iterationQueryResponse.getResults().get(i).getAsJsonObject();
String iterationName = iterationJsonObject.get("Name").getAsString();
System.out.println("iteration: " + iterationName);
QueryRequest storyRequest = new QueryRequest("HierarchicalRequirement");
storyRequest.setProject(projectRef);
storyRequest.setFetch(new Fetch(new String[] {"Name", "FormattedID","ScheduleState"}));
storyRequest.setLimit(1000);
storyRequest.setScopedDown(false);
storyRequest.setScopedUp(false);
storyRequest.setQueryFilter(new QueryFilter("Iteration.Name", "=", iterationName));
QueryResponse storyQueryResponse = restApi.query(storyRequest);
System.out.println("Number of stories in " + iterationName + " :" + storyQueryResponse.getTotalResultCount());
for (int j=0; j<storyQueryResponse.getResults().size();j++){
JsonObject storyJsonObject = storyQueryResponse.getResults().get(j).getAsJsonObject();
System.out.println("Name: " + storyJsonObject.get("Name") + " FormattedID: " + storyJsonObject.get("FormattedID") + " ScheduleState: " + storyJsonObject.get("ScheduleState"));
}
}
}
}
finally{
if (restApi != null) {
restApi.close();
}
}
}
}
UPDATE: as far as your question in the comment, the code above is equivalent of
https://rally1.rallydev.com/slm/webservice/v2.0/hierarchicalrequirement?query=((Iteration.Name = i1) AND (Project = /project/12352608219))
There are other ways to achive the same result. Iteration name may not be unique, hence the second condition by project ref. In the code the request's project is set first, that's why the query itself uses one condition, but effectively there are two. If you know your iteration's ref, or ObjectID then the same result will be returned from (Iteration = /iteration/123456789), and there is no need to filter by project since a reference or ObjectID are unique.
WS API doc is interactive.Test your queries in WS API and copy the resulting query URLs from the address bar if you want to see how queries are formed:
-Query in the context of the intended object: click on the work item type in the Object Model, e.g. Defect or HierarchicalRequirement before typing your query in the query box.
-Enter a query in a box, e.g (Iteration.Name = i1)
-Click on Query button
-Results are displayed in the window from which you can copy query URL from address bar of your browser.
I am trying to get all the followersIDs from an a twitter account with about 150.000 followers. I later want to map their location, but first I need all those IDs.
at the moment I am using this code:
long lCursorIDs = -1;
long[] fArray = new long[100];
do
{
fArray = twitter.getFollowersIDs(name, lCursorIDs).getIDs();
} while (twitter.getFollowersIDs(name, lCursorIDs).hasNext ());
try
{
PrintWriter pr = new PrintWriter(filenameOutput);
for (int i=0; i<fArray.length ; i++)
{
pr.println(fArray[i]);
}
pr.close();
System.out.println("Follower IDs collected and saved to file: " + filenameOutput );
}
catch (Exception e)
{
e.printStackTrace();
System.out.println("No such file exists.");
}
This works for User with less followers. but with that many it always returns an error message - rate limit exceeded.
I was thinking about getting only a certain number of followersIDs per hour, but I am not sure how to do that and not start every hour from the beginning with the first follower. also, I am not sure how many followers I can get with one request. maybe it is 100, as with the "lookupUser" method but I am not sure.. any ideas/suggestions?
EDIT: ok, I just tried to get the followerIDs of an account with 2700 followers and it stored them correctly in the text file. It also only "cost" one request. than I changed the account name to an account with 15500 followers and it crashes again with an rate limit exceeded message. I don´t get why since it´s only roughly 6 times as many followers but all the remaining requests get spend.. any ideas on what I´m doing wrong?
the answer:
int numberOfFollowers;
numberOfFollowers = user.getFollowersCount();
//CREATE ARRAYS FOR FOLLOWER IDS
long cursor = -1;
long[] fArray = new long[numberOfFollowers];
long[] local = new long[5000];
IDs ids = twitter.getFollowersIDs(name, cursor);
int j = 0;
int x = 5000;
int durchgang = 1;
int d_anzahl = 1 + numberOfFollowers / 5000;
//STROE FOLLOWER IDS IN ARRAYS
do
{
ids = twitter.getFollowersIDs(name, cursor);
local = twitter.getFollowersIDs(name, cursor).getIDs();
System.out.println("Durchgang: " + durchgang + " / " + d_anzahl );
System.arraycopy(local, 0, fArray, j * x , local.length);
j++;
durchgang++;
cursor = ids.getNextCursor();
} while (ids.hasNext());
this gets an array with all follower IDs of any twitter User. It calculates the number of loops needed to get all follower IDs and copys each array of 5000 IDs into new array which has all IDs at the end.
I'm trying to get a list of recent statuses from each user on a persons list of followers. I've got the following to get the users...
IDs list = twitter.getFriendsIDs(0);
for(long ID : list.getIDs()){
twitter4j.User TW_user = twitter.showUser(ID);
}
All I can get from this is getStatus() which is their most recent status. getHomeTimeline() is also insufficient as I need a list of recent tweets from each user. Is there anyway I can achieve this using Twitter4J?
I was just trying to find this answer myself. I had decent success using the getUserTimeline method. Looks like you're trying to look up a list of friend IDs, so this method below should take the long[] and spit out all the user statuses. lookupUsers also accepts a String[] of screen names if you want to look users up that way instead.
public static void lookupUsers(long[] usersList) {
try {
Twitter twitter = new TwitterFactory().getInstance();
ResponseList<User> users = twitter.lookupUsers(usersList);
Paging paging = new Paging(1, 100);
List<Status> statuses;
for (User user : users) {
statuses = twitter.getUserTimeline(user.getScreenName(), paging);
System.out.println("\nUser: #" + user.getScreenName());
for (Status s : statuses) {
System.out.println(s.getText());
}
}
} catch (TwitterException e) {
e.printStackTrace();
}
}
Alex's answer is close, but will only get you 100 tweets per user. The following will get you all (or at least the API's max limit):
IDs list = twitter.getFriendsIDs(0);
for(long ID : list.getIDs()) {
Status[] tweets = getAllTweets(twitter, ID);
System.out.println(ID + ": " + tweets.length);
}
Status[] getAllTweets(Twitter twitter, long userId)
{
int pageno = 1;
List statuses = new ArrayList();
while (true)
{
try
{
int size = statuses.size();
Paging page = new Paging(pageno++, 100);
statuses.addAll(twitter.getUserTimeline(userId, page));
if (statuses.size() == size)
break;
}
catch (TwitterException e)
{
e.printStackTrace();
}
}
return (Status[]) statuses.toArray(new Status[0]);
}