I'm trying to retrieve data out of an api calls with different tokens (credentials), each token represent a different division.
The api has a limit of row per page which is why is paginated. So far my job extract the data out of the API looping through the pages until record count is 0, but is only working for 1 token and I have 6 different tokens, so if I add another token it should retrieve all records for token 1 and then continue to token 2, etc...
So the only thing remaining is loop through each token but making sure it retrieve all records for each token.
Below is my job:
Tjava_1 code:
globalMap.put("page",new Integer("1"));
globalMap.put("enddata", new Boolean(true));
Tloop_1:
tFixedFlowInput_1, here is where I put all different tokens but like I explained above if I use 1 it works perfect but if I put more than 1 it doesn't work.:
tJavaRow_1:
globalMap.put("Token", new String(row1.Token));
globalMap.put("Division", new String(row1.Division));
System.out.println("Token: " + globalMap.get("Token"));
System.out.println("Division: " + globalMap.get("Division"));
tFlowToIterate_2:
tREST_2:
tExtractJSONFields_6:
tJavaRow_6:
System.out.println(row21.count);
globalMap.put("page",(Integer)globalMap.get("page")+1);
if(row21.count == null || row21.count == 0)
{
globalMap.put("enddata", new Boolean(false));
}
This is the result I get when I use 1 token in the tFixedFlowInput_1, which works as expected.:
This is the result if I add 2 or more tokens, which is what I want to resolve. Instead of doing this it should loop all pages for token 1 until there are no records and then continue with token 2 and so on...
Any help will be greatly appreciated!!
Related
In the firestore security settings you can set conditions for writing/reading data.
Currently I have this piece:
service cloud.firestore {
match /databases/{database}/documents {
match /{document=**} {
allow read: if request.auth != null && request.time <
resource.data.timeCreated + duration.value(1, 'h');
allow write: if request.auth != null;
}
}
}
Now I want to limit the writing; a user should only be able to send data every 5 minutes. How can I achieve this?
There is not a native way to do this.
You cannot for non-signed in users in any way that is not trivially bypassed. You could use Cloud Functions to achieve this for signed in users though.
Each user has a profile with next time they can write, along with the id of the document to write next.
Use Rules on writes to check that the id doesn't already exist and it's >= the allowed time
Use Cloud Functions on write to update the user profile document with a new allowed time and unique id for the next write
I am working on a java application that uses the jInstagram api. I can successfully login with my application, but when I want to get a list of users that follow a certain user, I can only gather 50 user Id's.
The code I am using to get 50 users that follow a user is this:
String userId = instagram.getCurrentUserInfo().getData().getId();
UserFeed feed = instagram.getUserFollowedByList(userId);
List<UserFeedData> users = feed.getUserList();
//iterate through the list and print each value. The print value is simply a user id.
I can then iterate through the list and print out 50 user Id's. This is fine, but I need to get a lot more user Id's.
From my research, in order to get more than 50 user Id's I must use the Pagination class. Here is what I put together.
String userId = instagram.getCurrentUserInfo().getData().getId();
UserFeed feed = instagram.getUserFollowList(userId);
UserFeed recentData = instagram.getUserFollowedByListNextPage(feed.getPagination());
int counter = 0;
while(recentData.getPagination() != null && counter < 10){
List<UserFeedData> a = recentData.getUserList();
for(int i = 0; i < a.size(); i++){
System.out.println(a.get(i));
}
counter++;
}
This Code works, but it gives an output like this for each user.
UserFeedData [id=316470004, profilePictureUrl=https://instagramimages-a.akamaihd.net/profiles/profile_316470004_75sq_1386826158.jpg, userName=thebeautifulwarrior, fullName=Dee Shower, website=http://www.totallifechanges.com/dns2015, bio=2015 the year to Reinvent & Restore! Lose 10 pounds in 10 days!!! Ask me how. Invest in yourself. E-mail: totallifechange2015#gmail.com]
For my program, I only want the id part. I know I can just parse the text and create a substring, but I want to do this more efficiently and retrieve the data from the api call instead. In the first snippet of code, that gives an output exactly how I need it. For example, the output for the code is "316470004" rather than the entire user information set.
Thanks for your help in advance!
What you're seeing is the string representation of the entire UserFeedData object, rather than just the ID. Instead of System.out.println(a.get(i)); use System.out.println(a.get(i).getId()); to pull the user's ID.
I am using DynamoDBMapper for a class, let's say "User" (username being the primary key) which has a field on it which says "Status". It is a Hash+Range key table, and everytime a user's status changes (changes are extremely infrequent), we add a new entry to the table alongwith the timestamp (which is the range key). To fetch the current status, this is what I am doing:
DynamoDBQueryExpression expr =
new DynamoDBQueryExpression(new AttributeValue().withS(userName))
.withScanIndexForward(false).withLimit(1);
PaginatedQueryList<User> result =
this.getMapper().query(User.class, expr);
if(result == null || result.size() == 0) {
return null;
}
for(final User user : result) {
System.out.println(user.getStatus());
}
This for some reason, is printing all the statuses a user has had till now. I have set scanIndexForward to false so that it is in descending order and I put limit of 1. I am expecting this to return the latest single entry in the table for that username.
However, when I even look into the wire logs of the same, I see a huge amount of entries being returned, much more than 1. For now, I am using:
final String currentStatus = result.get(0).getStatus();
What I am trying to understand here is, what is whole point of the withLimit clause in this case, or am I doing something wrong?
In March 2013 on the AWS forums a user complained about the same problem.
A representative from Amazon sent him to use the queryPage function.
It seems as if the limit is not preserved for elements but rather a limit on chunk of elements retrieved in a single API call, and the queryPage might help.
You could also look into the pagination loading strategy configuration
Also, you can always open a Github issue for the team.
I'm developing an application in which I pick rcode(numbers) from the database with the name. If the name is present it will display the rcode associated with it. The output like
10001
Suppose, if two names are present with the same name, it will display like
10001 10002
Here is my servlet code for this..
ResultSet rs=stmt.executeQuery("select * from newfarmer where rname='"+get+"'");
while(rs.next()){
username=rs.getString("rname");
if(get.equals(username)){
rcode=rs.getString("rcode");
out.println(rcode);
}
}
and,
I'm using the following code in my android application to get the data from it. That is I need to pick all the data something like where rname=" ". Now the problem is if two records with the same name are present, It will pickup only one record. But servlet outputs show as
10001 10002
In my application I count the total number of records and display it. But it shows only one
String line = null;
while ((line = reader.readLine()) != null) {
StringTokenizer st=new StringTokenizer(line);
count=st.countTokens();
Toast.makeText(getBaseContext(), count+" names found",
Toast.LENGTH_SHORT).show();
r_code=st.nextToken();
ff=1;
}
So, I need to access both the records and also count the total no.of records. Can anyone help to do this
out.println(rcode) adds a CRLF (which is ignored when you look at the output with a browser). What happens if you replace it with a space character ?
out.print(rcode);
out.print(" ");
use list to display the records and add one counter....,for list refer http://www.vogella.com/articles/AndroidSQLite/article.html
i am not able to understand why you are writting that inside if statement. you need not to write that statement as username will always be equal to get. remove that statement and try to run it again.
Here is what i'm trying to do :
I have a list a twitter user ID, for each one of them I need to retrieve a complete list of his followers ID and his friends ID. I don't need anything else, no screen name etc..
i'm using twitter4j btw
Here is how I'm doing it :
for each user i'm executing the following code in order to get a complete list of his followers IDs
long lCursor = -1
do{
IDs response = t.getFollowersIDs(id, lCursor);
long tab[] = response.getIDs();
for(long val : tab){
myIdList.add(val);
}
lCursor = response.getNextCursor();
}while(lCursor != 0);
My problem :
according to this page : https://dev.twitter.com/docs/api/1.1/get/followers/ids
the request rate limit for getFollowersIDs() is 15, considering this method return a maximum number of 5000 IDs, it means that it will be only possible to get 15*5000 IDs (or 15 users if they have less than 5000 followers).
This is really not enough for what i'm trying to do.
Am I doing something wrong ? Is there any solutions to improve that ? (even slightly)
Thanks for your help :)
The rate limit for that endpoint in v1.1 is 15 calls per 15 minutes per access token. See https://dev.twitter.com/docs/rate-limiting/1.1 for more information about the limits.
With that in mind, if you have an access token for each of your users, you should be able to fetch up to 75,000 (15*5000) follower IDs every 15 minutes for each access token.
If you only have one access token you'll, unfortunately, be limited in the manner you described and will just have to handle when your application hits the rate limit and continue processing once the 15 minutes is up.