what I am trying to do is reading the latest statuses from all my facebook friends
using graph api ,it takes too long , I am getting all my friends for facebook as json and I read the latest statuses from them , what I am getting is timeout , I know it will take too long to do , but what the efficient way to handle such thing?
break that into batches and probably do it in separate threads (since most of this will be IO work).
Related
I am calling an REST API endpoint which is hosting a very large amount of data. The data quantity is too much that my chrome tab crashes as well (It displays the data for a short time and it's loading even more data causing the tab to crash). Even postman fails to get the data and instead would only return 200 OK code without displaying any response body.
I'm trying to write a java program to consume the response from the API. Is there a way to consume the response without using a lot of memory?
Please let me know if the question is not clear. Thank you !!
A possibility is to use a JSON streaming parser like Jackson Streaming API (https://github.com/FasterXML/jackson-docs/wiki/JacksonStreamingApi) for example code see https://javarevisited.blogspot.com/2015/03/parsing-large-json-files-using-jackson.html
For JS there is https://github.com/DonutEspresso/big-json
If data is really so large then better to split task:
Download full data via ordinary http client to disk
Make bulk processing , using some streaming approach, similar to SAX parsing for XML:
JAVA - Best approach to parse huge (extra large) JSON file
With such split, you will not deal with possible network errors during processing and will keep data consistency.
I'm getting pretty familiar with using Asyn tasks to fetch data from API endpoints now. I can easily hit a url and parse the JSON data that returns.
However I've run into a problem in which this API has a lot of pages to it.
What's the best way to deal with an API that has a lot of pages, and has no option to change the results per page?
My particular endpoint has 40+ pages of data (12 results per page). I feel as if spinning up a new async task per each page endpoint is a bit ridiculous.
Any ideas?
Unfortunatly as everyone suggest there is no way around the api if it does not support a results per page argument. You could prefetch one or two pages and join them in one AsyncTask that way you minimize the amount of async task that fork from the main thread and have a strategy when you need to load more pages.
I would definitely suggest you, use retrofit HTTP client. I had the same issue almost 260+ calls and Retrofit work fine for me.Check it here
I want to have some data stored in Device in order my application to work offline. They are plain java objects that are returned from a JSON web service. I use retrofit + GSON . The issue is how to store them.
WHAT I have tried
Tried to use Serializable interface at my java classes and during my sync() method when I fetch them from webservice I store them at a file. Performance is a bit poor at slow devices during sync. Not much but it is an issue.
WHAT I attempted
I attemted to store JSON from Response Retrofit. However it seems a bit of pain. Because it doesnt return as string the result but as InputStream. Then I have to convert back again.
WHAT I didnt do
I didnt try SQLite and any ORM. Seems overkill to me for 7 simple list of readonly List of objects. And I guess that performance is going to suffer more.
In IOS core data is easy to achieve this out of the box. Is there a solution for this in android ?
Best approach to this is to write the JSON out to a file using one of the near infinite number of free libraries or code samples, such as: http://www.mkyong.com/java/json-simple-example-read-and-write-json.
I want to get time line data for 2 users. Currently I am calling getUserTimeline("username",paging) method for both the users and combining the data. Is there any optimal way of doing it, for example, using one twitter api call or is this the only way it can be done? Please help.
As far as I know Twitter APIs does not allow to query for multiple users' timelines.
I guess you already did it, but you should double check that I am not wrong by looking at the official Twitter APIs documentation.
I'm working on a project which uses Facebook Graph Api to download information about users of our application and their Facebook friends. There is a need to update this data, but as I understand Real-Time Update is not an option. For example I would like to have update of profile feed of friends of our app user, and I don't see a way to do this with Real-Time Update.
Could someone give me some advice on this update mechanism? I need to update app users, their friend connections and profile feeds of users and their friends. I understand I'll have to poll Facebook servers to retrieve this data. What I'm trying to find out is some good practices when doing these things. Update frequency? Way to recognize that data has changed? If anyone has experience with this kind of things every advice would mean a lot.
Thanks.
You can use the since= query string parameter of the Graph API call. Here's some pseudocode to help you along
var usersLastPostDate = GetLastPostDateFromDataStore(userId);
if(usersLastPostDate not populated) {
streamItems = GraphApiGet(userId, "me/feed")
lastStreamItemDate = GetNewestStreamItemDate(streamItems)
StoreLastPostDateIntoDataStore(userId, lastStreamItemDate )
}
else {
streamItems = GraphApiGet(userId, "me/feed?since=" + usersLastPostDate )
}
Not massively useful for your use case (as you're wanting to get data which changes frequently), but worth pointing out that the Graph API now supports ETags - https://developers.facebook.com/blog/post/627/.
ETags will tell you if the data has changed since the last time you requested it. This won't stop you from hitting Facebooks API throttling limits, but is a quick and easy way to tell if the data has changed since you last asked for it.
There is no one answer to your question, as it depends on what your application is doing. How often do you need to get the updated information? If your data is stale for 5 minutes, is that really a problem? Can you grab the data from Facebook lazily, when some user action requires that you have it?
If you do need to do a lot of polling try and use non-blocking IO, especially if you're expecting to have a lot of open HTTP requests to Facebook whilst you're polling. Build a reliable queueing mechanism and HTTP poker to ensure requests are being made as expected. Without any idea of what technology stack you're using it's hard to be more specific than that.
HTH
What about this: Open Graph Subscription system ?