Develop Reference

Pagination in CosmosDB Java SDK with continuation token

I'm trying to create from an async client a method to retrieve items from a CosmosDB but I'm afraid I'm full of questions and little to no documentation from Microsoft side
I've created a function that will read from a cosmosDB a list of items, page by page, which continuation will depend on a continuityToken. The methos looks like this. Please, be aware there could be some minor mistakes non related to the core functionality which is reading page by page:
#FunctionName("Feed")
public HttpResponseMessage getFeed(
#HttpTrigger(
name = "get",
methods = { HttpMethod.GET },
authLevel = AuthorizationLevel.ANONYMOUS,
route = "Feed"
) final HttpRequestMessage<Optional<String>> request,
#CosmosDBInput(
name = "Feed",
databaseName = Constants.DATABASE_NAME,
collectionName = Constants.LOG_COLLECTION_NAME,
sqlQuery = "SELECT * FROM c", // This won't be used actually as we use our own query
connectionStringSetting = Constants.CONNECTION_STRING_KEY
) final LogEntry[] logEntryArray,
final ExecutionContext context
) {
context
.getLogger()
.info("Query with paging and continuation token");
String query = "SELECT * FROM c"
int pageSize = 10; //No of docs per page
int currentPageNumber = 1;
int documentNumber = 0;
String continuationToken = null;
double requestCharge = 0.0;
// First iteration (continuationToken = null): Receive a batch of query response pages
// Subsequent iterations (continuationToken != null): Receive subsequent batch of query response pages, with continuationToken indicating where the previous iteration left off
do {
context
.getLogger()
.info("Receiving a set of query response pages.");
context
.getLogger()
.info("Continuation Token: " + continuationToken + "\n");
CosmosQueryRequestOptions queryOptions = new CosmosQueryRequestOptions();
Flux<FeedResponse<LogEntry>> feedResponseIterator =
container.queryItems(query, queryOptions, LogEntry.class).byPage(continuationToken,pageSize);
try {
feedResponseIterator.flatMap(fluxResponse -> {
context
.getLogger()
.info("Got a page of query result with " +
fluxResponse.getResults().size() + " items(s)"
+ " and request charge of " + fluxResponse.getRequestCharge());
context
.getLogger()
.info("Item Ids " + fluxResponse
.getResults()
.stream()
.map(LogEntry::getDate)
.collect(Collectors.toList()));
return Flux.empty();
}).blockLast();
} catch (Exception e) {
}
} while (continuationToken != null);
context
.getLogger()
.info(String.format("Total request charge: %f\n", requestCharge));
return request
.createResponseBuilder(HttpStatus.OK)
.header("Content-Type", "application/json")
.body("ALL READ")
.build();
}
For simplicity the read items are merely logged.
First question: We are using an async document client that returns a Flux. Will the client keep track of the token? It is a stateless client in principle. I understand that the sync client could take easily care of this case, but wouldn't the async client reset its memory of tokens after the first page and token has been generated?
Second: Is the while loop even appropriated? My assumption is a big no, as we need to send back the token in a header and the frontend UI will need to send the token to the Azure Function in a header or other similar fashion. The token should be extracted from the context then
Third: Is the flatMap and blockList way to read the flux appropriate? I was trying to play with the subscribe method but again I don't see how it could work for an async client.
Thanks a lot,
Alex.
UPDATE:
I have observed that Flux only uses the items per page value to set the number of items to be retrieved per batch, but after retrieval of one page it doesn't stop and keeps retrieving pages! I don't know how to stop it. I have tried substituting the Flux.empty() per Mono.empty() and setting a LIMIT clause in the sql query. The first option does the same and the second freezes the query and never returns apparently. How can I return one page an only one page along with the continuation token to do the following query once the user clicks on the next page button?

How to get the access token to a Facebook page from the link with Spring Social?

I have to get a management token from a page from the link provided by the user. Until then I have collected the username of the page as a substring from the provided link and request permission from the user as follows:
public String createFacebookAuthorizationURL(String pagename, Long churchId) {
FacebookConnectionFactory connectionFactory = new FacebookConnectionFactory(facebookAppId, facebookSecret);
OAuth2Operations oauthOperations = connectionFactory.getOAuthOperations();
OAuth2Parameters params = new OAuth2Parameters();
params.setRedirectUri("https://" + this.hostName + "/api/facebook/" + churchId + "/response"); params.setScope("public_profile,pages_show_list,publish_pages,manage_pages,user_events");
return oauthOperations.buildAuthorizeUrl(params);
}
public String createFacebookAccessToken(String code, Long churchId) {
FacebookConnectionFactory connectionFactory = new FacebookConnectionFactory(facebookAppId, facebookSecret);
AccessGrant accessGrant = connectionFactory.getOAuthOperations().exchangeForAccess(code, "https://" + this.hostName + "/api/facebook/" + churchId + "/response", null);
return accessGrant.getAccessToken();
}
I have already tested the above method and it works. The problem is that: this method requests a token for access to user pages, I want access to only one page, but I do not know how to put this in the requisitiond and token.
I want use this permissions to post text, links and images in fb pages via my system API...

How to get the next page on spring pagination

I have a resource in my service with pagination and I would like to know how could I process the next request to get the second page.
Here is the java resource:
#GetMapping(value = "/partner/codes")
public Page<String> getCodes(#PageableDefault(size = 5) Pageable pageable) {
final List<String> userIds = service.getIds();
int start = pageable.getOffset();
int end = (start + pageable.getPageSize()) > userIds.size() ? userIds.size() : (start + pageable.getPageSize());
return new PageImpl<String>(userIds.subList(start, end), pageable, userIds.size());
}
And the response the response with 5 results:
{
"content":[
"4a136aa6-00d4-44f0-bb48-d192fd8bc010",
"bebebaf2-b881-4733-8a65-1ecf80b5192e",
"1a0f9d07-1393-48a8-8883-37d87681e84b",
"d2580fdc-db6c-4fa3-89d4-2b52898a20bf",
"2c90e683-4ed4-45a4-b70b-614a3339670b"
],
"last":false,
"totalPages":3,
"totalElements":57,
"size":20,
"number":0,
"sort":null,
"numberOfElements":20,
"first":true
}

I'm sorry, as there was nothing explicit in the documentation, I hadn't noticed that just passing the parameters in the query string.
?page=2&size=20
And the client should create the rule using the response message.

Jsoup fetches wrong results

Working with Jsoup. The URL works well on the browser. But it fetches wrong result on the server. I set the maxBodySize "0" as well. But it still only gets first few tags. Moreover the data is even different from the browser one. Can you guys give me a hand?
String queryUrl = "http://www.juso.go.kr/addrlink/addrLinkApi.do?confmKey=U01TX0FVVEgyMDE3MDYyODE0MTYyMzIyMTcw&currentPage=1&countPerPage=20&keyword=연남동";
Document document = Jsoup.connect(queryUrl).maxBodySize(0).get();

Are you aware that this endpoint returns paginated data? Your URL asks for 20 entries from the first page. I assume that the order of these entries is not specified so you can get different data each time you call this endpoint - check if there is a URL parameter that can determine specific sort order.
Anyway to read all 2037 entries you have to do it sequentially. Examine following code:
final String baseUrl = "http://www.juso.go.kr/addrlink/addrLinkApi.do";
final String key = "U01TX0FVVEgyMDE3MDYyODE0MTYyMzIyMTcw";
final String keyword = "연남동";
final int perPage = 100;
int currentPage = 1;
while (true) {
System.out.println("Downloading data from page " + currentPage);
final String url = String.format("%s?confmKey=%s&currentPage=%d&countPerPage=%d&keyword=%s", baseUrl, key, currentPage, perPage, keyword);
final Document document = Jsoup.connect(url).maxBodySize(0).get();
final Elements jusos = document.getElementsByTag("juso");
System.out.println("Found " + jusos.size() + " juso entries");
if (jusos.size() == 0) {
break;
}
currentPage += 1;
}
In this case we are asking for 100 entries per page (that's the maximum number this endpoint supports) and we call it 21 times, as long as calling for a specific page return any <juso> element. Hope it helps solving your problem.

Using jQuery, how do I way attach a string array as a http parameter to a http request?

I have a spring controller with a request mapping as follows
#RequestMapping("/downloadSelected")
public void downloadSelected(#RequestParam String[] ids) {
// retrieve the file and write it to the http response outputstream
}
I have an html table of objects which for every row has a checkbox with the id of the object as the value. When they submit, I have a jQuery callback to serialize all ids. I want to stick those ids into an http request parameter called, "ids" so that I can grab them easily.
I figured I could do the following
var ids = $("#downloadall").serializeArray();
Then I would need to take each of the ids and add them to a request param called ids. But is there a "standard" way to do this? Like using jQuery?

I don't know about "standard way", but this is how I would do it.
var ids = $("#downloadall").serializeArray();
will give you a dataset on the form (only the checked items presented):
[{name:"foo1", value:"bar1"}, {name:"foo2", value:"bar2"}]
To feed this to jQuery's .ajax() just:
$.ajax({
url: <your url>,
data: ids.map(function (i) {return i.name+'='+i.value;}).join('&')
});
The Array.map() is not compatible with all browsers yet so you better have this code on your page too:
if (!Array.prototype.map) {
Array.prototype.map = function(fun /*, thisp*/) {
var len = this.length >>> 0;
if (typeof fun != "function")
throw new TypeError();
var res = new Array(len);
var thisp = arguments[1];
for (var i = 0; i < len; i++) {
if (i in this)
res[i] = fun.call(thisp, this[i], i, this);
}
return res;
};
}
This code snippet I got from mozilla developer center.
I didn't put them in a ?ids=... param, but this way they are easy to access on server side. You can always just modify the map function to fit your needs.