Develop Reference

How can I turn blocking code into a responsive style?

I have always been used to blocking programming and working in the Spring MVC framework. Recently, I considered learning reactive programming. I was full of doubts about how to convert the previous logic into a new style.
See the following processing(Pseudocode):
public Mono<List<String>> a() {
// 1...
List<String> strings = new ArrayList<>();
for (int i = 0; i < 100; i++) {
strings.add("hello " + i);
}
Mono<List<String>> mono = Mono.just(strings);
// 2...
mono.subscribe(e -> {
b();
});
// 3...
mono.subscribe(e -> {
c();
});
mono.subscribeOn(Schedulers.boundedElastic());
return mono;
}
// Simulate a time-consuming process.
public void b() {
try {
Thread.sleep(100);
} catch (InterruptedException err) {
throw new RuntimeException(err);
}
}
// Simulate the process of requesting an external HTTP interface once.
public int c() {
try {
Thread.sleep(300);
} catch (InterruptedException err) {
throw new RuntimeException(err);
}
return 1;
}
I tried to convert it into code that conforms to the responsive programming style, but found that the time-consuming code logic has blocked the current thread, which is inconsistent with my expectation.
I tested Webflux and Tomcat respectively, and the results show that the performance of the former is very poor. I suspect that the IO thread is blocked, which can be seen from the thread sleep time.

Thread.sleep() will pause the JVM thread running the request. You can't call this method with a Spring WebFlux application. By design WebFlux uses very few threads to handle the requests to avoid context switching but if your code intentionally blocks them you break the whole design.
In practice Spring WebFlux can be faster than a regular Spring MVC if the application workload is I/O bound e.g. a micro-service that calls multiple external APIs and doesn't perform significant calculations.
I'd suggest that you try simulating the I/O operations by making a network call to an actual server using a reactive library like Reactor Netty. Otherwise you would have to dig into the code and figure out how to create a meaningful mock for a network I/O operation, which might be tricky.

Reactive programming is good for juggling work across threads when "someone else other than this JVM" is busy.
So, handover to OS for file write, or to DB for record update or to another remote server over API call. Let them call you back when they have the result. Eventually, you should be juggling, but work goes on somewhere else and results flow in. This is where reactivity shines.
So, it's difficult to simulate with Thread.sleep or for loop running 10_000 times.
Also, it means the whole flow has to be reactive - your disk IO library should be reactive, your DB library should be reactive, your Rest client for calling other networked services should be reactive. Reactor should be helping with solutions for each of these.
Also, its not all-or-nothing. Even if your disk IO is blocking, you will still gain benefits if atleast the Rest client is non-blocking.

How to process files in batch with PDF OCR?

I would like to process 20000 PDFS in batch asynchronously with Google OCR, but I did not find documentation releated with it, I already tried using client.asyncBatchAnnotateFilesAsync fuction;
List<AsyncAnnotateFileRequest> requests = new ArrayList<>();
for (MultipartFile file : files) {
GcsSource gcsSource = GcsSource.newBuilder().setUri(gcsSourcePath + file.getOriginalFilename()).build();
InputConfig inputConfig = InputConfig.newBuilder().setMimeType("application/pdf").setGcsSource(gcsSource)
.build();
GcsDestination gcsDestination = GcsDestination.newBuilder()
.setUri(gcsDestinationPath + file.getOriginalFilename()).build();
OutputConfig outputConfig = OutputConfig.newBuilder().setBatchSize(2).setGcsDestination(gcsDestination)
.build();
AsyncAnnotateFileRequest request = AsyncAnnotateFileRequest.newBuilder().addFeatures(feature)
.setInputConfig(inputConfig).setOutputConfig(outputConfig).build();
requests.add(request);
}
AsyncBatchAnnotateFilesRequest request = AsyncBatchAnnotateFilesRequest.newBuilder().addAllRequests(requests)
.build();
AsyncBatchAnnotateFilesResponse response = client.asyncBatchAnnotateFilesAsync(request).get();
System.out.println("Waiting for the operation to finish.");
But what I get is an error message
io.grpc.StatusRuntimeException: INVALID_ARGUMENT: At this time, only single requests are supported for asynchronous processing.
If google does not provide batch process why de they provide asyncBatchAnnotateFilesAsync? Maybe am I using an old version? Does asyncBatchAnnotateFilesAsync function work in other beta version?

Multiple requests on a single call are not supported by the Vision service.
This may be confusing because, according to the RPC API documentation you could indeed provide multiple requests on a single service call (1 file per request), still, according to this issue tracker there is a known limitation on the Vision service and currently it can only take one request at a time.

Since they limit to just 1 file per request, can you just send in 20k requests? They are async requests so it should be pretty fast to send those in.

How to balance the load when building cache from App Engine?

I currently have the following situation, which has bothered me for a couple of months right now.
The case
I have build a Java (FX) application which serves as a cash registry for my shop. The application contains a lot of classes (such as Customer, Customer, Transaction etc.), which are shared with the server API. The server API is hosted on Google App Engine.
Because we also have an online shop, I have chosen to build the cache of the entire database on startup of the application. To do this I call the GET of my Data API for each class/table:
protected QueryBuilder performGet(HttpServletRequest req, HttpServletResponse res)
throws ServletException, IOException, ApiException, JSONException {
Connection conn = connectToCloudSQL();
log.info("Parameters: "+Functions.parameterMapToString(req.getParameterMap()));
String tableName = this.getTableName(req);
log.info("TableName: "+tableName);
GetQueryBuilder queryBuilder = DataManager.executeGet(conn, req.getParameterMap(), tableName, null);
//Get the correct method to create the objects
String camelTableName = Functions.snakeToCamelCase(tableName);
String parsedTableName = Character.toUpperCase(camelTableName.charAt(0)) + camelTableName.substring(1);
List<Object> objects = new ArrayList<>();
try {
log.info("Parsed Table Name: "+parsedTableName);
Method creationMethod = ObjectManager.class.getDeclaredMethod("create"+parsedTableName, ResultSet.class, boolean.class);
while (queryBuilder.getResultSet().next()) {
//Create new objects with the ObjectManager
objects.add(creationMethod.invoke(null, queryBuilder.getResultSet(), false));
}
log.info("List of objects created");
creationMethod = null;
}
catch (Exception e) {
camelTableName = null;
parsedTableName = null;
objects = null;
throw new ApiException(e, "Something went wrong while iterating through ResultSet.", ErrorStatus.NOT_VALID);
}
Functions.listOfObjectsToJson(objects, res.getOutputStream());
log.info("GET Request succeeded");
//Clean up objects
camelTableName = null;
parsedTableName = null;
objects = null;
closeConnection(conn);
return queryBuilder;
}
It simples gets every row from the requested table in my Cloud SQL database. Then it creates the objects, with the class that is shared with the client application. Lastly, it converts these classes to JSON using GSON. Some of my tables have 10.000+ rows, and then it takes approx. 5-10 sec to do this.
At the client, I convert this JSON back to a list of objects by using the same shared class. First I load the essential classes sequentially (because else the application won't start), and after that I load the rest of the classes in the background with separate threads.
The problem
Every time I load up the cache, there is a chance (1 on 4) that the server responds with a DeadlineExceededException on some of the bigger tables. I believe this has something to do with Google App Engine not being able to fire up a new instance in time, and therefore the computation time exceeds the limit.
I know it has something to do with loading the objects in background threads, because these all start at the same time. When I delay the start of these threads with 3 seconds, the error occurs a lot less, but is still present. Because the application loads 15 classes in the background, delaying them is not ideal because the application will only work partly until it is done. It is also not an option to load everything before starting, because this will take more than 2 minutes.
Does anyone know how to set up some load balancing on Google App Engine for this? I would like to solve this server side.

You clearly have an issue with warm up requests and a query that takes quite long. You have the usual options:
Do some profiling and reduce the cost of your method invocations
use caching (memcache) to cache some of the result
If those options don't work for you, you should parallelize your computations. One thing that comes to my mind is that you could reliably reduce request times if you simply split your request into multiple parallel requests like so:
Let's say your table contains 5k rows.
Then you create 50 requests with each handleing 100 rows.
Aggregate the results on server or client side and respond
It'll be quite tough to do this on just the server side but it should be possible if your now (much) smaller taks return within a couple of seconds.
Alternatively you could return a job id at once and make the client poll for the result in a couple of seconds. This would however require a small change on the client side. It's the better option though imho, especially if you want to use a task queue for creating your response.

Proper way of handling DB calls in a TCP Application

I'm currently in the progress of working on a telephone application, and for easy portability I'm making use of Unity3D. The application's design looks nice and crisp and scales well into all of my target resolutions, but the networking is giving me an issue.
I'm using Java for the server backend, and I'm using JDBC to mananage database connections. The problem is that this application is sure to have a few thousand users at minimum (Based on my current following, blogs, and marketing techniques) and I would like to make sure that I'm doing this correctly to avoid any lockups from SQL being used at the same time.
This application pulls everything that is needed from the database. For security and glitch prevention reasons if the database cannot be connected the server lets the client(application) know that there was an error.
Here's what I'm worried about: When a user logs in a few things are done almost instantly. The database is checked for their login credentials, if it's successful the client then loads the next stage of the application then sends a packet to the server. The server then grabs more information from the server (This is done through a total of three queries in the shortest form I can possibly come up with.) However; What happens if a 2-300 people are logging in, 3-400 are spending tokens (Requires db calls) and 2-300 are requesting database data elsewhere. That's around 1,000 requested database calls coming in.
I don't want the application to seem really laggy.
Here's how I'm currently handling it after a little research, but it doesn't feel necessarily right. Looking for advice and corrections. (decodeTCP is called when a packet with the header id of X is received.)
public void decodeTcp(Session session) throws IOException {
try {
ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1);
ScheduledFuture<?> scheduledFuture = scheduledExecutorService.schedule(new Callable<Object>() {
public Object call() throws Exception {
return Database.getSingleton().fetch((User)session.getAttachment());
}
}, 0, TimeUnit.SECONDS);
int results = (int) scheduledFuture.get();scheduledFuture.get());
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
Where Database#fetch(Object) returns an int.

How do I build a single access web service?

I have a Jax-RS Jersey web service on Weblogic. It runs fine but returns a large amount of data. The problem is if I have more that 4 calls for the service at the same time I use up all the JVM memory on the server, then everything stops working and I have to reboot.
How can I limit the service to only run 2 or 3 instances and have other calls wait?

How are you returning your data? Do you create the entire object to return in memory? If so then you may want to look into streaming the response instead.
Can you give us an idea as to what your service is doing?
Edit:
You can stream stuff (like data from a ResultSet) like this;
#GET
public MyResultStream getData()
{
ResultSet rs = queryDatabase();
return new MyResultStream(rs);
}
You'll need to provide MyResultStream yourself;
public class MyResultStream implements javax.ws.rs.core.StreamingOutput
{
private ResultSet rs;
public MyResultStream (ResultSet rs)
{
this.rs = rs;
}
public void write(OutputStream output)
{
//write any document pre-able
// for example <results>
while (rs.next())
{
//get the data from the ResultSet and write it to the output in XML form
// for example <result><foo>bar</foo></result>
}
//write any document post-amble
// for example </results>
}
}
Remember you'll have to close your ResultSet somehow.

If you are crashing the JVM with only three or four requests, then I would probably start looking at the architecture of what is done. As Qwerky mentioned, are you creating all the objects you are returning? Are those object heavy? Could you use lighter objects for returning the data? Do you have to return all the data at once? How much (number of records/object) data are you potentially returning? How big (size in KB, MB, etc)? Does the server have enough memory? Is the min and max memory of the server set on startup to values that help it perform better? Is there a leak? Am I creating too many objects on the heap too fast?
And pending it isn't something with the application in itself, what about clustering and load balancing (to spread out the requests).
Also...depending on the amount of data you are returning...sometimes a web service is NOT a good thing. At my company we had a case where we tried to develop a web service that needed to process send back around 30K rows of data. It was just to much to handle for it. We actually ended up turning that process into a batch process. The results of that process were then used by our web UI and web services. Since the users didn't need the data until the next day anyway it just made it easier to manage the processing of the data.
Just some differing thoughts to help give you some different angles to pursue this with.