Develop Reference

Concurrency on Vertx

i have joined to one of those Vertx lovers , how ever the single threaded main frame may not be working for me , because in my server there might be 50 file download requests at a moment , as a work around i have created this class
public abstract T onRun() throws Exception;
public abstract void onSuccess(T result);
public abstract void onException();
private static final int poolSize = Runtime.getRuntime().availableProcessors();
private static final long maxExecuteTime = 120000;
private static WorkerExecutor mExecutor;
private static final String BG_THREAD_TAG = "BG_THREAD";
protected RoutingContext ctx;
private boolean isThreadInBackground(){
return Thread.currentThread().getName() != null && Thread.currentThread().getName().equals(BG_THREAD_TAG);
}
//on success will not be called if exception be thrown
public BackgroundExecutor(RoutingContext ctx){
this.ctx = ctx;
if(mExecutor == null){
mExecutor = MyVertxServer.vertx.createSharedWorkerExecutor("my-worker-pool",poolSize,maxExecuteTime);
}
if(!isThreadInBackground()){
/** we are unlocking the lock before res.succeeded , because it might take long and keeps any thread waiting */
mExecutor.executeBlocking(future -> {
try{
Thread.currentThread().setName(BG_THREAD_TAG);
T result = onRun();
future.complete(result);
}catch (Exception e) {
GUI.display(e);
e.printStackTrace();
onException();
future.fail(e);
}
/** false here means they should not be parallel , and will run without order multiple times on same context*/
},false, res -> {
if(res.succeeded()){
onSuccess((T)res.result());
}
});
}else{
GUI.display("AVOIDED DUPLICATE BACKGROUND THREADING");
System.out.println("AVOIDED DUPLICATE BACKGROUND THREADING");
try{
T result = onRun();
onSuccess((T)result);
}catch (Exception e) {
GUI.display(e);
e.printStackTrace();
onException();
}
}
}
allowing the handlers to extend it and use it like this
public abstract class DefaultFileHandler implements MyHttpHandler{
public abstract File getFile(String suffix);
#Override
public void Handle(RoutingContext ctx, VertxUtils utils, String suffix) {
new BackgroundExecutor<Void>(ctx) {
#Override
public Void onRun() throws Exception {
File file = getFile(URLDecoder.decode(suffix, "UTF-8"));
if(file == null || !file.exists()){
utils.sendResponseAndEnd(ctx.response(),404);
return null;
}else{
utils.sendFile(ctx, file);
}
return null;
}
#Override
public void onSuccess(Void result) {}
#Override
public void onException() {
utils.sendResponseAndEnd(ctx.response(),404);
}
};
}
and here is how i initialize my vertx server
vertx.deployVerticle(MainDeployment.class.getCanonicalName(),res -> {
if (res.succeeded()) {
GUI.display("Deployed");
} else {
res.cause().printStackTrace();
}
});
server.requestHandler(router::accept).listen(port);
and here is my MainDeployment class
public class MainDeployment extends AbstractVerticle{
#Override
public void start() throws Exception {
// Different ways of deploying verticles
// Deploy a verticle and don't wait for it to start
for(Entry<String, MyHttpHandler> entry : MyVertxServer.map.entrySet()){
MyVertxServer.router.route(entry.getKey()).handler(new Handler<RoutingContext>() {
#Override
public void handle(RoutingContext ctx) {
String[] handlerID = ctx.request().uri().split(ctx.currentRoute().getPath());
String suffix = handlerID.length > 1 ? handlerID[1] : null;
entry.getValue().Handle(ctx, new VertxUtils(), suffix);
}
});
}
}
}
this is working just fine when and where i need it , but i still wonder if is there any better way to handle concurencies like this on vertx , if so an example would be really appreciated . thanks alot

I don't fully understand your problem and reasons for your solution. Why don't you implement one verticle to handle your http uploads and deploy it multiple times? I think that handling 50 concurrent uploads should be a piece of cake for vert.x.
When deploying a verticle using a verticle name, you can specify the number of verticle instances that you want to deploy:
DeploymentOptions options = new DeploymentOptions().setInstances(16);
vertx.deployVerticle("com.mycompany.MyOrderProcessorVerticle", options);
This is useful for scaling easily across multiple cores. For example you might have a web-server verticle to deploy and multiple cores on your machine, so you want to deploy multiple instances to take utilise all the cores.
http://vertx.io/docs/vertx-core/java/#_specifying_number_of_verticle_instances

vertx is a well-designed model so that a concurrency issue does not occur.
generally, vertx does not recommend the multi-thread model.
(because, handling is not easy.)
If you select multi-thread model, you have to think about shared data..
Simply, if you just only want to split EventLoop Area,
first of all, you make sure Check your a number of CPU Cores.
and then Set up the count of Instances .
DeploymentOptions options = new DeploymentOptions().setInstances(4);
vertx.deployVerticle("com.mycompany.MyOrderProcessorVerticle", options);
But, If you have 4cores of CPU, you don't set up over 4 instances.
If you set up to number four or more, the performance won't improve.
vertx concurrency reference
http://vertx.io/docs/vertx-core/java/

Apache Camel: Get move path after file transfer

I am creating a file transfer route which is using move to set a dynamic path where the file is moved after successful file transfer. I have also setup a notifier to keep track of file transfer events.
As the move path is dynamic, I need to get the evaluated path where file was moved after the file transfer. How can this path inside the notifier?
public class MyFtpServiceBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
getContext()
.getManagementStrategy()
.addEventNotifier(new MyFtpServiceNotifier());
from("file:C:/tmp/inputfolder?move=archive/${date:now:yyyyMMdd}/${file:onlyname}")
.routeId("myRoute")
.to("file:C:/tmp/outputfolder")
}
}
public class MyFtpServiceNotifier extends EventNotifierSupport {
#Override
public void notify(EventObject event) throws Exception {
Exchange exchange = ((AbstractExchangeEvent) event).getExchange();
if (event instanceof ExchangeSentEvent) {
// Want to get here the path where file was moved
}
}
#Override
public boolean isEnabled(EventObject event) {
return event instanceof AbstractExchangeEvent;
}
}

You could try to add a header to the exchange, something like:
.setHeader("FILE_PROCESSED").simple("archive/${date:now:yyyyMMdd}/${file:onlyname}")

I'm not sure how to do it in a notifier, but you can always add a Processor and use the FileEndpoint.getMove().evaluate() method on the exchange to obtain the final moved file path.
For example:
#Override
public void process(final Exchange exchange) throws Exception {
File movedFile = null;
if (exchange.getFromEndpoint() instanceof FileEndpoint) {
FileEndpoint fileEndpoint = (FileEndpoint) exchange.getFromEndpoint();
String movePath = fileEndpoint.getMove().evaluate(exchange, String.class);
File inputDir = fileEndpoint.getFile();
movedFile = new File(inputDir, movePath);
}
}

How to keep initial log file in rotating log handler

I am using Java logging to log the memory static in my file and use java.util.logging.FileHandler to implement rotating log. Now I have a situation where my manager wants to keep the initial logging file and rotate the rest of the file. Is there any way I can keep the initial log file but yet rotate the rest of the file.
public class TopProcessor extends Handler {
Handler handler;
public TopProcessor() throws IOException{
File dir = new File(System.getProperty("user.home"), "logs");
dir.mkdirs();
File fileDir = new File(dir,"metrics");
fileDir.mkdirs();
String pattern = "metrics-log-%g.json";
int count = 5;
int limit = 500000;
handler = new TopProcessorHandler(fileDir.getAbsolutePath()+File.separator+pattern, limit, count);
}
class TopProcessorHandler extends FileHandler{
public TopProcessorHandler(String pattern, int limit, int count)
throws IOException {
super(pattern, limit, count);
}
}
private void writeInformationToFile(String information) {
handler.publish(new LogRecord(Level.ALL, information));
}
#Override
public void close() {
handler.close();
}
#Override
public void flush() {
handler.flush();
}
#Override
public void publish(LogRecord record) {
handler.publish(record);
}
}

Create 2 files one initial log file and other rotating log file..You can merge two files when you want to read logs

Vaadin Upload component how get fileName before submitUpload?

I try make implementation for comparing the files before they are uploaded.
If file whith name is exist in system ask about create new version or just override it.
Here is the problem, how to get file name?
I can't use receiveUpload(), because after this method file is remove from upload component ?

The problem is that once you start an upload using the Upload component, it can only be interrupted by calling the interruptUpload() method, and you cannot resume anytime later.
The interruption is permanent.
This means you cannot pause in the middle of the upload to see if you already have the file in your system. You have to upload the file all the way.
Considering this drawback, you can sill check in your system if you have the file, after the upload finishes. If you have the file, you can show a confirmation dialog in which you decide wether to keep the file or overwrite.
The following is an example in which I check in the "system" (I just keep a String list with the filenames) if the file has already been uploaded:
public class RestrictingUpload extends Upload implements Upload.SucceededListener, Upload.Receiver {
private List<String> uploadedFilenames;
private ByteArrayOutputStream latestUploadedOutputStream;
public RestrictingUpload() {
setCaption("Upload");
setButtonCaption("Upload file");
addSucceededListener(this);
setReceiver(this);
uploadedFilenames = new ArrayList<String>();
}
#Override
public OutputStream receiveUpload(String filename, String mimeType) {
latestUploadedOutputStream = new ByteArrayOutputStream();
return latestUploadedOutputStream;
}
#Override
public void uploadSucceeded(SucceededEvent event) {
if (fileExistsInSystem(event.getFilename())) {
confirmOverwrite(event.getFilename());
} else {
uploadedFilenames.add(event.getFilename());
}
}
private void confirmOverwrite(final String filename) {
ConfirmDialog confirmDialog = new ConfirmDialog();
String message = String.format("The file %s already exists in the system. Overwrite?", filename);
confirmDialog.show(getUI(), "Overwrite?", message, "Overwrite", "Cancel", new ConfirmDialog.Listener() {
#Override
public void onClose(ConfirmDialog dialog) {
if (dialog.isConfirmed()) {
copyFileToSystem(filename);
}
}
});
}
private void copyFileToSystem(String filename) {
try {
IOUtils.write(latestUploadedOutputStream.toByteArray(), new FileOutputStream(filename));
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e2) {
e2.printStackTrace();
}
}
private boolean fileExistsInSystem(String filename) {
return uploadedFilenames.contains(filename);
}
}
Note that I have used 2 external libraries:
Apache Commons IO 2.4 (http://mvnrepository.com/artifact/commons-io/commons-io/2.4) for writing to streams
ConfirmDialog from Vaadin Directory (https://vaadin.com/directory#addon/confirmdialog)
You can get the code snippet for this class from Gist: https://gist.github.com/gabrielruiu/9960772 which you can paste into your UI and test it out.

Camel file unit test

I am new to Apache Camel, I have written a simple route to scan a directory (/test), file will be processed when it was copied into the directory. Anyone has an idea on how to write a camel unit test to test the following route? Is there a way to mock the process of copying the file into the /test directory so that the route will be triggered.
public void configure() {
from( "file:/test?preMove=IN_PROGRESS" +
"&move=completed/${date:now:yyyyMMdd}/${file:name}" +
"&moveFailed=FAILED/${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}" )
.process(new Processor() {
public void process(Exchange exchange) throws IOException {
File file = (File) exchange.getIn().getBody();
// read file content ......
}
});
}

You have done the routing by one of many correct ways. But there exist some more important pieces to make your code run - you should create a context, create a router with this your configure(), add it to a context, and run this context.
Sorry, I prefer beans to processors, so you have also to register a bean. And make you processing a normal named method in a named class.
I think, the most compact info is here. JUnit test is a standalone app and you need to run Camel as a standalone app for JUnit testing.

I think the basic idea is that you mock the end endpoint so you can check what is coming out your route. There are a few different ways, but you could test your route as follows:
public class MyRouteTest extends CamelSpringTestSupport {
private static final String INPUT_FILE = "myInputFile.xml";
private static final String URI_START = "direct:start";
private static final String URI_END = "mock:end";
#Override
public boolean isUseAdviceWith() {
return true;
}
#Override
protected AbstractApplicationContext createApplicationContext() {
return new AnnotationConfigApplicationContext(CamelTestConfig.class); // this is my Spring test config, where you wire beans
}
#Override
protected RouteBuilder createRouteBuilder() {
MyRoute route = new MyRoute();
route.setFrom(URI_START); // I have added getter and setters to MyRoute so I can mock 'start' and 'end'
route.setTo(URI_END);
return route;
}
#Test
public void testMyRoute() throws Exception {
MockEndpoint result = getMockEndpoint(URI_END);
context.start();
// I am just checking I receive 5 messages, but you should actually check the content with expectedBodiesReceived() depending on what your processor does to the those files.
result.expectedMessageCount(5);
// I am just sending the same file 5 times
for (int i = 0; i < 5; i++) {
template.sendBody(URI_START, getInputFile(INPUT_FILE));
}
result.assertIsSatisfied();
context.stop();
}
private File getInputFile(String name) throws URISyntaxException, IOException {
return FileUtils.getFile("src", "test", "resources", name);
}
I am sure you already solved your issue is 2013, but this is how I would solve it in 2017. Regards