Develop Reference

Interacting with a large java program as a service?

How can I do the following?
What I want to do is load Stanford NLP ONCE, then interact with it via an HTTP or other endpoint. The reason is that it takes a long time to load, and loading for every string to analyze is out of the question.
For example, here is Stanford NLP loading in a simple C# program that loads the jars... I'm looking to do what I did below, but in java:
Reading POS tagger model from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [9.3 sec].
Loading classifier from D:\Repositories\StanfordNLPCoreNLP\stanford-corenlp-3.6.0-models\edu\stanford\nlp\models\ner\english.all.3class.distsim.crf.ser.gz ... done [12.8 sec].
Loading classifier from D:\Repositories\StanfordNLPCoreNLP\stanford-corenlp-3.6.0-models\edu\stanford\nlp\models\ner\english.muc.7class.distsim.crf.ser.gz ... done [5.9 sec].
Loading classifier from D:\Repositories\StanfordNLPCoreNLP\stanford-corenlp-3.6.0-models\edu\stanford\nlp\models\ner\english.conll.4class.distsim.crf.ser.gz ... done [4.1 sec].
done [8.8 sec].
Sentence #1 ...
This is over 30 seconds. If these all have to load each time, yikes. To show what I want to do in java, I wrote a working example in C#, and this complete example may help someone some day:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using java.io;
using java.util;
using edu.stanford.nlp;
using edu.stanford.nlp.pipeline;
using Console = System.Console;
namespace NLPConsoleApplication
{
class Program
{
static void Main(string[] args)
{
// Path to the folder with models extracted from `stanford-corenlp-3.6.0-models.jar`
var jarRoot = #"..\..\..\..\StanfordNLPCoreNLP\stanford-corenlp-3.6.0-models";
// Text for intial run processing
var text = "Kosgi Santosh sent an email to Stanford University. He didn't get a reply.";
// Annotation pipeline configuration
var props = new Properties();
props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner, parse, sentiment");
props.setProperty("ner.useSUTime", "0");
// We should change current directory, so StanfordCoreNLP could find all the model files automatically
var curDir = Environment.CurrentDirectory;
Directory.SetCurrentDirectory(jarRoot);
var pipeline = new StanfordCoreNLP(props);
Directory.SetCurrentDirectory(curDir);
// loop
while (text != "quit")
{
// Annotation
var annotation = new Annotation(text);
pipeline.annotate(annotation);
// Result - Pretty Print
using (var stream = new ByteArrayOutputStream())
{
pipeline.prettyPrint(annotation, new PrintWriter(stream));
Console.WriteLine(stream.toString());
stream.close();
}
edu.stanford.nlp.trees.TreePrint tprint = new edu.stanford.nlp.trees.TreePrint("words");
Console.WriteLine();
Console.WriteLine("Enter a sentence to evaluate, and hit ENTER (enter \"quit\" to quit)");
text = Console.ReadLine();
} // end while
}
}
}
So it takes the 30 seconds to load, but each time you give it a string on the console, it takes the smallest fraction of a second to parse & tokenize that string.
You can see that I loaded the jar files prior to the while loop.
This may end up being a socket service, HTML, or something else that will entertain requests (in the form of strings), and spit back the parsing.
My ultimate goal is to use a mechanism in Nifi, via a processor that can send strings to be parsed, and have them returned in less than a second, versus 30+ seconds if a traditional web server threaded example (for instance) is used. Every request would load the whole thing for 30 seconds, THEN get down to business. I hope I made this clear!
How to do this?

Any of the mechanisms you list are perfectly reasonable routes forward for leveraging that service with Apache NiFi. Depending on your needs, some of the processors and extensions that are bundled with the standard release of NiFi may be sufficient to interact with your proposed web service or similar offering.
If you are striving for performing all of this within NiFi itself, a custom Controller Service might be a great path to provide this resource to NiFi that falls within the lifecycle of the application itself.
NiFi can be extended with items like controller services and custom processors and we have some documentation to get you started down that path.
Additional details could certainly help to provide some more information. Feel free to follow up on here with additional comments and/or reach out to the community via our mailing lists.
One item I did want to call out if it was unclear that NiFi is JVM driven and work would be done in Java or JVM friendly languages.

You should look at the new CoreNLP Server which Stanford NLP introduced in version 3.6.0. It seems like it does just what you want? Some other people such as ETS have done similar things.
Fine point: If using this heavily, you might (at present) want to grab the latest CoreNLP code from github HEAD, since it contains a few fixes to the server which will be in the next release.

Programatically embed a video in a slideshow using Apache Open Office API

I want to create a plugin that adds a video on the current slide in an open instance of Open Office Impress by specifying the location of the video automatically. I have successfully added shapes to the slide. But I cannot find a way to embed a video.
Using the .uno:InsertAVMedia I can take user input to choose a file and it works. How do I want to specify the location of the file programmatically?
CONCLUSION:
This is not supported by the API. Images and audio can be inserted without user intervention but videos cannot be done this way. Hope this feature is released in subsequent versions.

You requested information about an extension, even though the code you are using is quite different, using a file stream reader and POI.
If you really do want to develop an extension, then start with one of the Java samples. An example that uses Impress is https://wiki.openoffice.org/wiki/File:SDraw.zip.
Inserting videos into an Impress presentation can be difficult. First be sure you can get it to work manually. The most obvious way to do that seems to be Insert -> Media -> Audio or Video. However many people use links to files instead of actually embedding the file. See also https://ask.libreoffice.org/en/question/1898/how-to-embed-video-into-impress-presentation/.
If embedding is working for your needs and you want to automate the embedding by using an extension (which seems to be what your question is asking), then there is a dispatcher method called InsertAVMedia that does this.
I do not know offhand what the parameters are for the call. See https://forum.openoffice.org/en/forum/viewtopic.php?f=20&t=61127 for how to look up parameters for dispatcher calls.
EDIT
Here is some Basic code that inserts a video.
sub insert_video
dim document as object
dim dispatcher as object
document = ThisComponent.CurrentController.Frame
dispatcher = createUnoService("com.sun.star.frame.DispatchHelper")
dispatcher.executeDispatch(document, ".uno:InsertAVMedia", "", 0, Array())
end sub
From looking at InsertAVMedia in sfx.sdi, it seems that this call does not take any parameters.
EDIT 2
Sorry but InsertVideo and InsertImage do not take parameters either. From svx.sdi it looks like the following calls take parameters of some sort: InsertGalleryPic, InsertGraphic, InsertObject, InsertPlugin, AVMediaToolBox.
However according to https://wiki.openoffice.org/wiki/Documentation/OOoAuthors_User_Manual/Getting_Started/Sometimes_the_macro_recorder_fails, it is not possible to specify a file for InsertObject. That documentation also mentions that you never know what will work until you try it.
InsertGraphic takes a FileName parameter, so I would think that should work.

It is possible to add an XPlayer on the current slide. It looks like this will allow you to play a video, and you can specify the file's URL automatically.
Here is an example using createPlayer: https://forum.openoffice.org/en/forum/viewtopic.php?f=20&t=57699.
EDIT:
This Basic code works on my system. To play the video, simply call the routine.
sub play_video
If Video_flag = 0 Then
video =converttoURL( _
"C:\Users\JimStandard\Downloads\H264_test1_Talkinghead_avi_480x360.avi")
Video_flag = 1
'for windows:
oManager = CreateUnoService("com.sun.star.media.Manager_DirectX")
' for Linux
' oManager = CreateUnoService("com.sun.star.media.Manager_GStreamer")
oPlayer = oManager.createPlayer( video )
' oPlayer.CreatePlayerwindow(array()) ' crashes?
'oPlayer.setRate(1.1)
oPlayer.setPlaybackLoop(False)
oPlayer.setMediaTime(0.0)
oPlayer.setVolumeDB(GetSoundVolume())
oPlayer.start() ' Lecture
Player_flag = 1
Else
oPlayer.start() ' Lecture
Player_flag = 1
End If
End Sub

Testing HLS using JMeter

I am using JMeter to test HLS playback from a Streaming Server. So, the first HTTP request is for a master manifest file(m3u8). Say,
http://myserver/application1/subpath1/file1.m3u8
The reply to this will result in a playlist something like,
subsubFolder/360p/file1.m3u8
subsubFolder/480p/file1.m3u8
subsubFolder/720p/file1.m3u8
So, next set of URLs become
http://myserver/application1/subpath1/subsubFolder/360p/file1.m3u8
http://myserver/application1/subpath1/subsubFolder/480p/file1.m3u8
http://myserver/application1/subpath1/subsubFolder/720p/file1.m3u8
Now, individual reply to these further will be an index of chunks, like
0/file1.ts
1/file1.ts
2/file2.ts
3/file3.ts
Again, we have next set of URLs as
http://myserver/application1/subpath1/subsubFolder/360p/0/file1.ts
http://myserver/application1/subpath1/subsubFolder/360p/1/file1.ts
http://myserver/application1/subpath1/subsubFolder/360p/2/file1.ts
http://myserver/application1/subpath1/subsubFolder/360p/3/file1.ts
This is just the case of one set(360p). There will be 2 more sets like these(for 480p, 720p).
I hope the requirement statement is clear uptill this.
Now, the problem statement.
Using http://myserver/application1 as static part, regex(.+?).m3u8 is applied at 1st reply which gives subpath1/subsubFolder/360p/file1. This, is then added to the static part again, to get http://myserver/application1/subpath1/subsubFolder/360p/file1 + .m3u8
The problem comes at the next stage. As, you can see, with parts extracted previously, all I'm getting is
http://myserver/application1/subpath1/subsubFolder/360p/file1/0/file1.ts
The problem is obvious, an extra file1, 360p/file1 in place of 360p/0.
Any suggestions, inputs or alternate approaches appreciated.

If I understood the problem correctly, all you need is the file name as the other URLs can be constructed with it. Rather than using http://myserver/application1 as static part of your regex, I would try to get the filename directly:
([^\/.]+)\.m3u8$
# match one or more characters that are not a forward slash or a period
# followed by a period
# followed by the file extension (m3u8)
# anchor the whole match to the end
Now consider your urls, e.g. http://myserver/application1/subpath1/subsubFolder/360p/file1.m3u8, the above regex will capture file1, see a working demo here. Now you can construct the other URLs, e.g. (pseudo code):
http://myserver/application1/subpath1/subsubFolder/360p/ + filename + .m3u8
http://myserver/application1/subpath1/subsubFolder/360p/ + filename + /0/ + filename + .ts
Is this what you were after?

Make sure you use:
(.*?) - as Regular Expression (change plus to asterisk in your regex)
-1 - as Match No.
$1$- as template
See How to Load Test HTTP Live Media Streaming (HLS) with JMeter article for detailed instructions.

If you are ready to pay for a commercial plugin, then there is an easy and much more realistic solution which is a plugin for Apache JMeter provided by UbikLoadPack:
Besides doing this job for you, it will simulate the way a player would read the file. It will also scale much better than any custom script or player solution.
It supports VOD and Live which are quite difficult to script.
See:
http://www.ubik-ingenierie.com/blog/easy-and-realistic-load-testing-of-http-live-streaming-hls-with-apache-jmeter/
http://www.ubik-ingenierie.com/blog/ubikloadpack-http-live-streaming-plugin-jmeter-videostreaming-mpegdash/
Disclaimer, we are the providers of this solution

Include SVN revision number in source code

My requirement is simple. At the beginning of each file there should be a block comment like this:
/*
* This file was last modified by {username} at {date} and has revision number {revisionnumber}
*/
I want to populate the {username}, {date} and {revisionnumber} with the appropriate content from SVN.
How can I achieve this with NetBeans and Subversion? I have searched a lot but I can't find exactly what I need.

I looked at this question and got some useful information. It is not exactly duplicate because I am working with NetBeans but the idea is the same. This is my header:
/*
* $LastChangedDate$
* $LastChangedRevision$
*/
Then I go to Team > Subversion > Svn properties and add svn:keywords as property name and LastChangedDate LastChangedRevision as property value.
And when I commit from NetBeans it looks like this:
/*
* $LastChangedDate: 2012-02-13 17:38:57 +0200 (Пн, 13 II 2012) $
* $LastChangedRevision: 27 $
*/
Thanks all for the support! I will accept my answer because other answers do not include the NetBeans information. Nevertheless I give +1 to the other answers.

As this data only exists after the file was committed it should be set by SVN itself, not a client program. (And client-side processing tends to get disabled or not configured at all.) This means there is no simple template/substitute like you want, because then after the first replacement the template variables would be lost.
You can find information abut SVN's keyword substitution here. Then things like $Rev$ can be replaced by $Rev: 12 $.

You can do this with The SubWCRev Program.
SubWCRev is Windows console program which can be used to read the
status of a Subversion working copy and optionally perform keyword
substitution in a template file. This is often used as part of the
build process as a means of incorporating working copy information
into the object you are building. Typically it might be used to
include the revision number in an “About” box.
This is typically done during the build process.
If you use Linux, you can find a Linux binary here. If you wish, you could also write your own using the output of svn log.

I followed Petar Minchev's suggestions, only I put the $LastChangedRevision$ tag not in a comment block but embedded it in a string. Now it is available to programmatically display the revision number in a Help -> About dialog.
String build = "$LastChangedRevision$";
I can later display the revision value in the about dialog using a String that has all of the fluff trimmed off.
String version = build.replace("$LastChangedRevision:", "").replace("$", "").trim();

I recommend a slightly different approach.
Put the following header at the top of your source files.
/*
* This file was last modified by {username} at {date} and has revision number {revisionnumber}
*/
Then add a shell script like this
post update, checkout script
USERNAME=# // use svnversion to get username
DATE=# // use svnversion to get revisio nnumber
sed -e "s#{username}#${USERNAME}#" -e "s#{date}#${DATE}#" ${SOURCE_CONTROL_FILE} > ${SOURCE_FILE}
pre commit script
cat standard_header.txt > ${SOURCE_CONTROL_FILE}
tail --lines $((${LENGTH}-4)) ${SOURCE_FILE} >> ${SOURCE_CONTROL_FILE}

java gate api. Creating pipeline with success, how can i get the annotationsets from the docs processed?

sorry in advance for my poor grammar.
I have created a pipeline with GATE API, i run it successfully.
I created a serialanalysercontroller like this: pipeline = (SerialAnalyserController)Factory.createResource("gate.creole.SerialAnalyserController");
, then i load a corpus of files (previously populated)
pipeline.setCorpus(foo)
and last, pipeline.execute().
It all works great and i see the results. My problem is that i cannot find the way to get the AnnotationSet for each document that was processed in the corpus. For example i want to find the AnnotationSet ("sentences") to find in which offsets the sentences start and stop in the original text file. The API does not tell how I will get the annotations from the SerialAnalyserController - how to get each gate.Document after the process pipeline has finished.
Thanks in advance

Ok, found it!
I get the corpus back, then because the Corpus is a list, with method get(x) i get which document I want and then I get the annotationSets.
Thanks