Configure Tensorflow on GPU Java - java

How to set following config "gpu_options.allow_growth = True" for Tensorflow, using Java API?
I tried in this way:
model.session().runner()
.setOptions("gpu_options.allow_growth = True".getBytes())
.feed("image_tensor", input).fetch("detection_scores")
.fetch("detection_classes").fetch("detection_boxes").fetch("num_detections").run();
I get the following error: Unparseable RunOptions proto.

I think this is the way to configure:
ConfigProto config = ConfigProto.newBuilder()
.setGpuOptions(GPUOptions.newBuilder().setAllowGrowth(true))
.build();
model.session().runner()
.setOptions(config.toByteArray())
.feed("image_tensor", input).fetch("detection_scores")
.fetch("detection_classes").fetch("detection_boxes").fetch("num_detections").run();

The API documentation says:
public Session.Runner setOptions (byte[] options)
(Experimental method): set options (typically for debugging) for this run.
The options are presented as a serialized RunOptions protocol buffer.
Still looking for an exact example, but I assume: just taking a string and turning that into an array of bytes isn't what you should be doing.
These examples indicate that you need something like:
Session.Run r = runner.setOptions(RunStats.runOptions()).runAndFetchMetadata();
fetchTensors = r.outputs;
if (runStats == null) {
runStats = new RunStats();
}
Long story short: you have to dig into RunStats to figure how to get your options in there, to then provide an object of that class to the setOptions() method.

Related

How to enable the Barcode recognization in Abbyy Fine Reader Engine 12?

Barcode recognition is disabled by default in Abbyy Fine Reader Engine 12.
In order to enable it, I need to set the DetectBarcodes property of the PageAnalysisParams Object to TRUE.
Can anyone please help me, how can I set this property true in my java code sdk?
This is the property which we have to set:
public native void setDetectBarcodes(boolean arg0);
How can we call the native function from the java code?
Because calling directly with the object it is giving error.
Error: The local variable pageAnalysisParams may not have been initializedJava(536870963)
To get/initalize an instance of IPageAnalysisParams you can:
IPageAnalysisParams pageAnalysisParams = engine.CreatePageAnalysisParams();
You can also obtain an instance from "document processing params", like:
IDocumentProcessingParams documentparams = engine.CreateDocumentProcessingParams();
IPageAnalysisParams pageAnalysisParams = documentparams.getPageProcessingParams().getPageAnalysisParams();
source: https://github.com/search?q=IPageAnalysisParams&type=code
Looking at the public code samples, you should:
Obtain an instance of IDocumentProcessingParams (dpParams).
Tune that object (and sub-objects(page analysis params)).
And pass that to: document.Process(dpParams);
As #xerx593 suggested, programatically tuning document processing params is a perfectly valid answer.
Another valid answer is to use a configuration file, for example custom_barcode_profile.ini, and fill it according to your needs. This allows you to have better control and readibility over your profiles:
...
[PageAnalysisParams]
DetectBarcodes = TRUE
...
Use your ABBYY SDK documentation and/or ABBYY java wrapper classes to fine tune other parameters, then instead of using document.Process(dpParams);, instantiate an engine object and pass your custom_barcode_profile.ini file to it:
IEngine engine = Engine.InitializeEngine(<your sdk & license params>);
engine.LoadProfile("custom_barcode_profile.ini");
IFRDocument document = engine.CreateFRDocument();
document.AddImageFile("document.png", null, null);
document.Process(null);
document.Export("result.xml", FileExportFormatEnum.FEF_XML, null);
You cannot programatically "mix" multiple predefined profiles into one, you need to add parameters to a custom profile or even create another one and pass it to your engine object.
To enable table detection in the profile we defined later, look for parameters that affects table detection in the documentation, such as DetectTables, and add it to your custom profile:
...
[PageAnalysisParams]
DetectBarcodes = TRUE
DetectTables = TRUE
...

how to use "closure-compiler " inside the java

I'm using closure-compiler which is provided by Google. I have JavaScript's in the string variable. need to compress the string using closure-compiler in java
I already tried the code from the following link http://blog.bolinfest.com/2009/11/calling-closure-compiler-from-java.html
This is the code I used "source" variable has the value of the javascript
Compiler compiler = new Compiler();
CompilerOptions options = new CompilerOptions();
// Advanced mode is used here, but additional options could be set, too.
CompilationLevel.SIMPLE_OPTIMIZATIONS.setOptionsForCompilationLevel(options);
compiler.compile(source, options);
return compiler.toSource();
I have error in the following line: compiler.compile(source, options);
Compiler.complier() method does not require 2 parameters but it requires 3 parameters.
Have a look at this link.
You will understand the number and kind of parameters required for the method you are calling.

How to create a Script Mapper in Keycloak?

I need to create a Protocol Mapper of type Script Mapper in Keycloak. The script should get a user attribute, check its size, and put it on the token. I found no documentation or examples of how a script should be created. From the bits and pieces I could gather, I guess I the script would need to look something like:
var value = user.getAttribute("myAttribute");
if (value.length > LIMIT) {
value = value.substring(0,LIMIT);
}
token.setOtherClaims("myAttribute",value);
Is this right? I made up user.getAttribute("myAttribute"). Is there a source of documentation where I can find how to get a Keycloak user attribute?
does the script need to return anything?
Any help would be mostly welcome.
The magic of Script Mappers can be understood by looking at the keycloak sources here: Source
The script can return something by using the exports variable like this
exports = "Claim Value"
The different types:
user: Source JavaDoc
realm: Source JavaDoc
token: Source JavaDoc
userSession: Source JavaDoc
keycloakSession: Source JavaDoc
Here is an example script:
// you can set standard fields in token
token.setAcr("test value");
// you can set claims in the token
token.getOtherClaims().put("claimName", "claim value");
// multi-valued claim (thanks to #ErwinRooijakkers)
token.getOtherClaims().put('foo', Java.to(['bars'], "java.lang.String[]"))
// work with variables and return multivalued token value
var ArrayList = Java.type("java.util.ArrayList");
var roles = new ArrayList();
var client = keycloakSession.getContext().getClient();
var forEach = Array.prototype.forEach;
forEach.call(user.getClientRoleMappings(client).toArray(), function(roleModel) {
roles.add(roleModel.getName());
});
exports = roles;
Hope it helps!
I needed this feature but could not find this "script mapper" thing in my freshly installed 10.0.2. Turns out it is not enabled by default, as seen in the docs here : https://www.keycloak.org/docs/latest/server_installation/#profiles
To enable it, you can either :
Create a file standalone/configuration/profile.properties with feature.scripts=enabled
or
start the server with bin/standalone.sh|bat -Dkeycloak.profile.feature.scripts=enabled
And it seems from the source code
public boolean isSupported() {
return Profile.isFeatureEnabled(Profile.Feature.SCRIPTS) && Profile.isFeatureEnabled(Profile.Feature.UPLOAD_SCRIPTS);
}
that the upload_scripts should be enabled likewise
I hope it will help someone

Calling a Java Web Service from a .Net Project - Handling arrays

I am trying to call some web services written in Java from my asp.net code. I don't know Java.
There are several methods each with multiple properties. These methods are then passed to one controlling method. The problem occurs with how to handle arrays. I cannot seem to get the syntax quite right.
For example, there can be from 1 to n locations. Each location has multiple properties.
I can build one location ok. But how do I build an array of locations? I tried several approaches. Here is the closest I’ve gotten. It complies but crashes on the third to the last line below. I’ve removed the code that is not relevant to explaining the problem.
The error says: Object reference not set to an instance of an object.
WebReferenceMERegistration.getMERegistration _myMERegistration =
new WebReferenceMERegistration.getMERegistration();
WebReferenceMERegistration.Locations _myLocation =
new WebReferenceMERegistration.Locations();
WebReferenceMERegistration.Locations[] _myLocations = null;
_myLocation.AddressLine1 = txtEmployerAddress1.Text;
_myLocation.AddressLine2 = txtEmployerAddress2.Text;
// more properties set here
_myLocations[0] = _myLocation;
_myMERegistration.Locations = _myLocations;
_Results = _myRegistrationService.getMERegistration(_myMERegistration);
int length = ...;
WebReferenceMERegistration.Locations[] _myLocations =
new WebReferenceMERegistration.Locations[length];

Is Scala/Java not respecting w3 "excess dtd traffic" specs?

I'm new to Scala, so I may be off base on this, I want to know if the problem is my code. Given the Scala file httpparse, simplified to:
object Http {
import java.io.InputStream;
import java.net.URL;
def request(urlString:String): (Boolean, InputStream) =
try {
val url = new URL(urlString)
val body = url.openStream
(true, body)
}
catch {
case ex:Exception => (false, null)
}
}
object HTTPParse extends Application {
import scala.xml._;
import java.net._;
def fetchAndParseURL(URL:String) = {
val (true, body) = Http request(URL)
val xml = XML.load(body) // <-- Error happens here in .load() method
"True"
}
}
Which is run with (URL doesn't matter, this is a joke example):
scala> HTTPParse.fetchAndParseURL("http://stackoverflow.com")
The result invariably:
java.io.IOException: Server returned HTTP response code: 503 for URL: http://www.w3.org/TR/html4/strict.dtd
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1187)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:973)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.startEntity(XMLEnti...
I've seen the Stack Overflow thread on this with respect to Java, as well as the W3C's System Team Blog entry about not trying to access this DTD via the web. I've also isolated the error to the XML.load() method, which is a Scala library method as far as I can tell.
My Question: How can I fix this? Is this something that is a by product of my code (cribbed from Raphael Ferreira's post), a by product of something Java specific that I need to address as in the previous thread, or something that is Scala specific? Where is this call happening, and is it a bug or a feature? ("Is it me? It's her, right?")
I've bumped into the SAME issue, and I haven't found an elegant solution (I'm thinking into posting the question to the Scala mailing list) Meanwhile, I found a workaround: implement your own SAXParserFactoryImpl so you can set the f.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true); property. The good thing is it doesn't require any code change to the Scala code base (I agree that it should be fixed, though).
First I'm extending the default parser factory:
package mypackage;
public class MyXMLParserFactory extends SAXParserFactoryImpl {
public MyXMLParserFactory() throws SAXNotRecognizedException, SAXNotSupportedException, ParserConfigurationException {
super();
super.setFeature("http://xml.org/sax/features/validation", false);
super.setFeature("http://apache.org/xml/features/disallow-doctype-decl", false);
super.setFeature("http://apache.org/xml/features/nonvalidating/load-dtd-grammar", false);
super.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
}
}
Nothing special, I just want the chance to set the property.
(Note: that this is plain Java code, most probably you can write the same in Scala too)
And in your Scala code, you need to configure the JVM to use your new factory:
System.setProperty("javax.xml.parsers.SAXParserFactory", "mypackage.MyXMLParserFactory");
Then you can call XML.load without validation
Without addressing, for now, the problem, what do you expect to happen if the function request return false below?
def fetchAndParseURL(URL:String) = {
val (true, body) = Http request(URL)
What will happen is that an exception will be thrown. You could rewrite it this way, though:
def fetchAndParseURL(URL:String) = (Http request(URL)) match {
case (true, body) =>
val xml = XML.load(body)
"True"
case _ => "False"
}
Now, to fix the XML parsing problem, we'll disable DTD loading in the parser, as suggested by others:
def fetchAndParseURL(URL:String) = (Http request(URL)) match {
case (true, body) =>
val f = javax.xml.parsers.SAXParserFactory.newInstance()
f.setNamespaceAware(false)
f.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
val MyXML = XML.withSAXParser(f.newSAXParser())
val xml = MyXML.load(body)
"True"
case _ => "False"
}
Now, I put that MyXML stuff inside fetchAndParseURL just to keep the structure of the example as unchanged as possible. For actual use, I'd separate it in a top-level object, and make "parser" into a def instead of val, to avoid problems with mutable parsers:
import scala.xml.Elem
import scala.xml.factory.XMLLoader
import javax.xml.parsers.SAXParser
object MyXML extends XMLLoader[Elem] {
override def parser: SAXParser = {
val f = javax.xml.parsers.SAXParserFactory.newInstance()
f.setNamespaceAware(false)
f.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
f.newSAXParser()
}
}
Import the package it is defined in, and you are good to go.
This is a scala problem. Native Java has an option to disable loading the DTD:
f.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
There are no equivalent in scala.
If you somewhat want to fix it yourself, check scala/xml/parsing/FactoryAdapter.scala and put the line in
278 def loadXML(source: InputSource): Node = {
279 // create parser
280 val parser: SAXParser = try {
281 val f = SAXParserFactory.newInstance()
282 f.setNamespaceAware(false)
<-- insert here
283 f.newSAXParser()
284 } catch {
285 case e: Exception =>
286 Console.err.println("error: Unable to instantiate parser")
287 throw e
288 }
GClaramunt's solution worked wonders for me. My Scala conversion is as follows:
package mypackage
import org.xml.sax.{SAXNotRecognizedException, SAXNotSupportedException}
import com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl
import javax.xml.parsers.ParserConfigurationException
#throws(classOf[SAXNotRecognizedException])
#throws(classOf[SAXNotSupportedException])
#throws(classOf[ParserConfigurationException])
class MyXMLParserFactory extends SAXParserFactoryImpl() {
super.setFeature("http://xml.org/sax/features/validation", false)
super.setFeature("http://apache.org/xml/features/disallow-doctype-decl", false)
super.setFeature("http://apache.org/xml/features/nonvalidating/load-dtd-grammar", false)
super.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false)
}
As mentioned his the original post, it is necessary to place the following line in your code somewhere:
System.setProperty("javax.xml.parsers.SAXParserFactory", "mypackage.MyXMLParserFactory")
It works. After some detective work, the details as best I can figure them:
Trying to parse a developmental RESTful interface, I build the parser and get the above (rather, a similar) error. I try various parameters to change the XML output, but get the same error. I try to connect to an XML document I quickly whip up (cribbed stupidly from the interface itself) and get the same error. Then I try to connect to anything, just for kicks, and get the same (again, likely only similar) error.
I started questioning whether it was an error with the sources or the program, so I started searching around, and it looks like an ongoing issue- with many Google and SO hits on the same topic. This, unfortunately, made me focus on the upstream (language) aspects of the error, rather than troubleshoot more downstream at the sources themselves.
Fast forward and the parser suddenly works on the original XML output. I confirmed that there was some additional work has been done server side (just a crazy coincidence?). I don't have either earlier XML but suspect that it is related to the document identifiers being changed.
Now, the parser works fine on the RESTful interface, as well any well formatted XML I can throw at it. It also fails on all XHTML DTD's I've tried (e.g. www.w3.org). This is contrary to what #SeanReilly expects, but seems to jive with what the W3 states.
I'm still new to Scala, so can't determine if I have a special, or typical case. Nor can I be assured that this problem won't re-occur for me in another form down the line. It does seem that pulling XHTML will continue to cause this error unless one uses a solution similar to those suggested by #GClaramunt $ #J-16 SDiZ have used. I'm not really qualified to know if this is a problem with the language, or my implementation of a solution (likely the later)
For the immediate timeframe, I suspect that the best solution would've been for me to ensure that it was possible to parse that XML source-- rather than see that other's have had the same error and assume there was a functional problem with the language.
Hope this helps others.
There are two problems with what you are trying to do:
Scala's xml parser is trying to physically retrieve the DTD when it shouldn't. J-16 SDiZ seems to have some advice for this problem.
The Stack overflow page you are trying to parse isn't XML. It's Html4 strict.
The second problem isn't really possible to fix in your scala code. Even once you get around the dtd problem, you'll find that the source just isn't valid XML (empty tags aren't closed properly, for example).
You have to either parse the page with something besides an XML parser, or investigate using a utility like tidy to convert the html to xml.
My knowledge of Scala is pretty poor, but couldn't you use ConstructingParser instead?
val xml = new java.io.File("xmlWithDtd.xml")
val parser = scala.xml.parsing.ConstructingParser.fromFile(xml, true)
val doc = parser.document()
println(doc.docElem)
For scala 2.7.7 I managed to do this with scala.xml.parsing.XhtmlParser
Setting Xerces switches only works if you are using Xerces. An entity resolver works for any JAXP parser.
There are more generalized entity resolvers out there, but this implementation does the trick when all I'm trying to do is parse valid XHTML.
http://code.google.com/p/java-xhtml-cache-dtds-entityresolver/
Shows how trivial it is to cache the DTDs and forgo the network traffic.
In any case, this is how I fix it. I always forget. I always get the error. I always go fetch this entity resolver. Then I'm back in business.

Categories

Resources