Develop Reference

Why am I getting StringIndexOutOfBoundsException error in this substring method?

protected Void doInBackground(Void... voids) {
...
ABV = elem.select("td > span.muted").text();
Log.d("myLOG_ABV", ABV);
Log.d("myLOG_ABVlength", String.valueOf(ABV.length()));
/*String temp_ABV = ABV.substring(ABV.length()-6, ABV.length());*/
... }
Result
D/myLOG_ABV: Russian River Brewing Company American Wild Ale | 7.50%
D/myLOG_ABVlength: 55
and then, I cancled the annotation code.
...
ABV = elem.select("td > span.muted").text();
Log.d("myLOG_ABV", ABV);
Log.d("myLOG_ABVlength", String.valueOf(ABV.length()));
***String temp_ABV = ABV.substring(ABV.length()-6, ABV.length());***
...
Result
Caused by: java.lang.StringIndexOutOfBoundsException: length=0;
index=-6
Why am I getting StringIndexOutOfBoundsException error in this substring method?
I got the result that 'ABVlength : 55' in my code with annotation.
But after cancellation of annotation, I got StringIndexOutOfBoundsException.
Seriously, I am fighting with this code for 7hours 30minutes.

So we have:
***String temp_ABV = ABV.substring(ABV.length()-6, ABV.length());***
Caused by: java.lang.StringIndexOutOfBoundsException: length=0; index=-6
How can you get an index of -6 in this situation? ABV must have zero length, i.e. "".
It may be the case that ABV should have at least six characters, but that indicates a bug elsewhere.
What to do?
First and foremost, validate your inputs.
Somewhere you would normally want something like:
if (abv.length() < 6) {
throw new IllegalArgumentException("...");
}
Next find where that value is coming from and fix the bug.
Looks like you're parsing some kind of external document. In this case you may want the handling to be more lenient:
int abvLen = abv.length();
int abvEndIndex = abvLen - 6;
if (abvEndIndex < 0) {
log.error("ABV length less than 6", abv);
abvEndIndex = 0;
}
String abvEnd = abv.substring(abvEndIndex, abvLen);

How can I remove the output layer of pre-trained model with TensorFlow java api?

I have the pre-trained model like Inception-v3. I want to remove the output layer and use it in image cognition. Here is the example given by tensorflow:
Just like the python framework Keras, it has a method like model.layers.pop(). I tried do it with tensorflow java api. First I tried to use dl4j, but when I imported the keras model, I got an error like this:
2017-06-15 21:15:43 INFO KerasInceptionV3Net:52 - Importing Inception model from data/inception-model.json
2017-06-15 21:15:43 INFO KerasInceptionV3Net:53 - Importing Weights model from data/inception_v3_complete
Exception in thread "main" java.lang.RuntimeException: Unknown exception.
at org.bytedeco.javacpp.hdf5$H5File.allocate(Native Method)
at org.bytedeco.javacpp.hdf5$H5File.<init>(hdf5.java:12713)
at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.<init>(Hdf5Archive.java:61)
at org.deeplearning4j.nn.modelimport.keras.KerasModel$ModelBuilder.weightsHdf5Filename(KerasModel.java:603)
at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasModelAndWeights(KerasModelImport.java:176)
at edu.usc.irds.dl.dl4j.examples.KerasInceptionV3Net.<init>(KerasInceptionV3Net.java:55)
at edu.usc.irds.dl.dl4j.examples.KerasInceptionV3Net.main(KerasInceptionV3Net.java:108)
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 0:
#000: C:\autotest\HDF5110ReleaseRWDITAR\src\H5F.c line 579 in H5Fopen(): unable to open file
major: File accessibilty
minor: Unable to open file
#001: C:\autotest\HDF5110ReleaseRWDITAR\src\H5Fint.c line 1100 in H5F_open(): unable to open file: time = Thu Jun 15 21:15:44 2017,name = 'data/inception_v3_complete', tent_flags = 0
major: File accessibilty
minor: Unable to open file
#002: C:\autotest\HDF5110ReleaseRWDITAR\src\H5FD.c line 812 in H5FD_open(): open failed
major: Virtual File Layer
minor: Unable to initialize object
#003: C:\autotest\HDF5110ReleaseRWDITAR\src\H5FDsec2.c line 348 in H5FD_sec2_open(): unable to open file: name = 'data/inception_v3_complete', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
major: File accessibilty
minor: Unable to open file
So I went back to tensorflow. I'm going to modify the model in keras and convert the model to tensor. Here is my conversion script:
input_fld = './'
output_node_names_of_input_network = ["pred0"]
write_graph_def_ascii_flag = True
output_node_names_of_final_network = 'output_node'
output_graph_name = 'test2.pb'
from keras.models import load_model
import tensorflow as tf
import os
import os.path as osp
from keras.applications.inception_v3 import InceptionV3
from keras.applications.vgg16 import VGG16
from keras.models import Sequential
from keras.layers.core import Flatten, Dense, Dropout
from keras.layers.convolutional import Convolution2D, MaxPooling2D, ZeroPadding2D
from keras.optimizers import SGD
output_fld = input_fld + 'tensorflow_model/'
if not os.path.isdir(output_fld):
os.mkdir(output_fld)
net_model = InceptionV3(weights='imagenet', include_top=True)
num_output = len(output_node_names_of_input_network)
pred = [None]*num_output
pred_node_names = [None]*num_output
for i in range(num_output):
pred_node_names[i] = output_node_names_of_final_network+str(i)
pred[i] = tf.identity(net_model.output[i], name=pred_node_names[i])
print('output nodes names are: ', pred_node_names)
from keras import backend as K
sess = K.get_session()
if write_graph_def_ascii_flag:
f = 'only_the_graph_def.pb.ascii'
tf.train.write_graph(sess.graph.as_graph_def(), output_fld, f, as_text=True)
print('saved the graph definition in ascii format at: ', osp.join(output_fld, f))
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io
constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), pred_node_names)
graph_io.write_graph(constant_graph, output_fld, output_graph_name, as_t ext=False)
print('saved the constant graph (ready for inference) at: ', osp.join(output_fld, output_graph_name))
I got the model as .pb file, but when I put it into the tensor example, The LabelImage example, I got this error:
Exception in thread "main" java.lang.IllegalArgumentException: You must feed a value for placeholder tensor 'batch_normalization_1/keras_learning_phase' with dtype bool
[[Node: batch_normalization_1/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
at org.tensorflow.Session.run(Native Method)
at org.tensorflow.Session.access$100(Session.java:48)
at org.tensorflow.Session$Runner.runHelper(Session.java:285)
at org.tensorflow.Session$Runner.run(Session.java:235)
at com.dlut.cmh.sheng.LabelImage.executeInceptionGraph(LabelImage.java:98)
at com.dlut.cmh.sheng.LabelImage.main(LabelImage.java:51)
I don't know how to solve this. Can anyone help me? Or you have another way to do this?

The error message you get from the TensorFlow Java API:
Exception in thread "main" java.lang.IllegalArgumentException: You must feed a value for placeholder tensor 'batch_normalization_1/keras_learning_phase' with dtype bool
[[Node: batch_normalization_1/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
suggests that the model is constructed in a way that requires you to feed a boolean value for the tensor named batch_normalization_1/keras_learning_phase.
So, you'd have to include that in your call to run by changing:
try (Session s = new Session(g);
Tensor result = s.runner().feed("input",image).fetch("output").run().get(0)) {
to something like:
try (Session s = new Session(g);
Tensor learning_phase = Tensor.create(false);
Tensor result = s.runner().feed("input", image).feed("batch_normalization_1/keras_learning_phase", learning_phase).fetch("output").run().get(0)) {
The names of nodes you feed and fetch depend on the model, so it's possible that the names of the 'input' and 'output' nodes are different as well.
You might also want to consider using the TensorFlow SavedModel format (see also https://github.com/tensorflow/serving/issues/310#issuecomment-297015251)
Hope that helps

How to pass java.util.Date to Java library with help of php-java bridge

Im using javabridge to connect php to jasper reports and Im trying to pass two parameters but I get warnings and errors
Warning: Unchecked exception detected: [[o:Response$UndeclaredThrowableErrorMarker]:"FATAL: Undeclared java.lang.RuntimeException detected. java.lang.Exception: CreateInstance failed: new java.util.Date((o:String)[o:String]). Cause: java.lang.IllegalArgumentException VM: 1.7.0_79#http://java.oracle.com/" at: #-10 java.util.Date.parse(Unknown Source) #-9 java.util.Date.<init>(Unknown Source) #-8 sun.reflect.GeneratedConstructorAccessor57.newInstance(Unknown Source) #-7 sun.reflect.DelegatingConstructorAccessor[...]/java/Java.inc(361): java_Arg->getResult(false) #2 http://localhost:8080/JavaBridgeTemplate/java/Java.inc(364): java_Client->getWrappedResult(false) #3 http://localhost:8080/JavaBridgeTemplate/java/Java.inc(536): java_Client->getInternalResult() #4 http://localhost:8080/JavaBridgeTemplate/java/Java.inc(1930): java_Client->createObject('java.util.Date', Array) #5 C:\wamp\www\advanced\backend\javabridge\generate.php(49): Java->Java('java.util.Date', '12/Feb/16') #6 {main}] in http://localhost:8080/JavaBridgeTemplate/java/Java.inc on line 202
Fatal error: Uncaught [[o:Exception]:"java.lang.Exception: Invoke failed: [[c:JasperFillManager]]->fillReport((o:JasperReport)[o:JasperReport], (i:Map)[o:HashMap], (i:Connection)[o:Connection]). Cause: net.sf.jasperreports.engine.JRException: Incompatible php.java.bridge.Response$UndeclaredThrowableErrorMarker value assigned to parameter FInicio in the Reubicados dataset. VM: 1.7.0_79#http://java.oracle.com/" at: #-16 net.sf.jasperreports.engine.fill.JRFillDataset.setParameter(JRFillDataset.java:903) #-15 net.sf.jasperreports.engine.fill.JRFillDataset.setFillParameterValues(JRFillDataset.java:642) #-14 net.sf.jasperreports.engine.fill.JRFillDataset.setParameterValues(JRFillDataset.java:585) #-13 net.sf.jasperreports.engine.fill.JRBaseFiller.setParameters(JRBaseFiller.java:1280) #-12 net.sf.jasperreports.engine.fill.JRBaseFiller.fill(JRBaseFiller.java:901) #-11 net.sf.jasperreports.engine.fill.JRBaseFiller.fill(JRBaseFiller.java:845) #-10 net.sf.jasperreports.engine.fill.JRFiller.fillReport(JRFiller.java:58) #-9 net.sf.jas in http://localhost:8080/JavaBridgeTemplate/java/Java.inc on line 195
The problem is when it tries to create java.util.Date instance. Here´s php file:
<?php
require_once("http://localhost:8080/JavaBridgeTemplate/java/Java.inc");
try {
$Param1 = date('d/M/y', strtotime($_POST['FInicio']));
$Param2 = date('d/M/y', strtotime($_POST['FFin']));
$jasperxml = new java("net.sf.jasperreports.engine.xml.JRXmlLoader");
$jasperDesign = $jasperxml->load(realpath("Reubicados.jrxml"));
$query = new java("net.sf.jasperreports.engine.design.JRDesignQuery");
$jasperDesign->setQuery($query);
$compileManager = new JavaClass("net.sf.jasperreports.engine.JasperCompileManager");
$report = $compileManager->compileReport($jasperDesign); } catch (JavaException $ex) {
echo $ex; }
$fillManager = new JavaClass("net.sf.jasperreports.engine.JasperFillManager"); //aqui se pasan los parametros (Fecha Inicio y Fecha Fin)
$params = new Java("java.util.HashMap");
$date=new Java('java.util.Date',$Param1);
$date1=new Java('java.util.Date',$Param2);
$params->put("FInicio",$date);
$params->put("FFin",$date1);
$class = new JavaClass("java.lang.Class"); $class->forName("com.mysql.jdbc.Driver"); $driverManager = new JavaClass("java.sql.DriverManager");
//db username and password
$conn = $driverManager->getConnection("jdbc:mysql://localhost/viajestrafico?zeroDateTimeBehavior=convertToNull", "root", "root"); $jasperPrint = $fillManager->fillReport($report, $params, $conn);
$exporter = new java("net.sf.jasperreports.engine.JRExporter");
And sql query in ireports:
SELECT
ayudante_situacion_laboral.`FechaInicio` AS ayudante_situacion_laboral_FechaInicio,
ayudante_situacion_laboral.`FechaFin` AS ayudante_situacion_laboral_FechaFin,
ayudante_situacion_laboral.`Cant_Horas` AS ayudante_situacion_laboral_Cant_Horas,
ayudante_situacion_laboral.`Descripcion` AS ayudante_situacion_laboral_Descripcion,
ayudante.`Registro` AS ayudante_Registro,
ayudante.`Nombre` AS ayudante_Nombre,
situacion_laboral.`Estado` AS situacion_laboral_Estado
FROM
`ayudante` ayudante INNER JOIN `ayudante_situacion_laboral` ayudante_situacion_laboral ON ayudante.`Ayudante_ID` = ayudante_situacion_laboral.`AyudanteAyudante_ID`
INNER JOIN `situacion_laboral` situacion_laboral ON ayudante_situacion_laboral.`Situacion_LaboralSitL_ID` = situacion_laboral.`SitL_ID`
WHERE
situacion_laboral.Estado = 'Reubicado' and $P{FInicio}<= ayudante_situacion_laboral.`FechaInicio` and $P{FFin}>=ayudante_situacion_laboral.`FechaFin`
UNION
SELECT
chofer_situacion_laboral.`FechaInicio` AS chofer_situacion_laboral_FechaInicio,
chofer_situacion_laboral.`FechaFin` AS chofer_situacion_laboral_FechaFin,
chofer_situacion_laboral.`Cant_Horas` AS chofer_situacion_laboral_Cant_Horas,
chofer_situacion_laboral.`Descripcion` AS chofer_situacion_laboral_Descripcion,
chofer.`Registro` AS chofer_Registro,
chofer.`Nombre` AS chofer_Nombre,
situacion_laboral.`Estado` AS situacion_laboral_Estado
FROM
`chofer` chofer INNER JOIN `chofer_situacion_laboral` chofer_situacion_laboral ON chofer.`Chofer_ID` = chofer_situacion_laboral.`ChoferChofer_ID`
INNER JOIN `situacion_laboral` situacion_laboral ON chofer_situacion_laboral.`Situacion_LaboralSitL_ID` = situacion_laboral.`SitL_ID`
WHERE
(situacion_laboral.Estado = 'Reubicado' and $P{FInicio}<= chofer_situacion_laboral.`FechaInicio` and $P{FFin}>=chofer_situacion_laboral.`FechaFin`)

Looking to your error message, the java.util.Date cannot be created with '12/Feb/16':
java_Client->createObject('java.util.Date', Array)
#5 C:\wamp\www\advanced\backend\javabridge\generate.php(49):
Java->Java('java.util.Date', '12/Feb/16') #6 {main}] in
If you want to instanciate a java date object, use the java.util.Date(long date) constructor which accepts a timestamp expressed in milliseconds :
<?php
$startDate = $_POST['FInicio']; // better to filter this :)
$Param1 = strtotime($startDate) * 1000; // to get milliseconds
if ($Param1 == 0) {
throw new Exception("FInicio date parameter could not be parsed");
}
// ...
$javaDate = new Java('java.util.Date',$Param1); // This is a java date
Your $date parameter should now be a valid java.util.Date object.
You can test it :
$simpleDateFormat = new Java("java.text.SimpleDateFormat", 'yyyy-MM-dd');
echo $simpleDateFormat->format($javaDate);
// should print your date in Y-m-d format
Alternatively, you can parse the date in Java through the java.text.SimpleDateFormatter object:
$date = '2016-12-21';
$simpleDateFormat = new Java("java.text.SimpleDateFormat", 'yyyy-MM-dd');
$javaDate = $simpleDateFormat->parse($date); // This is a Java date
Both approaches works...
JasperReport issue with date
Not exactly linked to your question, but if you want to use your Date as a query parameter you should use the java.sql.Date(long date) object instead of java.util.Date... Here's a little dirty snippet to summarize changes:
// php
$sqlDate = new Java('java.sql.Date', strtotime($_POST['FInicio']) * 1000));
$params->put('FInicio', $sqlDate);
// in your report header (.jrxml):
<parameter name="FInicio" class="java.sql.Date">
// in your report query (.jrxml):
$P{FInicio} <= chofer_situacion_laboral.`FechaInicio`
You can also have a look to the soluble-japha javabridge client (refactored Java.inc client), the syntax differs a bit but there's a documentation about dates that might reveal useful.

It was simple, just a problem I had from the beginning in php code.
$query = new java("net.sf.jasperreports.engine.design.JRDesignQuery");
$jasperDesign->setQuery($query);
This code was preventing xrml query from executing, because it was creating a query object empty.

SoapUI: response with CDATA is giving null. searched various article but no luck

Hello I have below SOAP reponse.
`
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<soap:Body>
<GetWeatherResponse xmlns="http://www.webserviceX.NET">
<GetWeatherResult><![CDATA[<?xml version="1.0" encoding="utf-16"?>
<CurrentWeather>
<Location>Cape Town, Cape Town International Airport, South Africa (FACT) 33-59S 018-36E 0M</Location>
<Time>Jun 04, 2016 - 05:00 AM EDT / 2016.06.04 0900 UTC</Time>
<Wind> from the SE (130 degrees) at 21 MPH (18 KT):0</Wind>
<Visibility> greater than 7 mile(s):0</Visibility>
<SkyConditions> mostly clear</SkyConditions>
<Temperature> 60 F (16 C)</Temperature>
<DewPoint> 44 F (7 C)</DewPoint>
<RelativeHumidity> 55%</RelativeHumidity>
<Pressure> 30.39 in. Hg (1029 hPa)</Pressure>
<Status>Success</Status>
</CurrentWeather>]]></GetWeatherResult>
</GetWeatherResponse>
</soap:Body>
</soap:Envelope>"`
for the following request: http://www.webservicex.net/globalweather.asmx.
I want to read XML Data in above response using script. but it is giving null.
I have tried script as below
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def holder = groovyUtils.getXmlHolder(messageExchange.responseContent)
holder.namespaces["ns"] = "http://www.webserviceX.NET/" def
weatherinfo= holder.getNodeValue("//ns:GetWeatherResult/text()")
log.info weatherinfo
bit Instead of getting above reponse I am getting NULL. I have read the SOAPUI documentation on CDATA but its not working.

I got answer. It was just one extra slash in namespace that was giving me null.
Both the below scripts are working now.
> def respXmlHolder = new
> com.eviware.soapui.support.XmlHolder(messageExchange.getResponseContentAsXml())
> respXmlHolder.namespaces["ns1"] ="http://www.webserviceX.NET" def
> CDATAXml= respXmlHolder.getNodeValue("//ns1:GetWeatherResult/text()")
> log.info CDATAXml
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def holder = groovyUtils.getXmlHolder(messageExchange.responseContent)
holder.namespaces["ns"] = "http://www.webserviceX.NET"
def weatherinfo= holder.getNodeValue("//ns:GetWeatherResult/text()")
log.info weatherinfo

R XLSX java.lang.OutOfMemoryError: GC overhead limit exceeded

I wrote this r function that uses the xlsx package to write one or more data frames out to a .xlsx file. When given the same input (3 data frames: 6185 obs of 23 variables, 4 of 17 and 2 of 3) it throws an error most of the time, but not all of the time.
Can anyone tell me how to optimize my code, get the same outcome more elegantly or continue in spite of the error?
Here is the console output:
Running: WriteToFile()
WriteToFile:1
WriteToFile:4
Error in: Example Report
java.lang.OutOfMemoryError: GC overhead limit exceededReturning from: Example Report
and here is the function:
WriteToFile <- function() {
# Write a data frame(s) out to .xlsx file
if(debug > 2) {message("WriteToFile:1")}
# If the file already exists today, then delete it
if(file.exists(paste0(report.name, '(', today(), ')', ".xlsx"))) {
if(debug > 2) {message("WriteToFile:2")}
writeLines(paste0("File '", report.name, '(', today(), ')', ".xlsx", "' already exists and will be replaced."))
flush.console()
file.remove(paste0(report.name, '(', today(), ')', ".xlsx"))
}
# Get tab names that were generated in ProcessOutputs()
tabs <- ls(pattern=paste0("\\.data$"), name=.GlobalEnv)
# If no tabs, send fail-mail
if(length(tabs) == 0) {
if(debug > 2) {message("WriteToFile:3")}
SendPerlMail(fail.mail=TRUE, fail.msg="No data for tabs in WriteToFile()")
return(-1)
}
# Write first tab to first sheet, then write any remaining tabs to additional sheets
else {
if(debug > 2) {message("WriteToFile:4")}
write.xlsx2(x=get(tabs[1]), file=paste0(report.name, ' (', today(), ')', ".xlsx"),
sheetName=substr(x=tabs[1], start=0, stop=nchar(tabs[1])-5), col.names=TRUE, row.names=FALSE, append=FALSE)
if(length(tabs) > 1) {
if(debug > 2) {message("WriteToFile:5")}
for(t in mget(tabs[2:length(tabs)])) {
write.xlsx2(x=t, file=paste0(report.name, ' (', today(), ')', ".xlsx"),
sheetName=substr(x=substitute(t), start=0, stop=nchar(t)-5), col.names=TRUE, row.names=FALSE, append=TRUE)
}
}
}
email.file <<- paste0(getwd(), "/", report.name, ' (', today(), ')', ".xlsx")
return(1)
}
The outside variables referenced in the function are:
options(java.parameters = "-Xms2048m -Xmx4096m", "-XX:-UseGCOverheadLimit") (I set these Java parameters because they worked for someone in another post with a the same GC error)
debug = 3
report.name = "Example Report"
Example Query 1.data = data frame of 6185x23
Example Query 2.data = data frame of 4x17
Example Query 3.data = data frame of 2x3
Sometimes it outputs WriteToFile:5 to the console and sometimes it succeeds without throwing the error. Any help is greatly appreciated--I've been trying to figure out why this doesn't work reliably for a few hours now.