I have to call example.py (having another dependencies windrose, matlab, numpy ) python file from java to create wind graph but it unable to import any dependency while calling from java but its runs fine independently using python, I got
Error: no module name windrose
but this module is already in folder please let me know how to do it
windroseGraph ie = new windroseGraph();
ie.execfile("E:\\example.py");
PyInstance hello = ie.createClass("test", "None");
hello.invoke("run");
python file example.py
from windrose import WindroseAxes
from numpy.random import random
from numpy import arange
from matplotlib import pyplot as plt
import matplotlib.cm as cm
class test:
def new_axes():
fig = plt.figure(figsize=(8, 8), dpi=80, facecolor='w', edgecolor='w')
rect = [0.1, 0.1, 0.8, 0.8]
ax = WindroseAxes(fig, rect, axisbg='w')
fig.add_axes(ax)
return ax
def set_legend(ax):
l = ax.legend(borderaxespad=-0.10)
plt.setp(l.get_texts(), fontsize=8)
def run(self,wd,ws):
ax = new_axes()
ax.bar(wd, ws, normed=True, opening=0.8, edgecolor='white')
set_legend(ax)
##print ax._info
plt.show()
example.py is python file having function with parameter and generate wind graph using windrose.py module (python lib online available )
Related
I have a Java application which has a functionality to take a screenshot. It does it by running Powershell script:
Add-Type -AssemblyName System.Windows.Forms,System.Drawing
$screens = [Windows.Forms.Screen]::AllScreens
$top = ($screens.Bounds.Top | Measure-Object -Minimum).Minimum
$left = ($screens.Bounds.Left | Measure-Object -Minimum).Minimum
$width = ($screens.Bounds.Right | Measure-Object -Maximum).Maximum
$height = ($screens.Bounds.Bottom | Measure-Object -Maximum).Maximum
$bounds = [Drawing.Rectangle]::FromLTRB($left, $top, $width, $height)
$bmp = New-Object System.Drawing.Bitmap ([int]$bounds.width), ([int]$bounds.height)
$graphics = [Drawing.Graphics]::FromImage($bmp)
$graphics.CopyFromScreen($bounds.Location, [Drawing.Point]::Empty, $bounds.size)
$memStream = New-Object System.IO.MemoryStream
$bmp.Save($memStream, [Drawing.Imaging.ImageFormat]::Jpeg)
Write-Host $memStream.ToArray()
$graphics.Dispose()
$bmp.Dispose()
$memStream.Dispose()
Java application listens to the output of it and does some operations on it. The problem is that sometimes Write-Host $memStream.ToArray() takes too much time (Sometimes in 2 minutes, sometimes 3, or even 5). I'm not familiar with Powershell. Is there any analog of Write-Host which is faster? Or maybe I can take a screenshot using some other functionality faster? Thanks
You stated a solution using other functionality would be acceptable, so why not perform the screen capture directly with the Java application instead? Java is fully capable of this natively:
import java.awt.Robot;
import java.awt.Rectangle;
import java.awt.image.BufferedImage;
import java.awt.GraphicsDevice;
import java.awt.GraphicsEnvironment;
import java.awt.AWTException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
// Set up Robot and other vars
Robot robot = new Robot();
String imgFormat = "jpg";
BufferedImage screenBuffer;
Rectangle screenBounds;
// Enumerate all screens
GraphicsEnvironment graphEnv = GraphicsEnvironment.getLocalGraphicsEnvironment();
GraphicsDevice[] screens = graphEnv.getScreenDevices();
// Variables only used for generating filename
String fnameFormat = "%s-%s-screencap.%s";
String dtNowString = new SimpleDateFormat("yyyyMMddHHmmss").format(Calendar.getInstance().getTime());
String filename = String.format(fnameFormat, dtNowString, "all", imgFormat);
Rectangle allScreenBounds = new Rectangle();
int num = 0;
for(GraphicsDevice screen : screens) {
screenBounds = screen.getDefaultConfiguration().getBounds();
allScreenBounds.x = Math.min(allScreenBounds.x, screenBounds.x);
allScreenBounds.y = Math.min(allScreenBounds.y, screenBounds.y);
// Make sure we only add extra pixels to the total width and height, subtracting overlapping dimensions
// Does not take into account non-continuous display area, normally impossible on Windows
allScreenBounds.width += Math.abs(allScreenBounds.width - (screenBounds.width + screenBounds.x));
allScreenBounds.height += Math.abs(allScreenBounds.height - (screenBounds.height + screenBounds.y));
System.out.println(String.format("Display %d: X=%d, Y=%d, Height=%d, Width=%d", num++, screenBounds.x, screenBounds.y, screenBounds.height, screenBounds.width));
}
System.out.println(String.format("Screen Area: X=%d, Y=%d, Height=%d, Width=%d", allScreenBounds.x, allScreenBounds.y, allScreenBounds.height, allScreenBounds.width));
screenBuffer = robot.createScreenCapture(allScreenBounds);
// Save the screencap to file
ImageIO.write(screenBuffer, imgFormat, new File(filename));
There is file-writing code there for testing but if this is performed by your application you can remove the filename variables, import javax.imageio.ImageIO, and the ImageIO.write call as you'll have the screenshot data in screenBuffer instead.
I have the pre-trained model like Inception-v3. I want to remove the output layer and use it in image cognition. Here is the example given by tensorflow:
Just like the python framework Keras, it has a method like model.layers.pop(). I tried do it with tensorflow java api. First I tried to use dl4j, but when I imported the keras model, I got an error like this:
2017-06-15 21:15:43 INFO KerasInceptionV3Net:52 - Importing Inception model from data/inception-model.json
2017-06-15 21:15:43 INFO KerasInceptionV3Net:53 - Importing Weights model from data/inception_v3_complete
Exception in thread "main" java.lang.RuntimeException: Unknown exception.
at org.bytedeco.javacpp.hdf5$H5File.allocate(Native Method)
at org.bytedeco.javacpp.hdf5$H5File.<init>(hdf5.java:12713)
at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.<init>(Hdf5Archive.java:61)
at org.deeplearning4j.nn.modelimport.keras.KerasModel$ModelBuilder.weightsHdf5Filename(KerasModel.java:603)
at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasModelAndWeights(KerasModelImport.java:176)
at edu.usc.irds.dl.dl4j.examples.KerasInceptionV3Net.<init>(KerasInceptionV3Net.java:55)
at edu.usc.irds.dl.dl4j.examples.KerasInceptionV3Net.main(KerasInceptionV3Net.java:108)
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 0:
#000: C:\autotest\HDF5110ReleaseRWDITAR\src\H5F.c line 579 in H5Fopen(): unable to open file
major: File accessibilty
minor: Unable to open file
#001: C:\autotest\HDF5110ReleaseRWDITAR\src\H5Fint.c line 1100 in H5F_open(): unable to open file: time = Thu Jun 15 21:15:44 2017,name = 'data/inception_v3_complete', tent_flags = 0
major: File accessibilty
minor: Unable to open file
#002: C:\autotest\HDF5110ReleaseRWDITAR\src\H5FD.c line 812 in H5FD_open(): open failed
major: Virtual File Layer
minor: Unable to initialize object
#003: C:\autotest\HDF5110ReleaseRWDITAR\src\H5FDsec2.c line 348 in H5FD_sec2_open(): unable to open file: name = 'data/inception_v3_complete', errno = 2, error message = 'No such file or directory', flags = 0, o_flags = 0
major: File accessibilty
minor: Unable to open file
So I went back to tensorflow. I'm going to modify the model in keras and convert the model to tensor. Here is my conversion script:
input_fld = './'
output_node_names_of_input_network = ["pred0"]
write_graph_def_ascii_flag = True
output_node_names_of_final_network = 'output_node'
output_graph_name = 'test2.pb'
from keras.models import load_model
import tensorflow as tf
import os
import os.path as osp
from keras.applications.inception_v3 import InceptionV3
from keras.applications.vgg16 import VGG16
from keras.models import Sequential
from keras.layers.core import Flatten, Dense, Dropout
from keras.layers.convolutional import Convolution2D, MaxPooling2D, ZeroPadding2D
from keras.optimizers import SGD
output_fld = input_fld + 'tensorflow_model/'
if not os.path.isdir(output_fld):
os.mkdir(output_fld)
net_model = InceptionV3(weights='imagenet', include_top=True)
num_output = len(output_node_names_of_input_network)
pred = [None]*num_output
pred_node_names = [None]*num_output
for i in range(num_output):
pred_node_names[i] = output_node_names_of_final_network+str(i)
pred[i] = tf.identity(net_model.output[i], name=pred_node_names[i])
print('output nodes names are: ', pred_node_names)
from keras import backend as K
sess = K.get_session()
if write_graph_def_ascii_flag:
f = 'only_the_graph_def.pb.ascii'
tf.train.write_graph(sess.graph.as_graph_def(), output_fld, f, as_text=True)
print('saved the graph definition in ascii format at: ', osp.join(output_fld, f))
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io
constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), pred_node_names)
graph_io.write_graph(constant_graph, output_fld, output_graph_name, as_t ext=False)
print('saved the constant graph (ready for inference) at: ', osp.join(output_fld, output_graph_name))
I got the model as .pb file, but when I put it into the tensor example, The LabelImage example, I got this error:
Exception in thread "main" java.lang.IllegalArgumentException: You must feed a value for placeholder tensor 'batch_normalization_1/keras_learning_phase' with dtype bool
[[Node: batch_normalization_1/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
at org.tensorflow.Session.run(Native Method)
at org.tensorflow.Session.access$100(Session.java:48)
at org.tensorflow.Session$Runner.runHelper(Session.java:285)
at org.tensorflow.Session$Runner.run(Session.java:235)
at com.dlut.cmh.sheng.LabelImage.executeInceptionGraph(LabelImage.java:98)
at com.dlut.cmh.sheng.LabelImage.main(LabelImage.java:51)
I don't know how to solve this. Can anyone help me? Or you have another way to do this?
The error message you get from the TensorFlow Java API:
Exception in thread "main" java.lang.IllegalArgumentException: You must feed a value for placeholder tensor 'batch_normalization_1/keras_learning_phase' with dtype bool
[[Node: batch_normalization_1/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=<unknown>, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
suggests that the model is constructed in a way that requires you to feed a boolean value for the tensor named batch_normalization_1/keras_learning_phase.
So, you'd have to include that in your call to run by changing:
try (Session s = new Session(g);
Tensor result = s.runner().feed("input",image).fetch("output").run().get(0)) {
to something like:
try (Session s = new Session(g);
Tensor learning_phase = Tensor.create(false);
Tensor result = s.runner().feed("input", image).feed("batch_normalization_1/keras_learning_phase", learning_phase).fetch("output").run().get(0)) {
The names of nodes you feed and fetch depend on the model, so it's possible that the names of the 'input' and 'output' nodes are different as well.
You might also want to consider using the TensorFlow SavedModel format (see also https://github.com/tensorflow/serving/issues/310#issuecomment-297015251)
Hope that helps
In my project, I will use h2o's machine learning algorithm. While I don't load the train date.
I use the folloing ways.
var f = FileUtils.getFile("D:\\from_2017_2_13\\untitled2\\src\\main\\resources\\extdata\\iris_wheader.csv")
println(11111)
var frame = FrameUtils.parseFrame(Key.make("iris_weather.hex"),f)
println(22222)
The 11111 was output, then the program will being runing, and not stopping
11111
other way
var f = FileUtils.getFile("D:\\from_2017_2_13\\untitled2\\src\\main\\resources\\extdata\\iris_wheader.csv")
val parserSetup = H2OFrame.defaultParserSetup()
parserSetup.setSeparator(',').setCheckHeader(ParseSetup.HAS_HEADER).setNumberColumns(5)
val f3 = new H2OFrame(parserSetup, f)
f3
the error
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 65535
at water.DKV.get(DKV.java:202)
at water.DKV.get(DKV.java:175)
at water.parser.ParseSetup.createHexName(ParseSetup.java:594)
at water.fvec.H2OFrame.<init>(H2OFrame.scala:56)
at water.fvec.H2OFrame.<init>(H2OFrame.scala:84)
To load data into Scala as H2O Frame you can do the following:
import org.apache.spark.h2o._
import water.support.SparkContextSupport.addFiles
import org.apache.spark.SparkFiles
import java.io.File
val hc = H2OContext.getOrCreate(sc)
addFiles(sc, "/Users/avkashchauhan/smalldata/iris/iris.csv")
val irisData = new H2OFrame(new File(SparkFiles.get("iris.csv")))
Once data is loaded you can see the data frame as below:
scala> irisData
res1: water.fvec.H2OFrame =
Frame key: iris.hex
cols: 5
rows: 150
chunks: 1
size: 2454
Once you have ingested the data frame you can build model with it. If you are looking for a sample of using H2O library in Scala you can look for this blog for full end to end Scala based deep learning sample in H2O.
I am trying to compile sample Spark scala file through sbt and have built maven project in Eclipse IDE
Image
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
object simpleSpark {
def main(args : Arrayt[String]){
val logfile = "C:\\spark-1.6.1-bin-hadoop2.6\spark-1.6.1-bin-hadoop2.6\README.md"
val conf = new SparkConf().setAppName("Simple Application").setMaster("local[2]").set("spark.executor.memory", "1g")
val sc = new SparkContext(conf)
val logData = sc.textFile(logFile, 2).cache()
val numHadoops = logData.filter(line => line.contains("Hadoop")).count()
val numSparks = logData.filer(line => line.contains("Spark")).count()
println("Lines with Hadoop: %s, Lines with Spark: %s".format(numHadoops, numHadoops))
}
}
The error says you have illegal start of expression here set("spark.executor.memory",) . Are you sure you set spark.executor.memory correctly in actual code ?
If yes , can you show what you wrote is .sbt file ?
Here's a [python code][1] that I would like to know if can also be used for GAE Java (when code is migrated). So the question is, is the python code below something that can converted to Java without any python "dependencies" that Java can't have:
# stdlib
from collections import defaultdict
from datetime import datetime, timedelta
import os
import time
# 3p
import simplejson as json
# google api
from google.appengine.api import app_identity, logservice, memcache, taskqueue
from google.appengine.ext.db import stats as db_stats
# framework
import webapp2
class DatadogStats(webapp2.RequestHandler):
def get(self):
api_key = self.request.get('api_key')
if api_key != os.environ.get('DATADOG_API_KEY'):
self.abort(403)
FLAVORS = ['requests', 'services', 'all']
flavor = self.request.get('flavor')
if flavor not in FLAVORS:
self.abort(400)
def get_task_queue_stats(queues=None):
if queues is None:
queues = ['default']
else:
queues = queues.split(',')
task_queues = [taskqueue.Queue(q).fetch_statistics() for q in queues]
q_stats = []
for q in task_queues:
stats = {
'queue_name': q.queue.name,
'tasks': q.tasks,
'oldest_eta_usec': q.oldest_eta_usec,
'executed_last_minute': q.executed_last_minute,
'in_flight': q.in_flight,
'enforced_rate': q.enforced_rate,
}
q_stats.append(stats)
return q_stats
def get_request_stats(after=None):
if after is None:
one_minute_ago = datetime.utcnow() - timedelta(minutes=1)
after = time.mktime(one_minute_ago.timetuple())
else:
# cast to float
after = float(after)
logs = logservice.fetch(start_time=after)
stats = defaultdict(list)
for req_log in logs:
stats['start_time'].append(req_log.start_time)
stats['api_mcycles'].append(req_log.api_mcycles)
stats['cost'].append(req_log.cost)
stats['finished'].append(req_log.finished)
stats['latency'].append(req_log.latency)
stats['mcycles'].append(req_log.mcycles)
stats['pending_time'].append(req_log.pending_time)
stats['replica_index'].append(req_log.replica_index)
stats['response_size'].append(req_log.response_size)
stats['version_id'].append(req_log.version_id)
return stats
stats = {
'project_name': app_identity.get_application_id()
}
if flavor == 'services' or flavor == 'all':
stats['datastore'] = db_stats.GlobalStat.all().get()
stats['memcache'] = memcache.get_stats()
stats['task_queue'] = get_task_queue_stats(self.request.get('task_queues', None))
if flavor == 'requests' or flavor == 'all':
stats['requests'] = get_request_stats(self.request.get('after', None))
self.response.headers['Content-Type'] = 'application/json'
self.response.write(json.dumps(stats))
app = webapp2.WSGIApplication([
('/datadog', DatadogStats),
])
[1]: https://github.com/DataDog/gae_datadog/blob/master/datadog.py
Yes, the code can be converted and will work in Java, but you will have to do it manually (I don't know of any tools to "translate" from Python to Java).
Looking at all the imports you have, there's nothing there that can't be used in Java.