I'm running the following code in Rhadoop:
Sys.setenv(HADOOP_HOME="/home/ashkan/Downloads/hadoop-1.0.3/")
Sys.setenv(HADOOP_BIN="/home/ashkan/Downloads/hadoop-1.0.3/bin/")
Sys.setenv(HADOOP_CONF_DIR="/home/ashkan/Downloads/hadoop-1.0.3/conf")
Sys.setenv(HADOOP_CMD="/home/ashkan/Downloads/hadoop-1.0.3/bin/hadoop")
library (Rhipe)
library(rhdfs)
library(rmr2)
hdfs.init()
small.ints = to.dfs(1:10)
mapreduce(
input = small.ints,
map = function(k, v)
{
lapply(seq_along(v), function(r){
x <- runif(v[[r]])
keyval(r,c(max(x),min(x)))
})})
How ever, I get the following error:
Error in mr(map = map, reduce = reduce, combine = combine, vectorized.reduce, :
hadoop streaming failed with error code 1
Does anyone know what the problem is?
Thanks a lot.
To fix the problem you'll have to set the HADOOP_STREAMING environment variable. The below code worked fine for me. Note that your code is not using Rhipe so no need to load.
R Code (I'm using hadoop 2.4.0)
Sys.setenv("HADOOP_CMD"="/usr/local/hadoop/bin/hadoop")
Sys.setenv("HADOOP_STREAMING"="/usr/local/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.4.0.jar")
library(rhdfs)
# Initialise
hdfs.init()
library(rmr2)
hdfs.init()
small.ints = to.dfs(1:10)
mapreduce(
input = small.ints,
map = function(k, v)
{
lapply(seq_along(v), function(r){
x <- runif(v[[r]])
keyval(r,c(max(x),min(x)))
})})
I'm guessing that your hadoop streaming path will be as below:
Sys.setenv("HADOOP_STREAMING"="/home/ashkan/Downloads/hadoop-1.0.3/contrib/streaming/hadoop-streaming-1.0.3.jar")
Hope this helps.
Related
I'm having trouble with the substitution using lightbend config library .
I have an application.conf file with this content:
property.a = "propA"
list =
[
{
nameProp=one,
propToReplace = ${property.a}
},
{
nameProp=two,
propToReplace = ${property.a}
}
]
some.env {
property.a = "propEnvironment"
}
At some point in the code, I'm loading the property file using Configuration.load().
My goal is to subtitute the propToReplace with the value of property.a inside some.env, but after I run it I gets replace for the value outside (property.a = "propA").
Does anybody have an idea how to solve this?
Thanks in advance
You can substitute it by using environment variables, like running your program with:
-Dproperty.a=mySubstituteValue
I'm trying to run MaxEnt in biomod2 but I keep getting a java error...
This error seems to be related to a certain "status 1" that I can't understand.
This is my workflow:
# Defining MAXENT Mododelling options
myBiomodOption <- BIOMOD_ModelingOptions(
MAXENT.Phillips = list( path_to_maxent.jar = "C:/Users/thai/Documents/ORCHIDACEAE/Ecologicos/w2/biomod_1",
maximumiterations = 5000,
memory_allocated = 1024,
visible = FALSE,
linear = TRUE,
quadratic = TRUE,
product = TRUE,
threshold = FALSE,
hinge = TRUE,
lq2lqptthreshold = 80,
l2lqthreshold = 10,
hingethreshold = 15,
beta_threshold = -1,
beta_categorical = -1,
beta_lqp = -1,
beta_hinge = -1,
defaultprevalence = 0.5))
This runs with no problems. Then, I run a code for computing the models:
# Computing the models
myBiomodModelOut <- BIOMOD_Modeling(
myBiomodData,
models = c('MAXENT.Phillips'), #'SRE','RF',
models.options = myBiomodOption,
NbRunEval=1,
DataSplit=80,
Yweights=NULL,
VarImport=3,
models.eval.meth = c('TSS','ROC'),
SaveObj = TRUE,
rescal.all.models = TRUE)
I get the running log up to the end. But then, the follownng message comes, in red:
Warning message: running command 'java' had status 1
The model appears to run despite this "status 1" error. When I evaluate the model, there is indeed an evaluation:
> get_evaluations(myBiomodModelOut)
, , MAXENT.Phillips, RUN1, AllData
Testing.data Cutoff Sensitivity Specificity
TSS 0.982 939.0 100 98.202
ROC 0.988 946.5 100 98.494
, , MAXENT.Phillips, Full, AllData
Testing.data Cutoff Sensitivity Specificity
TSS 0.774 687.0 83.333 94.035
ROC 0.926 689.5 83.333 94.085
Well, this is a model evaluation, so a model was made, right?
But when I try to project, I get:
# Project our models over studied area
myBiomomodProj <- BIOMOD_Projection(modeling.output = myBiomodModelOut,
new.env = myExpl,
proj.name = 'current',
selected.models = 'all',
binary.meth = 'TSS',
compress = 'xz',
clamping.mask = F,
output.format = '.grd'
)
*** in setMethod('BinaryTransformation', signature(data='RasterLayer')
*** in setMethod('BinaryTransformation', signature(data='RasterLayer')
> Projecting Anguloa.virginalis_AllData_RUN1_MAXENT.Phillips ...
Error in .local(.Object, ...) :
In addition: Warning message:
running command 'java -mx1024m -cp "C:/Users/thai/Documents/ORCHIDACEAE/Ecologicos/w2/biomod_1/maxent.jar" density.Project "Anguloa.virginalis/models/1506620550/Anguloa.virginalis_AllData_RUN1_MAXENT.Phillips_outputs/Anguloa.virginalis_AllData_RUN1.lambdas" "Anguloa.virginalis/./m_46332684/part1" "Anguloa.virginalis/./m_46332684/part1/projMaxent.asc" doclamp=false visible=false autorun nowarnings notooltips' had status 1
Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer", :
Cannot create a RasterLayer object from this file. (file does not exist)
You see? that "had status 1" part makes me believe that there is a problem being carried out since the model
I Did a lot of research and found out that some people managed maxent to work with biomod2 by moving maxent.ar to a folder with no spaces in the folder name... But this is not my case. Also, some people got it by chanfing the memory allocation... I also tryied several different memory amounts, and I just get different errors, but I still cant't get projections.
Any tips?
My guess is that is due to a memory issue. I suspect you are working with high resolution environmental data on (quite) large area.
One thing to try should be:
split the environmental data into smaller pieces/area
make the projections on each area independently
stick projections back together
I am trying to enable file upload in a webview for android using nativescript.
The problem i am facing is when i am calling my file path callback function "onReceiveValue" where i am supposed to pass by a "android.net.Uri[]".
var mFilePathCallback: android.webkit.ValueCallback<android.net.Uri[]>;
This is how the array is made:
let results: android.net.Uri[] = [];
let dataString = data.getDataString();
results.push(android.net.Uri.parse(dataString));
And then the call:
mFilePathCallback.onReceiveValue(results);
The error i am getting:
Error: Cannot convert array to Ljava/lang/Object; at index 0
I am trying to make this implementation: https://github.com/OpenGeeksMe/Android-File-Chooser/blob/master/app/src/main/java/it/floryn90/webapp/MainActivity.java
Your code for generating an array should look like this:
let results = Array.create(android.net.Uri, 1);
let dataString = data.getDataString();
results[0] = android.net.Uri.parse(dataString);
Where the 1 in it is the number of elements the array is going to hold...
I'm trying to create a multi-value parameter in SpagoBI.
Here is my data set query whose last line appears to be causing an issue.
select C."CUSTOMERNAME", C."CITY", D."YEAR", P."NAME"
from "CUSTOMER" C, "DAY" D, "PRODUCT" P, "TRANSACTIONS" T
where C."CUSTOMERID" = T."CUSTOMERID"
and D."DAYID" = T."DAYID"
and P."PRODUCTID" = T."PRODUCTID"
and _CITY_
I created before open script in my dataset which looks like this:
this.queryText = this.queryText.replace(_CITY_, " CUSTOMER.CITY in ( "+params["cp"].value+" ) ");
My parameter is set as string, display type dynamic list box.
When I run the report I'm getting that error.
org.eclipse.birt.report.engine.api.EngineException: There are errors evaluating script "
this.queryText = this.queryText.replace(_CITY_, " CUSTOMER.CITY in ( "+params["cp"].value+" ) ");
":
Fail to execute script in function __bm_beforeOpen(). Source:
Could anyone please help me?
Hello I managed to solve the problem. Here is my code:
var substring = "" ;
var strParamValsSelected=reportContext.getParameterValue("citytext");
substring += "?," + strParamValsSelected ;
this.queryText = this.queryText.replace("'xxx'",substring);
As You can see the "?" is necessary before my parameter. Maybe It will help somebody. Thank You so much for Your comments.
If you are using SpagoBI server and High charts (JFreeChart Engine) / JSChat Engine you can just use ($P{param_url}) in query,
or build dynamic query using Java script / groovy Script
so your query could also be:
select C."CUSTOMERNAME", C."CITY", D."YEAR", P."NAME"
from "CUSTOMER" C, "DAY" D, "PRODUCT" P, "TRANSACTIONS" T
where C."CUSTOMERID" = T."CUSTOMERID"
and D."DAYID" = T."DAYID"
and P."PRODUCTID" = T."PRODUCTID"
and CUSTOMER."CITY" in ('$P{param_url}')
I'm using azure-documentdb java SDK in order to create and use "User Defined Functions (UDFs)"
So from the official documentation I finally find the way (with a Java client) on how to create an UDF:
String regexUdfJson = "{"
+ "id:\"REGEX_MATCH\","
+ "body:\"function (input, pattern) { return input.match(pattern) !== null; }\","
+ "}";
UserDefinedFunction udfREGEX = new UserDefinedFunction(regexUdfJson);
getDC().createUserDefinedFunction(
myCollection.getSelfLink(),
udfREGEX,
new RequestOptions());
And here is a sample query :
SELECT * FROM root r WHERE udf.REGEX_MATCH(r.name, "mytest_.*")
I had to create the UDF one time only because I got an exception if I try to recreate an existing UDF:
DocumentClientException: Message: {"Errors":["The input name presented is already taken. Ensure to provide a unique name property for this resource type."]}
How should I do to know if the UDF already exists ?
I try to use "readUserDefinedFunctions" function without success. Any example / other ideas ?
Maybe for the long term, should we suggest a "createOrReplaceUserDefinedFunction(...)" on azure feedback
You can check for existing UDFs by running query using queryUserDefinedFunctions.
Example:
List<UserDefinedFunction> udfs = client.queryUserDefinedFunctions(
myCollection.getSelfLink(),
new SqlQuerySpec("SELECT * FROM root r WHERE r.id=#id",
new SqlParameterCollection(new SqlParameter("#id", myUdfId))),
null).getQueryIterable().toList();
if (udfs.size() > 0) {
// Found UDF.
}
An answer for .NET users.
`var collectionAltLink = documentCollections["myCollection"].AltLink; // Target collection's AltLink
var udfLink = $"{collectionAltLink}/udfs/{sampleUdfId}"; // sampleUdfId is your UDF Id
var result = await _client.ReadUserDefinedFunctionAsync(udfLink);
var resource = result.Resource;
if (resource != null)
{
// The UDF with udfId exists
}`
Here _client is Azure's DocumentClient and documentCollections is a dictionary of your documentDb collections.
If there's no such UDF in the mentioned collection, the _client throws a NotFound exception.