How to fix UnsupportedOperationException while using spark joinWith to create Tuple2 - java

I am using Java with Spark. I need to create a Tuple2 Dataset by combining two separate Datasets. I am using joinWith as I want the individual objects to remain intact (cannot use join). However this is failing with:
Exception in thread "main" java.lang.UnsupportedOperationException: Cannot evaluate expression: NamePlaceholder
I tried it with and without Alias but am still getting the same error. What am I doing wrong?
Dataset<MyObject1> dsOfMyObject1;
Dataset<MyObject2> dsOfMyObject2;
Dataset<Tuple2<MyObject1, MyObject2>> tuple2Dataset =
dsOfMyObject1.as("A").
joinWith(dsOfMyObject2.as("B"),col("A.keyfield")
.equalTo(col("B.keyfield")));
Exception in thread "main" java.lang.UnsupportedOperationException: Cannot evaluate
expression: NamePlaceholder
at org.apache.spark.sql.catalyst.expressions.Unevaluable$class.eval(Expression.scala:255)
at org.apache.spark.sql.catalyst.expressions.NamePlaceholder$.eval(complexTypeCreator.scala:243)
at org.apache.spark.sql.catalyst.expressions.CreateNamedStructLike$$anonfun$names$1.apply(complexTypeCreator.scala:289)
at org.apache.spark.sql.catalyst.expressions.CreateNamedStructLike$$anonfun$names$1.apply(complexTypeCreator.scala:289)
at scala.collection.immutable.List.map(List.scala:274)

Related

Getting version mismatch error while updating into DCTM using java

I am trying to update into DCTM through java code, below is the code snippet
IDfDocument communication = (IDfDocument) getDfSession().getObject(DfId.valueOf(communicationId));
communication.setString(ATTR_STATUS, status);
communication.save();
but I am getting the below error
Caused by: DfException:: THREAD: be.ing.ca.xpression.DCTM001P-1; MSG: [DM_OBJ_MGR_E_VERSION_MISMATCH]error: "save of object
090283e589bf689d of type xx_document failed because of version
mismatch: old version was 4"; ERRORCODE: 100; NEXT: null
I thinki am getting this error because there is another process which is trying to modify the object ,and when more than one process try to modify anyobject DCTM throws this exception,
But after lot of searching i dident found any solution which can solve this error
If anyone knows the solution please reply..
Link that i refer
http://www.javablog.fr/?s=version+mismatch

Try calling a fetch() on the object before doing updates.
communication.fetch()
There are some optional parameters AFAIK, but it's been a while since I've been fiddling with DCTM.
Best of luck!

How to load saved model created by (Weka GUI) into my java application and browse the predicted result

My model is done using "FilteredClassifier" algorithm, then SMO as a
"classifier" parameter. "weka.classifiers.functions.SMO".
I tried to load my model into java using this code but it is not work
SupportVector SOM = (SupportVector) SerializationHelper.read(new
FileInputStream("C:\\Users\\HP\\Desktop\\SOM.model"));
and this code
FilteredClassifier SOM = (FilteredClassifier )
SerializationHelper.read(new
FileInputStream("C:\\Users\\HP\\Desktop\\SOM.model"));
both not working
then I want to browse the data used in building this model (actual value and predicted value).
how I can do it? Once I have created the model, do I need to load the dataset again?
This is the error
Exception in thread "main" java.lang.ClassCastException: weka.classifiers.meta.FilteredClassifier cannot be cast to weka.core.pmml.jaxbbindings.SupportVector
at weka.api.Model.main(Model.java:28)
This is the error
Exception in thread "main" java.lang.ClassCastException: weka.classifiers.meta.FilteredClassifier cannot be cast to weka.core.pmml.jaxbbindings.SupportVector
at weka.api.Model.main(Model.java:28)

weka.classifiers.meta.FilteredClassifier cannot be cast to weka.core.pmml.jaxbbindings.SupportVector
pmml and jaxb are XML related classes, you appear to have imported the wrong package.

Error in generating Java Wrapper in Web3j

I am trying to generate java Wrapper for a smart contract, but it failed because of this error:
Exception in thread "main" java.lang.IllegalArgumentException: not a valid name: Main.sol:main
at com.squareup.javapoet.Util.checkArgument(Util.java:64)
at com.squareup.javapoet.TypeSpec$Builder.<init>(TypeSpec.java:383)
at com.squareup.javapoet.TypeSpec$Builder.<init>(TypeSpec.java:362)
at com.squareup.javapoet.TypeSpec.classBuilder(TypeSpec.java:90)
at org.web3j.codegen.SolidityFunctionWrapper.createClassBuilder(SolidityFunctionWrapper.java:172)
at org.web3j.codegen.SolidityFunctionWrapper.generateJavaFiles(SolidityFunctionWrapper.java:103)
at org.web3j.codegen.SolidityFunctionWrapper.generateJavaFiles(SolidityFunctionWrapper.java:91)
at org.web3j.codegen.SolidityFunctionWrapperGenerator.generate(SolidityFunctionWrapperGenerator.java:123)
at org.web3j.codegen.SolidityFunctionWrapperGenerator.main(SolidityFunctionWrapperGenerator.java:87)
at org.web3j.codegen.SolidityFunctionWrapperGenerator.run(SolidityFunctionWrapperGenerator.java:48)
at org.web3j.console.Runner.main(Runner.java:38)
I followed what this guy said to remove this error
https://github.com/web3j/web3j/issues/16]
and give a default name (letters alphabetically) for each empty property as the following:
[{"constant":true,"inputs":
[{"name”:”a”,”type":"bytes32"}],"name":"ApprovedTAs","outputs":
[{"name":"Trade_id","type":"bytes32"},{"name":"Trade_producer","type":"address"},{"name":"Trade_consumer","type":"address"},{"name":"Trade_rate","type":"uint256"},{"name":"Trade_from","type":"uint256"},{"name":"Trade_to","type":"uint256"},{"name":"Trade_broker","type":"address"},{"name":"Trade_topic","type":"uint256"},{"name":"approvedByA","type":"bool"},{"name":"approvedByB","type":"bool"}],"payable":false,"type":"function"},{"constant":true,"inputs":[{"name”:”b”,”type":"address"},{"name”:”c”,”type":"uint256"}],"name":"offersByProducer","outputs":[{"name":"offer_id","type":"uint256"},{"name":"offer_producer","type":"address"},{"name":"offer_topic","type":"uint256"},{"name":"offer_rate","type":"uint256"},{"name":"offer_from","type":"uint256"},{"name":"offer_to","type":"uint256"},{"name":"offer_broker","type":"address"}],"payable":false,"type":"function"},{"constant":true,"inputs":[],"name":"getAll","outputs":[{"name”:”d”,”type":"address[]"},{"name”:”e”,”type":"address[]"},{"name”:”f”,”type":"address[]"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"mk_producer","type":"address"},{"name":"mk_topic","type":"uint256"},{"name":"mk_rate","type":"uint256"},{"name":"mk_from","type":"uint256"},{"name":"mk_to","type":"uint256"},{"name":"mk_broker","type":"address"}],"name":"mkOrder","outputs":[],"payable":false,"type":"function"},{"constant":true,"inputs":[{"name”:”g”,”type":"uint256"}],"name":"offers","outputs":[{"name":"offer_id","type":"uint256"},{"name":"offer_producer","type":"address"},{"name":"offer_topic","type":"uint256"},{"name":"offer_rate","type":"uint256"},{"name":"offer_from","type":"uint256"},{"name":"offer_to","type":"uint256"},{"name":"offer_broker","type":"address"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"ofr_topic","type":"uint256"},{"name":"ofr_rate","type":"uint256"},{"name":"ofr_from","type":"uint256"},{"name":"ofr_to","type":"uint256"},{"name":"ofr_broker","type":"address"}],"name":"offer","outputs":[],"payable":false,"type":"function"},{"constant":true,"inputs":[{"name”:”h”,”type":"uint256"},{"name”:”ii”,”type":"uint256"}],"name":"allAdd","outputs":[{"name”:”j”,”type":"address"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"sndC_id","type":"bytes32"},{"name":"sndC_producer","type":"address"},{"name":"sndC_broker","type":"address"},{"name":"sndC_topic","type":"uint256"},{"name":"sndC_from","type":"uint256"},{"name":"sndC_to","type":"uint256"},{"name":"sndC_count","type":"uint256"}],"name":"sendCCube","outputs":[{"name”:”l”,”type":"bool"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"id","type":"bytes32"}],"name":"agreeTA","outputs":[{"name”:”m”,”type":"bool"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"sndP_id","type":"bytes32"},{"name":"sndP_broker","type":"address"},{"name":"sndP_topic","type":"uint256"},{"name":"sndP_from","type":"uint256"},{"name":"sndP_to","type":"uint256"},{"name":"sndP_count","type":"uint256"}],"name":"sendPCube","outputs":[{"name”:”n”,”type":"bool"}],"payable":false,"type":"function"},{"constant":true,"inputs":[{"name”:”o”,”type":"bytes32"}],"name":"CreatedTAs","outputs":[{"name":"Trade_id","type":"bytes32"},{"name":"Trade_producer","type":"address"},{"name":"Trade_consumer","type":"address"},{"name":"Trade_rate","type":"uint256"},{"name":"Trade_from","type":"uint256"},{"name":"Trade_to","type":"uint256"},{"name":"Trade_broker","type":"address"},{"name":"Trade_topic","type":"uint256"},{"name":"approvedByA","type":"bool"},{"name":"approvedByB","type":"bool"}],"payable":false,"type":"function"},{"constant":false,"inputs":[{"name":"nkname","type":"string"},{"name":"code","type":"uint256"}],"name":"register","outputs": [{"name”:”p”,”type":"bool"}],"payable":false,"type":"function"}]
But the error still the same. I also changed the name of the contract rather than "main" but nothing change.
I also changed every variable name has numbers, special characters or even starting with _ but still doesn't work.
Any idea what I should change to solve this error
Thanks

This error is triggered if you are declaring a function o event with only the argument type without a name.for example :
event myevent(string)
should be
event myevent(string message);

How to fix Google DataFlow Pipeline (args) null pointer exception?

I'm trying to run a really simple dataflow job, just taking some data in BigQuery, processing it a bit and putting it in a new bigquery table
Pipeline p = Pipeline.create(
PipelineOptionsFactory.fromArgs(args).withValidation().create());
p.apply(BigQueryIO.Read.fromQuery("SELECT * FROM realtime.status_6_output_11"));
p.run();
However whenever I run it I get the following pretty undescriptive NullPointerException:
Exception in thread "main" java.lang.NullPointerException
at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
at java.util.regex.Matcher.reset(Matcher.java:309)
at java.util.regex.Matcher.<init>(Matcher.java:229)
at java.util.regex.Pattern.matcher(Pattern.java:1093)
at com.google.cloud.dataflow.sdk.util.IOChannelUtils.getFactory(IOChannelUtils.java:174)
at com.google.cloud.dataflow.sdk.io.BigQueryIO$Read$Bound.apply(BigQueryIO.java:553)
at com.google.cloud.dataflow.sdk.io.BigQueryIO$Read$Bound.apply(BigQueryIO.java:387)
at com.google.cloud.dataflow.sdk.runners.PipelineRunner.apply(PipelineRunner.java:74)
at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.apply(DirectPipelineRunner.java:247)
at com.google.cloud.dataflow.sdk.Pipeline.applyInternal(Pipeline.java:367)
at com.google.cloud.dataflow.sdk.Pipeline.applyTransform(Pipeline.java:274)
at com.google.cloud.dataflow.sdk.values.PBegin.apply(PBegin.java:47)
at com.google.cloud.dataflow.sdk.Pipeline.apply(Pipeline.java:156)
at com.noraway.conductor.NormalizedPipeline.main(NormalizedPipeline.java:42)
I think there's a problem with my command line arguments (don't have any right now) but I'm not sure what that would be.

It looks like there is a missing --tempLocation for BigQuery to use. The obscure error message is fixed as part of https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/313.

java.lang.ClassCastException for TDBFactory

I made a tdb index with Jena for the data I have.
In order to refer to the data while querying, I tried using TDBFactory and Model (both statements given below). I am getting the same exception for both, so this seems to be independent of the statement I write.
Dataset dataset = TDBFactory.createDataset(directory) ;
and
Model model=FileManager.get().loadModel(directory);
The runtime exception is:
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.hp.hpl.jena.rdf.model.impl.RDFReaderFImpl.reset(RDFReaderFImpl.java:81)
at com.hp.hpl.jena.rdf.model.impl.RDFReaderFImpl.<clinit>(RDFReaderFImpl.java:74)
at com.hp.hpl.jena.rdf.model.impl.ModelCom.<clinit>(ModelCom.java:54)
at com.hp.hpl.jena.rdf.model.ModelFactory.createDefaultModel(ModelFactory.java:114)
at com.hp.hpl.jena.vocabulary.OWL.<clinit>(OWL.java:36)
at com.hp.hpl.jena.sparql.graph.NodeConst.<clinit>(NodeConst.java:29)
at com.hp.hpl.jena.sparql.engine.optimizer.reorder.ReorderFixed.<clinit>(ReorderFixed.java:23)
at com.hp.hpl.jena.sparql.engine.optimizer.reorder.ReorderLib.fixed(ReorderLib.java:53)
at com.hp.hpl.jena.tdb.sys.SystemTDB.<clinit>(SystemTDB.java:187)
at com.hp.hpl.jena.tdb.TDB.<clinit>(TDB.java:90)
at com.hp.hpl.jena.tdb.setup.DatasetBuilderStd.<clinit>(DatasetBuilderStd.java:64)
at com.hp.hpl.jena.tdb.StoreConnection.make(StoreConnection.java:227)
at com.hp.hpl.jena.tdb.transaction.DatasetGraphTransaction.<init>(DatasetGraphTransaction.java:75)
at com.hp.hpl.jena.tdb.sys.TDBMaker._create(TDBMaker.java:57)
at com.hp.hpl.jena.tdb.sys.TDBMaker.createDatasetGraphTransaction(TDBMaker.java:45)
at com.hp.hpl.jena.tdb.TDBFactory._createDatasetGraph(TDBFactory.java:104)
at com.hp.hpl.jena.tdb.TDBFactory.createDatasetGraph(TDBFactory.java:73)
at com.hp.hpl.jena.tdb.TDBFactory.createDataset(TDBFactory.java:52)
at com.hp.hpl.jena.tdb.TDBFactory.createDataset(TDBFactory.java:48)
Caused by: java.lang.ClassCastException: org.apache.xerces.dom.DeferredTextImpl cannot be cast to org.w3c.dom.Element
at sun.util.xml.PlatformXmlPropertiesProvider.importProperties(PlatformXmlPropertiesProvider.java:118)
at sun.util.xml.PlatformXmlPropertiesProvider.load(PlatformXmlPropertiesProvider.java:90)
at java.util.Properties$XmlSupport.load(Properties.java:1201)
at java.util.Properties.loadFromXML(Properties.java:881)
at com.hp.hpl.jena.util.Metadata.read(Metadata.java:76)
at com.hp.hpl.jena.util.Metadata.addMetadata(Metadata.java:54)
at com.hp.hpl.jena.util.Metadata.<init>(Metadata.java:48)
at com.hp.hpl.jena.JenaRuntime.<clinit>(JenaRuntime.java:34)
The jar files that I am using are:
arq-2.8.7.jar,
commons-cli-1.2.jar,
commons-codec-1.6.jar,
commons-collections-3.2.1.jar,
commons-csv-1.0.jar,
commons-io-2.4.jar,
commons-lang3-3.1.jar,
commons-math3-3.0.jar,
httpclient-4.2.6.jar,
httpclient-cache-4.2.6.jar,
httpcore-4.2.5.jar,
jackson-annotations-2.3.0.jar,
jackson-core-2.3.3.jar,
jackson-databind-2.3.3.jar,
jcl-over-slf4j-1.7.6.jar,
jena-arq-2.12.1.jar,
jena-core-2.12.1.jar,
jena-iri-1.1.1.jar,
jena-sdb-1.5.1.jar,
jena-tdb-1.1.1.jar,
jgraph.jar,
jsonld-java-0.5.0.jar,
libthrift-0.9.1.jar,
log4j-1.2.17.jar,
slf4j-api-1.7.6.jar,
slf4j-log4j12-1.7.6.jar,
xercesImpl-2.11.0.jar,
xml-apis-1.4.01.jar
How do I fix this?

Categories

Resources