We use scala programing language and use spark as a data processing. we have 3 different projects
Project A // where we have defined this annotation
package Common {
case class TimeStamp(format: DateFormat) extends StaticAnnotation
}
Project B // where we have this case class which is using above annotation
case class CommonSchema(
#Common.TimeStamp(DateFormat.DateTime)
__datetime: Timestamp
)
Project C // where we use above case class as type
val events = events.map(c => CommonSchema(
c.__datetime,
))
events.as[CommonSchema]
We are hitting the below exception in project C
User class threw exception: java.lang.RuntimeException: error reading Scala signature of commonschema: assertion failed: unsafe symbol TimeStamp (child of <none>) in runtime reflection universe
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:46)
at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:619)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply$mcV$sp(SymbolLoaders.scala:28)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.load(SymbolLoaders.scala:33)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:140)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:133)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:168)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.typeParams(SynchronizedSymbols.scala:132)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.typeParams(SynchronizedSymbols.scala:168)
at scala.reflect.internal.Types$NoArgsTypeRef.typeParams(Types.scala:1931)
at scala.reflect.internal.Types$NoArgsTypeRef.isHigherKinded(Types.scala:1930)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType2(TypeComparers.scala:377)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType1(TypeComparers.scala:320)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType(TypeComparers.scala:278)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:16)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:784)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:260)
at org.apache.spark.sql.catalyst.ScalaReflection$.isSubtype(ScalaReflection.scala:83)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType$1.apply$mcZ$sp(ScalaReflection.scala:677)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType$1.apply(ScalaReflection.scala:676)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType$1.apply(ScalaReflection.scala:676)
at scala.reflect.internal.tpe.TypeConstraints$UndoLog.undo(TypeConstraints.scala:56)
at org.apache.spark.sql.catalyst.ScalaReflection$class.cleanUpReflectionObjects(ScalaReflection.scala:926)
at org.apache.spark.sql.catalyst.ScalaReflection$.cleanUpReflectionObjects(ScalaReflection.scala:49)
at org.apache.spark.sql.catalyst.ScalaReflection$.optionOfProductType(ScalaReflection.scala:675)
at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.apply(ExpressionEncoder.scala:51)
at org.apache.spark.sql.Encoders$.product(Encoders.scala:275)
Similar questions are answered in link1 and link2, but We already build uber jar for project C, so all the classes should be included in the package. locally the unit test that hits this code works fine but why we get the runtime errors?
This happened because were using maven-shade-plugin plugin and minimizing the jar. We made a config change so that this classes does not get removed from jar. And it has fixed the issue
Related
When I package my stand-alone-java-application, that compiles, renders and prints Jasper-Reports, in a fat jar with all dependencies, I get the error that the function TEXT – a function used in the report and defined in the net.sf.jasperreports-functions-library is not found.
The method TEXT(Integer, String) is undefined for the type [...]
value = IF(LEN(TEXT(((java.lang.Integer)parameter_X.getValue()),"#"))==2,"***",REPT("*",6-LEN(TEXT(((java.lang.Integer)parameter_X.getValue()),"#")))) //$JR_EXPR_ID=58$
Other functions from the same library work. When I run the application with gradle run, the function is found and the report can be printed. The library is on my classpath.
There are several similar questions on this site and Stackoverflow, but neither provided answers, that worked for me. In the following, I will explain what I tried:
In my build.gradle-File I defined dependencies to the following libraries:
implementation('net.sf.jasperreports:jasperreports:6.18.1')
implementation('net.sf.jasperreports:jasperreports-functions:6.18.1')
In my java-Files, I created a DesignFile and statically imported the functions into it. I tried two methods:
Method 1 as given in an answer in this thread: Unresolved symbols when compiling jasper report
/*…*/
designFile = JRXmlLoader.load(sourceFile.getAbsolutePath());
designFile.addImport("static net.sf.jasperreports.functions.standard.TextFunctions.*");
designFile.addImport("static net.sf.jasperreports.functions.standard.MathFunctions.*");
designFile.addImport("static net.sf.jasperreports.functions.standard.DateTimeFunctions.*");
designFile.addImport("static net.sf.jasperreports.functions.standard.LogicalFunctions.*");
JasperReport compiledReport = JasperCompileManager.getInstance(ctx).compile(designFile);
/*…*/
Method 2: importing each function on its own
public static String[] textFunctions(){
return new String[]{"BASE", "CHAR", "CLEAN", "CODE", "CONCATENATE", "DOUBLE_VALUE",
"EXACT", "FIND", /*"FIXED", */"FLOAT_VALUE", "INTEGER_VALUE", "LEFT", "LEN", "LONG_VALUE",
"LOWER", "LTRIM", "MID", "PROPER", "REPLACE", "REPT", "RIGHT", "RTRIM", "SEARCH", "SUBSTITUTE",
"T", /*"TEXT",*/ "TRIM", "UPPER"};
}}
/*…*/
designFile = JRXmlLoader.load(sourceFile.getAbsolutePath());
for (String textFunction : textFunctions()){
designFile.addImport("static net.sf.jasperreports.functions.standard.TextFunctions." + textFunction);
}/* similarly for logical functions, numeric functions and datetime functions*/
}
JasperReport compiledReport = JasperCompileManager.getInstance(ctx).compile(designFile);
/*…*/
As you can see, the function TEXT is commented out. That’s because the program fails at runtime when run with “gradle run”, when I comment it in:
net.sf.jasperreports.engine.JRException: Errors were encountered when compiling report expressions class file:
1. The import net.sf.jasperreports.functions.standard.TextFunctions.TEXT cannot be resolved
import static net.sf.jasperreports.functions.standard.TextFunctions.TEXT;
The same thing is true with most datetime-functions, in fact, only TIME can be imported to the DesignObject without error at runtime
In addition to that, when I type net.sf.jasperreports.functions.standard.TextFunctions. in my IDE, the content-assist shows a list of available functions, yet TEXT is missing.
I suspected, that the function is simply missing in the library, but it is there. In the .gradle/cache/modules-2/files-2.1/… directory, there is the jar with the TextFunctions.java, that contains the function TEXT:
When building the FatJar with dependencies, I have a TextFunctions.class-File in it, that contains the TEXT-Function as well.
In my jar, there are two jasperreports_extension.properties on the root level:
with the smaller one (from the jasperreport-functions-dependency) containing the following lines:
net.sf.jasperreports.extension.registry.factory.functions=net.sf.jasperreports.functions.FunctionsRegistryFactory
net.sf.jasperreports.extension.functions.datetime=net.sf.jasperreports.functions.standard.DateTimeFunctions
net.sf.jasperreports.extension.functions.math=net.sf.jasperreports.functions.standard.MathFunctions, net.sf.jasperreports.functions.standard.LogicalFunctions
net.sf.jasperreports.extension.functions.text=net.sf.jasperreports.functions.standard.TextFunctions
net.sf.jasperreports.extension.functions.report=net.sf.jasperreports.functions.standard.ReportFunctions
Funnily enough, commenting out TEXT, running it locally with gradle run, it works without problems. But packaging it and running it with java -jar it fails to find this function. And statically importing this function leads to gradle run failing as well.
What is happening and how can I fix this? I want to distribute my FatJar on a server and run it, but this problems makes it impossible for me.
Following the setup described in Simple sharing of artifacts between projects, we are in the special case where we have a multi module gradle build that produce different types of jars and we would like to declare a dependency to those jar in a configuration.
dependencies {
instrumentedClasspath(project(path: ":producer", configuration: 'instrumentedJars'))
}
from the docs works great.
In the project dependency-tests I have a project that reproduce the setup (with different names, but the idea is the same).
But I am doing this in a Gradle-plugin and I would like to have the same declaration in java.
DependencyHandler dependencyHandler = project.getDependencies();
// this adds a dependency to the main jar of the 'producer' project:
dependencyHandler.add("instrumentedClasspath", project.getRootProject().findProject(":producer"));
// this is not working:
dependencyHandler.add("instrumentedClasspath", project.getRootProject().findProject(":producer").getConfigurations().getByName("instrumentedJars"));
Failing with:
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':printConf'.
> Could not resolve all dependencies for configuration ':instrumentedJars'.
> Cannot convert the provided notation to an object of type Dependency: configuration ':producer:instrumentedJars' artifacts.
The following types/formats are supported:
- Instances of Dependency.
- String or CharSequence values, for example 'org.gradle:gradle-core:1.0'.
- Maps, for example [group: 'org.gradle', name: 'gradle-core', version: '1.0'].
- FileCollections, for example files('some.jar', 'someOther.jar').
- Projects, for example project(':some:project:path').
- ClassPathNotation, for example gradleApi().
Comprehensive documentation on dependency notations is available in DSL reference for DependencyHandler type.
* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
project(...)
inside the dependencies block comes from the DependencyHandler, and
path: ":producer", configuration: 'instrumentedJars'
is actually a map {"path"=":producer", "configuration"="instrumentedJars"}.
So something like that should work in Java:
Map<String, String> map = new HashMap<>();
map.put("path", ":producer");
map.put("configuration", "instrumentedJars");
dependencyHandler.add("instrumentedClasspath", dependencyHandler.project(map));
Note: When using Kotlin build script you can easily see types and declarations of functions and might be easier for discovering API. So in Kotlin project(...) in the dependencies block is an extension method defined as:
fun DependencyHandler.project(
path: String,
configuration: String? = null
): ProjectDependency =
uncheckedCast(
project(
if (configuration != null) mapOf("path" to path, "configuration" to configuration)
else mapOf("path" to path)
)
)
I'm trying to use lmdbjava in Clojure, but I'm struggling.
(import '[org.lmdbjava Env])
(def path (clojure.java.io/file "/tmp"))
(.open (.setMaxDbs (.setMapSize (Env/create) 10485760) 1) path)
(p.s. I realise there are cleaner ways. This is just for testing purposes.)
This is the error:
IllegalArgumentException No matching method found: open for class java.lang.Class clojure.lang.Reflector.invokeMatchingMethod (Reflector.java:53)
I've also tried this:
(.open (.setMaxDbs (.setMapSize (Env/create) 10485760) 1) path org.lmdbjava.EnvFlags/MDB_NOLOCK)
and this:
(.. (Env/create) (setMapSize 10485760) (setMaxDbs 1) (open path org.lmdbjava.EnvFlags/MDB_NOLOCK))
And I get this error:
ClassCastException org.lmdbjava.EnvFlags (in module: Unnamed Module) cannot be cast to [Lorg.lmdbjava.EnvFlags; (in module: Unnamed Module) user/eval1339 (form-init2868059116743223586.clj:1)
I realise I'm probably doing something daft, because I'm new to both Java and Clojure. Any help would be greatly appreciated!
By the way, this is the tutorial I'm following:
https://github.com/lmdbjava/lmdbjava/blob/master/src/test/java/org/lmdbjava/TutorialTest.java
Thanks!
From the signature of the Env.Builder class:
Env<T> open(File path, int mode, EnvFlags... flags)
you also have to supply a EnvFlags varargs parameter. This is how you do it in Clojure:
(.open (.setMaxDbs (.setMapSize (Env/create) 10485760) 1) path (into-array org.lmdbjava.EnvFlags []))
Also see How to handle java variable length arguments in clojure?
I'm using jode's obfuscator in an attempt to obfuscate all of the class files in my jar. I've packaged all the class files in the jar, so that I can use the config to specify what I need loading, and what I need preserving.
Here is my package hierarchy:
src
src.loading
src.loading.org
src.loading.sign
Here's what I've specified as my wildcards:
load = new WildCard { value = "src.*" }
preserve = new WildCard { value = "src.loading.org.*" }
And this is the error I get (Decompiling too. It's the only class.)
C:\Users\Kieran\Desktop\jode>java jode.obfuscator.Main project.jos
Jode (c) 1998-2001 Jochen Hoenicke <jochen#gnu.org>
used before: 479080
Loading and preserving classes
Exception in thread "main" java.lang.NoClassDefFoundError: src.loading.Class48
at jode.bytecode.ClassInfo.loadInfo(ClassInfo.java:631)
at jode.obfuscator.ClassIdentifier.initClass(ClassIdentifier.java:371)
Exception while decompiling:java.lang.NoClassDefFoundError: src.loading.Class48
at jode.bytecode.ClassInfo.loadInfo(ClassInfo.java:631)
at jode.decompiler.ClassAnalyzer.<init>(ClassAnalyzer.java:86)
at jode.decompiler.ClassAnalyzer.<init>(ClassAnalyzer.java:123)
at jode.decompiler.Decompiler.decompile(Decompiler.java:191)
at jode.swingui.Main.run(Main.java:204)
at java.lang.Thread.run(Unknown Source)
The class does exist, and I the jar file is fully functional. Why am I getting this error?
This specific class is using reflection, to access java.lang.Runtime, could this be the issue...?
I have a problem while transforming the xslt to pdf in java, i am following the same process posted in this link
"Java Transformation to PDF".
Error:
`
Caused by: java.lang.NoSuchMethodError: org.apache.xmlgraphics.java2d.GraphicContext.<init>(Lorg/apache/xmlgraphics/java2d/GraphicContext;)V
at org.apache.fop.render.intermediate.IFGraphicContext.<init>(IFGraphicContext.java:50)
at org.apache.fop.render.intermediate.IFGraphicContext.clone(IFGraphicContext.java:56)
at org.apache.fop.render.intermediate.IFRenderer.saveGraphicsState(IFRenderer.java:632)
at org.apache.fop.render.intermediate.IFRenderer.startViewport(IFRenderer.java:885)
at org.apache.fop.render.intermediate.IFRenderer.startVParea(IFRenderer.java:878)
at org.apache.fop.render.AbstractRenderer.renderRegionViewport(AbstractRenderer.java:289)
at org.apache.fop.render.intermediate.IFRenderer.renderRegionViewport(IFRenderer.java:731)
at org.apache.fop.render.AbstractRenderer.renderPageAreas(AbstractRenderer.java:249)
at org.apache.fop.render.AbstractRenderer.renderPage(AbstractRenderer.java:230)
at org.apache.fop.render.intermediate.IFRenderer.renderPage(IFRenderer.java:580)
at org.apache.fop.area.RenderPagesModel.addPage(RenderPagesModel.java:114)
at org.apache.fop.layoutmgr.AbstractPageSequenceLayoutManager.finishPage(AbstractPageSequenceLayoutManager.java:312)
at org.apache.fop.layoutmgr.PageSequenceLayoutManager.finishPage(PageSequenceLayoutManager.java:167)
at org.apache.fop.layoutmgr.PageSequenceLayoutManager.activateLayout(PageSequenceLayoutManager.java:109)
at org.apache.fop.area.AreaTreeHandler.endPageSequence(AreaTreeHandler.java:238)
at org.apache.fop.fo.pagination.PageSequence.endOfNode(PageSequence.java:120)
at org.apache.fop.fo.FOTreeBuilder$MainFOHandler.endElement(FOTreeBuilder.java:349)
at org.apache.fop.fo.FOTreeBuilder.endElement(FOTreeBuilder.java:177)
at net.sf.saxon.event.ContentHandlerProxy.endElement(ContentHandlerProxy.java:391)
at net.sf.saxon.event.ProxyReceiver.endElement(ProxyReceiver.java:174)
at net.sf.saxon.event.NamespaceReducer.endElement(NamespaceReducer.java:213)
at net.sf.saxon.event.ComplexContentOutputter.endElement(ComplexContentOutputter.java:417)
at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:301)
at net.sf.saxon.instruct.Block.processLeavingTail(Block.java:409)
at net.sf.saxon.instruct.Instruction.process(Instruction.java:94)
at net.sf.saxon.instruct.ElementCreator.processLeavingTail(ElementCreator.java:298)
at net.sf.saxon.instruct.Template.applyLeavingTail(Template.java:175)
at net.sf.saxon.instruct.ApplyTemplates.applyTemplates(ApplyTemplates.java:343)
at net.sf.saxon.Controller.transformDocument(Controller.java:1736)
at net.sf.saxon.Controller.transform(Controller.java:1560)
at mypackage.v2.business.pdf.XMLtoPDF.convertXMLPDF(XMLtoPDF.java:103)
... 51 more
java.lang.ClassCastException: java.lang.NoSuchMethodError cannot be cast to java.lang.Exception
`
Please let me know what could be the problem.
As described in the documentation
Thrown if an application tries to call a specified method of a class (either static or instance), and that class no longer has a definition of that method.
Here's how this can happen. Imagine you built some code that used a library with a method called foo(). You compiled your project and then later decided to upgrade the library. You do this by overwriting the old jar file with the new one. You don't recompile your code.
But what you didn't know is the new library removed the foo() method. Now if you run your code, it will throw this exception because your compile code calls this method and it's no longer there.
In your specific case, this isn't necessarily your code with the problem. It could be that you have one library that uses another library and the problem is there (for example, Library X uses version 2 of Library Y, but you only have Library Y version 1 in your classpath). If this is happening, you need to make sure the libraries are using the version they expect.
For your specific problem, you need to find the version of the jar that has org.apache.xmlgraphics.java2d.GraphicContext in it and it needs to have a constructor that takes a... list of org/apache/xmlgraphics/java2d/GraphicContext... maybe. I forget what (L...;)V means.