Develop Reference

assertion failed: unsafe symbol TimeStamp (child of <none>) in runtime reflection universe

We use scala programing language and use spark as a data processing. we have 3 different projects
Project A // where we have defined this annotation
package Common {
case class TimeStamp(format: DateFormat) extends StaticAnnotation
}
Project B // where we have this case class which is using above annotation
case class CommonSchema(
#Common.TimeStamp(DateFormat.DateTime)
__datetime: Timestamp
)
Project C // where we use above case class as type
val events = events.map(c => CommonSchema(
c.__datetime,
))
events.as[CommonSchema]
We are hitting the below exception in project C
User class threw exception: java.lang.RuntimeException: error reading Scala signature of commonschema: assertion failed: unsafe symbol TimeStamp (child of <none>) in runtime reflection universe
at scala.reflect.internal.pickling.UnPickler.unpickle(UnPickler.scala:46)
at scala.reflect.runtime.JavaMirrors$JavaMirror.unpickleClass(JavaMirrors.scala:619)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply$mcV$sp(SymbolLoaders.scala:28)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter$$anonfun$complete$1.apply(SymbolLoaders.scala:25)
at scala.reflect.internal.SymbolTable.slowButSafeEnteringPhaseNotLaterThan(SymbolTable.scala:263)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.complete(SymbolLoaders.scala:25)
at scala.reflect.runtime.SymbolLoaders$TopClassCompleter.load(SymbolLoaders.scala:33)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:140)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anonfun$typeParams$1.apply(SynchronizedSymbols.scala:133)
at scala.reflect.runtime.Gil$class.gilSynchronized(Gil.scala:19)
at scala.reflect.runtime.JavaUniverse.gilSynchronized(JavaUniverse.scala:16)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:123)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.gilSynchronizedIfNotThreadsafe(SynchronizedSymbols.scala:168)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$class.typeParams(SynchronizedSymbols.scala:132)
at scala.reflect.runtime.SynchronizedSymbols$SynchronizedSymbol$$anon$8.typeParams(SynchronizedSymbols.scala:168)
at scala.reflect.internal.Types$NoArgsTypeRef.typeParams(Types.scala:1931)
at scala.reflect.internal.Types$NoArgsTypeRef.isHigherKinded(Types.scala:1930)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType2(TypeComparers.scala:377)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType1(TypeComparers.scala:320)
at scala.reflect.internal.tpe.TypeComparers$class.isSubType(TypeComparers.scala:278)
at scala.reflect.internal.SymbolTable.isSubType(SymbolTable.scala:16)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:784)
at scala.reflect.internal.Types$Type.$less$colon$less(Types.scala:260)
at org.apache.spark.sql.catalyst.ScalaReflection$.isSubtype(ScalaReflection.scala:83)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType$1.apply$mcZ$sp(ScalaReflection.scala:677)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType$1.apply(ScalaReflection.scala:676)
at org.apache.spark.sql.catalyst.ScalaReflection$$anonfun$optionOfProductType$1.apply(ScalaReflection.scala:676)
at scala.reflect.internal.tpe.TypeConstraints$UndoLog.undo(TypeConstraints.scala:56)
at org.apache.spark.sql.catalyst.ScalaReflection$class.cleanUpReflectionObjects(ScalaReflection.scala:926)
at org.apache.spark.sql.catalyst.ScalaReflection$.cleanUpReflectionObjects(ScalaReflection.scala:49)
at org.apache.spark.sql.catalyst.ScalaReflection$.optionOfProductType(ScalaReflection.scala:675)
at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$.apply(ExpressionEncoder.scala:51)
at org.apache.spark.sql.Encoders$.product(Encoders.scala:275)
Similar questions are answered in link1 and link2, but We already build uber jar for project C, so all the classes should be included in the package. locally the unit test that hits this code works fine but why we get the runtime errors?

This happened because were using maven-shade-plugin plugin and minimizing the jar. We made a config change so that this classes does not get removed from jar. And it has fixed the issue

Weka DiscriminantAnalysis NoClassDefFoundError

I'm trying to use MultiClassFLDA in discriminant analysis package but I always get an error on running the code and defining a new instance of the MultiClassFLDA class
Exception in thread "main" java.lang.NoClassDefFoundError: no/uib/cipr/matrix/Vector
at assignment2.face.tryLDA(face.java:141)
at assignment2.Assignment2.main(Assignment2.java:106)
Caused by: java.lang.ClassNotFoundException: no.uib.cipr.matrix.Vector
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

It seems to be linked to some dynamic class loading in newer versions of Weka, presumably within the Weka Package Manager: that class is defined in the mtj.jar, which is bundled inside weka.jar. In that other question, an answer suggested to extract the mtj.jar and add it to your classpath.
As I didn't have this problem with other Filters, my guess is that MultiClassFLDA is not implemented correctly: I found out is that if you use another Filter before, that specific class will get loaded:
// Run a dummy Filter for correct initialization
Filter f = new Standardize();
f.setInputFormat(data);
Filter.useFilter(new Instances("", params, 0), f); // Dummy run on empty dataset
// Now run the MultiClassFLDA
f = new MultiClassFLDA();
f.setInputFormat(data);
data = Filter.useFilter(data, f);
N.B. that is a really ugly hack! I used it to be able to work. I'll edit my answer when I find the appropriate way to do it (other than extracting the jar from Weka itslef).

Why is this an "illegal class name" in Groovy / Java? [duplicate]

I'm using the following code in post-build step of my Jenkins job:
evaluate(new File("Set-BuildBadge.groovy"));
So it runs a script successfully if it does not contain functions.
If inside the script I define a function for example:
def addSummaryWithText(Icon, Text) {
manager.createSummary(Icon).appendText(Text, false)
}
...
addSummaryWithText("installer.gif", "Project: " + ProjectName)
then I get the following error:
FATAL: Illegal class name "Set-BuildBadge$addSummaryWithText" in class
file Set-BuildBadge$addSummaryWithText java.lang.ClassFormatError:
Illegal class name "Set-BuildBadge$addSummaryWithText" in class file
Set-BuildBadge$addSummaryWithText at
java.lang.ClassLoader.defineClass1(Native Method) ...
I'm not getting how GroovyShell.evaluate works.
Can anyone help me?

Looks like the JVM doesn't like class names with a hyphen in them.
By calling your script Set-BuildBadge.groovy internally it is compiled into a class that isn't allowed when you add a function to the script.
Changing the name of the script to SetBuildBadge.groovy will fix it :-)

Modified classes are not picked by Tomcat

I have modified few classes (like HiveConnection.class, HiveDatabaseMetaData.class, HiveResultSetMetaData.class, HiveStatement.class) in hive-jdbc-0.13.1-cdh5.3.3.jar, and created a new jar which contains only modified classes and named it as 0_modified_hive_jdbc.jar.
Those classes has some overriden methods which was throwing method not supported exception. Instead we created class in the same name, we logged message, and not throwing exception.
Placed hive-jdbc-0.13.1-cdh5.3.3.jar inside tomcat/lib (because org.apache.hive.jdbc.HiveDriver is referred as a driver name for a JNDI in server.xml)
I have placed all external jars like this,
/home/nages/external_jar/0_override/0_modified_hive_jdbc.jar
/home/nages/external_jar/hive/hive-jdbc-0.13.1-cdh5.3.3.jar
/home/nages/external_jar/others/commons-lang-2.5.jar
/home/nages/external_jar/others/spring-beans-2.5.3.jar
/home/nages/external_jar/others/spring-context-2.5.3.jar
/home/nages/external_jar/others/spring-core-2.5.3.jar
/home/nages/external_jar/others/spring-jdbc-2.5.3.jar
/home/nages/external_jar/others/spring-tx-2.5.3.jar
etc..
Referred these jars like this
File /home/nages/tomcat/conf/Catalina/localhost/myapp.xml
<Context reloadable="true" path="/myapp">
<Loader className="org.apache.catalina.loader.VirtualWebappLoader" virtualClasspath="/home/nages/external_jar/0_overrider/*.jar;/home/nages/external_jar/hive/*.jar;/home/nages/external_jar/others/*.jar;/etc/hadoop/conf/;"/>
</Context>
Deployed my application (myapp.war) in tomcat.
find.jsp presented in myapp.war.
/home/nages/tomcat/webapps/myapp/find.jsp
Here is the piece of code presented in find.jsp
ClassLoader loader = clazz.getClassLoader();
if (loader == null){
throw new Exception("Class Loader is null.");
}
String fileName = className.replace('.','/')+".class";
URL fileUrl = loader.getResource(fileName);
out.println("<b>Location : </b>" + fileUrl);
When I request for org.apache.hive.jdbc.HiveConnection in find.jsp, it returns,
Location : jar:file:/home/nages/external_jar/0_override/0_modified_hive_jdbc.jar!/org/apache/hive/jdbc/HiveConnection.class.
Inside war there is a java class, which has the following code
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.support.rowset.SqlRowSet;
JdbcTemplate template = new JdbcTemplate(getDataSource("Hive"));
SqlRowSet sqs = template.queryForRowSet(sqlQuery);
I am getting following exception
Caused by: java.sql.SQLException: Method not supported
at org.apache.hive.jdbc.HiveResultSetMetaData.isSigned(HiveResultSetMetaData.java:141)
at com.sun.rowset.CachedRowSetImpl.initMetaData(CachedRowSetImpl.java:726)
at com.sun.rowset.CachedRowSetImpl.populate(CachedRowSetImpl.java:640)
at org.springframework.jdbc.core.SqlRowSetResultSetExtractor.createSqlRowSet(SqlRowSetResultSetExtractor.java:66)
at org.springframework.jdbc.core.SqlRowSetResultSetExtractor.extractData(SqlRowSetResultSetExtractor.java:49)
at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:447)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:395)
... 48 more
Question is why it is not picking org.apache.hive.jdbc.HiveResultSetMetaData.class from overriden jar? find.jsp says the classloader picks this from overriden jar, whereas while executing query it picks from somewhere else.
Java : 1.7_45u
tomcat : 7.0.35

unable to use package-info. javax.xml.ws.WebServiceException: class XXX do not have a property of the name {YYY}document

short: Is it possible to change classes in my package test (add some headers) so I don't have to use packege-info.java?
longer version:
In eclipse my program works fine. But after convertion to exe with commercial encrypting program I am getting error.
javax.xml.ws.WebServiceException: class test.xsd.SendDocument do not have a property of the name {https://YYY/xsd}document
at com.sun.xml.internal.ws.client.sei.BodyBuilder$DocLit.<init>(BodyBuilder.java:191)
at com.sun.xml.internal.ws.client.sei.SEIMethodHandler.<init>(SEIMethodHandler.java:105)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.<init>(SyncMethodHandler.java:73)
at com.sun.xml.internal.ws.client.sei.SEIStub.<init>(SEIStub.java:71)
at com.sun.xml.internal.ws.client.WSServiceDelegate.createEndpointIFBaseProxy(WSServiceDelegate.java:628)
at com.sun.xml.internal.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:331)
at com.sun.xml.internal.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:313)
at com.sun.xml.internal.ws.client.WSServiceDelegate.getPort(WSServiceDelegate.java:295)
at javax.xml.ws.Service.getPort(Service.java:92)
at test.GateService.getGateServiceSOAP12Port(GateService.java:74)
at com.myprogram.model.sign.Declar2TestWS.<init>(Declar2TestWS.java:108)
at com.myprogram.sap.jco.server.SAPServerConnection$SignXMLFunctionHandler.createWS(SAPServerConnection.java:393)
at com.myprogram.sap.jco.server.SAPServerConnection$SignXMLFunctionHandler.handleRequest(SAPServerConnection.java:284)
at com.sap.conn.jco.rt.DefaultServerWorker$FunctionDispatcher.handleRequest(DefaultServerWorker.java:1009)
at com.sap.conn.jco.rt.DefaultServerWorker$FunctionDispatcher.handleRequest(DefaultServerWorker.java:937)
at com.sap.conn.jco.rt.DefaultServerWorker.dispatchRequest(DefaultServerWorker.java:153)
at com.sap.conn.jco.rt.MiddlewareJavaRfc$JavaRfcServer.dispatchRequest(MiddlewareJavaRfc.java:3300)
at com.sap.conn.jco.rt.MiddlewareJavaRfc$JavaRfcServer.listen(MiddlewareJavaRfc.java:2350)
at com.sap.conn.jco.rt.DefaultServerWorker.dispatch(DefaultServerWorker.java:277)
at com.sap.conn.jco.rt.DefaultServerWorker.loop(DefaultServerWorker.java:337)
at com.sap.conn.jco.rt.DefaultServerWorker.run(DefaultServerWorker.java:238)
at java.lang.Thread.run(Thread.java:619)
Caused by: javax.xml.bind.JAXBException: {https://YYY/xsd}document is not a valid property on class test.xsd.SendDocument
at com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.getElementPropertyAccessor(JAXBContextImpl.java:931)
at com.sun.xml.internal.ws.client.sei.BodyBuilder$DocLit.<init>(BodyBuilder.java:188)
... 21 more
I manage to get the same error in eclipse by deleting packege-info.java
packege-info.java:
#javax.xml.bind.annotation.XmlSchema(namespace = "https://YYY/xsd", elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED)
package test;
Is it possible to change classes in my package test (add some headers) so I don't have to use packege-info.java?
Thank you for the help!