Develop Reference

SOLR 7.4.0 exception: java.lang.NoClassDefFoundError: org/apache/solr/handler/dataimport/FieldStreamDataSource

I am trying to implement a custom SOLR FieldStreamDataSource. This is the code of my Java class:
package MySOLR;
import org.apache.solr.handler.dataimport.*;
public class MyFieldStreamDataSource extends FieldStreamDataSource {
}
My SOLR is installed in C:\solr\solr-7.4.0. So I added the following jar to my NetBeans project
C:\solr\solr-7.4.0\dist\solr-dataimporthandler-7.4.0.jar
and compiled it to MySOLR.jar
Then I placed MySOLR.jar into C:\solr\solr-7.4.0\server\lib\ext and added the following to my SOLR DIH config file:
...
<dataSource name="fieldStreamDataSource" type="MySOLR.MyFieldStreamDataSource" />
...
<entity name="tika_RTF_NOTES" onError="continue" processor="TikaEntityProcessor" dataField="ts0_tika.RTF_NOTES" dataSource="fieldStreamDataSource">
<field name="text" column="text" />
</entity>
...
I added the following entries to my SOLR core's solrconfig.xml file:
<lib dir="C:/solr/solr-7.4.0/contrib/dataimporthandler/lib" regex=".*\.jar" />
<lib dir="C:/solr/solr-7.4.0/dist/" regex="solr-dataimporthandler-\d.*\.jar" />
<!--lib path="C:/solr/solr-7.4.0/dist/solr-dataimporthandler-7.4.0.jar" /-->
<lib dir="C:/solr/solr-7.4.0/contrib/dataimporthandler-extras/lib" regex=".*\.jar" />
<lib dir="C:/solr/solr-7.4.0/dist/" regex="solr-dataimporthandler-extras-\d.*\.jar" />
After re-starting my SOLR instance (Windows service) I try to run the DIH via the SOLR web interface. I get the following exception in the solr.log file (and in the web interface):
2019-03-18 11:08:12.164 ERROR (qtp1986417638-24) [ x:iET] o.a.s.s.HttpSolrCall null:java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/solr/handler/dataimport/FieldStreamDataSource
at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:662)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:530)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
...
I am pretty sure that my entries in solrconfig.xml are correct as when I remove the entries I see that SOLR is picking up less libraries. I can see that in the solr.log file when looking at
o.a.s.c.SolrResourceLoader [...] Added ... libs to classloader, from paths: ...
What am I missing?
Many thanks in advance,
Michael

By chance I found the solution. There are 2 alternatives (for me):
Place MySOLR.jar in c:\solr\solr-7.4.0\contrib\dataimporthandler\lib
Place MySOLR.jar in c:\solr\solr-7.4.0\dist and add a reference to it in solrconfig.xml

Log4j jdbc appender for DB2 in Tomcat 8

I'm trying to use Log4j 2 to save the logs generated from a web application into a table in a DB2 10.5 database. As a servlet container i'm using Tomcat 8.
My data source declaration, inside Tomcat's context.xml:
<Resource
type="javax.sql.DataSource"
name="jdbc/NCRDS"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.ibm.db2.jcc.DB2Driver"
url="jdbc:db2://192.168.1.58:50000/NCR"
username="admin"
password="admin"
initialSize="0"
maxActive="80"
maxIdle = "30"
minIdle="20"
timeBetweenEvictionRunsMillis="30000"
minEvictableIdleTimeMillis="60000"
testOnBorrow="true"
validationQuery="VALUES 1"
validationInterval="30000"
removeAbandoned="true"
removeAbandonedTimeout="60"
logAbandoned="true"
abandonWhenPercentageFull="60"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ResetAbandonedTimer"/>
The appender declared insinde log4j2.xml:
<JDBC name="databaseAppender" tableName="admin.logs">
<DataSource jndiName="java:/comp/env/jdbc/NCRDS" />
<Column name="evendDate" isEventTimestamp="true" />
<Column name="Level" pattern="%level" />
<Column name="Logger" pattern="%logger" />
<Column name="Message" pattern="%msg" />
</JDBC>
The jdbc resource is also declared in my application's web.xml file. While i've managed to insert the logs on a MySQL database (changing of course the required information in my configuration files), i'm not able to achieve the same in DB2. The following stack trace part is what seems to be causing the problem:
Caused by: java.lang.AbstractMethodError: com.ibm.db2.jcc.am.po.setNString(ILjava/lang/String;)V
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.tomcat.jdbc.pool.interceptor.AbstractQueryReport$StatementProxy.invoke(AbstractQueryReport.java:233)
at com.sun.proxy.$Proxy40.setNString(Unknown Source)
at org.apache.logging.log4j.core.appender.db.jdbc.JdbcDatabaseManager.writeInternal(JdbcDatabaseManager.java:110)
at org.apache.logging.log4j.core.appender.db.AbstractDatabaseManager.write(AbstractDatabaseManager.java:167)
at org.apache.logging.log4j.core.appender.db.AbstractDatabaseAppender.append(AbstractDatabaseAppender.java:105)
at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:99)
at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:430)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:409)
at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:367)
at org.apache.logging.log4j.core.Logger.logMessage(Logger.java:112)
at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:738)
at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:708)
at org.apache.logging.log4j.spi.AbstractLogger.debug(AbstractLogger.java:237)
at com.ikubinfo.fileservice.FileSystemFileManager.<init>(FileSystemFileManager.java:53)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147)
... 24 more
I can't seem to find any information regarding this error and i really can explain to myself what's going wrong. I'd be very grateful for your help.
What i'm using: Eclipse Luna, Tomcat 8, DB2 10.5, db2jcc.jar , jdk1.7, log4j required jars.

This: Caused by: java.lang.AbstractMethodError: com.ibm.db2.jcc.am.po.setNString basically says that the DB2 driver does not implement the method setNString(), which Log4j is trying to use at org.apache.logging.log4j.core.appender.db.jdbc.JdbcDatabaseManager.writeInternal(JdbcDatabaseManager.java:110).
Now, what to do about it I can't tell. May be there is a parameter that can instruct Log4j to use setString() instead, or may be, Log4j being open source, you could edit the JdbcDatabaseManager class.

The problem here, as mustaccio correctly stated, was that the setNString() method was called which was trying to set var args, but instead, the log4j2 implementation could offer only a setString() method for a DB2 configuration . This call was caused by the defined column names. My first case:
<Column name="Date" isEventTimestamp="true" />
<Column name="Level" pattern="%level" isUnicode="false"/>
<Column name="Logger" pattern="%logger" isUnicode="false"/>
<Column name="Message" pattern="%msg" isUnicode="false"/>
You can see the names had both upper and lower case chars. Now i've got two solutions:
1. I could have added the HTML entity for the quotation mark, because DB2 needs quotation marks to read lower-case chars. So the final form would have been: <Column name=""Date""
2. The one which i'm following, rename the table names to upper-cases and simply do the same on the column references in log4j2.xml: <Column name="DATE"

com.spatial4j.core.exception.InvalidShapeException: java.lang.NumberFormatException

I am using following field type with solr 4.2 & its working fine.
<field name="latlong" type="location_rpt" indexed="true" stored="true" required="false" multiValued="false" />
<fieldType name="location_rpt" class="solr.SpatialRecursivePrefixTreeFieldType"
distErrPct="0.025"
maxDistErr="0.000009"
units="degrees"
/>
But, when I am upgrading solr to solr 4.7.1, it is reporting following errors while posting new docs:
Caused by: com.spatial4j.core.exception.InvalidShapeException: java.lang.NumberFormatException: For input string: "78.42968,30.73333,78.45"
at org.apache.solr.util.SpatialUtils.parsePoint(SpatialUtils.java:68)
at org.apache.solr.util.SpatialUtils.parsePointSolrException(SpatialUtils.java:84)
... 45 more
Caused by: java.lang.NumberFormatException: For input string: "78.42968,30.73333,78.45"
I am posting following doc value to solr.
<field name='latlong'><![CDATA[30.9308,78.42968,30.73333,78.45]]></field>

There are 3 Values in the String. If I read the documentation correct it will only expect x,y.

Solr 4.3.0 Http Status 500

OS: Ubuntu 12.04
I just upgraded my system from using Solr 1.4 to Solr 4.3.0 and can't seem to get the MySQL driver to work (or so I suspect). Solr seems to work fine (accessing it through the browser, etc) until I add the following lines to the solrconfig.xml
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">/opt/solr/core01/conf/data-config.xml</str>
</lst>
</requestHandler>
After adding those lines into the config, restart Tomcat7 and refreshing my localhost:8080/solr, I get the following error:
HTTP Status 500 - {msg=SolrCore 'core01' is not available due to init failure: org/apache/solr/util/plugin/SolrCoreAware,trace=org.apache.solr.common.SolrException:
SolrCore 'core01' is not available due to init failure: org/apache/solr/util/plugin/SolrCoreAware at
And then a bunch of misc. filters, etc. I only see this error via the browser - I can't seem to find it in any of the logs for Tomcat.
Another thing to note is that I have included the required JAR files for the:
solr-dataimporthandler-extras-4.3.0.jar
solr-dataimporthandler-4.3.0.jar
as they seem to not come with the download for Solr 4.3.0
This problem has caused so much time to be wasted so hopefully someone on here can lend a hand and see what is wrong.
Thanks in advance!
EDIT:
This is what my data-config.xml looks like
<?xml version="1.0" encoding="UTF-8"?>
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost/db_name" user="db_user" password="db_password" />
<document>
<entity name="table" query="SELECT * FROM table">
<field column="id" name="id" />
<field column="name" name="name" />
</entity>
</document>
</dataConfig>
With the obvious variables replaced with real data.

After looking at the DIHQuickStart page and looking at the error meessage, you might want to remove the path from your config setting, as Solr will look in the conf folder of the core where you have defined the DIH first by default. So change it to:
<str name="config">data-config.xml</str>

Data Import Handler in Solr

I am trying the Data Import Handler for MySQL Database.
I added the DIhandler in solrconfig.xml, created a data-config.xml according to my database scheme and also added a field in the schema.xml which was different. I am connecting with MySQL database
After i connect and I run the dataimport?command=full-import i get this response
"00C:\solr\conf\data-config.xmlfull-importidle1102011-03-05 15:01:04Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.2011-03-05 15:01:042011-03-05 15:01:040:0:0.400This response format is experimental. It is likely to change in the future."
The xml files are in this http://pastebin.com/iKebKGSZ

<field column="manu" name="manu" />
<field column="id" name="id" />
<field column="weight" name="weight" />
<field column="price" name="price" />
<field column="popularity" name="popularity" />
<field column="instock" name="inStock" />
<field column="includes" name="includes" />
Are these fields also in your schema.xml?
I couldnt see them in the pastebin link. Make sure you have all fields in your schema as well.