Querying Solr server using SolrJ - java

I'm trying to query my solr database based off the code provided in this question with SolrJ but it keeps throwing a null pointer exception.
My code is:
#PUT
#Produces(MediaType.TEXT_PLAIN)
public String returnText(String url) throws MalformedURLException, SolrServerException{
SolrServer server = new HttpSolrServer("http://localhost:8080/apache-solr-1.4.0");
ModifiableSolrParams params = new ModifiableSolrParams();
params.set("?q", "*:*");
params.set("facet", true);
params.set("rows", 5);
QueryResponse response = server.query(params);
System.out.println(response);
return "success";
}
and if I run this url: http://localhost:8080/apache-solr-1.4.0/select/?q=*:*&facet=true&rows=5 in my browser then it returns the correct values however if I enter: http://localhost:8080/apache-solr-1.4.0/select/q=*:*&facet=true&rows=5 then I get the exact same error as when I run it in eclipse, the error is:
19/12/2012 1:09:01 PM com.sun.jersey.spi.container.ContainerResponse mapMappableContainerException
SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
org.apache.solr.client.solrj.SolrServerException: Server at http://localhost:8080/apache-solr-1.4.0 returned non ok status:500, message:null java.lang.NullPointerException at java.io.StringReader.<init>(StringReader.java:33) at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:197) at org.apache.solr.search.LuceneQParser.parse(LuceneQParserPlugin.java:78) at org.apache.solr.search.QParser.getQuery(QParser.java:131) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:89) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188) at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641) at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97) at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185) at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:332) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:233) at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165) at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791) at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693) at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954) at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170) at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102) at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88) at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76) at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53) at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57) at com.sun.grizzly.ContextTask.run(ContextTask.java:69) at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330) at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309) at java.lang.Thread.run(Thread.java:619)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:328)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:211)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:89)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:311)
at geoportal.webservice.download.returnText(download.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
.......
If I change my code to params.set("q","*:*"); then I get this error:
19/12/2012 1:13:30 PM com.sun.jersey.spi.container.ContainerResponse mapMappableContainerException
SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
org.apache.solr.client.solrj.SolrServerException: Error executing query
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:95)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:311)
at geoportal.webservice.download.returnText(download.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
.......
Any ideas as to what I've done wrong?
Thanks heaps
EDIT
Here is my new code for the solrQuery based off this:
#PUT
#Produces(MediaType.TEXT_PLAIN)
public String returnText(String url) throws MalformedURLException, SolrServerException{
SolrServer server = new HttpSolrServer("http://localhost:8080/apache-solr-1.4.0");
SolrQuery query = new SolrQuery();
query.setQuery("*:*");
query.setFacet(true);
query.set("wt", "json");
query.setRows(5);
query.setStart(0);
QueryResponse response = server.query(query);
System.out.println(response);
return "success";
}
But it still isn't working :(
Console output:
19/12/2012 2:24:33 PM com.sun.jersey.spi.container.ContainerResponse mapMappableContainerException
SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
org.apache.solr.client.solrj.SolrServerException: Error executing query
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:95)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:311)
at geoportal.webservice.download.returnText(download.java:39)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
......
EDIT
I put a System.out.println(query.toString()); in my code and it outputted:
q=*%3A*&facet=true&wt=json&rows=5&start=0
If I enter this into my browser like
http://localhost:8080/apache-solr-1.4.0/select/q=*%3A*&facet=true&wt=json&rows=5&start=0
then it throws a null pointer exception again. However if I manually enter a ? in front of q=*%3A*... then it works. So I assume that my solr server needs this ? (is this because it's so old?), is there any way to hard code one in?
Thanks

Try use the SolrQuery object instead of ModifiableSolrParams. Maybe it will help.
If you cant use SolrQuery for any reason, try using statics names, like "CommonParams.Q" instead of hardcoded ones like "?q"
EDITED
I tested your problem and I think you are lacking configurations at your Application Server.
Are you using JBoss 7.1? You need to add a line to .standalone.sh or standalone.bat telling where solr is. For example, in Jboss 7.1, in default configurations, you have to add \"-Dsolr.solr.home=$SOLR_HOME/example/solr\" \ to the standalone.sh
I dont know about others Application Servers, but you can search a little and see how you can do that in another AS.

Try using SolrQuery, you have option to set everything like query, filter, facet, start, rows etc. Below is a sample code
on the latest solr 6.2.1 you can create solr client like below:
SolrClient solr = new HttpSolrClient.Builder("<url>").build();
SolrQuery query = new SolrQuery();
query.setQuery("collector:" + input);
query.addFilterQuery("(doc_type:" + entity + ")");
query.addSort("lastUpdatedAt", ORDER.desc);
query.setFacet(true);
query.setStart(pagenumber);
query.setRows(pagesize);
QueryResponse response = solr.query(query);

You might not have the same version of Solr and SolrJ.
You might be making a bad decision by using such an ancient version of Solr at all.
You are failing to check the Solr log to find out where the 500 is coming from.
You are using ModifiableSolrParams instead SolrQuery.

As the previous comments suggests try using SolrQuery.
Here is the link that may help solrj example
I think you're missing response.getResults();
this will return a SolrDocumentList which you can handle using an Iterator and loop through the results
like:
SolrDocumentList docs = response.getResults();
Iterator<SolrDocument> iter = docs.iterator();
while (iter.hasNext())
{
// handle results
}

Related

Quarkus: The return value of "org.jboss.resteasy.reactive.server.mapping.RuntimeResource.getProduces()" is null

I created Reactive SQL client on Quarkus using this official guide, but while making GET request on http://localhost:8080/hello trying to query data from database, I get such an error:
2022-03-06 13:18:53,051 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (vert.x-eventloop-thread-5) HTTP Request to /hello failed, error id: 2796a874-61d9-43ee-bd87-6f098cffa484-2: java.lang.NullPointerException: Cannot invoke "org.jboss.resteasy.reactive.common.util.ServerMediaType.getSortedMediaTypes()" because the return value of "org.jboss.resteasy.reactive.server.mapping.RuntimeResource.getProduces()" is null
at org.jboss.resteasy.reactive.server.handlers.PublisherResponseHandler.handle(PublisherResponseHandler.java:203)
at org.jboss.resteasy.reactive.server.handlers.PublisherResponseHandler.handle(PublisherResponseHandler.java:30)
at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:141)
at org.jboss.resteasy.reactive.server.handlers.RestInitialHandler.beginProcessing(RestInitialHandler.java:49)
at org.jboss.resteasy.reactive.server.vertx.ResteasyReactiveVertxHandler.handle(ResteasyReactiveVertxHandler.java:17)
at org.jboss.resteasy.reactive.server.vertx.ResteasyReactiveVertxHandler.handle(ResteasyReactiveVertxHandler.java:7)
at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1212)
at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:163)
at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:141)
at io.quarkus.vertx.http.runtime.StaticResourcesRecorder$2.handle(StaticResourcesRecorder.java:67)
at io.quarkus.vertx.http.runtime.StaticResourcesRecorder$2.handle(StaticResourcesRecorder.java:55)
at io.vertx.ext.web.impl.RouteState.handleContext(RouteState.java:1212)
at io.vertx.ext.web.impl.RoutingContextImplBase.iterateNext(RoutingContextImplBase.java:126)
at io.vertx.ext.web.impl.RoutingContextImpl.next(RoutingContextImpl.java:141)
at io.vertx.ext.web.handler.impl.StaticHandlerImpl.lambda$sendStatic$1(StaticHandlerImpl.java:274)
at io.vertx.core.impl.future.FutureImpl$3.onSuccess(FutureImpl.java:141)
at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
You need to add something like:
#Produces(MediaType.APPLICATION_JSON)
// Each element will be sent as JSON
#RestSseElementType(MediaType.APPLICATION_JSON)
to your JAX-RS method.
However, you most likely also need to include quarkus-resteasy-reactive-jackson.
Furthermore, you might want to revisit the use of Multi as a return type as you are likely looking for something like Uni<List<Address>>.

DocumentBuilder.parse() throwing java.net.UnknownHostException after changing a webservice URL from http to https

Previously , we had following code in our application and it used to fetch the required data. We just used to read the required fields by forming a web-service URL by passing username , password and search parameter (DEA number). The same URL (with parameters) could also be hit from browser directly to see the results :
{ DocumentBuilderFactory factory =
DocumentBuilderFactory.newInstance();
DocumentBuilder parser = factory.newDocumentBuilder();
System.setProperty("http.proxyHost", getProxyHost());
System.setProperty("http.proxyPort", getProxyPort());
System.out.println("Before URL formation");
URL url = new URL(
"http://www.deanumber.com/Websvc/deaWebsvc.asmx/GetQuery?UserName=" + getDeaUsername() + "&Password=" + getDeaPassword() + "&DEA=" +
deaNumber +
"&BAC=&BASC=&ExpirationDate=&Company=&Zip=&State=&PI=&MaxRows=");
System.out.println("After URL formation");
System.out.println("URL formed is: "+url);
Document document = null;
try {
System.out.println("toExternalForm");
String strURL = url.toExternalForm();
System.out.println("toExternalForm done: "+strURL);
**document = parser.parse(strURL);** //This is causing exception
}
catch (SAXParseException spe)
{ System.out.println("Inside SAXParseException");
spe.printStackTrace();
}
}
Now, the web-service URL has got changed from http to https with rest of the things same. The new URL also works from the browser. Moreover, even if I enter the http URL in the browser , it automatically redirects to https and shows the data. However, the above code is not doing the same and it gave following exception : java.net.ProtocolException: Unsupported protocol: https'
So, I changed the code to use https in the url, but now a new exception is coming - java.net.UnknownHostException. From the console log, I can see that the exception is coming at ==> document = parser.parse(strURL). Here , parser is a reference variable for a DocumentBuilder object as you can see from the code above. I am pasting the complete stack trace below.
Complete stack trace for UnknownHostException:
{java.net.UnknownHostException: www.deanumber.com
[2017-23-28 08:23, 0]ERROR[[ACTIVE] ExecuteThread: '6' for queue: 'weblogic.kernel.Default (self-tuning)'](DeaFetcher.java:123) - Exception in DeaFetcher.getDeaFromUrl java.net.UnknownHostException: www.deanumber.com
at java.net.InetAddress.getAllByName0(InetAddress.java:1250)
at java.net.InetAddress.getAllByName(InetAddress.java:1162)
at java.net.InetAddress.getAllByName(InetAddress.java:1098)
at weblogic.net.http.HttpsClient.openServer(HttpsClient.java:265)
at weblogic.net.http.HttpsClient.openServer(HttpsClient.java:350)
at weblogic.net.http.HttpsClient.New(HttpsClient.java:553)
at weblogic.net.http.HttpsURLConnection.getHttpClient(HttpsURLConnection.java:332)
at weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:711)
at weblogic.net.http.SOAPHttpsURLConnection.getInputStream(SOAPHttpsURLConnection.java:37)
at com.sun.org.apache.xerces.internal.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:643)
at com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:189)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:812)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
at weblogic.xml.jaxp.RegistryDocumentBuilder.parse(RegistryDocumentBuilder.java:163)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:177)
at com.company.project.service.DeaFetcher.fetchDeaFromUrl(DeaFetcher.java:358)
at com.company.project.service.DeaFetcher.fetchDeas(DeaFetcher.java:330)
at com.company.project.service.DeaFetcher.fetchDeas(DeaFetcher.java:239)
at com.company.project.service.DeaFetcher.fetchDeaMastersList(DeaFetcher.java:46)
at com.company.project.service.DeaFetcher$$FastClassByCGLIB$$8f6b7575.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:191)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:695)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:144)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:107)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:166)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:630)
at com.company.project.service.DeaFetcher$$EnhancerByCGLIB$$a5bf2c5a.fetchDeaMastersList(<generated>)
at com.company.project.web.MainFormController.onSubmit(EntryFormController.java:137)
at org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:267)
at org.springframework.web.servlet.mvc.AbstractFormController.handleRequestInternal(AbstractFormController.java:250)
at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:857)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:792)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:475)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:440)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:751)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:844)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:280)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:254)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:136)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:346)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:243)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3432)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3402)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
at weblogic.servlet.internal.WebAppServletContext.doSecuredExecute(WebAppServletContext.java:2285)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2201)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2179)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1572)
at weblogic.servlet.provider.ContainerSupportProviderImpl$WlsRequestExecutor.run(ContainerSupportProviderImpl.java:255)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:311)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:263)
}
It looks like you are still trying call a web service using the same URL, but this time with HTTPS in the URL. If that is the case, then you may need to invoke the web service using the client code generated by the WSDL of the web service along with proper authentication code.
If you are calling the web service using a valid client code generated from the WSDL, then please provide the client code details in your question to see what exactly it is trying to do here.
You write in the comments that you have the WSDL, then let eclipse generate the client for this, for example following: How do you convert wsdl to java classes using Eclipse? (or search in the internet for "generate client from wsdl")

RestTemplate postForEntity experiencing java.io.IOException: Premature EOF error

Would appreciate any help regarding my issue on one of my maven projects.
Exception in thread "main" org.springframework.web.client.ResourceAccessException: I/O error on POST request for "https://test-services.domain.ph/campaign/": Premature EOF; nested exception is java.io.IOException: Premature EOF
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:666)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.postForEntity(RestTemplate.java:407)
at homecredit.ph.CampaignConnector.call(CampaignConnector.java:46)
Caused by: java.io.IOException: Premature EOF
at sun.net.www.http.ChunkedInputStream.readAheadBlocking(ChunkedInputStream.java:565)
at sun.net.www.http.ChunkedInputStream.readAhead(ChunkedInputStream.java:609)
at sun.net.www.http.ChunkedInputStream.read(ChunkedInputStream.java:696)
at java.io.FilterInputStream.read(FilterInputStream.java:133)
Origin:
ResponseEntity<ApiResponse> response = restTemplate.postForEntity(url, entity, ApiResponse.class);
Destination:
#RequestMapping(value="/campaign", method = RequestMethod.POST, consumes=MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<ApiResponse> insertCampaignRecord(
#Valid #RequestBody CampaignRecordInsertRequest campaignRecordInsertRequest){
logInfo("Incoming insert request. " + DescriptorUtility.convertToString(campaignRecordInsertRequest));
campaignDataService.insertNewRecord(CampaignRecordConverter.convertToCampaignRecord(campaignRecordInsertRequest));
return ResponseUtility.defaultResponse();
}
ResponseUtility
public static ResponseEntity<ApiResponse> defaultResponse(){
ApiResponse apiResponse = new ApiResponse();
apiResponse.setTimestamp(DateUtility.currentDateString());
apiResponse.setMessage(ResponseMessages.SUCCESS);
return new ResponseEntity<>(apiResponse, HttpStatus.OK);
}
CampaignData Service
#Async("AsyncExecutor")
public void insertNewRecord(CampaignRecord campaignRecord) {
try {
campaignRecordRepository.save(campaignRecord);
} catch (Exception e) {
logError(e);
}
}
Server Log
2017-09-11 11:11:11 INFO 18383 [http-nio-8773-exec-10] [CampaignRecordController] - Incoming insert request. {"dateCampaign":1504656000000,"cuid":...
2017-09-11 11:11:11 WARN 18383 [http-nio-8773-exec-10] [SqlExceptionHelper] - SQL Error: 1062, SQLState: 23000
2017-09-11 11:11:11 ERROR 18383 [http-nio-8773-exec-10] [SqlExceptionHelper] - Duplicate entry 'CMP_CLX##1208637#20170906' for key 'UNIQUE_KEY'
2017-09-11 11:11:11 ERROR 18383 [http-nio-8773-exec-10] [CampaignDataService] - could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
2017-09-11 11:11:11 ERROR 18383 [http-nio-8773-exec-10] [CampaignDataService] - could not execute statement
PS. Server logs is normal(return a successful response either record successfully saved or not)
Issue is intermittent. Occurs randomly when sending bulk requests.
Thanks in advance! :)
I had the same problem in Spring Boot 2.1. In my case I had 3 apps (call them A, B, and C) where B was really just a proxy between A and C:
A --> B --> C
The Premature EOF was occurring on the response from B to A. All indications were a successful HTTP response (200), but inspecting the body of the response using a debugger revealed it had a new line character in the middle of the serialized DTO data, instead of at the end where I expected it:
(Notice the return character after the id field, and lack of any content length; ignore the unreadable boxes at the end, they're part of the byte array that are not initialized/used)
In my case, Service B is both a server and a client. The code looked something like this:
public ResponseEntity<String> handle(String request, HttpHeaders headers) {
// Do some validation and then call Service C, and pass the response
// back to Service A
return restTemplate.postForEntity(
urlForServiceC,
new HttpEntity<>(request, headers),
String.class);
}
I didn't dive too far into the guts of RestTemplate or its message converters, but what tipped me off that there might be an issue with the response buffering is that I was using a Spring filter to log the responses of each service. This filter has to copy the response stream to avoid exceptions from other filters related to the body already being consumed.
What I noticed is that when I ran with this filter enabled, the Premature EOF exceptions went away. And when I disabled it, the exceptions came back. Something about copying the response stream had solved the Premature EOF errors.
This led me to try the following in Service B:
public ResponseEntity<String> handle(String request, HttpHeaders headers) {
// Do some validation and then call Service C, and pass the response
// back to Service A
String response = restTemplate.postForEntity(
urlForServiceC,
new HttpEntity<>(request, headers),
String.class).getBody();
return ResponseEntity.ok(response);
}
The subtle change is that I'm saving the response first to a local variable, which requires me to call ResponseEntity.getBody(). This forces the entire response body from Service C to be consumed before returning to Service A. After making this change my Premature EOF errors have not returned.
Based on the server logs seems like, the server is trying save some record and its failing (due to Unique key violation).
2017-09-11 11:11:11 WARN 18383 [http-nio-8773-exec-10] [SqlExceptionHelper] - SQL Error: 1062, SQLState: 23000
2017-09-11 11:11:11 ERROR 18383 [http-nio-8773-exec-10] [SqlExceptionHelper] - Duplicate entry 'CMP_CLX##1208637#20170906' for key 'UNIQUE_KEY'
2017-09-11 11:11:11 ERROR 18383 [http-nio-8773-exec-10] [CampaignDataService] - could not execute statement; SQL [n/a]; constraint [null]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
2017-09-11 11:11:11 ERROR 18383 [http-nio-8773-exec-10] [CampaignDataService] - could not execute statement
Looks like the server is not able to handle the exception gracefully and the whole flow breaks, causing a HTTP-500 response code (probably) with an empty response.
Two actions you can take:
Handle the exception gracefully.
Verify why unique key is getting violated and fix that if possible.
For anyone who might be experiencing this.
Issue was caused by spring boot eureka.
Seems like there is a bug when it comes to passing ResponseEntity response in a massive scale (bulk processing) that causes the response status to be malformed.
Current workaround is to switch from ResponseEntity to the object body instead.
I have the same problem when i use RestTemplate with default http client factory.I found it missing 'Accept-Encoding:gzip' in the headers when I capture packets.Finally i get it by replace the default http client factory with apache's http client factory,like this:
HttpComponentsClientHttpRequestFactory clientHttpRequestFactory = new HttpComponentsClientHttpRequestFactory(HttpClientBuilder.create().build());
RestTemplate restTemplate = new RestTemplate(clientHttpRequestFactory);

How to convert Elasticsearch Query Builder response into CSV format in Java?

I am trying to Convert my Elasticsearch query response i.e. in Json format into CSV format by using the Json2Flat library but it is not working. Here is the Java code.
The query argument of Query Builder is coming from URL using Java Servlets and Jetty Server.
SearchResponse response = client.prepareSearch(index)
.setQuery(QueryBuilders.matchQuery("user", query))
.setSize(100)
.execute()
.actionGet();
System.out.println("searchKeyWord : " + response);
String doc=response.toString();
JFlat flatMe = new JFlat(doc);
//directly write the JSON document to CSV
flatMe.json2Sheet().write2csv("/home/alihas/Desktop/file.json");
//directly write the JSON document to CSV but with delimiter
flatMe.json2Sheet().write2csv("/home/alihas/Desktop/file1.json",'|');
The error that I got is:
HTTP ERROR 500
Problem accessing /. Reason:
com/google/gson/JsonElement
Caused by:
java.lang.NoClassDefFoundError: com/google/gson/JsonElement
at Elasticsearch.elasticsearch.searchKeyWord(elasticsearch.java:296)
at Elasticsearch.elasticsearch.mainMe(elasticsearch.java:98)
at Elasticsearch.QuerySelector.doGet(QuerySelector.java:39)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:426)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:931)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:361)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:867)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
at org.eclipse.jetty.server.Server.handle(Server.java:337)
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:581)
at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1005)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:560)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:222)
at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:474)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:437)
at java.lang.Thread.run(Thread.java:745)
Any help is highly appreciated.

getQueryString on ExternalContext's Request object returns null on WebSphere

I have a problem using JSF 1.1 (MyFaces 1.1.8) on WebSphere. When deploying my application on Tomcat 5.5, this behavior does not apply and everything works fine.
When I try to read parameters from an URL posted back from Ogone (payment gateway), joining my JSF-session again (session still exists), the QueryString is not available through
final HttpServletRequest request = (HttpServletRequest)FacesContext
.getCurrentInstance().getExternalContext().getRequest();
String qs = request.getQueryString(); // null on WebSphere, filled on Tomcat
To be a bit more complete, the query string looks something like this:
http://100.100.100.100:9080/my_app_context/pages/screen7.jsf?orderID=4208013442596271&currency=EUR&amount=60&PM=iDEAL&ACCEPTANCE=0000000000&STATUS=9&CARDNO=11-XXXX-11&ED=&CN=K%C3%A9m+Birdy&TRXDATE=08%2F06%2F12&PAYID=15879916&NCERROR=0&BRAND=iDEAL&COMPLUS=personID%3D01%26subscriptionID%3D02&IP=100.100.100.100&SHASIGN=9642468EE016BD49787B62EA7231124B3C24F9BBFC5B3B3B4D9E97D678979E57E5EED9B47D611441A6FCA2EB1D2DB87348C8111ACFF366A4E244106CCCFAFDA5
I'm thinking of some WebSphere webcontainer setting that might prevent the container from handling the external request properly. Or maybe there's some classloading issue here, but I haven't been able to track it down.
The stack trace on WebSphere is like this:
12-08-07 08:02:26 ERROR -
java.lang.NullPointerException
at ... own code trying to getQueryString()...
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:618)
at org.apache.myfaces.el.PropertyResolverImpl.getProperty(PropertyResolverImpl.java:459)
at org.apache.myfaces.el.PropertyResolverImpl.getValue(PropertyResolverImpl.java:85)
at org.apache.myfaces.custom.security.SecurityContextPropertyResolver.getValue(SecurityContextPropertyResolver.java:101)
at org.apache.myfaces.el.ELParserHelper$MyPropertySuffix.evaluate(ELParserHelper.java:539)
at org.apache.commons.el.ComplexValue.evaluate(ComplexValue.java:145)
at org.apache.myfaces.el.ValueBindingImpl.getValue(ValueBindingImpl.java:386)
at javax.faces.component.UIOutput.getValue(UIOutput.java:118)
at org.apache.myfaces.shared_impl.renderkit.RendererUtils.getValue(RendererUtils.java:267)
at org.apache.myfaces.shared_impl.renderkit.RendererUtils.getStringValue(RendererUtils.java:226)
at org.apache.myfaces.shared_impl.renderkit.html.HtmlTextRendererBase.renderOutput(HtmlTextRendererBase.java:69)
at org.apache.myfaces.shared_impl.renderkit.html.HtmlTextRendererBase.encodeEnd(HtmlTextRendererBase.java:57)
at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:799)
at javax.faces.webapp.UIComponentTag.encodeEnd(UIComponentTag.java:678)
at javax.faces.webapp.UIComponentTag.doEndTag(UIComponentTag.java:546)
at com.ibm._jsp._aanmeldScherm7._jspx_meth_h_outputText_1(_aanmeldScherm7.java:204)
at com.ibm._jsp._aanmeldScherm7._jspx_meth_f_subview_0(_aanmeldScherm7.java:1870)
at com.ibm._jsp._aanmeldScherm7._jspService(_aanmeldScherm7.java:85)
at com.ibm.ws.jsp.runtime.HttpJspBase.service(HttpJspBase.java:88)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1213)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1154)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:118)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:848)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:691)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:654)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:526)
at com.ibm.wsspi.webcontainer.servlet.GenericServletWrapper.handleRequest(GenericServletWrapper.java:122)
at com.ibm.ws.jsp.webcontainerext.AbstractJSPExtensionServletWrapper.handleRequest(AbstractJSPExtensionServletWrapper.java:248)
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.include(WebAppRequestDispatcher.java:648)
at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:1042)
at org.apache.jasper.runtime.PageContextImpl.include(PageContextImpl.java:484)
at org.apache.struts.tiles.TilesUtilImpl.doInclude(TilesUtilImpl.java:99)
at org.apache.struts.tiles.TilesUtil.doInclude(TilesUtil.java:135)
at org.apache.struts.taglib.tiles.InsertTag.doInclude(InsertTag.java:760)
at org.apache.struts.taglib.tiles.InsertTag$InsertHandler.doEndTag(InsertTag.java:892)
at org.apache.struts.taglib.tiles.InsertTag.doEndTag(InsertTag.java:462)
at com.ibm._jsp._defaultLayout2._jspx_meth_tiles_insert_2(_defaultLayout2.java:392)
at com.ibm._jsp._defaultLayout2._jspx_meth_h_form_0(_defaultLayout2.java:597)
at com.ibm._jsp._defaultLayout2._jspx_meth_f_view_0(_defaultLayout2.java:715)
at com.ibm._jsp._defaultLayout2._jspService(_defaultLayout2.java:92)
at com.ibm.ws.jsp.runtime.HttpJspBase.service(HttpJspBase.java:88)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1213)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1154)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:118)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:848)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:691)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:654)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:526)
at com.ibm.wsspi.webcontainer.servlet.GenericServletWrapper.handleRequest(GenericServletWrapper.java:122)
at com.ibm.ws.jsp.webcontainerext.AbstractJSPExtensionServletWrapper.handleRequest(AbstractJSPExtensionServletWrapper.java:248)
at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.forward(WebAppRequestDispatcher.java:325)
at org.apache.myfaces.context.servlet.ServletExternalContextImpl.dispatch(ServletExternalContextImpl.java:426)
at org.apache.myfaces.tomahawk.application.jsp.JspTilesViewHandlerImpl.dispatch(JspTilesViewHandlerImpl.java:236)
at org.apache.myfaces.tomahawk.application.jsp.JspTilesViewHandlerImpl.renderView(JspTilesViewHandlerImpl.java:222)
at org.ajax4jsf.framework.ViewHandlerWrapper.renderView(ViewHandlerWrapper.java:101)
at org.ajax4jsf.framework.ajax.AjaxViewHandler.renderView(AjaxViewHandler.java:221)
at org.apache.myfaces.lifecycle.RenderResponseExecutor.execute(RenderResponseExecutor.java:41)
at org.apache.myfaces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:146)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:147)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1213)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1154)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:145)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:75)
at org.ajax4jsf.framework.ajax.xmlfilter.BaseFilter.doFilter(BaseFilter.java:213)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:301)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:190)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:130)
at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:87)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:848)
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:691)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:654)
at com.ibm.ws.wswebcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:526)
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:90)
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:764)
at com.ibm.ws.wswebcontainer.WebContainer.handleRequest(WebContainer.java:1478)
at com.ibm.ws.webcontainer.channel.WCChannelLink.ready(WCChannelLink.java:133)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:457)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.handleNewRequest(HttpInboundLink.java:515)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.processRequest(HttpInboundLink.java:300)
at com.ibm.ws.http.channel.inbound.impl.HttpInboundLink.ready(HttpInboundLink.java:271)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:165)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:136)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:196)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:751)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:881)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1560)
If anyone could point me in the right direction, please.
WebSphere version is 6.1.0.43.
Ok, this might come in handy for others.
Diving deeper into WebSphere Application Server's custom web container properties, I came accross this one:
Make query string available to the welcome file [Fix Pack 25 or later]
Typically, when a request is initially sent to the context root of the
application, the request is forwarded to a welcome file. If a query
string is included in an initial request, it is not available to the
welcome file if you included the request.getQueryString() attribute in
the welcome file. However, the query string is available to the
welcome file if you included the javax.servlet.forward.query_string
attribute in the welcome file.
If you need to use the request.getQueryString() attribute, instead of
the javax.servlet.forward.query_string attribute, to make the query
string available to the welcome file, add the
com.ibm.ws.webcontainer.provideQStringToWelcomeFile custom property to
your Web container configuration and set the property to true. The
default value for this property is false. Name Value
com.ibm.ws.webcontainer.provideQStringToWelcomeFile true
For WebSphere 6.1 these custom properties can be found here: http://pic.dhe.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=%2Fcom.ibm.websphere.base.doc%2Finfo%2Faes%2Fae%2Frweb_custom_props.html
The clue is that once forwarding a request, the query string itself is stripped from the Request object, leaving room for "internal" query strings. The initial query string is still available using
request.getAttribute("javax.servlet.forward.query_string");
All a bit tricky, but our code is now robust and capable of running inside WebSphere and other servlet containers, as well as managing UTF-8 and ISO-8859-1 encoded query string parameters sent by Ogone.
Thanks for listening.

Categories

Resources