Develop Reference

Native memory allocation (mmap) failed to map when using jdbc8.jar and Oracle client 18

I have a java app that extracts large xml from an Oracle 11g database Release 11.2.0.4.0 (using spring 4) and stores them in files. We had a problem extracting data containing multibyte characters. Depending on where in the xml the multibyte was it would sometimes split the multibyte into 2 parts. It looked like the problem was to do with the version of jdbc and the oracle client installed. So we migrated to Oracle client 18 and ojdb8.jar, leaving the code untouched. The multibyte problem was solved, but now instead we have memory issues that were not happening before. The error I get is:
There is insufficient memory for the Java Runtime Environment to continue.
Native memory allocation (mmap) failed to map 256376832 bytes for committing reserved memory.
I have played around with the java command parameters but to no avail. This is what I am running:
java -Dfile.encoding=UTF8 -Doracle.jdbc.timezoneAsRegion=false -Xmx10240m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp -XX:NativeMemoryTracking=detail -XX:+StartAttachListener -XX:+UnlockDiagnosticVMOptions -XX:+PrintNMTStatistics
Decreasing Xmx to 512m ended up with running out of heap memory.
I have monitored the application's memory using jcmd VM.native_memory baseline/summary.diff and GC.class_stats and one of the biggest memory consumers are String objects. I was not able to make sense of the rest.
the sql is:
SELECT XML_DATA
FROM table where....
The xml_data column is defined as:
XML_DATA NOT NULL SYS.XMLTYPE STORAGE BINARY
mapping this in java to oracle.xdb.XMLType:
public List<XMLType> extractXmlDataList(String sqlExtractionQuery, Key key) {
MapSqlParameterSource namedSqlParams = createKeyParamMap(key);
List<XMLType> dataList = namedParameterJdbcTemplate.queryForList(sqlExtractionQuery, namedSqlParams, XMLType.class);
return dataList;
}
protected void extractXmlData(Key key) {
List<XMLType> xmlRecs = producerDao.extractXmlDataList(sqlExtractionQuery, patentKey);
if (xmlRecs != null && xmlRecs.size() > 0) {
for (XMLType xmlData : xmlRecs) {
String xmlText = xmlData.getStringVal();
//create nu.xom.Document
Builder parser = new Builder();
Document xmlDocument = parser.build(xmlText, null);
}
}
}
How can moving to Oracle client 18 and jdbc8.jar have affected the memory consumption so much?

The memory issue was solved by using xmlserialize rather than Oracle Java XMLType object. Still using Oracle client 18 and ojdbc8.jar. No further problems with multibyte UTF8 splitting.

Java heap Error while running R code

I'm trying to do a feature selection using the chi.squared function in FSelector package in R.
My dataset is about 132 variables X 192,000 rows.
chisquared.fs <- chi.squared(fo,df)
where fo contains the class variable: class ~.
I'm getting this error while running the code:
Error in .jcall("weka/filters/Filter", "Lweka/core/Instances;", "useFilter",
:java.lang.OutOfMemoryError: Java heap space
I know it is a Java memory leak error and I have already tried this before calling any libraries:
options( java.parameters = "-Xmx6g")
Any pointers would be really welcome.

Guys update: I had done what #copeg suggested without restarting R. I restarted R and with the options statement at the beginning before calling the libraries and it worked. Thanks for your suggestions.

Stanford POS tag out of memory

I trying to load the my own trained tagger call mytagger.tagger(size: 265KB) and it throw error below
Loading default properties from tagger taggers/mytagger.tagger
Reading POS tagger model from taggers/mytagger.tagger ... Whoops! Incomplete read. Rewrite the code!
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
Below is coding:
MaxentTagger tagger = new MaxentTagger("taggers/mytagger.tagger");
My Java is 64bit and using NetBeans 7.3.1. I try increase the memory size to -Xms4096M but it still the throw the same error message.
Anyone have solution for this issue. Thank you....
Keith

Memory leakage in cluster environment in tomcat

I am getting OutOfMemory Exception in my app. I have taken the heap dump and ananlyzed through MAT. While analyzing my app memory usage I found following suspect. I am unable to understand the main cause behind these suspects.
Please help me in understanding this leakage suspects and what is relevant solution for it.
Suspect 1
The thread org.apache.tomcat.util.threads.TaskThread # 0x2bdf5ff8 "ajp-bio-9002"-exec-5 keeps local variables with total size 113,973,288 (50.72%) bytes.
The memory is accumulated in one instance of "org.apache.tomcat.util.threads.TaskThread" loaded by "org.apache.catalina.loader.StandardClassLoader # 0x293b4488".
Thread Stack
"ajp-bio-9002"-exec-5
at java.util.Arrays.copyOf([CI)[C (Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(I)V (AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(C)Ljava/lang/AbstractStringBuilder; (AbstractStringBuilder.java:572)
at java.lang.StringBuffer.append(C)Ljava/lang/StringBuffer; (StringBuffer.java:320)
at org.apache.myfaces.renderkit.html.util.ReducedHTMLParser.consumeString(C)Ljava/lang/String; (ReducedHTMLParser.java:303)
at org.apache.myfaces.renderkit.html.util.ReducedHTMLParser.consumeAttrValue()Ljava/lang/String; (ReducedHTMLParser.java:327)
at org.apache.myfaces.renderkit.html.util.ReducedHTMLParser.parse()V (ReducedHTMLParser.java:579)
at org.apache.myfaces.renderkit.html.util.ReducedHTMLParser.parse(Ljava/lang/CharSequence;Lorg/apache/myfaces/renderkit/html/util/CallbackListener;)V (ReducedHTMLParser.java:66)
at org.apache.myfaces.renderkit.html.util.DefaultAddResource.parseResponse(Ljavax/servlet/http/HttpServletRequest;Ljava/lang/String;Ljavax/servlet/http/HttpServletResponse;)V (DefaultAddResource.java:699)
at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;Ljavax/servlet/FilterChain;)V (ExtensionsFilter.java:157)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(Ljavax/servlet/ServletRequest;Ljavax/servlet/ServletResponse;)V (ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardWrapperValve.java:240)
at org.apache.catalina.core.StandardContextValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardContextValve.java:164)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (AuthenticatorBase.java:462)
at org.apache.catalina.core.StandardHostValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardHostValve.java:164)
at org.apache.catalina.valves.ErrorReportValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (AccessLogValve.java:562)
at org.apache.catalina.core.StandardEngineValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (StandardEngineValve.java:118)
at org.apache.catalina.ha.session.JvmRouteBinderValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (JvmRouteBinderValve.java:218)
at org.apache.catalina.ha.tcp.ReplicationValve.invoke(Lorg/apache/catalina/connector/Request;Lorg/apache/catalina/connector/Response;)V (ReplicationValve.java:333)
at org.apache.catalina.connector.CoyoteAdapter.service(Lorg/apache/coyote/Request;Lorg/apache/coyote/Response;)V (CoyoteAdapter.java:395)
at org.apache.coyote.ajp.AjpProcessor.process(Lorg/apache/tomcat/util/net/SocketWrapper;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState; (AjpProcessor.java:301)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapper;Lorg/apache/tomcat/util/net/SocketStatus;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState; (AjpProtocol.java:183)
at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(Lorg/apache/tomcat/util/net/SocketWrapper;)Lorg/apache/tomcat/util/net/AbstractEndpoint$Handler$SocketState; (AjpProtocol.java:169)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run()V (JIoEndpoint.java:302)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Ljava/lang/Runnable;)V (ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run()V (ThreadPoolExecutor.java:908)
at java.lang.Thread.run()V (Thread.java:662)
Suspect 2
One instance of "java.lang.StringBuffer" loaded by "" occupies 59,216,088 (26.35%) bytes. The instance is referenced by org.apache.myfaces.renderkit.html.util.ReducedHTMLParser # 0x276990e8 , loaded by "org.apache.catalina.loader.WebappClassLoader # 0x29592038". The memory is accumulated in one instance of "char[]" loaded by "".

You can go to the "dominator_tree" tab of memory analyzer (MAT) and expand the TaskThread. This will show you the retained heap of all the objects in that taskthread. This might help you reach the part (code) in your application causing the issue.

It looks like org.apache.myfaces.renderkit.html.util.ReducedHTMLParser is the culprit. The javadoc for ReducedHTMLParser explains how it works. It buffers the entire HTML response in memory and then processes. It looks like it it trying - and failing - to process a very large response.

java.lang.OutOfMemoryError: Java heap space:Trying to use jasper exportreport method

Getting memory out of error while using jaspers exportreport method. The stack trace is given below.
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.text.RuleBasedBreakIterator.readFile(Unknown Source)
at java.text.RuleBasedBreakIterator.readTables(Unknown Source)
at java.text.RuleBasedBreakIterator.<init>(Unknown Source)
at java.text.BreakIterator.createBreakInstance(Unknown Source)
at java.text.BreakIterator.getBreakInstance(Unknown Source)
at java.text.BreakIterator.getLineInstance(Unknown Source)
at java.text.BreakIterator.getLineInstance(Unknown Source)
at java.awt.font.LineBreakMeasurer.<init>(Unknown Source)
at net.sf.jasperreports.engine.fill.TextMeasurer.renderParagraph(TextMeasurer.java:655)
at net.sf.jasperreports.engine.fill.TextMeasurer.measure(TextMeasurer.java:367)
at net.sf.jasperreports.engine.fill.JRFillTextElement.chopTextElement(JRFillTextElement.java:511)
at net.sf.jasperreports.engine.fill.JRFillTextField.prepare(JRFillTextField.java:607)
at net.sf.jasperreports.engine.fill.JRFillElementContainer.prepareElements(JRFillElementContainer.java:328)
at net.sf.jasperreports.engine.fill.JRFillBand.fill(JRFillBand.java:393)
at net.sf.jasperreports.engine.fill.JRFillBand.fill(JRFillBand.java:352)
at net.sf.jasperreports.engine.fill.JRVerticalFiller.fillColumnBand(JRVerticalFiller.java:2023)
at net.sf.jasperreports.engine.fill.JRVerticalFiller.fillDetail(JRVerticalFiller.java:755)
at net.sf.jasperreports.engine.fill.JRVerticalFiller.fillReportContent(JRVerticalFiller.java:285)
at net.sf.jasperreports.engine.fill.JRVerticalFiller.fillReport(JRVerticalFiller.java:132)
at net.sf.jasperreports.engine.fill.JRBaseFiller.fill(JRBaseFiller.java:836)
at net.sf.jasperreports.engine.fill.JRBaseFiller.fill(JRBaseFiller.java:765)
at net.sf.jasperreports.engine.fill.JRFiller.fillReport(JRFiller.java:84)
at net.sf.jasperreports.engine.JasperFillManager.fillReport(JasperFillManager.java:624)
at net.sf.jasperreports.engine.JasperFillManager.fillReport(JasperFillManager.java:540)
at com.aricent.aircell.logpatternanalyzer.report.JasperReportUtils.exportReport(JasperReportUtils.java:86)
Also I have tried memory leak analyser and as well as jvisualvm to find out the root cause, but it shows char[] occupying more than 90%. how to find out the exact root cause of this problem.

Seems like the Xmx settings is the problem. You should make sure that it's set high enough. If you suspect that anything else might be the problem you can always create a heapdump when you run oom and check that there is no leaking objects. This can be done by adding two parameters.
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/path/to/heapdump
You can then analyze it with jvisualvm or Eclipse MAT (Memory Analyzer Tool)

Have you tried to increase the heap size (Xmx)? It could happen that the report export simply need more memory. Is it working for smaller reports?

Are you try to use Virtualizer for fillReport?
http://community.jaspersoft.com/questions/520261/large-report-virtualizer-solution