I try to read a PDF document using the itext 7.1.9 and I get an exception that looks like the following:
com.itextpdf.kernel.pdf.canvas.parser.util.InlineImageParsingUtils$InlineImageParseException: Unexpected ColorSpace: /R9.
at com.itextpdf.kernel.pdf.canvas.parser.util.InlineImageParsingUtils.getComponentsPerPixel(InlineImageParsingUtils.java:257)
at com.itextpdf.kernel.pdf.canvas.parser.util.InlineImageParsingUtils.computeBytesPerRow(InlineImageParsingUtils.java:271)
at com.itextpdf.kernel.pdf.canvas.parser.util.InlineImageParsingUtils.parseUnfilteredSamples(InlineImageParsingUtils.java:298)
at com.itextpdf.kernel.pdf.canvas.parser.util.InlineImageParsingUtils.parseSamples(InlineImageParsingUtils.java:345)
at com.itextpdf.kernel.pdf.canvas.parser.util.InlineImageParsingUtils.parse(InlineImageParsingUtils.java:163)
at com.itextpdf.kernel.pdf.canvas.parser.util.PdfCanvasParser.parse(PdfCanvasParser.java:119)
at com.itextpdf.kernel.pdf.canvas.parser.PdfCanvasProcessor.processContent(PdfCanvasProcessor.java:283)
at com.itextpdf.kernel.pdf.canvas.parser.PdfCanvasProcessor.processPageContent(PdfCanvasProcessor.java:306)
at com.itextpdf.kernel.pdf.canvas.parser.PdfDocumentContentParser.processContent(PdfDocumentContentParser.java:77)
at com.itextpdf.kernel.pdf.canvas.parser.PdfDocumentContentParser.processContent(PdfDocumentContentParser.java:90)
I suppose this method must be fixed in itext 7 library:
private static int getComponentsPerPixel(PdfName colorSpaceName, PdfDictionary colorSpaceDic) {
if (colorSpaceName == null) {
return 1;
} else if (colorSpaceName.equals(PdfName.DeviceGray)) {
return 1;
} else if (colorSpaceName.equals(PdfName.DeviceRGB)) {
return 3;
} else if (colorSpaceName.equals(PdfName.DeviceCMYK)) {
return 4;
} else {
if (colorSpaceDic != null) {
PdfArray colorSpace = colorSpaceDic.getAsArray(colorSpaceName);
if (colorSpace != null) {
if (PdfName.Indexed.equals(colorSpace.getAsName(0))) { // the fail is on here. colorSpace.getAsName(0) returns /ICCBased
return 1;
}
} else {
PdfName tempName = colorSpaceDic.getAsName(colorSpaceName);
if (tempName != null) {
return getComponentsPerPixel(tempName, colorSpaceDic);
}
}
}
throw (new InlineImageParseException("Unexpected ColorSpace: {0}.")).setMessageParams(new Object[]{colorSpaceName});
}
}
I don't understand why I get it and how to fix it.
This issue has been fixed in iText more than a year ago. The fix has first been released in version 7.1.16.
Thus, please update your iText version.
For details, the fix has been added in commit df4013c8f141059a91373008ac4a7013f6be0852 authored on 2021-04-14 10:36:24 and committed on 2021-05-25 15:11:36 with the comment
Add support processing inline image with ICCBased color space in resources
DEVSIX-5295
i am using pdfbox to render a pdf file to image, but i got ArrayIndexOutOfBoundsException when run TrueTypeFont.getUnicodeCmap method, cmapTable is empty and cmapTable.getCmaps()[0] led to out of bounds, here is the call stack
java.lang.ArrayIndexOutOfBoundsException: 0
at org.apache.fontbox.ttf.TrueTypeFont.getUnicodeCmap(TrueTypeFont.java:566)
at org.apache.pdfbox.pdmodel.font.PDCIDFontType2.<init>(PDCIDFontType2.java:183)
at org.apache.pdfbox.pdmodel.font.PDCIDFontType2.<init>(PDCIDFontType2.java:70)
at org.apache.pdfbox.pdmodel.font.PDFontFactory.createDescendantFont(PDFontFactory.java:125)
at org.apache.pdfbox.pdmodel.font.PDType0Font.<init>(PDType0Font.java:128)
at org.apache.pdfbox.pdmodel.font.PDFontFactory.createFont(PDFontFactory.java:83)
at org.apache.pdfbox.pdmodel.PDResources.getFont(PDResources.java:123)
at org.apache.pdfbox.contentstream.operator.text.SetFontAndSize.process(SetFontAndSize.java:60)
at org.apache.pdfbox.contentstream.PDFStreamEngine.processOperator(PDFStreamEngine.java:815)
at org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:472)
at org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:446)
at org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:149)
at org.apache.pdfbox.rendering.PageDrawer.drawPage(PageDrawer.java:189)
at org.apache.pdfbox.rendering.PDFRenderer.renderPage(PDFRenderer.java:208)
at org.apache.pdfbox.rendering.PDFRenderer.renderImage(PDFRenderer.java:139)
at org.apache.pdfbox.rendering.PDFRenderer.renderImageWithDPI(PDFRenderer.java:80)
and here is the getUnicodeCmap method in pdfbox
public CmapSubtable getUnicodeCmap(boolean isStrict) throws IOException
{
CmapTable cmapTable = getCmap();
if (cmapTable == null)
{
if (isStrict)
{
throw new IOException("The TrueType font does not contain a 'cmap' table");
}
else
{
return null;
}
}
CmapSubtable cmap = cmapTable.getSubtable(CmapTable.PLATFORM_UNICODE,
CmapTable.ENCODING_UNICODE_2_0_FULL);
if (cmap == null)
{
cmap = cmapTable.getSubtable(CmapTable.PLATFORM_UNICODE,
CmapTable.ENCODING_UNICODE_2_0_BMP);
}
if (cmap == null)
{
cmap = cmapTable.getSubtable(CmapTable.PLATFORM_WINDOWS,
CmapTable.ENCODING_WIN_UNICODE_BMP);
}
if (cmap == null)
{
// Microsoft's "Recommendations for OpenType Fonts" says that "Symbol" encoding
// actually means "Unicode, non-standard character set"
cmap = cmapTable.getSubtable(CmapTable.PLATFORM_WINDOWS,
CmapTable.ENCODING_WIN_SYMBOL);
}
if (cmap == null)
{
if (isStrict)
{
throw new IOException("The TrueType font does not contain a Unicode cmap");
}
else
{
// fallback to the first cmap (may not be Unicode, so may produce poor results)
cmap = cmapTable.getCmaps()[0];
}
}
return cmap;
}
i found this file contains fonts with custom encoding, and there is a "warm" comment in the pdfbox's code: fallback to the first cmap (may not be Unicode, so may produce poor results). So I suspect that the custom font encoding caused this problem, is this correct?
I'm using apache PDFBox from java, and I have a source PDF with multiple optional content groups. What I am wanting to do is export a version of the PDF that includes only the standard content and the optional content groups that were enabled. It is important for my purposes that I preserve any dynamic aspects of the original.... so text fields are still text fields, vector images are still vector images, etc. The reason that this is required is because I intend to ultimately be using a pdf form editor program that does not know how to handle optional content, and would blindly render all of them, so I want to preprocess the source pdf, and use the form editing program on a less cluttered destination pdf.
I've been trying to find something that could give me any hints on how to do this with google, but to no avail. I don't know if I'm just using the wrong search terms, or if this is just something that is outside of what the PDFBox API was designed for. I rather hope it's not the latter. The info shown here does not seem to work (converting the C# code to java), because despite the pdf I'm trying to import having optional content, there does not seem to be any OC resources when I examine the tokens on each page.
for(PDPage page:pages) {
PDResources resources = page.getResources();
PDFStreamParser parser = new PDFStreamParser(page);
parser.parse();
Collection tokens = parser.getTokens();
...
}
I'm truly sorry for not having any more code to show what I've tried so far, but I've just been poring over the java API docs for about 8 hours now trying to figure out what I might need to do this, and just haven't been able to figure it out.
What I DO know how to do is add text, lines, and images to a new PDPage, but I do not know how to retrieve that information from a given source page to copy it over, nor how to tell which optional content group such information is part of (if any). I am also not sure how to copy form fields in the source pdf over to the destination, nor how to copy the font information over.
Honestly, if there's a web page out there that I wasn't able to find with google with the searches that I tried, I'd be entirely happy to read up more about it, but I am really quite stuck here, and I don't know anyone personally that knows about this library.
Please help.
EDIT:
Trying what I understand from what was suggested below, I've written a loop to examine each XObject on the page as follows:
PDResources resources = pdPage.getResources();
Iterable<COSName> names = resources.getXObjectNames();
for(COSName name:names) {
PDXObject xobj = resources.getXObject(name);
PDFStreamParser parser = new PDFStreamParser(xobj.getStream().toByteArray());
parser.parse();
Object [] tokens = parser.getTokens().toArray();
for(int i = 0;i<tokens.length-1;i++) {
Object obj = tokens[i];
if (obj instanceof COSName && obj.equals(COSName.OC)) {
i++;
Object obj = tokens[i];
if (obj instanceof COSName) {
PDPropertyList props = resources.getProperties((COSName)obj);
if (props != null) {
...
However, after an OC key, the next entry in the tokens array is always an Operator tagged as "BMC". Nowhere am I finding any info that I can recognize from the named optional content groups.
Here's a robust solution for removing marked content blocks (open to feedback if anyone finds anything that isn't working right). You should be able to adjust for OC blocks...
This code properly handles nesting and removal of resources (xobject, graphics state and fonts - easy to add others if needed).
public class MarkedContentRemover {
private final MarkedContentMatcher matcher;
/**
*
*/
public MarkedContentRemover(MarkedContentMatcher matcher) {
this.matcher = matcher;
}
public int removeMarkedContent(PDDocument doc, PDPage page) throws IOException {
ResourceSuppressionTracker resourceSuppressionTracker = new ResourceSuppressionTracker();
PDResources pdResources = page.getResources();
PDFStreamParser pdParser = new PDFStreamParser(page);
PDStream newContents = new PDStream(doc);
OutputStream newContentOutput = newContents.createOutputStream(COSName.FLATE_DECODE);
ContentStreamWriter newContentWriter = new ContentStreamWriter(newContentOutput);
List<Object> operands = new ArrayList<>();
Operator operator = null;
Object token;
int suppressDepth = 0;
boolean resumeOutputOnNextOperator = false;
int removedCount = 0;
while (true) {
operands.clear();
token = pdParser.parseNextToken();
while(token != null && !(token instanceof Operator)) {
operands.add(token);
token = pdParser.parseNextToken();
}
operator = (Operator)token;
if (operator == null) break;
if (resumeOutputOnNextOperator) {
resumeOutputOnNextOperator = false;
suppressDepth--;
if (suppressDepth == 0)
removedCount++;
}
if (OperatorName.BEGIN_MARKED_CONTENT_SEQ.equals(operator.getName())
|| OperatorName.BEGIN_MARKED_CONTENT.equals(operator.getName())) {
COSName contentId = (COSName)operands.get(0);
final COSDictionary properties;
if (operands.size() > 1) {
Object propsOperand = operands.get(1);
if (propsOperand instanceof COSDictionary) {
properties = (COSDictionary) propsOperand;
} else if (propsOperand instanceof COSName) {
properties = pdResources.getProperties((COSName)propsOperand).getCOSObject();
} else {
properties = new COSDictionary();
}
} else {
properties = new COSDictionary();
}
if (matcher.matches(contentId, properties)) {
suppressDepth++;
}
}
if (OperatorName.END_MARKED_CONTENT.equals(operator.getName())) {
if (suppressDepth > 0)
resumeOutputOnNextOperator = true;
}
else if (OperatorName.SET_GRAPHICS_STATE_PARAMS.equals(operator.getName())) {
resourceSuppressionTracker.markForOperator(COSName.EXT_G_STATE, operands.get(0), suppressDepth == 0);
}
else if (OperatorName.DRAW_OBJECT.equals(operator.getName())) {
resourceSuppressionTracker.markForOperator(COSName.XOBJECT, operands.get(0), suppressDepth == 0);
}
else if (OperatorName.SET_FONT_AND_SIZE.equals(operator.getName())) {
resourceSuppressionTracker.markForOperator(COSName.FONT, operands.get(0), suppressDepth == 0);
}
if (suppressDepth == 0) {
newContentWriter.writeTokens(operands);
newContentWriter.writeTokens(operator);
}
}
if (resumeOutputOnNextOperator)
removedCount++;
newContentOutput.close();
page.setContents(newContents);
resourceSuppressionTracker.updateResources(pdResources);
return removedCount;
}
private static class ResourceSuppressionTracker{
// if the boolean is TRUE, then the resource should be removed. If the boolean is FALSE, the resource should not be removed
private final Map<COSName, Map<COSName, Boolean>> tracker = new HashMap<>();
public void markForOperator(COSName resourceType, Object resourceNameOperand, boolean preserve) {
if (!(resourceNameOperand instanceof COSName)) return;
if (preserve) {
markForPreservation(resourceType, (COSName)resourceNameOperand);
} else {
markForRemoval(resourceType, (COSName)resourceNameOperand);
}
}
public void markForRemoval(COSName resourceType, COSName refId) {
if (!resourceIsPreserved(resourceType, refId)) {
getResourceTracker(resourceType).put(refId, Boolean.TRUE);
}
}
public void markForPreservation(COSName resourceType, COSName refId) {
getResourceTracker(resourceType).put(refId, Boolean.FALSE);
}
public void updateResources(PDResources pdResources) {
for (Map.Entry<COSName, Map<COSName, Boolean>> resourceEntry : tracker.entrySet()) {
for(Map.Entry<COSName, Boolean> refEntry : resourceEntry.getValue().entrySet()) {
if (refEntry.getValue().equals(Boolean.TRUE)) {
pdResources.getCOSObject().getCOSDictionary(COSName.XOBJECT).removeItem(refEntry.getKey());
}
}
}
}
private boolean resourceIsPreserved(COSName resourceType, COSName refId) {
return getResourceTracker(resourceType).getOrDefault(refId, Boolean.FALSE);
}
private Map<COSName, Boolean> getResourceTracker(COSName resourceType){
if (!tracker.containsKey(resourceType)) {
tracker.put(resourceType, new HashMap<>());
}
return tracker.get(resourceType);
}
}
}
Helper class:
public interface MarkedContentMatcher {
public boolean matches(COSName contentId, COSDictionary props);
}
Optional Content Groups are marked with BDC and EMC. You will have to navigate through all of the tokens returned from the parser and remove the "section" from the array. Here is some C# Code that was posted a while ago - [1]: How to delete an optional content group alongwith its content from pdf using pdfbox?
I investigated that (converting to Java) but couldn't get it work as expected. I managed to remove the content between BDC and EMC and then save the result using the same technique as the sample but the PDF was corrupted. Perhaps that is my lack of C# Knowledge (related to Tuples etc.)
Here is what I came up with, as I said it doesn't work perhaps you or someone else (mkl, Tilman Hausherr) can spot the flaw.
OCGDelete (PDDocument doc, int pageNum, String OCName) {
PDPage pdPage = (PDPage) doc.getDocumentCatalog().getPages().get(pageNum);
PDResources pdResources = pdPage.getResources();
PDFStreamParser pdParser = new PDFStreamParser(pdPage);
int ocgStart
int ocgLength
Collection tokens = pdParser.getTokens();
Object[] newTokens = tokens.toArray()
try {
for (int index = 0; index < newTokens.length; index++) {
obj = newTokens[index]
if (obj instanceof COSName && obj.equals(COSName.OC)) {
// println "Found COSName at "+index /// Found Optional Content
startIndex = index
index++
if (index < newTokens.size()) {
obj = newTokens[index]
if (obj instanceof COSName) {
prop = pdRes.getProperties(obj)
if (prop != null && prop instanceof PDOptionalContentGroup) {
if ((prop.getName()).equals(delLayer)) {
println "Found the Layer to be deleted"
println "prop Name was " + prop.getName()
index++
if (index < newTokens.size()) {
obj = newTokens[index]
if ((obj.getName()).equals("BDC")) {
ocgStart = index
println("OCG Start " + ocgStart)
ocgLength = -1
index++
while (index < newTokens.size()) {
ocgLength++
obj = newTokens[index]
println " Loop through relevant OCG Tokens " + obj
if (obj instanceof Operator && (obj.getName()).equals("EMC")) {
println "the next obj was " + obj
println "after that " + newTokens[index + 1] + "and then " + newTokens[index + 2]
println("OCG End " + ocgLength++)
break
}
index++
}
if (endIndex > 0) {
println "End Index was something " + (startIndex + ocgLength)
}
}
}
}
}
}
}
}
}
}
catch (Exception ex){
println ex.message()
}
for (int i = ocgStart; i < ocgStart+ ocgLength; i++){
newTokens.removeAt(i)
}
PDStream newContents = new PDStream(doc);
OutputStream output = newContents.createOutputStream(COSName.FLATE_DECODE);
ContentStreamWriter writer = new ContentStreamWriter(output);
writer.writeTokens(newTokens);
output.close();
pdPage.setContents(newContents);
}
I'm using Lucene for an Eclipse plugin. Currently I iterate over my indexed terms like this:
I get a Terms instance using IndexReader.getTermVector(id, field)
I iterate over this instance using TermsEnum like this: while ((text = vectorEnum.next()) != null)
Now what I want additionally is to get the first n elements of a field. I figured I have to use PostingsEnum to accomplish this, but I don't get how to use it. I guess I can get it by calling postings() on my TermsEnum, but I don't know what to do with that.
Edit:
That's the important part of my code I guess:
Terms vector = indexReader.getTermVector(id, field);
BytesRef text = null;
if (vector != null) {
TermsEnum vectorEnum = vector.iterator();
while ((text = vectorEnum.next()) != null) {
String term = text.utf8ToString();
[do stuff]
}
And that's the FieldType:
FieldType fieldType = new FieldType();
fieldType.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS);
fieldType.setStored(true);
fieldType.setStoreTermVectors(true);
fieldType.setTokenized(true);
Not sure why but requesting positions using setIndexOptions doesn't seem to work so you have to explicitly set setStoreTermVectorPositions. You still have to set the index options to something other than NONE but it doesn't seem necessary to use DOCS_AND_FREQS_AND_POSITIONS, i.e.
fieldType.setIndexOptions(IndexOptions.DOCS);
fieldType.setStoreTermVectorPositions(true);
then you can access the positions:
Terms vector = indexReader.getTermVector(id, field);
if (vector != null) {
TermsEnum vectorEnum = vector.iterator();
BytesRef text;
while ((text = vectorEnum.next()) != null) {
String term = text.utf8ToString();
PostingsEnum postings = vectorEnum.postings(null, PostingsEnum.POSITIONS);
while (postings.nextDoc() != DocIdSetIterator.NO_MORE_DOCS) {
int freq = postings.freq();
while (freq-- > 0)
logger.info("Position: {}", postings.nextPosition());
}
}
}
A qustion about Eclipse PDE development: I write a small plugin for Eclipse and have the following
* an org.eclipse.ui.texteditor.ITextEditor
* a line number
How can I automatically jump to that line and mark it? It's a pity that the API seems only to support offsets (see: ITextEditor.selectAndReveal()) within the document but no line numbers.
The best would be - although this doesn't work:
ITextEditor editor = (ITextEditor)IDE.openEditor(PlatformUI.getWorkbench().getActiveWorkbenchWindow().getActivePage(), file, true );
editor.goto(line);
editor.markLine(line);
It this possible in some way? I did not find a solution
on the class DetailsView I found the following method.
private static void goToLine(IEditorPart editorPart, int lineNumber) {
if (!(editorPart instanceof ITextEditor) || lineNumber <= 0) {
return;
}
ITextEditor editor = (ITextEditor) editorPart;
IDocument document = editor.getDocumentProvider().getDocument(
editor.getEditorInput());
if (document != null) {
IRegion lineInfo = null;
try {
// line count internaly starts with 0, and not with 1 like in
// GUI
lineInfo = document.getLineInformation(lineNumber - 1);
} catch (BadLocationException e) {
// ignored because line number may not really exist in document,
// we guess this...
}
if (lineInfo != null) {
editor.selectAndReveal(lineInfo.getOffset(), lineInfo.getLength());
}
}
}
Even though org.eclipse.ui.texteditor.ITextEditor deals wiith offset, it should be able to take your line number with the selectAndReveal() method.
See this thread and this thread.
Try something along the line of:
((ITextEditor)org.eclipse.jdt.ui.JavaUI.openInEditor(compilationUnit)).selectAndReveal(int, int);