Develop Reference

PDFBox: No glyph for U+0050 in extracted font

I'm trying to create new page in document and write some text to it, while using the font contained in the file.
The font is extracted from the resources:
PDPage page = document.getPage(0);
PDResources res = page.getResources();
List<PDFont> fonts = new ArrayList<>();
for (COSName fontName : res.getFontNames()) {
PDFont font = res.getFont(fontName);
System.out.println(font);
fonts.add(font);
}
And later used to write some text:
stream.beginText();
stream.setFont(fonts.get(0), 12);
stream.setTextMatrix(Matrix.getTranslateInstance(20, 50));
stream.showText("Protokol");
stream.endText();
The showText method always fails with error
No glyph for U+0050 (P) in font QZHBRL+ArialMT
But the glyph is there, as verified by FontForge:
Also the method hasGlyph returns true.
The complete project including the PDF is available at github repository showing the issue

Here you actually ran into an open PDFBox TODO, your stream.showText eventually calls encode of the underlying CID font for each character and here we have:
public class PDCIDFontType2 extends PDCIDFont
{
...
public byte[] encode(int unicode)
{
int cid = -1;
if (isEmbedded)
{
...
// otherwise we require an explicit ToUnicode CMap
if (cid == -1)
{
//TODO: invert the ToUnicode CMap?
// see also PDFBOX-4233
cid = 0;
}
}
...
if (cid == 0)
{
throw new IllegalArgumentException(
String.format("No glyph for U+%04X (%c) in font %s", unicode, (char) unicode, getName()));
}
return encodeGlyphId(cid);
}
...
}
(org.apache.pdfbox.pdmodel.font.PDCIDFontType2)
Where PDFBox could not otherwise determine a mapping from Unicode to glyph code (if (cid == -1)), the JavaDoc comments indicate another way to determine a glyph code, an inverse lookup of the ToUnicode map. If this was implemented, PDFBox could have determined a glyph ID and written your text.
Unfortunately it is not implemented yet.

This has been fixed in issue PDFBOX-5103. This will be available in PDFBox 2.0.23 and until then, in a snapshot build.

Write cyrillic chars into PDF form fields with PDFBox

I am using pdfbox 2.0.5 to fill out form fields of a PDF document using this code:
doc = PDDocument.load(inputStream);
PDDocumentCatalog catalog = doc.getDocumentCatalog();
PDAcroForm form = catalog.getAcroForm();
for (PDField field : form.getFieldTree()){
field.setValue("должен");
}
I get this error: U+0434 ('afii10069') is not available in this font Times-Roman (generic: TimesNewRomanPSMT) encoding: StandardEncoding with differences
The PDF document itself contains cyrillic text which is displayed fine. I have tried using different fonts. For "Arial Unicode MS" it wants to download a 50MB "Adobe Acrobat Reader DC Font Pack". Is this a requirement for cyrillic characters?
Which font do I have to specify in the text field to handle cyrillic (or asian) characters?
Thanks,
Ropo

Adobe handles that by reusing the embedded font file in the {/Ubuntu} font and creates a new font resource from that. Here is a quick hack which can serve as a guide of how to achieve something similar. The code is specific to a sample I've got.
PDDocument doc = PDDocument.load(new File(...));
PDAcroForm acroForm = doc.getDocumentCatalog().getAcroForm();
PDResources formResources = acroForm.getDefaultResources();
PDTrueTypeFont font = (PDTrueTypeFont) formResources.getFont(COSName.getPDFName("Ubuntu"));
// here is the 'magic' to reuse the font as a new font resource
TrueTypeFont ttFont = font.getTrueTypeFont();
PDFont font2 = PDType0Font.load(doc, ttFont, true);
ttFont.close();
formResources.put(COSName.getPDFName("F0"), font2);
PDTextField formField = (PDTextField) acroForm.getField("Text2");
formField.setDefaultAppearance("/F0 0 Tf 0 g");
formField.setValue("öäüинформацию");
doc.save(...);
doc.close();

The solution was trivial:
form.setNeedAppearances(true);
And then I remove the blue box of the field with:
field.setReadOnly(true);

PDFBox Form fill - saveIncremental does not work

I have a pdf file with some form field that I want to fill from java. Right now I'm trying to fill just one form which I am finding by its name. My code looks like this:
File file = new File("c:/Testy/luxmed/Skierowanie3.pdf");
PDDocument document = PDDocument.load(file);
PDDocumentCatalog doc = document.getDocumentCatalog();
PDAcroForm Form = doc.getAcroForm();
String formName = "topmostSubform[0].Page1[0].pana_pania[0]";
PDField f = Form.getField(formName);
setField(document, formName, "Artur");
System.out.println("New value 2nd: " + f.getValueAsString());
document.saveIncremental(new FileOutputStream("c:/Testy/luxmed/nowy_pd3.pdf"));
document.close();
and this:
public static void setField(PDDocument pdfDocument, String name, String Value) throws IOException
{
PDDocumentCatalog docCatalog = pdfDocument.getDocumentCatalog();
PDAcroForm acroForm = docCatalog.getAcroForm();
PDField field = acroForm.getField(name);
if (field instanceof PDCheckBox){
field.setValue("Yes");
}
else if (field instanceof PDTextField){
System.out.println("Original value: " + field.getValueAsString());
field.setValue(Value);
System.out.println("New value: " + field.getValueAsString());
}
else{
System.out.println("Nie znaleziono pola");
}
}
As system.out states, the value was set correctly, but in new the generated pdf file, new value is not showing up (original String is presented) so I guess the incremental saving does not work properly. What am I missing?
I use 2.0.2 version of pdfbox, and here is pdf file with which I working: pdf

In general
When saving changes to a PDF as an incremental update with PDFBox 2.0.x, you have to set the property NeedToBeUpdated to true for every PDF object changed. Furthermore, the object must be reachable from the PDF catalog via a chain of references, and each PDF object in this chain also has to have the property NeedToBeUpdated set to true.
This is due to the way PDFBox saves incrementally, starting from the catalog it inspects the NeedToBeUpdated property, and if it is set to true, PDFBox stores the object, and only in this case it recurses deeper into the objects referenced from this object in search for more objects to store.
In particular this implies that some objects unnecessarily have to be marked NeedToBeUpdated, e.g. the PDF catalog itself, and in some cases this even defeats the purpose of the incremental update at large, see below.
In case of the OP's document
Setting the NeedToBeUpdated properties
On one hand one has to extend the setField method to mark the chain of field dictionaries up to and including the changed field and also the appearance:
public static void setField(PDDocument pdfDocument, String name, String Value) throws IOException
{
PDDocumentCatalog docCatalog = pdfDocument.getDocumentCatalog();
PDAcroForm acroForm = docCatalog.getAcroForm();
PDField field = acroForm.getField(name);
if (field instanceof PDCheckBox) {
field.setValue("Yes");
}
else if (field instanceof PDTextField) {
System.out.println("Original value: " + field.getValueAsString());
field.setValue(Value);
System.out.println("New value: " + field.getValueAsString());
}
else {
System.out.println("Nie znaleziono pola");
}
// vvv--- new
COSDictionary fieldDictionary = field.getCOSObject();
COSDictionary dictionary = (COSDictionary) fieldDictionary.getDictionaryObject(COSName.AP);
dictionary.setNeedToBeUpdated(true);
COSStream stream = (COSStream) dictionary.getDictionaryObject(COSName.N);
stream.setNeedToBeUpdated(true);
while (fieldDictionary != null)
{
fieldDictionary.setNeedToBeUpdated(true);
fieldDictionary = (COSDictionary) fieldDictionary.getDictionaryObject(COSName.PARENT);
}
// ^^^--- new
}
(FillInFormSaveIncremental method setField)
On the other hand the main code has to be extended to mark a chain from the catalog to the fields array:
PDDocument document = PDDocument.load(...);
PDDocumentCatalog doc = document.getDocumentCatalog();
PDAcroForm Form = doc.getAcroForm();
String formName = "topmostSubform[0].Page1[0].pana_pania[0]";
PDField f = Form.getField(formName);
setField(document, formName, "Artur");
System.out.println("New value 2nd: " + f.getValueAsString());
// vvv--- new
COSDictionary dictionary = document.getDocumentCatalog().getCOSObject();
dictionary.setNeedToBeUpdated(true);
dictionary = (COSDictionary) dictionary.getDictionaryObject(COSName.ACRO_FORM);
dictionary.setNeedToBeUpdated(true);
COSArray array = (COSArray) dictionary.getDictionaryObject(COSName.FIELDS);
array.setNeedToBeUpdated(true);
// ^^^--- new
document.saveIncremental(new FileOutputStream(...));
document.close();
(FillInFormSaveIncremental test testFillInSkierowanie3)
Beware: for use with generic PDFs one obviously should introduce some null tests...
Opening the result file in Adobe Reader one will unfortunately see that the program complains about changes which disable extended features in the file.
This is due to the quirk in PDFBox' incremental saving that it requires some unnecessary objects in the update section. In particular the catalog is saved there which contains a usage rights signature (the technology granting extended features). The re-saved signature obviously is not at its original position in its original revision anymore. Thus, is invalidated.
Most likely the OP OP wanted to save the PDF incrementally to not break this signature but PDFBox does not permit this. Oh well...
Thus, the only thing one can do is prevent the warning by completely removing the signature.
Removing the usage rights signature
We already have retrieved the catalog object in the additions above, so removing the signature is easy:
COSDictionary dictionary = document.getDocumentCatalog().getCOSObject();
// vvv--- new
dictionary.removeItem(COSName.PERMS);
// ^^^--- new
dictionary.setNeedToBeUpdated(true);
(FillInFormSaveIncremental test testFillInSkierowanie3)
Opening the result file in Adobe Reader one will unfortunately see that the program complains about missing extended features in the file to save it.
This is due to the fact that Adobe Reader requires extended features to save changes to XFA forms, extended features we had to remove in this step.
But the document at hand is a hybrid AcroForm & XFA form document, and Adobe Reader requires no extended features to save AcroForm documents. Thus, all we have to do is remove the XFA form. As our code only sets the AcroForm value, this is a good idea anyways...
Removing the XFA form
We already have retrieved the acroform object in the additions above, so removing the XFA form referenced from there is easy:
dictionary = (COSDictionary) dictionary.getDictionaryObject(COSName.ACRO_FORM);
// vvv--- new
dictionary.removeItem(COSName.XFA);
// ^^^--- new
dictionary.setNeedToBeUpdated(true);
(FillInFormSaveIncremental test testFillInSkierowanie3)
Opening the result file in Adobe Reader one will see that one now can without further ado edit the form and save the file.
Beware, a sufficiently new Adobe Reader version is required for this, earlier versions (up to at least version 9) did require extended features even for saving changes to an AcroForm form

Render Type3 font character as image using PDFBox

In my project, I'm stuck with necessity to parse PDF file, that contains some characters rendered by Type3 fonts. So, what I need to do is to render such characters into BufferedImage for further processing.
I'm not sure if I'm looking in correct way, but I'm trying to get PDType3CharProc for such characters:
PDType3Font font = (PDType3Font)textPosition.getFont();
PDType3CharProc charProc = font.getCharProc(textPosition.getCharacterCodes()[0]);
and the input stream of this procedure contains following data:
54 0 1 -1 50 43 d1
q
49 0 0 44 1.1 -1.1 cm
BI
/W 49
/H 44
/BPC 1
/IM true
ID
<some binary data here>
EI
Q
but unfortunately I don't have any idea how can I use this data to render character into an image using PDFBox (or any other Java libraries).
Am I looking in correct direction, and what can I do with this data?
If not, are there some other tools that can solve such problem?

Unfortunately PDFBox out-of-the-box does not provide a class to render contents of arbitrary XObjects (like the type 3 font char procs), at least as far as I can see.
But it does provide a class for rendering complete PDF pages; thus, to render a given type 3 font glyph, one can simply create a page containing only that glyph and render this temporary page!
Assuming, for example, the type 3 font is defined on the first page of a PDDocument document and has name F1, all its char procs can be rendered like this:
PDPage page = document.getPage(0);
PDResources pageResources = page.getResources();
COSName f1Name = COSName.getPDFName("F1");
PDType3Font fontF1 = (PDType3Font) pageResources.getFont(f1Name);
Map<String, Integer> f1NameToCode = fontF1.getEncoding().getNameToCodeMap();
COSDictionary charProcsDictionary = fontF1.getCharProcs();
for (COSName key : charProcsDictionary.keySet())
{
COSStream stream = (COSStream) charProcsDictionary.getDictionaryObject(key);
PDType3CharProc charProc = new PDType3CharProc(fontF1, stream);
PDRectangle bbox = charProc.getGlyphBBox();
if (bbox == null)
bbox = charProc.getBBox();
Integer code = f1NameToCode.get(key.getName());
if (code != null)
{
PDDocument charDocument = new PDDocument();
PDPage charPage = new PDPage(bbox);
charDocument.addPage(charPage);
charPage.setResources(pageResources);
PDPageContentStream charContentStream = new PDPageContentStream(charDocument, charPage);
charContentStream.beginText();
charContentStream.setFont(fontF1, bbox.getHeight());
charContentStream.getOutput().write(String.format("<%2X> Tj\n", code).getBytes());
charContentStream.endText();
charContentStream.close();
File result = new File(RESULT_FOLDER, String.format("4700198773-%s-%s.png", key.getName(), code));
PDFRenderer renderer = new PDFRenderer(charDocument);
BufferedImage image = renderer.renderImageWithDPI(0, 96);
ImageIO.write(image, "PNG", result);
charDocument.close();
}
}
(RenderType3Character.java test method testRender4700198773)
Considering the textPosition variable in the OP's code, he quite likely attempts this from a text extraction use case. Thus, he'll have to either pre-generate the bitmaps as above and simply look them up by name or adapt the code to match the available information in his use case (e.g. he might not have the original page at hand, only the font object; in that case he cannot copy the resources of the original page but instead may create a new resources object and add the font object to it).
Unfortunately the OP did not provide a sample PDF. Thus I used one from another stack overflow question, 4700198773.pdf from extract text with custom font result non readble for my test. There obviously might remain issues with the OP's own files.

I stumbled upon the same issue and I was able to render Type3 font by modifying PDFRenderer and the underlying PageDrawer:
class Type3PDFRenderer extends PDFRenderer
{
private PDFont font;
public Type3PDFRenderer(PDDocument document, PDFont font)
{
super(document);
this.font = font;
}
#Override
protected PageDrawer createPageDrawer(PageDrawerParameters parameters) throws IOException
{
FontType3PageDrawer pd = new FontType3PageDrawer(parameters, this.font);
pd.setAnnotationFilter(super.getAnnotationsFilter());//as done in the super class
return pd;
}
}
class FontType3PageDrawer extends PageDrawer
{
private PDFont font;
public FontType3PageDrawer(PageDrawerParameters parameters, PDFont font) throws IOException
{
super(parameters);
this.font = font;
}
#Override
public PDGraphicsState getGraphicsState()
{
PDGraphicsState gs = super.getGraphicsState();
gs.getTextState().setFont(this.font);
return gs;
}
}
Simply use Type3PDFRenderer instead of PDFRendered. Of course if you have multiple fonts this needs some more modification to handle them.
Edit: tested with pdfbox 2.0.9

How to resolve "...is not available in this font's encoding"?

So I am using PDFBox to fill in some pdfs. So far everything was great - I created a form in pdf with Avenir Light font, and I could fill it in. However, the problem that just now showed up, is that when I am trying to fill the pdf using letters such as ł, ą, ć ... I get the following error:
U+0142 is not available in this font's encoding: MacRomanEncoding with differences
with different numbers.
Now, my question is - how can I fix this, so that I can fill the form automatically? When I open the pdf in Acrobat Reader, I can insert those letters, and I dont get any errors. Here is how I set the field:
public void setField(PDDocument document, PDField field, String value ) throws IOException {
if( field != null && value != null) {
try{
field.setValue(value);
} catch (Exception e){
e.printStackTrace();
}
}
else {
System.err.println( "No field found with name:" + field.getPartialName() );
}
}
UPDATE
I've been trying to upload my own Avenir-Light.tff like this:
PDFont font = PDType1Font.HELVETICA;
PDResources res = new PDResources();
COSName fontName = res.add(font);
acroForm.setDefaultResources(res);
String da = "/" + fontName.getName() + " 12 Tf 0 g";
acroForm.setDefaultAppearance(da);
However, this doesn't seem to have any impact on the printed fields, and throws almost the same message:
U+0104 ('Aogonek') is not available in this font Helvetica (generic: ArialMT) encoding: WinAnsiEncoding

PDFBox define 14 standard fonts in PDType1Font :
PDType1Font.TIMES_ROMAN PDType1Font.TIMES_BOLD
PDType1Font.TIMES_ITALI PDType1Font.TIMES_BOLD_ITALIC
PDType1Font.HELVETICA PDType1Font.HELVETICA_BOLD
PDType1Font.HELVETICA_OBLIQUE
PDType1Font.HELVETICA_BOLD_OBLIQUE PDType1Font.COURIER
PDType1Font.COURIER_BOLD PDType1Font.COURIER_OBLIQUE
PDType1Font.COURIER_BOLD_OBLIQUE PDType1Font.SYMBOL
PDType1Font.ZAPF_DINGBATS
So if you want to use Avenir-Light you have to load it from a .ttf file. You can do this as #TilmanHausherr suggested PDType0Font.load(doc, new File("path/Avenir-Light.ttf"), false).
PDFont font = PDType0Font.load(doc, new File("path/Avenir-Light.ttf"), false);
PDResources res = new PDResources();
COSName fontName = res.add(font);
acroForm.setDefaultResources(res);
String da = "/" + fontName.getName() + " 12 Tf 0 g";
acroForm.setDefaultAppearance(da);
Update
Do you know why it also displays a warning if form of: OpenType Layout
tables used in font Avenir-Light are not implemented in PDFBox and
will be ignored?
Avenir-light font uses OpenType Layout tables (Advanced Typographic) that PDFBox does not support yet. This advaned typographics will be ignored