Develop Reference

Handle many unicode caracters with PDFBox

I am writing a Java function which takes a String as a parameter and produce a PDF as an output with PDFBox.
Everything is working fine as long as I use latin characters.
However, I don't know in advance what will be the input, and it might be some English as well as Chinese or Japanese characters.
In the case of non latin characters, here is the error I get:
Exception in thread "main" java.lang.IllegalArgumentException: U+3053 ('kohiragana') is not available in this font Helvetica encoding: WinAnsiEncoding
at org.apache.pdfbox.pdmodel.font.PDType1Font.encode(PDType1Font.java:426)
at org.apache.pdfbox.pdmodel.font.PDFont.encode(PDFont.java:324)
at org.apache.pdfbox.pdmodel.PDPageContentStream.showTextInternal(PDPageContentStream.java:509)
at org.apache.pdfbox.pdmodel.PDPageContentStream.showText(PDPageContentStream.java:471)
at com.mylib.pdf.PDFBuilder.generatePdfFromString(PDFBuilder.java:122)
at com.mylib.pdf.PDFBuilder.main(PDFBuilder.java:111)
If I understand correctly, I have to use a specific font for Japanese, another one for Chinese and so on, because the one that I am using (Helvetiva) doesn't handle all required unicode characters.
I could also use a font which handle all these unicode characters, such as Arial Unicode. However this font is under a specific license so I cannot use it and I haven't found another one.
I found some projects that want to overcome this issue, like the Google NOTO project.
However, this project provides multiple font files. So I would have to choose, at runtime, the correct file to load depending on the input I have.
So I am facing 2 options, one of which I don't know how to implement properly:
Keep searching for a font that handle almost every unicode character (where is this grail I am desperately seeking?!)
Try to detect which language is used and select a font depending on it.
Despite the fact that I don't know (yet) how to do that, I don't find it to be a clean implementation, as the mapping between the input and the font file will be hardcoded, meaning I will have to hardcode all the possible mappings.
Is there another solution?
Am I completely off tracks?
Thanks in advance for your help and guidance!
Here is the code I use to generate the PDF:
public static void main(String args[]) throws IOException {
String latinText = "This is latin text";
String japaneseText = "これは日本語です";
// This works good
generatePdfFromString(latinText);
// This generate an error
generatePdfFromString(japaneseText);
}
private static OutputStream generatePdfFromString(String content) throws IOException {
PDPage page = new PDPage();
try (PDDocument doc = new PDDocument();
PDPageContentStream contentStream = new PDPageContentStream(doc, page)) {
doc.addPage(page);
contentStream.setFont(PDType1Font.HELVETICA, 12);
// Or load a specific font from a file
// contentStream.setFont(PDType0Font.load(this.doc, new File("/fontPath.ttf")), 12);
contentStream.beginText();
contentStream.showText(content);
contentStream.endText();
contentStream.close();
OutputStream os = new ByteArrayOutputStream();
doc.save(os);
return os;
}
}

A better solution than waiting for a font or guessing a text's language is to have a multitude of fonts and selecting the correct font on a glyph-by-glyph base.
You already found the Google Noto Fonts which are a good base collection of fonts for this task.
Unfortunately, though, Google publishes the Noto CJK fonts only as OpenType fonts (.otf), not as TrueType fonts (.ttf), a policy that isn't likely to change, cf. the Noto fonts issue 249 and others. On the other hand PDFBox does not support OpenType fonts and isn't actively working on OpenType support either, cf. PDFBOX-2482.
Thus, one has to convert the OpenType font somehow to TrueType. I simply took the file shared by djmilch in his blog post FREE FONT NOTO SANS CJK IN TTF.
Font selection per character
So you essentially need a method which checks your text character by character and dissects it into chunks which can be drawn using the same font.
Unfortunately I don't see a better method to ask a PDFBox PDFont whether it knows a glyph for a given character than to actually try to encode the character and consider a IllegalArgumentException a "no".
I, therefore, implemented that functionality using the following helper class TextWithFont and method fontify:
class TextWithFont {
final String text;
final PDFont font;
TextWithFont(String text, PDFont font) {
this.text = text;
this.font = font;
}
public void show(PDPageContentStream canvas, float fontSize) throws IOException {
canvas.setFont(font, fontSize);
canvas.showText(text);
}
}
(AddTextWithDynamicFonts inner class)
List<TextWithFont> fontify(List<PDFont> fonts, String text) throws IOException {
List<TextWithFont> result = new ArrayList<>();
if (text.length() > 0) {
PDFont currentFont = null;
int start = 0;
for (int i = 0; i < text.length(); ) {
int codePoint = text.codePointAt(i);
int codeChars = Character.charCount(codePoint);
String codePointString = text.substring(i, i + codeChars);
boolean canEncode = false;
for (PDFont font : fonts) {
try {
font.encode(codePointString);
canEncode = true;
if (font != currentFont) {
if (currentFont != null) {
result.add(new TextWithFont(text.substring(start, i), currentFont));
}
currentFont = font;
start = i;
}
break;
} catch (Exception ioe) {
// font cannot encode codepoint
}
}
if (!canEncode) {
throw new IOException("Cannot encode '" + codePointString + "'.");
}
i += codeChars;
}
result.add(new TextWithFont(text.substring(start, text.length()), currentFont));
}
return result;
}
(AddTextWithDynamicFonts method)
Example use
Using the method and the class above like this
String latinText = "This is latin text";
String japaneseText = "これは日本語です";
String mixedText = "Tこhれiはs日 本i語sで すlatin text";
generatePdfFromStringImproved(latinText).writeTo(new FileOutputStream("Cccompany-Latin-Improved.pdf"));
generatePdfFromStringImproved(japaneseText).writeTo(new FileOutputStream("Cccompany-Japanese-Improved.pdf"));
generatePdfFromStringImproved(mixedText).writeTo(new FileOutputStream("Cccompany-Mixed-Improved.pdf"));
(AddTextWithDynamicFonts test testAddLikeCccompanyImproved)
ByteArrayOutputStream generatePdfFromStringImproved(String content) throws IOException {
try ( PDDocument doc = new PDDocument();
InputStream notoSansRegularResource = AddTextWithDynamicFonts.class.getResourceAsStream("NotoSans-Regular.ttf");
InputStream notoSansCjkRegularResource = AddTextWithDynamicFonts.class.getResourceAsStream("NotoSansCJKtc-Regular.ttf") ) {
PDType0Font notoSansRegular = PDType0Font.load(doc, notoSansRegularResource);
PDType0Font notoSansCjkRegular = PDType0Font.load(doc, notoSansCjkRegularResource);
List<PDFont> fonts = Arrays.asList(notoSansRegular, notoSansCjkRegular);
List<TextWithFont> fontifiedContent = fontify(fonts, content);
PDPage page = new PDPage();
doc.addPage(page);
try ( PDPageContentStream contentStream = new PDPageContentStream(doc, page)) {
contentStream.beginText();
for (TextWithFont textWithFont : fontifiedContent) {
textWithFont.show(contentStream, 12);
}
contentStream.endText();
}
ByteArrayOutputStream os = new ByteArrayOutputStream();
doc.save(os);
return os;
}
}
(AddTextWithDynamicFonts helper method)
I get
for latinText = "This is latin text"
for japaneseText = "これは日本語です"
and for mixedText = "Tこhれiはs日 本i語sで すlatin text"
Some asides
I retrieved the fonts as Java resources but you can use any kind of InputStream for them.
The font selection mechanism above can quite easily be combined with the line breaking mechanism shown in this answer and the justification extension thereof in this answer

Below is another implementation of splitting a plain text into the chunks of TextWithFont objects. Algorithm does character-by-character encoding and always tries to encode with a main font and only in the case of a failure will proceed with the next fonts in the list of fallback fonts.
Main classwith properties:
public class SplitByFontsProcessor {
/** Text to be processed */
private String text;
/** List of fonts to be used for processing */
private List<PDFont> fonts;
/** Main font to be used for processing */
private PDFont mainFont;
/** List of fallback fonts to be used for processing. It does not contain the main font. */
private List<PDFont> fallbackFonts;
........
}
Methods within the same class:
private List<TextWithFont> splitUsingFallbackFonts() throws IOException {
final List<TextWithFont> fontifiedText = new ArrayList<>();
final StringBuilder strBuilder = new StringBuilder();
boolean isHandledByMainFont = false;
// Iterator over Unicode codepoints in Java string
final PrimitiveIterator.OfInt iterator = text.codePoints().iterator();
while (iterator.hasNext()) {
int codePoint = iterator.nextInt();
final String stringCodePoint = new String(Character.toChars(codePoint));
// try to encode Unicode codepoint
try {
// Multi-byte encoding with 1 to 4 bytes.
mainFont.encode(stringCodePoint); // fails here if can not be handled by the font
strBuilder.append(stringCodePoint); // append if succeeded to encode
isHandledByMainFont = true;
} catch(IllegalArgumentException ex) {
// IllegalArgumentException is thrown if character can not be handled by a given Font
// Adding successfully handled characters so far
if (StringUtils.isNotEmpty(strBuilder.toString())) {
fontifiedText.add(new TextWithFont(strBuilder.toString(), mainFont));
strBuilder.setLength(0);// clear StringBuilder
}
handleByFallbackFonts(fontifiedText, stringCodePoint);
isHandledByMainFont = false;
} // end main font try-catch
}
// If this is the last successful run that was handled by main font, then add result
if (isHandledByMainFont) {
fontifiedText.add(new TextWithFont(strBuilder.toString(), mainFont));
}
return mergeAdjacents(fontifiedText);
}
Method handleByFallbackFonts():
private void handleByFallbackFonts(List<TextWithFont> fontifiedText, String stringCodePoint)
throws IOException {
final StringBuilder strBuilder = new StringBuilder();
boolean isHandledByFallbackFont = false;
// Retry with fallback fonts
final Iterator<PDFont> fallbackFontsIterator = fallbackFonts.iterator();
while(fallbackFontsIterator.hasNext()) {
try {
final PDFont fallbackFont = fallbackFontsIterator.next();
fallbackFont.encode(stringCodePoint); // fails here if can not be handled by the font
isHandledByFallbackFont = true;
strBuilder.append(stringCodePoint);
fontifiedText.add(new TextWithFont(strBuilder.toString(), fallbackFont));
break; // if successfully handled - break the loop
} catch(IllegalArgumentException exception) {
// do nothing, proceed to the next font
}
} // end while
// If character was not handled and this is the last font - throw an exception
if (!isHandledByFallbackFont) {
final String fontNames = fonts.stream()
.map(PDFont::getName)
.collect(Collectors.joining(", "));
int codePoint = stringCodePoint.codePointAt(0);
throw new TextProcessingException(
String.format("Unicode code point [%s] can not be handled by configured fonts: [%s]",
codePoint, fontNames));
}
}
Method splitUsingFallbackFonts() returns a list of TextWithFont objects in which adjacent objects with the same font will not be necessarily belong to the same object. This happens because an algorithm will always first retry to render a character by the main font, and in case it fails, it will create a new object with the font capable of rendering the character. So we need to call a utility method, mergeAdjacents(), which will merge them together.
private static List<TextWithFont> mergeAdjacents(final List<TextWithFont> fontifiedText) {
final Deque<TextWithFont> result = new LinkedList<>();
for (TextWithFont elem : fontifiedText) {
final TextWithFont resElem = result.peekLast();
if (resElem == null || !resElem.getFont().equals(elem.getFont())) {
result.addLast(elem);
} else {
result.addLast(merge(result.pollLast(), elem));
}
}
return new ArrayList<>(result);
}

How to append a text field in JFrame?

For instance using textField.setText it replaces the text currently in the text field however I want the output to go one step down from the previous output. Any help would be appreciated. Thanks.

Add the text directly to the Document:
Document doc = textField.getDocument();
doc.insertString(doc.getLength(), "the text", null);
Another option might be to use:
textField.replaceSelection(...);
and the text will be added at the current caret position.
Edit:
Here is the code used by the append(...) method of a JTextArea
public void append(String str) {
Document doc = getDocument();
if (doc != null) {
try {
doc.insertString(doc.getLength(), str, null);
} catch (BadLocationException e) {
}
}
}
I suggest you could use the replaceSelection(...) method since there in not need for the try/catch block.

JTextPane adding large spaces every time i insertString

Hey i'm making a chat application and was at first using a simple JTextPane for a basic, color supporting, chat view pane. I then wanted to add html link support to make them clickable by adding an HTML listener and setting the content type to text/html. The clickable links work perfectly, but now every time i insert a String the chat will add a large space. Here is the code i use below:
Constructor:
public JTextPaneTest() {
this.addHyperlinkListener(new LinkController());
this.setContentType("text/html");
this.setEditable(false);
}
Here is how i append regular text:
public void append(Color c, String s) {
SimpleAttributeSet sas = new SimpleAttributeSet();
StyleConstants.setForeground(sas, c);
StyledDocument doc = (StyledDocument)this.getDocument();
int len = getDocument().getLength();
try {
doc.insertString(len, s, sas);
} catch (BadLocationException e) {
e.printStackTrace();
}
setCaretPosition(len + s.length());
}
And Here is how i insertlinks
public void addHyperlink(URL url, String text) {
try {
Document doc = this.getDocument();
SimpleAttributeSet hrefAttr = new SimpleAttributeSet();
hrefAttr.addAttribute(HTML.Attribute.HREF, url.toString());
SimpleAttributeSet attrs = new SimpleAttributeSet();
attrs.addAttribute(HTML.Tag.A, hrefAttr);
StyleConstants.setUnderline(attrs, true);
StyleConstants.setForeground(attrs, Color.blue);
doc.insertString(doc.getLength(), text, attrs);
}
catch (BadLocationException e) {
e.printStackTrace(System.err);
}
}
For whatever reason with the content type set to just basic text, i don't get this space issue.
Here are some pictures of it:
http://i.stack.imgur.com/dpMBB.png
In the picture, the Name is inserted, then :, Then the rest of the text.
Edit: For whatever reason the JTextPane is automatically centering my InsertStrings.
Edit2: Is it possible to remove the margin between the HTML inserted strings? I've been trying everything for hours on end and simply can't find a solution. Only possible solution i can think of is reformatting the text via getText/setText every time i insert a string to insure no margins are added..

Instead of inserting text ith attribute try to create a dummy element and replace it with outer HTML containing th link
SimpleAttributeSet a=new SimpleAttributeSet();
a.addAttribute("DUMMY_ATTRIBUTE_NAME","DUMMY_ATTRIBUTE_VALUE");
doc.setCharacterAttributes(start, text.length(), a, false);
Element elem=doc.getCharacterElement(start);
String html="<a href='"+text+"'>"+text+"</a>";
doc.setOuterHTML(elem, html);
Se working example here

I ended up using just basic 'text' and using components for click detection.
To fix this issue i was having with HTML, i simply used
editorKit.insertHTML(doc, doc.getLength(), "html code", 0, 0, null);
Rather then inserting my code directly usings doc's 'insertString'

setContentType("text/html") for JTextPane doesn't work as it is expected

When you setContentType("text/html") it is applied only for the text that is set via JTextPane.setText(). All other text, that is put to the JTextPane via styles is "immune" to content type.
Here is what I mean:
private final String[] messages = {"first msg", "second msg <img src=\"file:src/test/2.png\"/> yeah", "<img src=\"file:src/test/2.png\"/> third msg"};
public TestGUI() throws BadLocationException {
JTextPane textPane = new JTextPane();
textPane.setEditable(false);
textPane.setContentType("text/html");
//Read all the messages
StringBuilder text = new StringBuilder();
for (String msg : messages) {
textext.append(msg).append("<br/>");
}
textPane.setText(text.toString());
//Add new message
StyledDocument styleDoc = textPane.getStyledDocument();
styleDoc.insertString(styleDoc.getLength(), messages[1], null);
JScrollPane scrollPane = new JScrollPane(textPane, JScrollPane.VERTICAL_SCROLLBAR_ALWAYS, JScrollPane.HORIZONTAL_SCROLLBAR_NEVER);
//add scrollPane to the main window and launch
//...
}
In general, I have a chat that is represented by JTextPane. I receive messages from server, process them - set text color for specific cases, change smile markers to images path etc. everything is made within the bounds of HTML. But as it can be clearly seen from example above, only the setText is the subject of setContentType("text/html") and the second part, where new message added is represented by "text/plain" (if I'm not mistaken).
Is it possible to apply "text/html" content type to all data that is inserted to JTextPane? Without it, it is almost impossible to process messages without implemention of very complex algorithm.

I don't think you should be using the insertString() method to add text. I think you should be using something like:
JTextPane textPane = new JTextPane();
textPane.setContentType( "text/html" );
textPane.setEditable(false);
HTMLDocument doc = (HTMLDocument)textPane.getDocument();
HTMLEditorKit editorKit = (HTMLEditorKit)textPane.getEditorKit();
String text = "hyperlink";
editorKit.insertHTML(doc, doc.getLength(), text, 0, 0, null);

REEDIT
Sorry, I misunderstood the problem: inserting a string as HTML.
For that one needs to resort to the HTMLEditorKit capabilities:
StyledDocument styleDoc = textPane.getStyledDocument();
HTMLDocument doc = (HTMLDocument)styleDoc;
Element last = doc.getParagraphElement(doc.getLength());
try {
doc.insertBeforeEnd(last, messages[1] + "<br>");
} catch (BadLocationException ex) {
} catch (IOException ex) {
}

Here is a much simpler way to do that.
JTextPane pane = new JTextPane();
pane.setContentType("text/html");
pane.setText("<html><h1>My First Heading</h1><p>My first paragraph.</p></body></html>");

JTextPane appending a new string

In an every article the answer to a question "How to append a string to a JEditorPane?" is something like
jep.setText(jep.getText + "new string");
I have tried this:
jep.setText("<b>Termination time : </b>" +
CriterionFunction.estimateIndividual_top(individual) + " </br>");
jep.setText(jep.getText() + "Processes' distribution: </br>");
And as a result I got "Termination time : 1000" without "Processes' distribution:"
Why did this happen???

I doubt that is the recommended approach for appending text. This means every time you change some text you need to reparse the entire document. The reason people may do this is because the don't understand how to use a JEditorPane. That includes me.
I much prefer using a JTextPane and then using attributes. A simple example might be something like:
JTextPane textPane = new JTextPane();
textPane.setText( "original text" );
StyledDocument doc = textPane.getStyledDocument();
// Define a keyword attribute
SimpleAttributeSet keyWord = new SimpleAttributeSet();
StyleConstants.setForeground(keyWord, Color.RED);
StyleConstants.setBackground(keyWord, Color.YELLOW);
StyleConstants.setBold(keyWord, true);
// Add some text
try
{
doc.insertString(0, "Start of text\n", null );
doc.insertString(doc.getLength(), "\nEnd of text", keyWord );
}
catch(Exception e) { System.out.println(e); }

A JEditorPane, just a like a JTextPane has a Document that you can use for inserting strings.
What you'll want to do to append text into a JEditorPane is this snippet:
JEditorPane pane = new JEditorPane();
/* ... Other stuff ... */
public void append(String s) {
try {
Document doc = pane.getDocument();
doc.insertString(doc.getLength(), s, null);
} catch(BadLocationException exc) {
exc.printStackTrace();
}
}
I tested this and it worked fine for me. The doc.getLength() is where you want to insert the string, obviously with this line you would be adding it to the end of the text.

setText is to set all text in a textpane. Use the StyledDocument interface to append, remove, ans so on text.
txtPane.getStyledDocument().insertString(
offsetWhereYouWant, "text you want", attributesYouHope);