Develop Reference

Not able to read from Parquet Reader along with Hadoop configuration using Java

I need to read parquet file from s3 using java & maven support.
public static void main(String[] args) throws IOException, URISyntaxException {
Path path = new Path("s3", "batch-dev", "/aman/part-e52b.c000.snappy.parquet");
Configuration conf = new Configuration();
conf.set("fs.s3.awsAccessKeyId", "xxx");
conf.set("fs.s3.awsSecretAccessKey", "xxxxx");
InputFile file = HadoopInputFile.fromPath(path, conf);
ParquetFileReader reader2 = ParquetFileReader.open(conf, path);
//MessageType schema = reader2.getFooter().getFileMetaData().getSchema();
//System.out.println(schema);
}
Using above code, give FileNotFoundException
Note that: Note that I am using s3 scheme and not s3a. Not sure whether we have support for s3 scheme in Hadoop.
Exception in thread "main" java.io.FileNotFoundException: s3://batch-dev/aman/part-e52b.c000.snappy.parquet: No such file or directory.
at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:334)
at org.apache.parquet.hadoop.util.HadoopInputFile.fromPath(HadoopInputFile.java:39)
at com.bidgely.cloud.core.cass.gb.S3GBRawDataHandler.main(S3GBRawDataHandler.java:505)
However, with the same path if I use s3Client, I am able to get the object. But the problem here is that I can not read parquet data from input stream getting from below code.
public static void main(String args[]) {
AWSCredentials credentials = new BasicAWSCredentials("XXXXX", "XXXXX");
AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withRegion("us-west-2").withCredentials(new AWSStaticCredentialsProvider(credentials)).build();
S3Object object = s3Client.getObject(new GetObjectRequest("batch-dev", "/aman/part-e52b.c000.snappy.parquet"));
System.out.println(object.getObjectContent());
}
Kindly help me with the solution. [I had to use java only].

Adding hyperlink to inner PDF files

I have to create a PDF file by adding two PDF files inside a generated PDF file as a tree structure using iText in Java.
I have to create bookmarks with PDF file names and add a hyperlink to the bookmark. When the bookmark is clicked, the respective PDF should be opened in that PDF file itself, not as a separate PDF.
PDFTREE
pdf1
pdf2

Such bookmarks are referred to as outline elements in the PDF specification (PDF 32000-1:2008, p.367):
The outline consists of a tree-structured hierarchy of outline items (sometimes called bookmarks), which serve as a visual table of contents to display the document’s structure to the user.
If you merge the documents with PdfMerger, the outlines are copied to the resulting PDF by default. However, you want a main-node per document and not a flat list of bookmarks. Since cloning and copying outlines in no trivial task, it is best to let iText handle this. Unfortunately, we have little direct control how outlines are being merged.
We can build a SpecialMerger as a wrapper around PdfMerger to extract the cloned outlines (first step) and get them into a hierarchical structure afterwards (second step). The outline of each merged PDF is temporarily stored in the outlineList together with the desired name of the main node and its reference (page number in the merged PDF). After all the PDFs are merged, we can attach the temporarily stored outlines back to the root-node.
public static class SpecialMerger {
private final PdfDocument outputPdf;
private final PdfMerger merger;
private final PdfOutline rootOutline;
private final List<DocumentOutline> outlineList = new ArrayList<>();
private int nextPageNr = 1;
public SpecialMerger(final PdfDocument outputPdf) {
if (outputPdf.getNumberOfPages() != 0) {
throw new IllegalArgumentException("PDF must be empty");
}
this.outputPdf = outputPdf;
this.merger = new PdfMerger(outputPdf, true, true);
this.rootOutline = outputPdf.getOutlines(false);
}
public void merge(PdfDocument from, int fromPage, int toPage, String filename) {
merger.merge(from, fromPage, toPage); // merge with normal PdfMerger
// extract and clone outline of merged document
final List<PdfOutline> children = new ArrayList<>(rootOutline.getAllChildren());
rootOutline.getAllChildren().clear(); // clear root outline
outlineList.add(new DocumentOutline(filename, nextPageNr, children));
nextPageNr = outputPdf.getNumberOfPages() + 1; // update next page number
}
public void writeOutline() {
outlineList.forEach(o -> {
final PdfOutline outline = rootOutline.addOutline(o.getName()); // bookmark with PDF name
outline.addDestination(PdfExplicitDestination.createFit(outputPdf.getPage(o.getPageNr())));
outline.setStyle(PdfOutline.FLAG_BOLD);
o.getChildern().forEach(outline::addOutline); // add all extracted child bookmarks
});
}
private static class DocumentOutline {
private final String name;
private final int pageNr;
private final List<PdfOutline> childern;
public DocumentOutline(final String pdfName, final int pageNr, final List<PdfOutline> childern) {
this.name = pdfName;
this.pageNr = pageNr;
this.childern = childern;
}
public String getName() {
return name;
}
public int getPageNr() {
return pageNr;
}
public List<PdfOutline> getChildern() {
return childern;
}
}
}
Now, we can use this custom merger to merge the PDFs and then add the outline with writeOutline:
public static void main(String[] args) throws IOException {
String filename1 = "pdf1.pdf";
String filename2 = "pdf2.pdf";
try (
PdfDocument generatedPdf = new PdfDocument(new PdfWriter("output.pdf"));
PdfDocument pdfDocument1 = new PdfDocument(new PdfReader(filename1));
PdfDocument pdfDocument2 = new PdfDocument(new PdfReader(filename2))
) {
final SpecialMerger merger = new SpecialMerger(generatedPdf);
merger.merge(pdfDocument1, 1, pdfDocument1.getNumberOfPages(), filename1);
merger.merge(pdfDocument2, 1, pdfDocument2.getNumberOfPages(), filename2);
merger.writeOutline();
}
}
The result looks like this (Preview and Adobe Acrobat Reader on macOS):
Another option is to make a portfolio by embedding the PDFs. However, this is not supported by all PDF viewers and most users are not accustomed to these portfolios.
public static void main(String[] args) throws IOException {
String filename1 = "pdf1.pdf";
String filename2 = "pdf2.pdf";
try (PdfDocument generatedPdf = new PdfDocument(new PdfWriter("portfolio.pdf"))) {
Document doc = new Document(generatedPdf);
doc.add(new Paragraph("This PDF contains embedded documents."));
doc.add(new Paragraph("Use a compatible PDF viewer if you cannot see them."));
PdfCollection collection = new PdfCollection();
collection.setView(PdfCollection.TILE);
generatedPdf.getCatalog().setCollection(collection);
addAttachment(generatedPdf, filename1, filename1);
addAttachment(generatedPdf, filename2, filename2);
}
}
private static void addAttachment(PdfDocument doc, String attachmentPath, String name) throws IOException {
PdfFileSpec fileSpec = PdfFileSpec.createEmbeddedFileSpec(doc, attachmentPath, name, name, null, null);
doc.addFileAttachment(name, fileSpec);
}
The result in Adobe Acrobat Reader on macOS:

How to change direction of Hebrew letters?

I'm using itext 7.1.8 and I need to save Hebrew text in my document. I found this solution here but it doesn't work for me.
My code looks like the following:
public class RunItextApp {
public static void main(String[] args) throws Exception {
final String filename = "simple.pdf";
final String hebrew = "שדג";
final String text = "\u05E9\u05D3\u05D2";
createSimplePdf(filename, hebrew);
}
private static void createSimplePdf(String filename, String text) throws Exception {
final String path = RunItextApp.class.getResource("/Arial.ttf").getPath();
final PdfFont font = PdfFontFactory.createFont(path, PdfEncodings.IDENTITY_H, true);
Style hebrewStyle = new Style()
.setBaseDirection(BaseDirection.RIGHT_TO_LEFT)
.setTextAlignment(TextAlignment.RIGHT)
.setFontSize(14)
.setFont(font);
final PdfWriter pdfWriter = new PdfWriter(filename);
final PdfDocument pdfDocument = new PdfDocument(pdfWriter);
final Document pdf = new Document(pdfDocument);
pdf.setBaseDirection(BaseDirection.RIGHT_TO_LEFT);
pdf.add(
new Paragraph(text)
.setFontScript(Character.UnicodeScript.HEBREW)
.addStyle(hebrewStyle)
);
pdf.close();
}
}
Why this code doesn't work?
How can I set text direction?

Please take a look at this page https://kb.itextpdf.com/home/it5kb/faq/how-to-set-rtl-direction-for-hebrew-when-converting-html-to-pdf. It is described the same problem you have. I think that the main trick is in text font.

Decoding QR code in photo using Zxing

I want to decode some photos with QR code.
I found this method:
public static String readQRCode(String filePath, String charset, Map hintMap) throws FileNotFoundException, IOException, NotFoundException {
BinaryBitmap binaryBitmap = new BinaryBitmap(new HybridBinarizer(
new BufferedImageLuminanceSource(
ImageIO.read(new FileInputStream(filePath)))));
Result qrCodeResult = new MultiFormatReader().decode(binaryBitmap, hintMap);
return qrCodeResult.getText();
}
It's working on pictures like this.
Unfortunately it doesn't work on all images, i.e: this. As you see it has poor quality.
Error: com.google.zxing.NotFoundException
Any ideas, what can I do?

PDF parser text contains

I want to verify PDF Document using TestNG and PDFBox.
I would ask, is PDF able to check contains text like this:
PDFParser parser = new PDFParser(stream);
parser.getDocument().conntains("ABC")

Try below code:-
public void ReadPDF() throws Exception {
URL TestURL = new URL("http://www.axmag.com/download/pdfurl-guide.pdf");
BufferedInputStream TestFile = new BufferedInputStream(TestURL.openStream());
PDFParser TestPDF = new PDFParser(TestFile);
TestPDF.parse();
String TestText = new PDFTextStripper().getText(TestPDF.getPDDocument());
Assert.assertTrue(TestText.contains("Open the setting.xml, you can see it is like this"));
}
Download libraries :- https://pdfbox.apache.org/index.html