Develop Reference

Convert html to Pdf with with very big data

I have implemented html to pdf conversion using openhtmltopdf and I use it in Struts 2 action and it works very well. However, in the case of very large data, e.g. the html data is > 3Mb (pdf file ~1.6Mb) when I test it with JMeter for 50 hits the application crashes with message java.lang.OutOfMemoryError: Java heap space.
If I increase the java limit with the -Xmx option I just get some extra hits
The code i use is like this:
First clean html
public class HtmlToXhtmlConverterHTMLCleaner2 extends AbstractHtmlToXhtmlConverter
implements IHtmlToXhtmlConverter {
public HtmlToXhtmlConverterHTMLCleaner2(String htmlData) {
super(htmlData);
}
#Override
public void convert() {
final HtmlCleaner cleaner = new HtmlCleaner();
CleanerProperties cleanerProperties = cleaner.getProperties();
cleanerProperties.setAdvancedXmlEscape(true);
cleanerProperties.setOmitXmlDeclaration(true);
cleanerProperties.setOmitDoctypeDeclaration(false);
cleanerProperties.setTranslateSpecialEntities(true);
cleanerProperties.setTransResCharsToNCR(true);
cleanerProperties.setRecognizeUnicodeChars(true);
cleanerProperties.setIgnoreQuestAndExclam(true);
cleanerProperties.setUseEmptyElementTags(false);
cleanerProperties.setPruneTags("script");
final XmlSerializer xmlSerializer = new PrettyXmlSerializer(cleanerProperties);
try {
final TagNode rootTagNode = cleaner.clean(htmlData);
this.xhtmlData = xmlSerializer.getAsString(rootTagNode);
} catch (Exception ex) {
ex.printStackTrace();
}
}
then convert cleaned html to pdf
public class PDFConverterHtmlToPdf extends AbstractPDFConverter implements IPDFConverter {
ByteArrayOutputStream psfData;
public PDFConverterHtmlToPdf(String xhtmlData, String cssFile) {
super();
this.xhtmlData = xhtmlData;
this.cssFile = cssFile;
}
#Override
public void convert() {
pdfData = new ByteArrayOutputStream();
try {
// There are more options on the builder than shown below.
PdfRendererBuilder builder = new PdfRendererBuilder();
if(cssFile != null && cssFile.length() > 0){
builder.withHtmlContent(xhtmlData, cssFile);
} else {
builder.withHtmlContent(xhtmlData, "");
}
builder.toStream(pdfData);
builder.run();
} catch (Exception e) {
e.printStackTrace();
}
}
}
then send data from strus2 action to request
private void buildPdfContent(String htmlContent) {
String pdfConverterCssFile = "http://localhost:8080/DocumentConverterApi/css/htmlToPdf.css";
PDFConverterHelp pdfConverterHelp = new PDFConverterHelp("demo.pdf",
htmlContent, pdfConverterCssFile);
pdfConverterHelp.build();
inputStream = new ByteArrayInputStream(pdfConverterHelp.getPDFFile().toByteArray());
pdfConverterHelp.closePdfData();
contentDisposition = "inline;filename=\"" + "demo.pdf\"";
}
I'm doing something wron?
Is there any other way to implement it without the risk of crashing the application?

How to convert this method from using java.io.File to java.nio.file?

Basically I have this method that I got off a tutorial (My main goal is to simply return the images from the spring boot server so that I could dynamically view them in Angular)
#RestController
public class FileController {
#Autowired
ServletContext context;
#GetMapping(path = "/allImages")
public ResponseEntity<List<String>> getImages(){
List<String> images = new ArrayList<String>();
String filesPath = context.getRealPath("/images");
File fileFolder = new File(filesPath);
if(fileFolder!=null) {
for(final File file : fileFolder.listFiles()) {
if(!file.isDirectory()) {
String encodeBase64 = null;
try {
String extention = FilenameUtils.getExtension(file.getName());
FileInputStream fileInputStream = new FileInputStream(file);
byte[] bytes = new byte[(int)file.length()];
encodeBase64 = Base64.getEncoder().encodeToString(bytes);
images.add("data:image/"+extention+";base64,"+encodeBase64);
fileInputStream.close();
} catch (Exception e) {
// TODO: handle exception
}
}
}
}
return new ResponseEntity<List<String>>(HttpStatus.OK);
}
With the current code, when I try to return the files, I get:
java.lang.NullPointerException: Cannot read the array length because the return value of "java.io.File.listFiles()" is null
I've been searching around and noticed that people recommend using java.nio.file instead but I'm a little lost on how would I implement this here. Any help is appreciated.

Example with nio:
public List<String> readImages() throws IOException {
return Files.list(Path.of("/images"))
.filter(Files::isRegularFile)
.map(this::encode)
.filter(Objects::nonNull)
.collect(Collectors.toList());
}
private String encode(Path file) {
try {
String extension = FilenameUtils.getExtension(file.getFileName().toString());
String encodeBase64 = Base64.getEncoder().encodeToString(Files.readAllBytes(file));
return "data:image/"+extension+";base64,"+encodeBase64;
} catch (Exception e) {
return null;
}
}

First get a Path to your folder:
Path folderPath = Paths.get(filesPath);
If your Path points to a directory, you can get a Stream<Path> of its contents using Files.list:
if (Files.isDirectory(folderPath)) {
List<Path> files = Files.list(folderPath)
.filter(path -> !Files.isDirectory(path))
.collect(Collectors.toList());
// Do something with the files.
}
It doesn't look like you are using the FileInputStream for anything, so you shouldn't need to translate that part. To get the file extension of your path, you probably need to convert the Path to a string, and extract the extension yourself.

I solved this issue with this code:
#Autowired
ServletContext context;
#GetMapping(path = "/allImages")
public List<String> readImages() throws IOException {
return Files.list(Paths.get(context.getRealPath("/images")))
.filter(Files::isRegularFile)
.map(this::encode)
.filter(Objects::nonNull)
.collect(Collectors.toList());
}
private String encode(Path file) {
try {
String extension = FilenameUtils.getExtension(file.getFileName().toString());
String encodeBase64 = Base64.getEncoder().encodeToString(Files.readAllBytes(file));
return "data:image/"+extension+";base64,"+encodeBase64;
} catch (Exception e) {
return null;
}
}
Thank you to everyone who helped.

How to load the correct language (set in config) instead of the language last in an array

I'm creating a little java app and I'm trying to load the yml files based on config.yml lang set (en/it) but I can't find a way to load them, only the last one in an array is loaded which is "it" for me.
I know that my method is probably the worst solution for a language file, I'm open to every method that will help me with the problem. But I prefer an external lang_en/it file instead of internal ones (Or is it better internal?)
After I set the language, the app will self-update every text in every class.
static final Properties props = new Properties();
static WelcomeMessage main = new WelcomeMessage();
static File file = null;
static File folder = null;
static boolean os = main.os.startsWith("Windows");
public static void create() {
String[] lang = {"en", "it"};
for (String s : lang) {
file = new File(WelcomeMessage.user + "/AppData/Roaming/MyApp/lang_" + s + ".yml");
folder = new File(file.getParent());
SetLanguages(s);
}
if (!file.exists()) {
try {
if (os) {
folder.mkdir();
file.createNewFile();
} else {
file = new File(main.user + "/Library/Application Support/MyApp/config.yml");
folder.mkdir();
file.createNewFile();
}
} catch (Exception e) {
System.out.println(e + " " + file);
}
}
}
public static void SetLanguages(String lang) {
if (lang.equals("en")) {
store("Settings.Save", "Save");
store("Settings.ConfigPath", "Config Path");
store("Settings.Language", "Language");
store("Settings.Title", "Settings");
} else if (lang.equals("it")) {
store("Settings.Save", "Salva");
store("Settings.ConfigPath", "Percorso config");
store("Settings.Language", "Lingua");
store("Settings.Title", "Impostazioni");
}
}
public static String get(String value) {
String key = null;
try {
FileInputStream in = new FileInputStream(file);
props.load(in);
key = props.getProperty(value);
in.close();
} catch (Exception fnf) {
System.out.println(fnf);
}
return key;
}
public static void store(String value, String key) {
try {
FileOutputStream out = new FileOutputStream(file);
props.setProperty(value, key);
props.store(out, null);
out.close();
} catch (Exception fnf) {
System.out.println(fnf);
}
}
This is how I get a text from yml:
path.setText(Language.get("Settings.ConfigPath"));
language.setText(Language.get("Settings.Language"));
f.setTitle(Language.get("Settings.Title"));
save.setText(Language.get("Settings.Save"));
And this my Language.get(key)
public static String get(String value) {
String key = null;
try {
FileInputStream in = new FileInputStream(file);
props.load(in);
key = props.getProperty(value);
in.close();
} catch (Exception fnf) {
System.out.println(fnf);
}
return key;
}

I suggest the following changes:
Create a Settings class to hold the properties save, configPath, language and title. Even better if this class uses an immutable builder pattern, because once set, the properties will never change.
Create a SettingsFactory class with method getSettings(language). This class shall also have a field Map<String, Settings>. In the constructor (or a static block), first check if a file exists on the disk, and if yes, load it into the map. If not, populate the map, one entry for each language, and persist to the disk.
getSettings would simply return the value from the map corresponding to the given language.
The format of the file written to the disk is a different matter. You say YAML, but I'm not seeing any YAML specific code in your snippet. If you don't know how to write a map to YAML, open a different question.

Making handleFileupload with parameter to share the methode

I'm using fileuplod primefaces. I have 3 buttons. Every button is responsible for uploading a file. My first steep is to use 3 methods on my bean to upload every file.
Is there a way to make the same method for all types? Every file has his own directory.
<h:form enctype="multipart/form-data" style="height:125px;width:75px;">
<p:fileUpload auto="true"
fileUploadListener="#{composantbean.handleFileUpload(???,1)}"
sizeLimit="2097152"
label="Choose"
allowTypes="/(\.|\/)(pdf)$/"
description="Images"/>
</h:form>
On my managed bean, I'm thinking of this solution:
public void handleFileUpload(FileUploadEvent event,int i) {
String lienPDN =destination+"PDN\\"+FilenameUtils.getName(event.getFile().getFileName());
File result = new File(lienPDN);
try {
FileOutputStream fileOutputStream = new FileOutputStream(result);
byte[] buffer = new byte[BUFFER_SIZE];
int bulk;
InputStream inputStream = event.getFile().getInputstream();
while (true) {
bulk = inputStream.read(buffer);
if (bulk < 0) {
break;
}
fileOutputStream.write(buffer, 0, bulk);
fileOutputStream.flush();
}
fileOutputStream.close();
inputStream.close();
FacesMessage msg = new FacesMessage("Succesful", event.getFile().getFileName()+ " is uploaded.");
FacesContext.getCurrentInstance().addMessage(null, msg);
selcetitem.setLienPdn(lienPDN);
} catch (IOException e) {
e.printStackTrace();
FacesMessage error = new FacesMessage(FacesMessage.SEVERITY_ERROR,"The files were not uploaded!", "");
FacesContext.getCurrentInstance().addMessage(null, error);
}
}

I think a better way might be to implements three handleFileUpload() methods. Each can handle their unique code (e.g. passing the right filepath). From there you can call a private void wrapUpUpload(String path, (...)).
Most of all this keeps your code readable. If also prevents the need for changing the default implementation of handleFileUpload().
E.g.: make sure to replace 1, 2, 3 with something meaningful
void handleFileUpload1(FileUploadEvent event) {
String path = "/uploads/1/";
wrapUpUpload(path);
}
void handleFileUpload2(FileUploadEvent event) {
String path = "/uploads/2/";
wrapUpUpload(path);
}
void handleFileUpload3(FileUploadEvent event) {
String path = "/uploads/3/";
wrapUpUpload(path);
}
private void wrapUpUpload(String path, (...)) {
// Upload the file
}

Download file using java apache commons?

How can I use the library to download a file and print out bytes saved? I tried using
import static org.apache.commons.io.FileUtils.copyURLToFile;
public static void Download() {
URL dl = null;
File fl = null;
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
copyURLToFile(dl, fl);
} catch (Exception e) {
System.out.println(e);
}
}
but I cannot display bytes or a progress bar. Which method should I use?
public class download {
public static void Download() {
URL dl = null;
File fl = null;
String x = null;
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
OutputStream os = new FileOutputStream(fl);
InputStream is = dl.openStream();
CountingOutputStream count = new CountingOutputStream(os);
dl.openConnection().getHeaderField("Content-Length");
IOUtils.copy(is, os);//begin transfer
os.close();//close streams
is.close();//^
} catch (Exception e) {
System.out.println(e);
}
}

If you are looking for a way to get the total number of bytes before downloading, you can obtain this value from the Content-Length header in http response.
If you just want the final number of bytes after the download, it is easiest to check the file size you just write to.
However if you want to display the current progress of how many bytes have been downloaded, you might want to extend apache CountingOutputStream to wrap the FileOutputStream so that everytime the write methods are called it counts the number of bytes passing through and update the progress bar.
Update
Here is a simple implementation of DownloadCountingOutputStream. I am not sure if you are familiar with using ActionListener or not but it is a useful class for implementing GUI.
public class DownloadCountingOutputStream extends CountingOutputStream {
private ActionListener listener = null;
public DownloadCountingOutputStream(OutputStream out) {
super(out);
}
public void setListener(ActionListener listener) {
this.listener = listener;
}
#Override
protected void afterWrite(int n) throws IOException {
super.afterWrite(n);
if (listener != null) {
listener.actionPerformed(new ActionEvent(this, 0, null));
}
}
}
This is the usage sample :
public class Downloader {
private static class ProgressListener implements ActionListener {
#Override
public void actionPerformed(ActionEvent e) {
// e.getSource() gives you the object of DownloadCountingOutputStream
// because you set it in the overriden method, afterWrite().
System.out.println("Downloaded bytes : " + ((DownloadCountingOutputStream) e.getSource()).getByteCount());
}
}
public static void main(String[] args) {
URL dl = null;
File fl = null;
String x = null;
OutputStream os = null;
InputStream is = null;
ProgressListener progressListener = new ProgressListener();
try {
fl = new File(System.getProperty("user.home").replace("\\", "/") + "/Desktop/Screenshots.zip");
dl = new URL("http://ds-forums.com/kyle-tests/uploads/Screenshots.zip");
os = new FileOutputStream(fl);
is = dl.openStream();
DownloadCountingOutputStream dcount = new DownloadCountingOutputStream(os);
dcount.setListener(progressListener);
// this line give you the total length of source stream as a String.
// you may want to convert to integer and store this value to
// calculate percentage of the progression.
dl.openConnection().getHeaderField("Content-Length");
// begin transfer by writing to dcount, not os.
IOUtils.copy(is, dcount);
} catch (Exception e) {
System.out.println(e);
} finally {
IOUtils.closeQuietly(os);
IOUtils.closeQuietly(is);
}
}
}

commons-io has IOUtils.copy(inputStream, outputStream). So:
OutputStream os = new FileOutputStream(fl);
InputStream is = dl.openStream();
IOUtils.copy(is, os);
And IOUtils.toByteArray(is) can be used to get the bytes.
Getting the total number of bytes is a different story. Streams don't give you any total - they can only give you what is currently available in the stream. But since it's a stream, it can have more coming.
That's why http has its special way of specifying the total number of bytes. It is in the response header Content-Length. So you'd have to call url.openConnection() and then call getHeaderField("Content-Length") on the URLConnection object. It will return the number of bytes as string. Then use Integer.parseInt(bytesString) and you'll get your total.