Programmatically Downloading CSV Files with Java - java

Scenario: A website I use to research stock data has a link on the page to Export Data to Spreadsheet. The URL displayed when hovering over the export link is of the form http://www.stocksite.com/historical/export.php?symbol=C .
Question: Rather, that manually visiting the page for each stock I would like to automate the task. From Java, how can I programmatically call the site with a stock symbol and save the exported csv file? The URL and URLConnection class seem like the obvious place to start, but I'm unsure where to go from there.

All you need to do is to get the CSV in flavor of an InputStream.
InputStream input = new URL("http://example.com/file.csv").openStream();
Then you can feed it to any decent Java CSV parser API. Almost any of them take input in flavor of InputStream or Reader. If necessary, you can easily decorate InputStream as a Reader using InputStreamReader which you can then feed to the CSV parser.
Reader reader = new InputStreamReader(input, "UTF-8");

Related

Save pdf report on database using BIRT

So, I'm trying to save the pdf report in database using service methode. I saw that there's a way to specify the output of the generated report by calling : pdfOptions.setOutputStream(output). But how can I call my save methode this way?
I saw this post but i'm stack at the persist point
I apreciate any advice
PDFRenderOption pdfOptions = new PDFRenderOption(options);
pdfOptions.setOutputFormat(FORMAT_PDF);
pdfOptions.setOption(IPDFRenderOption.PAGE_OVERFLOW, IPDFRenderOption.OUTPUT_TO_MULTIPLE_PAGES);
pdfOptions.setOutputStream(response.getOutputStream());//opens report on browser
runAndRenderTask.setRenderOption(pdfOptions);
You are streaming the output directly to the client with
pdfOptions.setOutputStream(response.getOutputStream());//opens report on browser
If you do this, your output gets consumed and you'll not be able to save it to the database.
I would use a "tee" like approach, you know, with one input stream and two output streams.
You could write that yourself, our you just use something like the Apache TeeOutputStream.
This could look like this:
OutputStream blobOutputStream = ...; // for writing to the DB as BLOB.
OutputStream teeStream = TeeOutputStream(response.getOutputStream(), blobOutputStream);
pdfOptions.setOutputStream(teeStream);

equivalent to Files.readAllLines() for InputStream or Reader?

I have a file that I've been reading into a List via the following method:
List<String> doc = java.nio.file.Files.readAllLines(new File("/path/to/src/resources/citylist.csv").toPath(), StandardCharsets.UTF_8);
Is there any nice (single-line) Java 7/8/nio2 way to pull off the same feat with a file that's inside an executable Jar (and presumably, has to be read with an InputStream)? Perhaps a way to open an InputStream via the classloader, then somehow coerce/transform/wrap it into a Path object? Or some new subclass of InputStream or Reader that contains an equivalent to File.readAllLines(...)?
I know I could do it the traditional way in a half page of code, or via some external library... but before I do, I want to make sure that recent releases of Java can't already do it "out of the box".
An InputStream represents a stream of bytes. Those bytes don't necessarily form (text) content that can be read line by line.
If you know that the InputStream can be interpreted as text, you can wrap it in a InputStreamReader and use BufferedReader#lines() to consume it line by line.
try (InputStream resource = Example.class.getResourceAsStream("resource")) {
List<String> doc =
new BufferedReader(new InputStreamReader(resource,
StandardCharsets.UTF_8)).lines().collect(Collectors.toList());
}
You can use Apache Commons IOUtils#readLines:
List<String> doc = IOUtils.readLines(inputStream, StandardCharsets.UTF_8);

Enterprise Architect scripting with java - create and modify linked document

my question: How can I create a new linked document and insert (or connect) it into an element (in my case a Note-Element of an activity diagram).
The Element-Class supports the three Methods:
GetLinkedDocument ()
LoadLinkedDocument (string Filename)
SaveLinkedDocument (string Filename)
I missing a function like
CreateLinkedDocument (string Filename)
My goal: I create an activity diagram programmatically and some notes are to big to display it pretty in the activity diagram. So my goal is to put this text into an linked document instead of directly in the activity diagram.
Regards
EDIT
Thank you very much to Uffe for the solution of my problem. Here is my solution code:
public void addLinkedDocumentToElement(Element element, String noteText) {
String filePath = "C:\\rtfNote.rtf";
PrintWriter writer;
//create new file on the disk
writer = new PrintWriter(filePath, "UTF-8");
//convert string to ea-rtf format
String rtfText = repository.GetFormatFromField("RTF", noteText);
//write content to file
writer.write(rtfText);
writer.close();
//create linked document to element by loading the before created rtf file
element.LoadLinkedDocument(filePath);
element.Update();
}
EDIT EDIT
It is also possible to work with a temporary file:
File f = File.createTempFile("rtfdoc", ".rtf");
FileOutputStream fos = new FileOutputStream(f);
String rtfText = repository.GetFormatFromField("RTF", noteText);
fos.write(rtfText.getBytes());
fos.flush();
fos.close();
element.LoadLinkedDocument(f.getAbsolutePath());
element.Update();
First up, let's separate the linked document, which is stored in the EA project and displayed in EA's built-in RTF viewer, from an RTF file, which is stored on disk.
Element.LoadLinkedDocument() is the only way to create a linked document. It reads an RTF file and stores its contents as the element's linked document. An element can only have one linked document, and I think it is overwritten if the method is called again but I'm not absolutely sure (you could get an error instead, but the EA API tends not to work that way).
In order to specify the contents of your linked document, you must create the file and then load it. The only other way would be to go hacking around in EA's internal and undocumented database, which people sometimes do but which I strongly advise against.
In .NET you can create RTF documents using Microsoft's Word API, but to my knowledge there is no corresponding API for Java. A quick search turns up jRTF, an open-source RTF library for Java. I haven't tested it but it looks as if it'll do the trick.
You can also use EA's API to create RTF data. You would then create your intended content in EA's internal display format and use Repository.GetFormatFromField() to convert it to RTF, which you would then save in the file.
If you need to, you can use Repository.GetFieldFromFormat() to convert plain-text or HTML-formatted text to EA's internal format.

Not able to get the pdf from internet

I am trying to download the pdf content from the internet and download it to a local file.
I am using iText for reading the pdf and using Java stream to write it.
After writing the file, new file can not be opened in the PDF reader.
PdfReader reader = new PdfReader(strURL);
FileOutputStream fos = new FileOutputStream(new File(fileName));
fos.write(reader.getPageContent(1));
fos.flush();
fos.close();
I am trying to get the PDf from this link
I am debugged few things. Here are the findings.
reader.getEofPos()
gives 291633, Which is same as file length.But
reader.getPageContent(1).length;
gives only 42360 bytes. Clearly byte read are less than the actual size.
Only one page of pdf is present
reader.getNumberOfPages() =1
Do i need to specify few more things to reader to read the entire pdf file?
If all you're trying to do is download a PDF from the internet and save it locally, this can be accomplished using a simple HTTP web request. An internet download is normally just an HTTP GET request, which you can accomplish by doing something like this:
URLConnection connection = new URL(url).openConnection();
connection.setRequestProperty("Accept-Charset", charset);
InputStream response = connection.getInputStream();
Once you get the response, you can save the bytes to a path of your choosing.

Reading php file from a website using Java?

I am aware that you can use Java to execute the php file from a website like below:
URLConnection conn = new URL("http://localhost/file.php").openConnection();
conn.connect();
However would it be possible to make Java read the php file as a text file and how? (Possibly via the use of the BufferedReader)
Thank you.
If you mean to read the php source, then the answer is no - you cannot read php source in a typical webserver configuration. In most cases it would be very insecure to allow it - passwords and other constants are often stored in php source code.
If however, you are just meaning to read out the result of executing the php document, as a browser would do, you can use something like;
URL u = new URL("http://www.example.com/my/php/doc.php");
URLConnection c = u.openConnection();
InputStream r = c.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(r));
for(String line; (line = reader.readLine()) != null;) System.out.println(line);
To read the source of a remote php document via http, the webserver needs to be configured to not parse & execute the file, rather just serve it blindly as a plain text file.
This can be achieved in a few ways;
Uninstall php - no php engine, no code execution.
Change the file extension - apache actually just identifies files by their file extension, so changing it to .phpsource would do the trick in many instances.
Alter the apache configuration - either globally, or via .htaccess you can alter webserver configurations for a single directory. You would want to do something like php_flag engine off or RemoveHandler .php
Note that only the first, and maybe the second, method above does not open up potential security holes in your server. So use with caution.

Categories

Resources