CSV file truncated after download - java

The legacy J2EE application details :
JSP + Servlets(2.4)
Websphere Application Server 7.0
The view is using IE frames, core javascript and so on
The user's action :
User's search returning > 900 rows takes some time to display (NO pagination)
User then clicks on 'Download' button which again triggers a form submit.
Following is the code snippet that is executed in the action servlet :
public class DownloadFileEvent extends ActionGeneric {
java.text.SimpleDateFormat df_file = new java.text.SimpleDateFormat("yyyyMMdd_HHmmss");
public void run(HttpServletRequest request, HttpServletResponse response) throws Exception {
String errormsg = null;
StringBuffer LineBuffer = null;
// read parameters.
String _v = request.getParameter("view");
// Start traitment.
try {
// get sessions.
ServletOutputStream out = response.getOutputStream();
// Create title columns.
response.setContentType("application/csv");
response.setHeader("Content-Disposition", "inline; filename = " + getFilename(_v));
LineBuffer = new StringBuffer();
//Get the string response from some business method
//String v_wrk = getOutflow(request, _v).toString();
LineBuffer.append(v_wrk);
LineBuffer.append("\r\n");
out.print(LineBuffer.toString());
out.flush();
// end.
}// fin try
catch (Exception e) {
errormsg = e.getMessage();
} finally {
// to do.
}
}// end run.
}// fin class
The issue :
Since the 'Download' takes some time, the user moves to other screen
When he comes back, the 'Open/Save/Save As' prompt is there already for some time. Now when user saves/opens the file but instead of 900 rows, there are less than 100 rows
Surprisingly, if the open/save is done immediately, all the rows are downloaded
In the catch block, I had put a log but there is no exception anywhere
The issue is not simulated on my local machine(Windows, WAS 7) or in the SYSTEM test environment(Linux, WAS 8.5) but surfaces on ACCEPTANCE (WAS 7, Linux) and PRODUCTION(WAS 7, Linux). The ACCEPTANCE and PRODUCTION have load balancers, web server set up but NOT in systest or local
How shall I proceed ?

Try a high SendBufferSize in your web servers. If the client does not read while the dialog is up, this will prevent the webserver from seeing writes eventually block then timeout.

Related

Java - Sending of ImageIcon through ObjectOutputStream works first time, but not after that

I'm working on a server/client project in Java where, at any point during the running of the program, the client can request a set of details related to a unique ID, and the server returns the relevant set of details. This is done through PrintWriter objects accessing the socket.getOutputStream, and works fine.
I am trying to also include the sending/receiving of an image from the server to the client, and have met some very strange behaviour from the program.
The methods that send and receive the images are shown below:
SERVER-SIDE:
//send image associated with item
//through ObjectOutputStream to client
private void sendItemImage(BidItem item)
{
try
{
//wrap object output stream around
//output stream to client at this socket
ObjectOutputStream imageOutput =
new ObjectOutputStream(client.getOutputStream());
//send image object to client
imageOutput.writeObject(item.getItemImage());
}
catch (IOException ioEx)
{
//alert server console
System.out.println("\nUnable to send image for item "
+ item.getItemCode() + " to "
+ bidderName + "!");
//no exit from system
//bidding can still continue
}
}
CLIENT-SIDE:
//to be displayed on GUI
private static void receiveItemImage()
{
try
{
//wrap ObjectInputStream around socket
//to receive objects sent from server
ObjectInputStream imageInput =
new ObjectInputStream(socket.getInputStream());
//read in image and store in ImageIcon instance
image = (ImageIcon) imageInput.readObject();
//re-create label object to be blank
imageLabel = new JLabel();
//remove label containing last image
imagePanel.remove(imageLabel);
//just ignores command if it does not already contain image
//apply image to label
imageLabel.setIcon(image);
//apply image to CENTER of panel
imagePanel.add(imageLabel, BorderLayout.CENTER);
}
//problem in input stream
catch (IOException ioEx)
{
//alert user
JOptionPane.showMessageDialog(
null, "Error receiving image!");
//allow system to continue
//no exit
}
//problem casting to ImageIcon type
catch (ClassNotFoundException cnfEx)
{
//alert user
JOptionPane.showMessageDialog(
null, "Error converting Object to ImageIcon!");
//allow system to continue
//no exit
}
}
So, each time there is a request, an ObjectOutpuStream and ObjectInputStream are created to handle the passing of the image, using the socket.getOutputStream/socket.getInputStream.
These methods are first called when a client connects to the server, and the first image and set of details are sent automatically. This works fine, but any subsequent attempts at requesting the image throughout the program result in the catch (IOException) clause being met, and the error messages shown above being displayed.
I cannot for the life of me work out why it would work the first time but not again after this. If anyone could point me in the right direction, that would be great!
Thanks,
Mark
You should only wrap a stream once. In this case, you can't wrap it again as this will not work for an Object Stream.
Once you are wrapping the stream only once, call ObjectOutputStream.reset() after sending an image. If you don't do this it will just pass the reference to the object again (and use a lot of memory)

Catch the download link automatically like IDM?

Here is my code for downloading a file I am passing a URL, but I want to make my download manager catch the link automatically
private void button1_Click_1(object sender, EventArgs e)
{
WebClient client = new WebClient();
client.DownloadProgressChanged += new DownloadProgressChangedEventHandler(ProgressChanged);
client.DownloadFileCompleted += new AsyncCompletedEventHandler(client_DownloadFileCompleted);
string []filename;
string fn ; fn=textBox1.Text;
int i;
filename =fn.Split('/');
for (i = 1; i <= filename.Length; i++) ;
string oname=filename[i-2];
//////// Starts the download
client.DownloadFileAsync(new Uri("http://dinirah.wen.ru/Internet-k-zariye-musalmano-gumrah-kia-ja-raha.mp3"), textBox2.Text+"\\"+oname);
//////// the above i passed the url to download.
label4.Text = i.ToString();
button1.Text = "Download In Process";
button1.Enabled = false;
}
Now I want to make my downloader automatically get the download url, and store it in a string which it then passes as an URL later.
You should develop browser extension(s) and then you would be able to intercept the file download process.
The other way is to monitor your computer's network connections and intercept when a desired file type is being requested. I have found that IDM intercepts my downloads even if I am downloading it using my programs (If enabled in IDM to intercept any download). Be careful with windows update files, diagnostic files and any other unknown file.

Apache Wicket: File download that would not lock the page

I want to create a link that would initiate a file download which would be asynchronous to the page itself, i.e. I want the page not to be locked during the file download. Should I make it be initiated outside wicket? Or is there something inside wicket that would let me set up a resource stream which would bypass the page locks?
Things I tried:
DownloadLink - locks the page, as stated in its doc. This was my starting point.
ResourceLink - did not state the locking explicitly in the doc, so I tried this, but it also locked the page.
At this point I've investigated the code of both links a bit and noticed they both schedule the download via ResourceStreamRequestHandler. Expecting that his kind of behavior could be just handler-specific I've attempted to schedule a custom handler I've written:
private void sendFile(final File file) throws IOException {
IRequestHandler fileDownloadHandler = new IRequestHandler() {
#Override
public void respond(IRequestCycle requestCycle) {
WebResponse response = (WebResponse) requestCycle.getResponse();
OutputStream outStream = response.getOutputStream();
response.setContentType("audio/x-wav");
response.setContentLength((int)file.length());
String fileName = "Somethingsomething.wav";
// sets HTTP header
response.setHeader("Content-Disposition", "attachment; filename=\"" + fileName + "\"");
byte[] byteBuffer = new byte[1024];
DataInputStream in = null;
try {
in = new DataInputStream(new FileInputStream(file));
int length = 0;
// reads the file's bytes and writes them to the response stream
while ((in != null) && ((length = in.read(byteBuffer)) != -1))
{
outStream.write(byteBuffer,0,length);
}
in.close();
outStream.close();
} catch (IOException e) {
throw new PortalError("IOException trying to write the response", e);
}
}
#Override
public void detach(IRequestCycle requestCycle) {
}
};
getRequestCycle().scheduleRequestHandlerAfterCurrent(fileDownloadHandler);
}
This did not quite work either, so I've investigated further. I've noticed that unlike I expected, the "scheduled" request handlers would not get executed on a separate request, as I expected, but on the same one. I figured that it must be that the page gets locked for the first handler and then remains locked while the second one is executing as well. So I've attempted to force the download handler into a separate request (via an ajax behaviour):
public void startDownload(AjaxRequestTarget target) throws DownloadTargetNotFoundException{
target.appendJavaScript("setTimeout(\"window.location.href='" + getCallbackUrl() + "'\", 100);");
}
#Override
public void onRequest() {
sendFile(getFile());
logger.debug("Download initiated");
}
I've found this here and hoped it could potentially be what I've been looking for. However, unsurprisingly so, the page gets locked still (I would imagine because the behaviour still has to be retrieved from the page, for which the page lock has to be acquired).
I'm at a loss where I should be looking next, especially after all this time trying to get a simple download link working. I was considering creating another web filter one layer above wicket, which could be signaled from within wicket to create the download after the wicket filter is finished with its work (and hence the page lock is already released), but that seems a bit excessive for a task like this.
Any suggestions are welcome.
You have to download from a resource, see
http://wicketinaction.com/2012/11/uploading-files-to-wicket-iresource/ and read http://wicket.apache.org/guide/guide/resources.html

HTML5 Server Side Events (SSE)

I am working on implementing SSE in a Web Application on Java Stack using Servlets. I have facing 2 key issues currently. Let me first place my code both for the Web page and the Servlet followed by the issue I am facing.
Web Page Code:
<script type="text/javascript">
function registerSSE() {
var source = new EventSource("http://www.sample.com/BootStrap/NotificationServlet");
source.addEventListener('StartProductionRun', function(e) {
// Get the data and identify the instrument Name/Id
var dataReceived = e.data;
document.getElementById(dataReceived + "_button").disabled = true;
}, false);
}
function StartProduction(instrument) {
var dataString = 'instrumentName='+ instrument;
// call ajax to submit the form and start the production run
$.ajax({
type: 'POST',
url: 'http://localhost:8080/BootStrap/ProductionRunServlet',
data: dataString,
success: function() {
$('#Status').html("<div id='message'></div>");
$('#message').html("<h4 aling=\"centre\">Prudction Run for Instrument " + instrument + " initiated.</h4>")
.hide()
.fadeIn(5000);
}
});
}
</script>
Servlet Code :
public void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
response.setContentType("text/event-stream;charset=UTF-8");
response.setHeader("Cache-Control", "no-cache");
PrintWriter out = response.getWriter();
NotificationService notificationService = null;
while (true) {
notificationService = NotificationService.getInstance();
if (notificationService.getNotificationCount() > 0 ) {
String notificationValue = notificationService.getNotification(0);
String[] keyValue = notificationValue.split(":");
out.print("event:" + keyValue[0] + "\n");
out.print("data: " + keyValue[1] + "\n");
out.print("retry:" + 1000 + "\n\n");
out.flush();
}
else {
out.print(": time stream \n");
out.print("retry:" + 1000 + "\n\n");
out.flush();
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
Now the issues:
The Web Page will be viewed by multiple users at the same time. And I want that the data to be pushed to all the users who viewing that page. Currently when I am running locally in my machine, even if I open Chrome and Firefox, I don't get the notification in both the Browsers. It comes only in one.
Also, If I leave the browser running for some time, I find that even if the servlet is pushing out data based on certain events. I don't get the notification on the Browser.
I need to make sure that:
The notification gets pushed to all the clients who are viewing that particular page irrespective of what they are doing on the page or the page is just used for viewing the information.
Looking forward to all the help I can get to make this working. Also, would be interested to know if there are other alternative which I can use.
Your servlet code is working perfectly, although i haven't worked so much on it(once did a jsp project). What i think is, you have missed something in javascript?
I think there should be a timer/thread/loop in javascript too, to get all pushed data continuously. i.e,
setInterval(
function(){
// code which needs to run every 5sec
},5000);
I hope this will help out a bit.
You should check if EventSource is available in that browser before using it. Maybe one of the browsers has not support for it.

Applet processing a file works locally but fails in website

I've been bashing my head in on this problem for a few days now. I've done my full share of Googling, and I'm hoping I can find someone here that is more experienced than I (not hard to find haha) that can decipher my problem.
Scenario: I developed a Java Applet that implements a Swing GUI. Background work: The Applet gathers records from a large "phone book" excel file (.csv) and stores them in a Map data structure. The phone book contains about 106,000 records, and on the 34,586th record, I get an ArrayIndexOutOfBoundsException that I cannot make sense of. The exception only occurs when I'm running the applet on my personal website. The applet runs perfectly fine with no errors while testing in my IDE (NetBeans) and running the .html file (the one that contains the applet) on my local machine. The output and exception that is thrown while running on my website is as follows (I cut most of the records to save space):
Java Console
Kary,Webber,2826 East 12th Ave.,Memphis,TN,38168,901-749-1834
Erinn,Rocha,2132 East Main Ave.,Memphis,TN,38168,865-414-5105
Gina,Lane,71 South First St. Apt. 11,Memphis,TN,38168,731-485-1129
Patsy,Hart,661 East 11th St.
java.lang.ArrayIndexOutOfBoundsException: 3
at Implementation.PersonnelDatabase.addRecordFromFields(PersonnelDatabase.java:192)
at Implementation.PersonnelDatabase.initDBFromFile(PersonnelDatabase.java:215)
at Implementation.PersonnelDatabase.processData(PersonnelDatabase.java:239)
at Implementation.PersonnelDatabaseApplet$2.doInBackground(PersonnelDatabaseApplet.java:78)
at Implementation.PersonnelDatabaseApplet$2.doInBackground(PersonnelDatabaseApplet.java:69)
at javax.swing.SwingWorker$1.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at javax.swing.SwingWorker.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
As you can see, on the 34,586th record (beginning with Patsy,Hart), it outputs midway through her address. The full record is as follows: Patsy,Hart,661 East 11th St. Apt. 195,Memphis,TN,38168,555-555-5555 .
Below are the sections of code that are most logically influenced by the exception.
Object tag in HTML file
<object type="application/x-java-applet" height="400" width="300">
<param name="codebase" value="classes" />
<param name="code" value="Implementation/PersonnelDatabaseApplet.class" />
<param name="archive" value="PersonnelDatabase.jar" />
Applet failed to run. No Java plug-in was found.
</object>
PersonnelDatabase class (processes background data):
/*
* Create a new record using an appropriately ordered set of fields and add it to the data base
*/
public void addRecordFromFields(String[] fields)
{
// Read record attributes in, one at a time
Record thisRecord = new Record();
thisRecord.setFirstName(fields[0]);
thisRecord.setLastName(fields[1]);
thisRecord.setAddress(fields[2]);
thisRecord.setCity(fields[3]);
thisRecord.setState(fields[4]);
thisRecord.setZipCode(fields[5]);
thisRecord.setPhoneNo(fields[6]);
addRecord(thisRecord);
}
// O( n )
/**
* Destroy the current data base and load new data from a file.
* #param filename the file to use as a source
* #throws IOException: Either file not found or IO error
*/
public void initDBFromFile(URL url) throws IOException
{
// Open and read the file
InputStream in = url.openStream();
BufferedReader filein = new BufferedReader(new InputStreamReader(in));
// Read record file, parse lines, and add records to data base
String line = filein.readLine();
while(line != null) {
System.err.println(line);
String[] fields = line.split(",");
addRecordFromFields(fields);
line = filein.readLine();
}
filein.close();
}
/**
* Loads the default library and provides for interaction with the data
* via the JPanel GUI inputs.
* #param args
* #throws IOException
*/
public String processData(String input, int selection, URL url)
{
//Create the main library object
PersonnelDatabase dbiLib = new PersonnelDatabase();
System.err.println(url);
// Try to read the default library
try
{
dbiLib.initDBFromFile(url);
}
catch (IOException e)
{
System.err.println("File IO Error");
e.printStackTrace();
System.exit(1);
}
// Queries can be simulated by typing into the console in Eclipse, and using Ctrl-d (Ctrl-z in Windows) when finished.
// For example: "searchLastName,Smith" would print a list of all people with the last name of Smith.
Iterable<Record> result = null;
String[] fields = new String[2];
if (input.contains(",")) {
fields = input.split(",");
}
switch(selection) {
case 0: result = dbiLib.searchByFirstName(input); break;
case 1: result = dbiLib.searchByLastName(input); break;
case 2: result = dbiLib.searchByFullName(fields[0].trim(), fields[1].trim()); break;
case 3: result = dbiLib.searchByCity(input); break;
case 4: result = dbiLib.searchByState(input); break;
case 5: result = dbiLib.searchByCityState(fields[0].trim(), fields[1].trim()); break;
case 6: result = dbiLib.searchByZip(input); break;
case 7: result = dbiLib.searchByPhoneNumber(input); break;
case 8: String[] newFields = new String[fields.length-1];
System.arraycopy(fields, 1, newFields, 0, fields.length-1);
dbiLib.addRecordFromFields(newFields);
return "Record added successfully!\nEnter a query or add another record.";
default: return "Invalid query.\nEnter another query or add a record.";
}
PersonnelDatabaseApplet class (initializes GUI, gathers input, and displays output):
public void init() {
/* Create and display the applet */
try {
java.awt.EventQueue.invokeAndWait(new Runnable() {
#Override
public void run() {
initComponents();
}
});
} catch (Exception e) {
System.err.println("Creation of GUI did not successfully complete.");
}
}
// Process inputs in the background.
SwingWorker worker = new SwingWorker<String, Void>() {
#Override
public String doInBackground() {
URL url = null;
try {
url = new URL(getCodeBase(), fileToRead);
}
catch(MalformedURLException e){}
personnelDatabase = new PersonnelDatabase();
final String output = personnelDatabase.processData(input, selection, url);
return output;
}
#Override
public void done() {
processingLabel.setVisible(true);
try {
textToDisplay = get(15, TimeUnit.SECONDS);
} catch (InterruptedException ignore) {
ignore.printStackTrace();
} catch (TimeoutException e) {
e.printStackTrace();
} catch (ExecutionException e) {
String why = null;
Throwable cause = e.getCause();
if(cause != null) {
why = cause.getMessage();
cause.printStackTrace();
} else {
why = e.getMessage();
e.printStackTrace();
}
System.err.println("Error retrieving request: " + why);
}
if(worker.isDone() && textToDisplay != null) {
processingLabel.setVisible(false);
outputTextArea.setText(textToDisplay);
}
}
};
private void searchButtonActionPerformed(java.awt.event.ActionEvent evt) {
selection = searchComboBox.getSelectedIndex();
input = valueTextField.getText();
processingLabel.setVisible(true);
worker.execute();
}
Link to the Applet on my personal website: http://www.ryan-taylor.me/Applied%20Maps/build/PersonnelDatabaseApplet.html
I'm fairly certain the error has nothing to do with the excel data itself as this program runs just fine in NetBeans and when running the html on my local machine. I'm guessing it has something to do with Swing (threads), but I'm not sure. I made changes to help with transferring data between Swing threads by making use of the SwingWorker, but I had no luck. I guess there's always the possibility I missed something when implementing it.
I also thought about signing the jar, but the file that I'm processing is stored online - not a local machine - so I didn't see a real need.
If anyone has any suggestions, I would greatly appreciate it!
Looks like the array index that's out of bounds is 3, because the input line contains only three fields and you're trying to access the fourth one (index 3) without checking to verify that it actually exists. The error is in
thisRecord.setCity(fields[3]);
because the array fields has only three elements. At
String[] fields = line.split(",");
addRecordFromFields(fields);
when you get to
Patsy,Hart,661 East 11th St.
array fields will be created with only 3 entries.
If the number of fields is expected to be constant then you should reject input lines that don't have the right number of fields. If the number of fields can vary, then you must check the actual number returned and extract only those elements that actually exist.
It looks like something is causing the file to be truncated when you run the applet in a browser. My guess is that you are fetching the file from a web server, and either the server or the browser is silently enforcing some download limit. (Or may be the file got truncated when you uploaded it ...)
I suspect a data race between your worker and another thread that is exposed by network latency. Two things may bear closer scrutiny:
Using invokeAndWait() to initialize the GUI is reasonable, as long as no further processing is done on the initial thread.
Your worker is unusual in not invoking publish(), as a way to process() records on the event dispatch thread, for example.

Categories

Resources