My application streams twitter data and writes them to files.
while(true){
Status status = queue.poll();
if (status == null) {
Thread.sleep(100);
}
if(status!=null){
list.add(status);
}
if(list.size()==10){
FileOutputStream fos = null;
ObjectOutputStream out = null;
try {
String uuid = UUID.randomUUID().toString();
String filename = "C:/path/"+topic+"-"+uuid+".ser";
fos = new FileOutputStream(filename);
out = new ObjectOutputStream(fos);
out.writeObject(list);
tweetsDownloaded += list.size();
if(tweetsDownloaded % 100==0)
System.out.println(tweetsDownloaded+" tweets downloaded");
// System.out.println("File: "+filename+" written.");
out.close();
} catch (IOException e) {
e.printStackTrace();
}
list.clear();
}
I have this code which gets data from files.
while(true){
File[] files = folder.listFiles();
if(files != null){
Arrays.sort(//sorting...);
//Here we manage each single file, from data-load until the deletion
for(int i = 0; i<files.length; i++){
loadTweets(files[i].getAbsolutePath());
//TODO manageStatuses
files[i].delete();
statusList.clear();
}
}
}
The method loadTweets() does the following operations:
private static void loadTweets(String filename) {
FileInputStream fis = null;
ObjectInputStream in = null;
try{
fis = new FileInputStream(filename);
in = new ObjectInputStream(fis);
statusList = (List<Status>) in.readObject();
in.close();
}
catch(IOException | ClassNotFoundException ex){
ex.printStackTrace();
}
}
Unfortunately, I don't know why sometimes it throws a
EOFException
when running this line
statusList = (List<Status>) in.readObject();
Anybody knows how I can solve this? Thank you.
I've seen that you're passing the file correctly with the getAbsolutePath() based on a previous question of yours
From what I've read that can be a couple of things, one of them the file being null.
Explaining this idea, you might have written the file but something caused the file to have nothing inside, this might cause an EOFException. The file in fact exists it's just empty
EDIT
Try to enclose the code in while(in.available() > 0)
It would look like this
private static void loadTweets(String filename) {
FileInputStream fis = null;
ObjectInputStream in = null;
try{
fis = new FileInputStream(filename);
in = new ObjectInputStream(fis);
while(in.available() > 0) {
statusList = (List<Status>) in.readObject();
}
in.close();
}
catch(IOException | ClassNotFoundException ex){
ex.printStackTrace();
}
}
Found out what was necessary to solve this. Thanks to #VGR's comment, I thought to pause the executing thread for 0.2 seconds if the file has been created less than a second ago.
if(System.currentTimeMillis()-files[i].lastModified()<1000){
Thread.sleep(200);
This prevents the exception and the application works now fine.
Related
The below code i am trying to execute is corrupting the excel file whenever an exception occurs before closing the FileInputStream and FileOutPutStream. So i tried keeping try catch block and in the catch block closed FileInputStream and FileOutPutStream, still excel is getting crashed.
So i thought something like this if FileInputStream is not closed then close in catch block . If FileOutPutStream is not closed then close in catch block. But how to achieve this?
or
Is there any better way to handle this situation?
public void checkingAccountNumber() throws InterruptedException, IOException {
WebDriverWait wait = new WebDriverWait(driver, 5);
FileInputStream fis= new FileInputStream("C:\\Users\\vyerrami\\Desktop\\MA_Conversion_Data.xlsx");
XSSFWorkbook wb= new XSSFWorkbook(fis);
XSSFSheet sheet = wb.getSheet("DEFERRED");
int noOfRows=sheet.getPhysicalNumberOfRows();
try {
for(int i=1; i<noOfRows; i++) {
DataFormatter d = new DataFormatter();
String cellvalue = d.formatCellValue(sheet.getRow(i).getCell(5));
String firstcellvalue=d.formatCellValue(sheet.getRow(i).getCell(0));
if(firstcellvalue.isEmpty()) {
driver.findElement(quickJump).click();
driver.findElement(quickJump).sendKeys("AccountSearch");
Thread.sleep(2000);
driver.findElement(quickJump).sendKeys(Keys.ENTER);
wait.until(ExpectedConditions.invisibilityOf(driver.findElement(loading)));
driver.findElement(accountSearchAccountNumberField).clear();
driver.findElement(accountSearchAccountNumberField).sendKeys(cellvalue);
driver.findElement(accountSearchAccountNumberField).sendKeys(Keys.TAB);
driver.findElement(accountSearchSearchButton).click();
wait.until(ExpectedConditions.invisibilityOf(driver.findElement(loading)));
if (driver.findElements(By.id("AccountSearch:AccountSearchScreen:AccountSearchResultsLV:0:AccountNumber")).size() > 0) {
if (driver.findElement(By.id("AccountSearch:AccountSearchScreen:AccountSearchResultsLV:0:AccountNumber")).getText().contains(cellvalue)) {
driver.findElement(By.id("AccountSearch:AccountSearchScreen:AccountSearchResultsLV:0:AccountNumber"))
.click();
fis.close();
FileOutputStream out = new FileOutputStream(
"C:\\Users\\vyerrami\\Desktop\\MA_Conversion_Data.xlsx");
if(driver.findElements(policyFilePolicyNumber).size()>0) {
sheet.getRow(i).createCell(0)
.setCellValue(driver.findElement(policyFilePolicyNumber).getText());
sheet.getRow(i).createCell(1)
.setCellValue(driver.findElement(policyFileEffectiveDate).getText());
wb.write(out);
out.close();
}
else {
sheet.getRow(i).createCell(0)
.setCellValue("Policy Number is not present");
wb.write(out);
out.close();
}
}
}
else {
FileOutputStream out = new FileOutputStream(
"C:\\Users\\vyerrami\\Desktop\\MA_Conversion_Data.xlsx");
sheet.getRow(i).createCell(0)
.setCellValue("Account Number is not present");
wb.write(out);
out.close();
}
}
}
}
catch(Exception e)
{
//i want to write the code if fis is not closed then i have to close
//i want to write the code is out is not closed then i have to close
}
}
EDIT CODE WHICH IS WORKING IS BELOW:
FileInputStream fis=null;
FileOutputStream out1=null;
XSSFWorkbook workbook= null;
XSSFSheet sheet = null;
try {
fis = new FileInputStream("C:\\Users\\vyerrami\\Desktop\\VINS.xlsx");
out1 = new FileOutputStream("C:\\Users\\vyerrami\\Desktop\\VINS.xlsx");
workbook = new XSSFWorkbook(fis);
sheet = workbook.getSheet("VINS");
int noOfRows = sheet.getPhysicalNumberOfRows();
for (int i = 0; i < 100; i++) {
driver.findElement(createVehicleButton).click();
wait.until(ExpectedConditions.invisibilityOf(driver.findElement(loading)));
driver.findElement(EnterVinNumber).clear();
driver.findElement(EnterVinNumber).sendKeys(sheet.getRow(i).getCell(0).getStringCellValue());
driver.findElement(EnterVinNumber).sendKeys(Keys.TAB);
wait.until(ExpectedConditions.invisibilityOf(driver.findElement(loading)));
driver.findElement(vehicleInquiryButton).click();
wait.until(ExpectedConditions.invisibilityOf(driver.findElement(loading)));
System.out.println(driver
.findElement(By.xpath(
"//*[#id='VehicleInquiryReportPopup:2-body']/div/div/table[2]/tbody/tr/td[2]/div"))
.getText());
System.out.println(driver
.findElement(By.xpath(
"//*[#id='VehicleInquiryReportPopup:2-body']/div/div/table[2]/tbody/tr/td[3]/div"))
.getText());
if (driver
.findElement(By.xpath(
"//*[#id='VehicleInquiryReportPopup:2-body']/div/div/table[2]/tbody/tr/td[2]/div"))
.getText().equalsIgnoreCase("null null")
|| !driver.findElement(By.xpath(
"//*[#id='VehicleInquiryReportPopup:2-body']/div/div/table[2]/tbody/tr/td[3]/div"))
.getText().equalsIgnoreCase("null null") && (sheet.getRow(i).getCell(1).getStringCellValue().isEmpty())) {
fis.close();
sheet.getRow(i).createCell(1).setCellValue("Not Leased");
workbook.write(out1);
out1.close();
workbook.close();
}
else {
fis.close();
sheet.getRow(i).createCell(1).setCellValue("Leased");
workbook.write(out1);
out1.close();
workbook.close();
}
driver.findElement(By.id("VehicleInquiryReportPopup:__crumb__")).click();
wait.until(ExpectedConditions.invisibilityOf(driver.findElement(loading)));
driver.findElement(By.xpath(
"//*[#id='SubmissionWizard:LOBWizardStepGroup:LineWizardStepSet:PAVehiclesScreen:PAVehiclesPanelSet:VehiclesListDetailPanel:VehiclesLV-body']/div/div/table/tbody/tr/td/div/img"))
.click();
driver.findElement(By.id(
"SubmissionWizard:LOBWizardStepGroup:LineWizardStepSet:PAVehiclesScreen:PAVehiclesPanelSet:VehiclesListDetailPanel_tb:Remove"))
.click();
wait.until(ExpectedConditions.invisibilityOf(driver.findElement(loading)));
}
}
catch (Exception e) {
e.printStackTrace();
}
finally {
if(fis!=null) {
fis.close();
}
if(out1!=null) {
out1.close();
}
}
}
I think, that you are looking for try with resource. It's sugar syntax for opening stream in try block and closing it in finally block. This way whatever happens in try block, stream is closed.
In most cases, you should open streams as late as possible and close them as soon as possible. Try to make smaller methods, so it is easier to debug them later.
I am trying to read one big file in chunk . So the read operation will be called multiple times being offset one of the parameter . The read is working perfectly fine .
But the real problem is starts when I try to delete the file after read is complete . It is throwing IO exception .
I do not want to forcefully garbage collect(System.gc()) .
Read Code :
public static GenericExcelRead ReadFileContent(String fileName, int offset, String status) throws IOException
{
GenericExcelRead aGenericExcelRead = new GenericExcelRead();
//FileInputStream fileStream = null;
FileChannel fileChannel = null;
MappedByteBuffer buffer;
try(FileInputStream fileStream = new FileInputStream(fileName)) {
fileChannel = fileStream.getChannel();
buffer = null;
if (status != "Completed")
{
if(fileChannel.size()>=(offset+1048756))
{
buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, offset, 1048756);
aGenericExcelRead.setStatus("Partial");
aGenericExcelRead.setEndOffset(offset+1048756);
}
else
{
buffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, offset, (fileChannel.size()-offset));
aGenericExcelRead.setStatus("Completed");
aGenericExcelRead.setEndOffset((int)fileChannel.size());
}
byte[] b = new byte[buffer.remaining()];
buffer.get(b);
String encodedcontent = new String(Base64.encodeBase64(b));
buffer.clear();
fileChannel.close();
aGenericExcelRead.setData(encodedcontent);
fileStream.close();
}
} catch (IOException e) {
throw new IOException("IO Exception/File not found");
}finally {
if(fileChannel != null)
fileChannel.close();
}
return aGenericExcelRead;
}
I am trying to query from database with Java JDBC and compress data in one column to gzip file in specific directory. I have tested my JDBC query and it working fine, but the Gzip code not going with the while loop, it's run with the loop firt row and stuck there. Why it's stuck? help me please!
These folders already existed: D:\Directory\My\year\id1\id2
//Some query JDBC code here, it's work well. I query all rows Data, year, id1,id2,id3
while (myRs1.next()){
String str = Data;
File myGzipFile = new File("D:\\Directory\\My\\"+year+"\\"+id1+"\\"+id2+"\\"+id3+".gzip");
GZIPOutputStream gos = null;
InputStream is = new ByteArrayInputStream(str.getBytes());
gos = new GZIPOutputStream(new FileOutputStream(myGzipFile));
byte[] buffer = new byte[1024];
int len;
while ((len = is.read(buffer)) != -1) {
gos.write(buffer, 0, len);
System.out.print("done for:"+id3);
}
try { gos.close(); } catch (IOException e) { }
}
Try formatting the source like this to catch exceptions.
public class InputStreamDemo {
public static void main(String[] args) throws Exception {
InputStream is = null;
int i;
char c;
try{
is = new FileInputStream("C://test.txt");
System.out.println("Characters printed:");
// reads till the end of the stream
while((i=is.read())!=-1)
{
// converts integer to character
c=(char)i;
// prints character
System.out.print(c);
}
}catch(Exception e){
// if any I/O error occurs
e.printStackTrace();
}finally{
// releases system resources associated with this stream
if(is!=null)
is.close();
}
}
}
I have run into quite a snag while writing my app. Here is my issue:
I am trying to initialize the file input stream like so:
FileInputStream fis
fis = openFileInput(selectedFile);
Then put this 1 line later:
byte[] input = new byte[fis.available()];
Problem is both bits of code need try/catch statements and the second block cannot recognize fis because it was initialized within a try/catch. Here is my code:
private void openFile(String selectedFile) {
String value = "";
FileInputStream fis;
try {
fis = openFileInput(selectedFile);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
try {
byte[] input = new byte[fis.available()];
} catch (IOException e) {
e.printStackTrace();
}
What should I do? (Thanks in advance)
The best approach in this situation is not to catch IOException at all.
private void openFile(String selectedFile) throws IOException {
FileInputStream fis = openFileInput(selectedFile);
byte[] input = new byte[fis.available()];
It does not make sense to continue after you got FileNotFoundException
Set FileInputStream fis = null; when you first declare the variable.
You could also run your code like this because IOException will also catch the file not found exception.
String value = "";
FileInputStream fis;
try {
fis = openFileInput(selectedFile);
byte[] input = new byte[fis.available()];
} catch (IOException e) {
e.printStackTrace();
}
Set the FileInputStream to a temporary value. null would be the best option, as in:
FileInputStream fis = null;
The reason for this is because if your try statement throws an error, then the fis will never me initialized. Then you'll have problems. If you don't exit the thing entirely, you should also add the statement after the try/catch blocks that tests if the value is null, just so that the program does not throw a null pointer exception.
So maybe something like:
if(fis == null) {
return; // Which will just end the method.
}
Also might want to put the try/catches together (you should still declare the other stuff outside of the try, at least anything you plan on using directly later on in the code) but it just might be more efficient coding wise), as in:
FileInputStream fis = null;
byte[] input = null;
try {
fis = openFileInput(selectedFile);
input = new byte[fis.available()];
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
I have a question regarding internal files in android.. I tried to write some data into a file and then read it back however, it seems like I can't write data to a file unless I cast it to an integer first.. is there anyway that I can save double or float values.. I added the code I'm trying to use below:
FormatCluster formatCluster = ((FormatCluster)objectCluster.returnFormatCluster(ofFormats,"Calibrated"));
if (formatCluster != null) {
//Obtain data for text view
calibratedDataArray[0] = formatCluster.mData;
calibratedUnits = formatCluster.mUnits;
A.setText("data: " + formatCluster.mData);
String filename = "myfile";
//String string = "Hello world!";
FileOutputStream outputStream;
try {
outputStream = openFileOutput(filename, Context.MODE_PRIVATE);
outputStream.write((int)formatCluster.mData);//here I don't want to cast the value to integer
outputStream.close();
} catch (Exception e) {
e.printStackTrace();
}
//testing.setText)
double ch;
StringBuffer fileContent = new StringBuffer("");
FileInputStream fis;
try {
fis = context.openFileInput( filename );
try {
while( (ch = fis.read()) != -1)
testing.setText(fileContent.append(ch));
} catch (IOException e) {
e.printStackTrace();
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
Envelope your InputStream and OutputStream with a DataInputStream and a DataOutputStream. These classes have the method you need