I have tried to use this question's answer to get a functioning implementation, but I get various errors and am now down to an EOFException and on debugging, it appears the file does not get written.
The goal is to download an image from a URL, save it to internal cache, then later fetch it from that cache for displaying. Where have I gone wrong? The EOFException is thrown in CachedImage.java on the line which reads byte[] data = (byte[]) ois.readObject();
CachedImage.java
package com.example.droid;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.nio.BufferUnderflowException;
import java.nio.ByteBuffer;
import android.graphics.Bitmap;
public class CachedImage implements Serializable {
private static final long serialVersionUID = -12345678987654321L;
private transient Bitmap _bmp;
public CachedImage(Bitmap bmp) {
this._bmp = bmp;
}
public void writeObject(ObjectOutputStream oos) throws IOException {
oos.defaultWriteObject();
if (this._bmp != null) {
int bytes = this._bmp.getWidth() * this._bmp.getHeight() * 4;
ByteBuffer buffer = ByteBuffer.allocate(bytes);
this._bmp.copyPixelsToBuffer(buffer);
if (buffer.hasArray()) {
try {
String configName = this._bmp.getConfig().name();
byte[] array = buffer.array();
oos.writeObject(array);
oos.writeInt(this._bmp.getWidth());
oos.writeInt(this._bmp.getHeight());
oos.writeObject(configName);
} catch (BufferUnderflowException e) {
}
}
} else {
oos.writeObject(null);
}
}
private void readObject(ObjectInputStream ois) throws IOException, ClassNotFoundException {
ois.defaultReadObject();
byte[] data = (byte[]) ois.readObject();
if (data != null) {
int w = ois.readInt();
int h = ois.readInt();
String configName = (String) ois.readObject();
Bitmap.Config configBmp = Bitmap.Config.valueOf(configName);
Bitmap bitmap_tmp = Bitmap.createBitmap(w, h, configBmp);
ByteBuffer buffer = ByteBuffer.wrap(data);
bitmap_tmp.copyPixelsFromBuffer(buffer);
this._bmp = bitmap_tmp.copy(configBmp, true);
bitmap_tmp.recycle();
} else {
this._bmp = null;
}
}
public Bitmap getBitmap() {
return this._bmp;
}
}
And here are the code segments which trigger the calls:
Async callback function for when the image is fetched from the URL to write the image to internal:
#Override
protected void onPostExecute(Bitmap result) {
if (result != null) {
FileOutputStream output = null;
ObjectOutputStream oos = null;
try {
output = ICEApplication.getAppContext().openFileOutput(filename, Context.MODE_PRIVATE);
oos = new ObjectOutputStream(output);
CachedImage cachedImage = new CachedImage(result);
oos.writeObject(cachedImage);
} catch (Exception e) {
Log.d("DEBUG", "Exception: " + e.getMessage());
} finally {
if (output != null) {
try {
oos.close();
output.close();
} catch (IOException e) {
}
}
}
}
}
The code to read the image from disk after downloaded and saved:
Bitmap image = null;
FileInputStream input = null;
ObjectInputStream ois = null;
try {
input = ICEApplication.getAppContext().openFileInput(urldisplay);
ois = new ObjectInputStream(input);
CachedImage cachedImage = (CachedImage)ois.readObject();
image = cachedImage.getBitmap();
} catch (Exception e) {
Log.d("DEBUG", "Exception: " + e.getMessage());
return null;
} finally {
if (input != null) {
try {
ois.close();
input.close();
} catch (IOException e) {
}
}
}
I read from
http://www.javablogging.com/what-are-writeobject-and-readobject-customizing-the-serialization-process/
ObjectOutputStream uses reflection to find out if those methods are declared. It uses getPrivateMethod so those methods have to be declared private in order to be used by the ObjectOutputStream
So, change CachedImage's method writeObject to private(because you posted it as public).
Related
I want to save a file to the internal storage by getting the text inputted from EditText. Then I want the same file to return the inputted text in String form and save it to another String which is to be used later.
Here's the code:
package com.omm.easybalancerecharge;
import android.app.Activity;
import android.content.Context;
import android.content.Intent;
import android.net.Uri;
import android.os.Bundle;
import android.telephony.TelephonyManager;
import android.view.Menu;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.EditText;
import android.widget.TextView;
public class MainActivity extends Activity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
final EditText num = (EditText) findViewById(R.id.sNum);
Button ch = (Button) findViewById(R.id.rButton);
TelephonyManager operator = (TelephonyManager) getSystemService(Context.TELEPHONY_SERVICE);
String opname = operator.getNetworkOperatorName();
TextView status = (TextView) findViewById(R.id.setStatus);
final EditText ID = (EditText) findViewById(R.id.IQID);
Button save = (Button) findViewById(R.id.sButton);
final String myID = ""; //When Reading The File Back, I Need To Store It In This String For Later Use
save.setOnClickListener(new OnClickListener() {
#Override
public void onClick(View v) {
// TODO Auto-generated method stub
//Get Text From EditText "ID" And Save It To Internal Memory
}
});
if (opname.contentEquals("zain SA")) {
status.setText("Your Network Is: " + opname);
} else {
status.setText("No Network");
}
ch.setOnClickListener(new OnClickListener() {
#Override
public void onClick(View v) {
// TODO Auto-generated method stub
//Read From The Saved File Here And Append It To String "myID"
String hash = Uri.encode("#");
Intent intent = new Intent(Intent.ACTION_CALL);
intent.setData(Uri.parse("tel:*141*" + /*Use The String With Data Retrieved Here*/ num.getText()
+ hash));
startActivity(intent);
}
});
}
I have included comments to help you further analyze my points as to where I want the operations to be done/variables to be used.
Hope this might be useful to you.
Write File:
private void writeToFile(String data,Context context) {
try {
OutputStreamWriter outputStreamWriter = new OutputStreamWriter(context.openFileOutput("config.txt", Context.MODE_PRIVATE));
outputStreamWriter.write(data);
outputStreamWriter.close();
}
catch (IOException e) {
Log.e("Exception", "File write failed: " + e.toString());
}
}
Read File:
private String readFromFile(Context context) {
String ret = "";
try {
InputStream inputStream = context.openFileInput("config.txt");
if ( inputStream != null ) {
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String receiveString = "";
StringBuilder stringBuilder = new StringBuilder();
while ( (receiveString = bufferedReader.readLine()) != null ) {
stringBuilder.append("\n").append(receiveString);
}
inputStream.close();
ret = stringBuilder.toString();
}
}
catch (FileNotFoundException e) {
Log.e("login activity", "File not found: " + e.toString());
} catch (IOException e) {
Log.e("login activity", "Can not read file: " + e.toString());
}
return ret;
}
For those looking for a general strategy for reading and writing a string to file:
First, get a file object
You'll need the storage path. For the internal storage, use:
File path = context.getFilesDir();
For the external storage (SD card), use:
File path = context.getExternalFilesDir(null);
Then create your file object:
File file = new File(path, "my-file-name.txt");
Write a string to the file
FileOutputStream stream = new FileOutputStream(file);
try {
stream.write("text-to-write".getBytes());
} finally {
stream.close();
}
Or with Google Guava
String contents = Files.toString(file, StandardCharsets.UTF_8);
Read the file to a string
int length = (int) file.length();
byte[] bytes = new byte[length];
FileInputStream in = new FileInputStream(file);
try {
in.read(bytes);
} finally {
in.close();
}
String contents = new String(bytes);
Or if you are using Google Guava
String contents = Files.toString(file,"UTF-8");
For completeness I'll mention
String contents = new Scanner(file).useDelimiter("\\A").next();
which requires no libraries, but benchmarks 50% - 400% slower than the other options (in various tests on my Nexus 5).
Notes
For each of these strategies, you'll be asked to catch an IOException.
The default character encoding on Android is UTF-8.
If you are using external storage, you'll need to add to your manifest either:
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
or
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
Write permission implies read permission, so you don't need both.
public static void writeStringAsFile(final String fileContents, String fileName) {
Context context = App.instance.getApplicationContext();
try {
FileWriter out = new FileWriter(new File(context.getFilesDir(), fileName));
out.write(fileContents);
out.close();
} catch (IOException e) {
Logger.logError(TAG, e);
}
}
public static String readFileAsString(String fileName) {
Context context = App.instance.getApplicationContext();
StringBuilder stringBuilder = new StringBuilder();
String line;
BufferedReader in = null;
try {
in = new BufferedReader(new FileReader(new File(context.getFilesDir(), fileName)));
while ((line = in.readLine()) != null) stringBuilder.append(line);
} catch (FileNotFoundException e) {
Logger.logError(TAG, e);
} catch (IOException e) {
Logger.logError(TAG, e);
}
return stringBuilder.toString();
}
Just a a bit modifications on reading string from a file method for more performance
private String readFromFile(Context context, String fileName) {
if (context == null) {
return null;
}
String ret = "";
try {
InputStream inputStream = context.openFileInput(fileName);
if ( inputStream != null ) {
InputStreamReader inputStreamReader = new InputStreamReader(inputStream);
int size = inputStream.available();
char[] buffer = new char[size];
inputStreamReader.read(buffer);
inputStream.close();
ret = new String(buffer);
}
}catch (Exception e) {
e.printStackTrace();
}
return ret;
}
The Kotlin way by using builtin Extension function on File
Write: yourFile.writeText(textFromEditText)
Read: yourFile.readText()
check the below code.
Reading from a file in the filesystem.
FileInputStream fis = null;
try {
fis = context.openFileInput(fileName);
InputStreamReader isr = new InputStreamReader(fis);
// READ STRING OF UNKNOWN LENGTH
StringBuilder sb = new StringBuilder();
char[] inputBuffer = new char[2048];
int l;
// FILL BUFFER WITH DATA
while ((l = isr.read(inputBuffer)) != -1) {
sb.append(inputBuffer, 0, l);
}
// CONVERT BYTES TO STRING
String readString = sb.toString();
fis.close();
catch (Exception e) {
} finally {
if (fis != null) {
fis = null;
}
}
below code is to write the file in to internal filesystem.
FileOutputStream fos = null;
try {
fos = context.openFileOutput(fileName, Context.MODE_PRIVATE);
fos.write(stringdatatobestoredinfile.getBytes());
fos.flush();
fos.close();
} catch (Exception e) {
} finally {
if (fos != null) {
fos = null;
}
}
I think this will help you.
I'm a bit of a beginner and struggled getting this to work today.
Below is the class that I ended up with. It works but I was wondering how imperfect my solution is. Anyway, I was hoping some of you more experienced folk might be willing to have a look at my IO class and give me some tips. Cheers!
public class HighScore {
File data = new File(Environment.getExternalStorageDirectory().getAbsolutePath() + File.separator);
File file = new File(data, "highscore.txt");
private int highScore = 0;
public int readHighScore() {
try {
BufferedReader br = new BufferedReader(new FileReader(file));
try {
highScore = Integer.parseInt(br.readLine());
br.close();
} catch (NumberFormatException | IOException e) {
e.printStackTrace();
}
} catch (FileNotFoundException e) {
try {
file.createNewFile();
} catch (IOException ioe) {
ioe.printStackTrace();
}
e.printStackTrace();
}
return highScore;
}
public void writeHighScore(int highestScore) {
try {
BufferedWriter bw = new BufferedWriter(new FileWriter(file));
bw.write(String.valueOf(highestScore));
bw.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Kotlin
class FileReadWriteService {
private var context:Context? = ContextHolder.instance.appContext
fun writeFileOnInternalStorage(fileKey: String, sBody: String) {
val file = File(context?.filesDir, "files")
try {
if (!file.exists()) {
file.mkdir()
}
val fileToWrite = File(file, fileKey)
val writer = FileWriter(fileToWrite)
writer.append(sBody)
writer.flush()
writer.close()
} catch (e: Exception) {
Logger.e(classTag, e)
}
}
fun readFileOnInternalStorage(fileKey: String): String {
val file = File(context?.filesDir, "files")
var ret = ""
try {
if (!file.exists()) {
return ret
}
val fileToRead = File(file, fileKey)
val reader = FileReader(fileToRead)
ret = reader.readText()
reader.close()
} catch (e: Exception) {
Logger.e(classTag, e)
}
return ret
}
}
the first thing we need is the permissions in AndroidManifest.xml
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
so in an asyncTask Kotlin class, we treat the creation of the file
import android.os.AsyncTask
import android.os.Environment
import android.util.Log
import java.io.*
class WriteFile: AsyncTask<String, Int, String>() {
private val mFolder = "/MainFolder"
lateinit var folder: File
internal var writeThis = "string to cacheApp.txt"
internal var cacheApptxt = "cacheApp.txt"
override fun doInBackground(vararg writethis: String): String? {
val received = writethis[0]
if(received.isNotEmpty()){
writeThis = received
}
folder = File(Environment.getExternalStorageDirectory(),"$mFolder/")
if(!folder.exists()){
folder.mkdir()
val readME = File(folder, cacheApptxt)
val file = File(readME.path)
val out: BufferedWriter
try {
out = BufferedWriter(FileWriter(file, true), 1024)
out.write(writeThis)
out.newLine()
out.close()
Log.d("Output_Success", folder.path)
} catch (e: Exception) {
Log.d("Output_Exception", "$e")
}
}
return folder.path
}
override fun onPostExecute(result: String) {
super.onPostExecute(result)
if(result.isNotEmpty()){
//implement an interface or do something
Log.d("onPostExecuteSuccess", result)
}else{
Log.d("onPostExecuteFailure", result)
}
}
}
Of course if you are using Android above Api 23, you must handle the request to allow writing to device memory. Something like this
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.os.Build
import androidx.appcompat.app.AppCompatActivity
import androidx.core.app.ActivityCompat
import androidx.core.content.ContextCompat
class ReadandWrite {
private val mREAD = 9
private val mWRITE = 10
private var readAndWrite: Boolean = false
fun readAndwriteStorage(ctx: Context, atividade: AppCompatActivity): Boolean {
if (Build.VERSION.SDK_INT < 23) {
readAndWrite = true
} else {
val mRead = ContextCompat.checkSelfPermission(ctx, Manifest.permission.READ_EXTERNAL_STORAGE)
val mWrite = ContextCompat.checkSelfPermission(ctx, Manifest.permission.WRITE_EXTERNAL_STORAGE)
if (mRead != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(atividade, arrayOf(Manifest.permission.READ_EXTERNAL_STORAGE), mREAD)
} else {
readAndWrite = true
}
if (mWrite != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(atividade, arrayOf(Manifest.permission.WRITE_EXTERNAL_STORAGE), mWRITE)
} else {
readAndWrite = true
}
}
return readAndWrite
}
}
then in an activity, execute the call.
var pathToFileCreated = ""
val anRW = ReadandWrite().readAndwriteStorage(this,this)
if(anRW){
pathToFileCreated = WriteFile().execute("onTaskComplete").get()
Log.d("pathToFileCreated",pathToFileCreated)
}
We can use this code to write String to a file
public static void writeTextToFile(final String filename, final String data) {
File file = new File(filename);
try {
FileOutputStream stream = new FileOutputStream(file);
stream.write(data.getBytes());
stream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
Then in the Main code, we use this, for example
writeTextToFile(getExternalFilesDir("/").getAbsolutePath() + "/output.txt", "my-example-text");
After that, check the file at Android/data/<package-name>/files.
The easiest way to append to a text file in kotlin:
val directory = File(context.filesDir, "LogsToSendToNextMunich").apply {
mkdirs()
}
val file = File(directory,"Logs.txt")
file.appendText("You new text")
If you want to just write to the file:
yourFile.writeText("You new text")
writing anything to the files, using bytes:
FileOutputStream(file).use {
it.write("Some text for example".encodeToByteArray())
}
I have a problem with my java code. Anyone know this problem. I tried to save my hashmap at the txt file and then read the txt file to the hashmap but it is not working. I think I saved successfully my hashmap but I cannot read it. could you guys please help my code?
Client_Database database = new Client_Database();
String filename = "C:\\bookCafeDatabase.txt";
File file = new File(filename);
byte [] contents = new byte[(int)file.length()];
database.setFileContents(contents);
HashMap<String, Client_Database> user_map = new HashMap<>();
user_map.put(database.setIdDB(Client_Signin.idMsg), database);
try{
ObjectOutputStream bos =
new ObjectOutputStream(
new BufferedOutputStream(
new FileOutputStream(file,true)));
bos.writeObject(user_map);
bos.flush();
bos.close();
ObjectInputStream bis =
new ObjectInputStream(
new BufferedInputStream(
new FileInputStream(file)));
bis.read(contents);
bis.close();
}catch (Exception e1) {
}
}
}
});
}
I think something wrong with this code but I cannot find any problem.
Client_Database database = new Client_Database();
String filename = "C:\\bookCafeDatabase.txt";
File file = new File(filename);
byte[] contents = new byte[(int) file.length()];
database.setFileContents(contents);
try {
ObjectOutputStream bos = new ObjectOutputStream(
new BufferedOutputStream(new FileOutputStream(file)));
bos.writeObject(result);
bos.close();
ObjectInputStream bis =
new ObjectInputStream(
new BufferedInputStream(
new FileInputStream(file)));
bis.read(contents);
result = (HashMap<String, Client_Database>)bis.readObject();
bis.close();
} catch (Exception e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.util.HashMap;
import java.util.Map;
import java.util.Scanner;
public class Client_Database implements Serializable {
private String nameDB;
private String idDB;
private String passwordDB;
private String addressDB;
private String emailDB;
private String fileName;
private byte [] fileContents;
private ObjectOutputStream out;
public String getNameDB() {
return nameDB;
}
public void setNameDB(String nameDB) {
this.nameDB = nameDB;
}
public String getIdDB() {
return idDB;
}
public String setIdDB(String idDB) {
return this.idDB = idDB;
}
public String getPasswordDB() {
return passwordDB;
}
public void setPasswordDB(String passwordDB) {
this.passwordDB = passwordDB;
}
public String getAddressDB() {
return addressDB;
}
public void setAddressDB(String addressDB) {
this.addressDB = addressDB;
}
public String getEmailDB() {
return emailDB;
}
public void setEmailDB(String emailDB) {
this.emailDB = emailDB;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public void setFileContents(byte [] fileContents) {
this.fileContents = fileContents;
}
public String getFileName() {
return this.fileName;
}
public byte [] getFileContents() {
return this.fileContents;
}
public int getFileSize() {
return this.fileContents.length;
}
Client_Database() {
nameDB = Client_Signin.nameMsg;
idDB = Client_Signin.idMsg;
passwordDB = Client_Signin.passwordMsg;
addressDB = Client_Signin.addressMsg;
emailDB = Client_Signin.emailMsg;
}
}
nameMsg = name.getText().trim();
idMsg = id.getText().trim();
passwordMsg = password.getText().trim();
rePasswordMsg = rePassword.getText().trim();
addressMsg = address.getText().trim();
emailMsg = email.getText().trim();
Client_Database database = new Client_Database();
byte[] contents = new byte[(int) file.length()];
database.setFileContents(contents);
try {
FileOutputStream f = new FileOutputStream(new File("C:\\bookCafeDatabase.txt"));
ObjectOutputStream o = new ObjectOutputStream(f);
o.writeObject(result);
o.close();
FileInputStream fi = new FileInputStream(new File("C:\\bookCafeDatabase.txt"));
ObjectInputStream oi = new ObjectInputStream(fi);
oi.read(contents);
result = (HashMap<String, Client_Database>)oi.readObject();
oi.close();
fi.close();
} catch (FileNotFoundException e) {
System.out.println("File not found");
} catch (IOException e) {
System.out.println("Error initializing stream");
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public class DataMiner {
private static BigData app = new BigData();
private static DomainOfConstants doc = new DomainOfConstants();
private static Logger log = Logger.getLogger(DataMiner.class);
private static DBManager conn = new DBManager();
private static java.sql.Connection con = null;
private static AmazonS3 s3Client;
private static Iterator<String> itr;
private static List<String> entries = new ArrayList<String>();
private static S3Object s3Object;
private static ObjectMetadata meta;
public static InputStream dataStream;
public static byte[] buffer = new byte[1024];
public static File file = new File(app.getCurrentPacsId()+".txt");
private static void obtainConnection(){
conn.connection();
entries = conn.grabDataSet();
conn.closeDb();
downloadBucket();
}
/*
*
* The Java heap size limits for Windows are:
* maximum possible heap size on 32-bit Java: 1.8 GB
* recommended heap size limit on 32-bit Java: 1.5 GB (or 1.8 GB with /3GB option)
*
* */
/*-------------Download and un-zip backup file-------------*/
private static void downloadBucket(){
try {
app.setAwsCredentials(doc.getAccessKey(), doc.getSecretKey());
s3Client = AmazonS3ClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(app.getAwsCredentials())).withRegion(Regions.US_EAST_1).build();
System.out.println("Connected to S3");
itr = entries.iterator();
while(itr.hasNext()){
app.setBucketKey(itr.next());
String key = app.getBucketKey();
app.setCurrentPacsId(key);
s3Object = s3Client.getObject(new GetObjectRequest(doc.getDesiredBucket(), app.getBucketKey()));
try {
ZipInputStream zis = new ZipInputStream(s3Object.getObjectContent());
ZipEntry entry = zis.getNextEntry();
extractObjects(buffer, s3Client, zis, entry);
} catch (AmazonServiceException e) {
log.error(e);
} catch (SdkClientException e) {
log.error(e);
} catch (IOException e) {
log.error(e);
}
}
System.out.println("Processing complete");
} catch (IllegalArgumentException e) {
e.printStackTrace();
}
}
public static void extractObjects(byte[] buffer, AmazonS3 s3Client, ZipInputStream zis, ZipEntry entry) throws IOException {
PipedOutputStream outputStream = null;
PipedInputStream is = null;
try {
while (entry != null)
{
String fileName = entry.getName();
if (fileName == "lib") {
fileName = entry.getName();
}
boolean containsBackup = fileName.contains(doc.getDesiredFile());
if (containsBackup == true) {
System.out.println("A back up file was found");
long start = System.currentTimeMillis();
formatSchemaName();
System.out.println("Extracting :" + app.getCurrentPacsId());
log.info("Extracting " + app.getCurrentPacsId() + ",
compressed: " + entry.getCompressedSize() + " bytes,
extracted: " +
entry.getSize() + " bytes");
//ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
outputStream = new PipedOutputStream();
is = new PipedInputStream(outputStream);
int len;
while ((len = zis.read(buffer)) >= 0)
{
outputStream.write(buffer, 0, len);
}
//InputStream is = new ByteArrayInputStream(outputStream.toByteArray());
meta = new ObjectMetadata();
meta.setContentLength(file.length());
fileName = app.getCurrentPacsId();
runDataConversion(is,s3Client,fileName);
recordTime(start);
is.close();
outputStream.close();
System.out.println("Unzip complete");
}
else{
System.out.println("No back up found");
}
entry = zis.getNextEntry();
}
zis.closeEntry();
zis.close();
} catch (AmazonServiceException e) {
log.error(e);
} catch (SdkClientException e) {
log.error(e);
}
}
/*------------Formating the replacment file name---------*/
private static void formatSchemaName(){
String s3Key = app.getCurrentPacsId();
String id = s3Key.replace(".zip", ".txt");
id = id.substring(id.indexOf("_"));
id = id.replaceFirst("_", "");
app.setCurrentPacsId(id);
}
/*---------------Process the data file----------------------*/
private static void runDataConversion(PipedInputStream is, AmazonS3 s3Client, String fileName) {
DataProcessor convert = new DataProcessor(s3Client);
convert.downloadBucket(is,fileName);
}
/*-------Records execution time of program in min/sec------*/
private static void recordTime(long start) throws IOException {
long end = System.currentTimeMillis();
long minutes = TimeUnit.MILLISECONDS.toMinutes(end - start);
long seconds = TimeUnit.MILLISECONDS.toSeconds(end - start);
System.out.println("Execution speed "+ minutes + ":" + (seconds % 60) +" min/sec\n");
}
And here is the class that does some text file processing.The code is very slow overall when processing files up to 3.5gb. It takes 3 hours to do so while running. I have tried using piped streams over byte streams. Java heap size set to -xms2800m on a 64 bit JDK.
public class DataProcessor {
private static AmazonS3 s3Client;
private static ObjectMetadata meta;
private static DomainOfConstants doc = new DomainOfConstants();
private static BigData app = new BigData();
public static File file = new File(app.getCurrentPacsId()+".txt");
private static Logger log = Logger.getLogger(DataProcessor.class);
//Construct connection
public DataProcessor (AmazonS3 s3Client){
this.s3Client = s3Client;
}
//
public void downloadBucket(PipedInputStream is, String fileName) {
try {
File dataStream = dataConversion(is);
s3Client.putObject(doc.getDestinationBucket(),FilenameUtils.getFullPath(doc.getDestinationKey()) + "Modified_"+ fileName, dataStream);
} catch (AmazonServiceException e) {
e.printStackTrace();
log.error(e);
} catch (SdkClientException e) {
e.printStackTrace();
log.error(e);
}
}
//Setup reading and writing streams
public static File dataConversion(PipedInputStream stream) {
BufferedReader reader = null;
BufferedOutputStream streamOut = null;
String line;
try {
reader = new BufferedReader(new InputStreamReader(stream,doc.getFileFormat()));
streamOut = new BufferedOutputStream(new FileOutputStream(file));
meta = new ObjectMetadata();
while(( line = reader.readLine() ) != null)
{
processLine(reader, streamOut, line);
}
}
catch (IOException e) {
e.printStackTrace();
} finally {
try {
streamOut.close();
reader.close();
} catch (IOException e) {
e.printStackTrace();
log.error(e);
}
}
return file;
}
/*---------------------------------------Data processing------------------------------------------------*/
/*-----------Process and print lines---------*/
private static void processLine(BufferedReader reader, BufferedOutputStream streamOut, String line) {
try {
String newLine = System.getProperty("line.separator");
while (reader.ready()) {
if (line.contains(doc.getInsert())) {
handleData(streamOut, line);
} else if (line.contains(doc.getUse())) {
handleSchemaName(streamOut, line);
} else {
streamOut.write(line.toLowerCase().getBytes(Charset.forName(doc.getFileFormat()).toString()));
streamOut.write(newLine.getBytes());
}
line = reader.readLine();
}
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
log.error(e);
} catch (IOException e) {
e.printStackTrace();
log.error(e);
}
}
/*-----------Replace-Schema-Name-----------*/
private static void handleSchemaName(BufferedOutputStream streamOut, String line) throws IOException {
line = line.replace(line, "USE " + "`" + doc.getSchemaName() + app.getCurrentPacsId() + "`;");
streamOut.write(line.getBytes(Charset.forName(doc.getFileFormat())));
}
/*--------Avoid-Formating-Data-Portion-of-file--------*/
private static void handleData(BufferedOutputStream streamOut, String line) throws IOException {
StringTokenizer tk = new StringTokenizer(line);
while (tk.hasMoreTokens()) {
String data = tk.nextToken();
if (data.equals(doc.getValue())) {
streamOut.write(data.toLowerCase().getBytes(Charset.forName(doc.getFileFormat()).toString()));
data = tk.nextToken();
while (tk.hasMoreTokens()) {
streamOut.write(data.getBytes(Charset.forName(doc.getFileFormat())));
data = tk.nextToken();
}
}
streamOut.write(line.toLowerCase().getBytes(Charset.forName(doc.getFileFormat().toString())));
streamOut.write(" ".getBytes(Charset.forName(doc.getFileFormat())));
}
}
Rule 1 is always to use a bigger buffer. 1024 is pitifully small. Try 32-64K.
You need to start the pipe reading thread before doing any writes to the pipe. In fact I'm surprised you don't get 'read end dead' errors. Does this code really work at all?
In fact get rid of the piped streams. Use a single thread and do all the processing as you go.
Get rid of the ready() test. It is an extra system call for nothing. Just read until end of stream.
Use a BufferedWriter instead of a BufferedOutputStream and stop converting all those strings to bytes (and use BufferedWriter.newLine() instead of the system property).
I spent really a lot of time to understand how to print a PCX image using CPCL on a Zebra Printer (via network) without downloading the image to the printer.
The sample on the documentation, in my opinion, is quite obscure.
I attach a sample class to show how to simply print an image.
It requires a "zebra.pcx" image on your classpath.
Hope it helps.
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.Socket;
public class PrintZebraPCXImage {
public static void main(String[] args) throws Exception {
PrintZebraPCXImage instance = new PrintZebraPCXImage();
instance.print("192.168.1.133", 6101);
}
public void print(String address, int port) throws Exception {
Socket socket = null;
DataOutputStream stream = null;
socket = new Socket(address, port);
try {
stream = new DataOutputStream(socket.getOutputStream());
ByteArrayOutputStream bos = readFileToString(this.getClass().getClassLoader().getResourceAsStream("zebra.pcx"));
stream.writeBytes("! 0 200 200 300 1\r\n");
stream.writeBytes("PCX 20 0\r\n");
stream.write(bos.toByteArray());
stream.writeBytes("PRINT\r\n");
} finally {
if (stream != null) {
stream.close();
}
if (socket != null) {
socket.close();
}
}
}
public ByteArrayOutputStream readFileToString(InputStream is) {
InputStreamReader isr = null;
ByteArrayOutputStream bos = null;
try {
isr = new InputStreamReader(is);
bos = new ByteArrayOutputStream();
byte[] buffer = new byte[2048];
int n = 0;
while (-1 != (n = is.read(buffer))) {
bos.write(buffer, 0, n);
}
return bos;
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
if (isr != null) {
try {
isr.close();
} catch (IOException e) {
}
}
if (is != null) {
try {
is.close();
} catch (IOException e) {
}
}
}
}
}
I am trying to write a class that can compress data. The below code fails (no exception is thrown, but the target .gz file is empty.)
Besides: I don't want to generate the .gz file directly like it is done in all examples. I only want to get the compressed
data, so that I can e.g. encrypt it before writting the data to a file.
If I write directly to a file everything works fine:
import java.io.*;
import java.util.zip.*;
import java.nio.charset.*;
public class Zipper
{
public static void main(String[] args)
{
byte[] dataToCompress = "This is the test data."
.getBytes(StandardCharsets.ISO_8859_1);
GZIPOutputStream zipStream = null;
FileOutputStream fileStream = null;
try
{
fileStream = new FileOutputStream("C:/Users/UserName/Desktop/zip_file.gz");
zipStream = new GZIPOutputStream(fileStream);
zipStream.write(dataToCompress);
fileStream.write(compressedData);
}
catch(Exception e)
{
e.printStackTrace();
}
finally
{
try{ zipStream.close(); }
catch(Exception e){ }
try{ fileStream.close(); }
catch(Exception e){ }
}
}
}
But, if I want to 'bypass' it to the byte array stream it does not produce a single byte - compressedData is always empty.
import java.io.*;
import java.util.zip.*;
import java.nio.charset.*;
public class Zipper
{
public static void main(String[] args)
{
byte[] dataToCompress = "This is the test data."
.getBytes(StandardCharsets.ISO_8859_1);
byte[] compressedData = null;
GZIPOutputStream zipStream = null;
ByteArrayOutputStream byteStream = null;
FileOutputStream fileStream = null;
try
{
byteStream = new ByteArrayOutputStream(dataToCompress.length);
zipStream = new GZIPOutputStream(byteStream);
zipStream.write(dataToCompress);
compressedData = byteStream.toByteArray();
fileStream = new FileOutputStream("C:/Users/UserName/Desktop/zip_file.gz");
fileStream.write(compressedData);
}
catch(Exception e)
{
e.printStackTrace();
}
finally
{
try{ zipStream.close(); }
catch(Exception e){ }
try{ byteStream.close(); }
catch(Exception e){ }
try{ fileStream.close(); }
catch(Exception e){ }
}
}
}
The problem is that you are not closing the GZIPOutputStream. Until you close it the output will be incomplete.
You just need to close it before reading the byte array. You need to reorder the finally blocks to achieve this.
import java.io.*;
import java.util.zip.*;
import java.nio.charset.*;
public class Zipper
{
public static void main(String[] args)
{
byte[] dataToCompress = "This is the test data."
.getBytes(StandardCharsets.ISO_8859_1);
try
{
ByteArrayOutputStream byteStream =
new ByteArrayOutputStream(dataToCompress.length);
try
{
GZIPOutputStream zipStream =
new GZIPOutputStream(byteStream);
try
{
zipStream.write(dataToCompress);
}
finally
{
zipStream.close();
}
}
finally
{
byteStream.close();
}
byte[] compressedData = byteStream.toByteArray();
FileOutputStream fileStream =
new FileOutputStream("C:/Users/UserName/Desktop/zip_file.gz");
try
{
fileStream.write(compressedData);
}
finally
{
try{ fileStream.close(); }
catch(Exception e){ /* We should probably delete the file now? */ }
}
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
I do not recommend inititalizing the stream variables to null, because it means your finally block can also throw a NullPointerException.
Also note that you can declare main to throw IOException (then you would not need the outermost try statement.)
There is little point in swallowing exceptions from zipStream.close();, because if it throws an exception you will not have a valid .gz file (so you should not proceed to write it.)
Also I would not swallow exceptions from byteStream.close(); but for a different reason - they should never be thrown (i.e. there is a bug in your JRE and you would want to know about that.)
I've improved JITHINRAJ's code - used try-with-resources:
private static byte[] gzipCompress(byte[] uncompressedData) {
byte[] result = new byte[]{};
try (ByteArrayOutputStream bos = new ByteArrayOutputStream(uncompressedData.length);
GZIPOutputStream gzipOS = new GZIPOutputStream(bos)) {
gzipOS.write(uncompressedData);
// You need to close it before using bos
gzipOS.close();
result = bos.toByteArray();
} catch (IOException e) {
e.printStackTrace();
}
return result;
}
private static byte[] gzipUncompress(byte[] compressedData) {
byte[] result = new byte[]{};
try (ByteArrayInputStream bis = new ByteArrayInputStream(compressedData);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
GZIPInputStream gzipIS = new GZIPInputStream(bis)) {
byte[] buffer = new byte[1024];
int len;
while ((len = gzipIS.read(buffer)) != -1) {
bos.write(buffer, 0, len);
}
result = bos.toByteArray();
} catch (IOException e) {
e.printStackTrace();
}
return result;
}
If you are still looking an answer you can use the below code to get the compressed byte[] using deflater and decompress it using inflater.
public static void main(String[] args) {
//Some string for testing
String sr = new String("fsdfesfsfdddddddsfdsfssdfdsfdsfdsfdsfdsdfggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggghghghghggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggfsdfesfsfdddddddsfdsfssdfdsfdsfdsfdsfdsdfggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggggghghghghggggggggggggggggggggggggggggggggggggggggg");
byte[] data = sr.getBytes();
System.out.println("src size "+data.length);
try {
compress(data);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static byte[] compress(byte[] data) throws IOException {
Deflater deflater = new Deflater();
deflater.setInput(data);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream(data.length);
deflater.finish();
byte[] buffer = new byte[1024];
while (!deflater.finished()) {
int count = deflater.deflate(buffer);
outputStream.write(buffer, 0, count);
}
outputStream.close();
byte[] output = outputStream.toByteArray();
System.out.println("Original: " + data.length );
System.out.println("Compressed: " + output.length );
return output;
}
To compress
private static byte[] compress(byte[] uncompressedData) {
ByteArrayOutputStream bos = null;
GZIPOutputStream gzipOS = null;
try {
bos = new ByteArrayOutputStream(uncompressedData.length);
gzipOS = new GZIPOutputStream(bos);
gzipOS.write(uncompressedData);
gzipOS.close();
return bos.toByteArray();
} catch (IOException e) {
e.printStackTrace();
}
finally {
try {
assert gzipOS != null;
gzipOS.close();
bos.close();
}
catch (Exception ignored) {
}
}
return new byte[]{};
}
To uncompress
private byte[] uncompress(byte[] compressedData) {
ByteArrayInputStream bis = null;
ByteArrayOutputStream bos = null;
GZIPInputStream gzipIS = null;
try {
bis = new ByteArrayInputStream(compressedData);
bos = new ByteArrayOutputStream();
gzipIS = new GZIPInputStream(bis);
byte[] buffer = new byte[1024];
int len;
while((len = gzipIS.read(buffer)) != -1){
bos.write(buffer, 0, len);
}
return bos.toByteArray();
} catch (IOException e) {
e.printStackTrace();
}
finally {
try {
assert gzipIS != null;
gzipIS.close();
bos.close();
bis.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
return new byte[]{};
}
You can use the below function, it is tested and working fine.
In general, your code has serious problem of ignoring the exceptions! returning null or simply not printing anything in the catch block will make it very difficult to debug
You do not have to write the zip output to a file if you want to process it further (e.g. encrypt it), you can easily modify the code to write the output to in-memory stream
public static String zip(File inFile, File zipFile) throws IOException {
FileInputStream fis = new FileInputStream(inFile);
FileOutputStream fos = new FileOutputStream(zipFile);
ZipOutputStream zout = new ZipOutputStream(fos);
try {
zout.putNextEntry(new ZipEntry(inFile.getName()));
byte[] buffer = new byte[BUFFER_SIZE];
int len;
while ((len = fis.read(buffer)) > 0) {
zout.write(buffer, 0, len);
}
zout.closeEntry();
} catch (Exception ex) {
ex.printStackTrace();
return null;
} finally {
try{zout.close();}catch(Exception ex){ex.printStackTrace();}
try{fis.close();}catch(Exception ex){ex.printStackTrace();}
}
return zipFile.getAbsolutePath();
}
Most of the examples have wrong exception handling.
public static byte[] gzipBytes(byte[] payload) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
try (GZIPOutputStream gzip = new GZIPOutputStream(baos)) {
gzip.write(payload);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
// note: toByteArray should be called after try-with-resources, not inside
return baos.toByteArray();
}
public static byte[] gunzipBytes(byte[] gzPayload) {
ByteArrayInputStream bais = new ByteArrayInputStream(gzPayload);
try (GZIPInputStream gzip = new GZIPInputStream(bais)) {
// java 9+ required for this method
return gzip.readAllBytes();
} catch (IOException e) {
throw new UncheckedIOException("Error while unpacking gzip content", e);
}
}
Try with this code..
try {
String inputFileName = "test.txt"; //may use your file_Path
String zipFileName = "compressed.zip";
//Create input and output streams
FileInputStream inStream = new FileInputStream(inputFileName);
ZipOutputStream outStream = new ZipOutputStream(new FileOutputStream(zipFileName));
// Add a zip entry to the output stream
outStream.putNextEntry(new ZipEntry(inputFileName));
byte[] buffer = new byte[1024];
int bytesRead;
//Each chunk of data read from the input stream
//is written to the output stream
while ((bytesRead = inStream.read(buffer)) > 0) {
outStream.write(buffer, 0, bytesRead);
}
//Close zip entry and file streams
outStream.closeEntry();
outStream.close();
inStream.close();
} catch (IOException ex) {
ex.printStackTrace();
}
Also may be helpful this one..
http://www.java-samples.com/java/zip_files_in_a_folder_using_java.htm