FileInputStream leaving weird garbage text after skip() call - java

The need:
I want to write data into compressed and normal format as well. When I'll have to write data into compressed format "useCompression" will be sent as "true" and "useCompression" will be false when data needs to be stored in normal(as it is given to the Writer class) format.
The problem here is, how will I identify whether the data is compressed or not later when Reader class is trying to read the data?
So, to solve the problem, I am writing "1" into file if is "useCompression" is true and "0" if "useCompression" is false.
writing is fine, but when we try to skip the first element using "fIn.skip(1)" cause it is the identifier and not actual data, it is leaving behind some garbage value.
For, example, I am trying to write "2019-07-31" into a file and "useCompression" is false, so my file will hold "02019-07-31" and post "fIn.skip(1)" call it should hold "2019-07-31" but it is holding "^#2019-07-31".
Please help me figure out what I am doing wrong here
I've tried to update Reader class's constructor as:
public Reader(String key)
{
mKey = key;
try {
FileInputStream fIn = new FileInputStream(streamFileForKey(key));
byte[] firstByte = new byte[1];
int read = fIn.read(firstByte);
boolean shouldUseDecompression = (1 == firstByte[0]);
if (shouldUseDecompression) {
mFin = new GZIPInputStream(fIn);
}
else {
mFin = fIn;
}
}
catch (Exception e) {
System.out.println("Failed to open (r) key " + key + " exception : " + e);
}
}
But it does not solve the problem.
The actual code is:
import java.io.InputStream;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.io.DataOutputStream;
import java.util.zip.GZIPOutputStream;
import java.util.zip.GZIPInputStream;
public class forStackOverflow {
public static void main(String []args) {
// Write to normal file
mReader1 = new Reader(mInputFileName);
mWriter1 = new Writer(mOutputFilename1, false);
int availableBytes = mReader1.availableBytes();
int readBytes = 1000;
while (availableBytes > 0)
{
if (availableBytes >= 1000) {
availableBytes = availableBytes - readBytes;
}
else {
readBytes = availableBytes;
availableBytes = availableBytes - readBytes;
}
mWriter1.write(mReader1.read(readBytes), 0, readBytes);
}
mReader1.close();
mWriter1.close();
}
private static File streamFileForKey(String key)
{
return new File(key);
}
static String mInputFileName = "~/Downloads/inputStream.txt";
static String mOutputFilename1 = "~/Downloads/outputStream.txt";
static Reader mReader1;
static Writer mWriter1;
private static class Writer
{
public Writer(String key, boolean useCompression) {
mKey = key;
try {
FileOutputStream fileOutput = new FileOutputStream(streamFileForKey(key));
if (useCompression) {
fileOutput.write(1);
gOutputStream = new GZIPOutputStream(fileOutput);
}
else {
fileOutput.write(0);
gOutputStream = new DataOutputStream(fileOutput);
}
}
catch (Exception e) {
System.out.println("Got error while opening stream for " + mKey + ". Ex: " + e);
}
}
public boolean write(byte[] bytes, int pos, int len)
{
boolean retVal = true;
if (gOutputStream == null) {
return false;
}
try {
gOutputStream.write(bytes, pos, len);
retVal = true;
}
catch (Exception e) {
System.out.println("Failed to write " + len + " bytes to key " +
mKey + e);
retVal = false;
}
return retVal;
}
public void close()
{
if (gOutputStream != null) {
try {
gOutputStream.close();
}
catch (Exception e) {
System.out.println("Failed to close key " + mKey + e);
}
gOutputStream = null;
}
}
private String mKey;
private OutputStream gOutputStream;
}
private static class Reader
{
public Reader(String key)
{
mKey = key;
try {
FileInputStream fIn = new FileInputStream(streamFileForKey(key));
if (shouldUseDecompression()) {
long skipped = fIn.skip(1);
mFin = new GZIPInputStream(fIn);
}
else {
long skipped = fIn.skip(1);
mFin = fIn;
}
}
catch (Exception e) {
System.out.println("Failed to open (r) key " + key + " exception : " + e);
}
}
public byte[] read(int len)
{
if (mFin == null) {
return null;
}
byte[] b = null;
try {
b = new byte[len];
int read;
read = mFin.read(b, 0, len);
if (read <= 0) {
return null;
}
}
catch (Exception e) {
System.out.println("Failed to read " + len + " bytes from key " +
mKey + " exception : " + e);
}
return b;
}
public void close()
{
if (mFin != null) {
try {
mFin.close();
}
catch (Exception e) {
System.out.println("Failed to close key " + mKey + " exception : " + e);
}
mFin = null;
}
}
private boolean shouldUseDecompression()
{
boolean retVal = false;
try {
FileInputStream fIn = new FileInputStream(streamFileForKey(mKey));
byte[] firstByte = new byte[1];
int read = fIn.read(firstByte);
// If first byte is `1` the we need to use decompression on it.
retVal = (1 == firstByte[0]);
fIn.close();
}
catch(Exception e) {
System.out.println("Exception in shouldUseDecompression() : " + e);
retVal = false;
}
return retVal;
}
public int availableBytes()
{
int available = 0;
try {
if (mFin != null) {
available = mFin.available();
}
}
catch (IOException e) {
System.out.println("Failed to read available bytes for " + mKey + ". Exception : " + e);
}
return available;
}
private String mKey;
private InputStream mFin;
}
}
The expected result should be, post "fIn.skip(1)" call file should hold "2019-07-31" and not "^#2019-07-31".

After a lot of struggle, I found out that it was the expected result.
^# was nothing but the binary 0 I was writing from Writer class's constructor incase useCompression was false.
From line fileOutput.write(0);.

Related

Android - Zip Path Traversal in play store

I am uploading my App on play store but get me bellow error:
Zip Path Traversal Your app contains an unsafe unzipping pattern that
may lead to a Path Traversal vulnerability. Please see this Google
Help Center article to learn how to fix the issue.
org.apache.cordova.Zip.unzipSync
I edited my source code like this LINK, but get me error.
Here is my source code changed:
public class Zip extends CordovaPlugin {
private static final String LOG_TAG = "Zip";
// Can't use DataInputStream because it has the wrong endian-ness.
private static int readInt(InputStream is) throws IOException {
int a = is.read();
int b = is.read();
int c = is.read();
int d = is.read();
return a | b << 8 | c << 16 | d << 24;
}
#Override
public boolean execute(String action, CordovaArgs args, final CallbackContext callbackContext) throws JSONException {
if ("unzip".equals(action)) {
unzip(args, callbackContext);
return true;
}
return false;
}
private void unzip(final CordovaArgs args, final CallbackContext callbackContext) {
this.cordova.getThreadPool().execute(new Runnable() {
public void run() {
unzipSync(args, callbackContext);
}
});
}
private void unzipSync(CordovaArgs args, CallbackContext callbackContext) {
InputStream inputStream = null;
try {
String zipFileName = args.getString(0);
String outputDirectory = args.getString(1);
// Since Cordova 3.3.0 and release of File plugins, files are accessed via cdvfile://
// Accept a path or a URI for the source zip.
Uri zipUri = getUriForArg(zipFileName);
Uri outputUri = getUriForArg(outputDirectory);
CordovaResourceApi resourceApi = webView.getResourceApi();
File tempFile = resourceApi.mapUriToFile(zipUri);
if (tempFile == null || !tempFile.exists()) {
String errorMessage = "Zip file does not exist";
callbackContext.error(errorMessage);
Log.e(LOG_TAG, errorMessage);
return;
}
File outputDir = resourceApi.mapUriToFile(outputUri);
outputDirectory = outputDir.getAbsolutePath();
outputDirectory += outputDirectory.endsWith(File.separator) ? "" : File.separator;
if (outputDir == null || (!outputDir.exists() && !outputDir.mkdirs())) {
String errorMessage = "Could not create output directory";
callbackContext.error(errorMessage);
Log.e(LOG_TAG, errorMessage);
return;
}
OpenForReadResult zipFile = resourceApi.openForRead(zipUri);
ProgressEvent progress = new ProgressEvent();
progress.setTotal(zipFile.length);
inputStream = new BufferedInputStream(zipFile.inputStream);
inputStream.mark(10);
int magic = readInt(inputStream);
if (magic != 875721283) { // CRX identifier
inputStream.reset();
} else {
// CRX files contain a header. This header consists of:
// * 4 bytes of magic number
// * 4 bytes of CRX format version,
// * 4 bytes of public key length
// * 4 bytes of signature length
// * the public key
// * the signature
// and then the ordinary zip data follows. We skip over the header before creating the ZipInputStream.
readInt(inputStream); // version == 2.
int pubkeyLength = readInt(inputStream);
int signatureLength = readInt(inputStream);
inputStream.skip(pubkeyLength + signatureLength);
progress.setLoaded(16 + pubkeyLength + signatureLength);
}
// The inputstream is now pointing at the start of the actual zip file content.
ZipInputStream zis = new ZipInputStream(inputStream);
inputStream = zis;
ZipEntry ze;
byte[] buffer = new byte[32 * 1024];
boolean anyEntries = false;
while ((ze = zis.getNextEntry()) != null) {
try {
anyEntries = true;
String compressedName = ze.getName();
if (ze.isDirectory()) {
try {
File dir = new File(outputDirectory + compressedName);
File f = new File(dir, ze.getName());
String canonicalPath = f.getCanonicalPath();
if (!canonicalPath.startsWith(dir.toString())){
dir.mkdirs();
}else {
if (inputStream != null) {
try {
inputStream.close();
} catch (IOException e) {
}
}
}
} catch (Exception e) {
String errorMessage = "An error occurred while unzipping.";
callbackContext.error(errorMessage);
Log.e(LOG_TAG, errorMessage, e);
}
} else {
File file = new File(outputDirectory + compressedName);
File f = new File(file, ze.getName());
String canonicalPath = f.getCanonicalPath();
if (!canonicalPath.startsWith(file.toString())) {
file.getParentFile().mkdirs();
if (file.exists() || file.createNewFile()) {
try {
Log.w("Zip", "extracting: " + file.getPath());
FileOutputStream fout = new FileOutputStream(file);
int count;
while ((count = zis.read(buffer)) != -1) {
fout.write(buffer, 0, count);
}
fout.close();
} catch (Exception e) {
String errorMessage = "An error occurred while unzipping.";
callbackContext.error(errorMessage);
Log.e(LOG_TAG, errorMessage, e);
}
}
}else {
if (inputStream != null) {
try {
inputStream.close();
} catch (IOException e) {
}
}
}
}
progress.addLoaded(ze.getCompressedSize());
updateProgress(callbackContext, progress);
zis.closeEntry();
} catch (Exception e) {
String errorMessage = "An error occurred while unzipping.";
callbackContext.error(errorMessage);
Log.e(LOG_TAG, errorMessage, e);
}
}
// final progress = 100%
progress.setLoaded(progress.getTotal());
updateProgress(callbackContext, progress);
if (anyEntries)
callbackContext.success();
else
callbackContext.error("Bad zip file");
} catch (Exception e) {
String errorMessage = "An error occurred while unzipping.";
callbackContext.error(errorMessage);
Log.e(LOG_TAG, errorMessage, e);
} finally {
if (inputStream != null) {
try {
inputStream.close();
} catch (IOException e) {
}
}
}
}
private void updateProgress(CallbackContext callbackContext, ProgressEvent progress) throws JSONException {
PluginResult pluginResult = new PluginResult(PluginResult.Status.OK, progress.toJSONObject());
pluginResult.setKeepCallback(true);
callbackContext.sendPluginResult(pluginResult);
}
private Uri getUriForArg(String arg) {
CordovaResourceApi resourceApi = webView.getResourceApi();
Uri tmpTarget = Uri.parse(arg);
return resourceApi.remapUri(
tmpTarget.getScheme() != null ? tmpTarget : Uri.fromFile(new File(arg)));
}
private static class ProgressEvent {
private long loaded;
private long total;
public long getLoaded() {
return loaded;
}
public void setLoaded(long loaded) {
this.loaded = loaded;
}
public void addLoaded(long add) {
this.loaded += add;
}
public long getTotal() {
return total;
}
public void setTotal(long total) {
this.total = total;
}
public JSONObject toJSONObject() throws JSONException {
return new JSONObject(
"{loaded:" + loaded +
",total:" + total + "}");
}
}
}

Flatbuffer writing in binary file in android give only single response

I am new at flatbuffer. I had created a schema file from outside the project and adding the user(Monster as in Flatbuffer document) inside the binary via java code in android. everything works fine. but the time of reading binary file data it only gives me the user length 1. I had added 3 people but it's give me the length of monsters is 1 at the time of reading. can anyone help to figure this out. Here is the code->
this is the full code ->
enter code here
builder = new FlatBufferBuilder(1024);
public void addNewData(String emailOffset, String nameOffset,String
contactNoOffset, String DOJOffset, String departmentOffset,
String empIdOffset, float[] embeddingOffset)
{
int storeembedd = SingleJson.createEmbeddingVector(builder,
embeddingOffset);
int hereEmailOffset = builder.createString(emailOffset);
int hereNameOffset = builder.createString(nameOffset);
int hereContactOffset = builder.createString(contactNoOffset);
int hereDOJOffset = builder.createString(DOJOffset);
int hereDepartmentOffset=builder.createString(departmentOffset);
int hereImpIdOffset = builder.createString(empIdOffset);
int test = SingleJson.createSingleJson(builder, hereEmailOffset,
hereNameOffset, hereContactOffset
, hereDOJOffset, hereDepartmentOffset, hereImpIdOffset,
storeembedd);
int [] offsetOfMonster = new int[1];
offsetOfMonster[0] = test;
int temp = Monsters.createMonstersVector(builder,
offsetOfMonster);
Monsters.startMonsters(builder);
Monsters.addMonsters(builder, temp);
int orc = Monsters.endMonsters(builder);
builder.finish(orc);
byte[] buf1 = builder.sizedByteArray();
openAndAppenedInBinary(buf1);
}
private void openAndAppenedInBinary(byte[] buf1) {
FileOutputStream output = null;
try {
InputStream inputStream = new FileInputStream(newFile());
byte[] buffer = new byte[inputStream.available()];
if (inputStream.available() == 0) {
output = new FileOutputStream(newFile(), true);
output.write(buf1);
} else {
while (inputStream.read(buffer) != -1) {
output = new FileOutputStream(newFile(), true);
output.write(buffer);
output.write(buf1);
}
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
output.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
private String newFile() {
File file = new File(Environment.getExternalStorageDirectory()
+ "/Android/data/"
+ context.getPackageName()
+ "/binFile");
if (!file.exists()) {
if (!file.mkdir()) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
try {
Path dir = Paths.get(file.getAbsolutePath());
Files.createDirectory(dir);
} catch (IOException e) {
Path parentDir = Paths.get(file.getParent());
if (!Files.exists(parentDir)) {
try {
Files.createDirectories(parentDir);
} catch (IOException e1) {
e1.printStackTrace();
}
}
}
}
}
}
File binFile = new File(file, "renamed.bin");
try {
FileWriter writer = new FileWriter(binFile);
writer.close();
} catch (IOException e) {
e.printStackTrace();
}
return binFile.getAbsolutePath();
}
here is my reading code of flatbiffer->
new FlatParsingForPerticularId().execute(readRawResource(R.raw.renamed));//renamed is my bin file
private class FlatParsingForPerticularId extends AsyncTask {
#Override
protected String doInBackground(Object... params) {
byte[] buffer = (byte[]) params[0];
long startTime = System.currentTimeMillis();
ByteBuffer bb = ByteBuffer.wrap(buffer);
Monsters monsterList = Monsters.getRootAsMonsters(bb);
int length = monsterList.monstersLength();
SingleJson monster = null;
for (int i = 0; i < length; i++) {//here I m getting length 1 intead of 3
monster = monsterList.monsters(i);
if (i == outputval[0]) {
outputval[0] = (int) monster.EmpNo();
break;
}
}
long endTime = System.currentTimeMillis() - startTime;
String textToShow = "Elements: " + monsterList.monstersLength() + ": load time: " + endTime + "ms";
String[] monsterArr = monster.Name().split(" ");
return monsterArr[0];
}

flac audio file to text file java program

I have code that can convert .wav to text file. But I want to convert .flac to text also. it seems like AudioInputStream doesn't support .flac type . What modification should I do in mycode so that it can convert .flac files to text? Here is the full code. I'm adding this line because stack overflow is asking me to do so.
import java.io.File;
import javax.sound.sampled.*;
public class AudioFileConvert01 {
public static void main(String[] args) {
if (args.length != 2) {
System.out.println("Usage: java AudioFileConvert01 " + "inputFile outputFile");
System.exit(0);
}
System.out.println("Input file: " + args[0]);
System.out.println("Output file: " + args[1]);
// Output file type depends on outputfile name extension.
String outputTypeStr = args[1].substring(args[1].lastIndexOf(".") + 1);
System.out.println("Output type: " + outputTypeStr);
AudioFileFormat.Type outputType = getTargetType(outputTypeStr);
if (outputType != null) {
System.out.println("Output type is supported");
} else {
System.out.println("Output type not supported.");
getTargetTypesSupported();
System.exit(0);
}
// Note that input file type does not depend on file name or extension.
File inputFileObj = new File(args[0]);
AudioInputStream audioInputStream = null;
try {
audioInputStream = AudioSystem.getAudioInputStream(inputFileObj);
} catch (Exception e) {
e.printStackTrace();
System.exit(0);
} // end catch
System.out.println("Input file format:");
showFileType(inputFileObj);
int bytesWritten = 0;
try {
bytesWritten = AudioSystem.write(audioInputStream, outputType, new File(args[1]));
} catch (Exception e) {
e.printStackTrace();
System.exit(0);
}
System.out.println("Bytes written: " + bytesWritten);
System.out.println("Output file format:");
showFileType(new File(args[1]));
}
private static void getTargetTypesSupported() {
AudioFileFormat.Type[] typesSupported = AudioSystem.getAudioFileTypes();
System.out.print("Supported audio file types:");
for (int i = 0; i < typesSupported.length; i++) {
System.out.print(" " + typesSupported[i].getExtension());
}
// for
// loop
System.out.println();
}
private static AudioFileFormat.Type getTargetType(String extension) {
AudioFileFormat.Type[] typesSupported = AudioSystem.getAudioFileTypes();
for (int i = 0; i < typesSupported.length; i++) {
if (typesSupported[i].getExtension().equals(extension)) {
return typesSupported[i];
}
}
return null;// no match
}
private static void showFileType(File file) {
try {
System.out.println(AudioSystem.getAudioFileFormat(file));
} catch (Exception e) {
e.printStackTrace();
System.exit(0);
}
}
}

Pass input to bash script using java input stream and collect bash script output to java output stream using thread

I am trying to pass input to bash script using java input stream and collect bash script output to java output stream using two different thread.
my bash script is:
#!/bin/sh
echo "added at start"
while read LINE; do
echo $LINE
done
and my Java code is:
public class NewRedirector implements Runnable {
private static final int BULK_BUFFER_SIZE = 500000;
private static final int READ_BUFFER_SIZE = 250000;
private static final int OFFSET = 0;
private final OutputStream targetStream;
private final InputStream sourceStream;
private final boolean useChannel;
public NewRedirector(InputStream sourceStream , OutputStream targetStream, boolean useChannel) {
this.sourceStream = sourceStream;
this.targetStream = targetStream;
this.useChannel = useChannel;
}
#Override
public void run() {
byte[] readbyte = new byte[READ_BUFFER_SIZE];
int dataSize = 0;
try {
if(targetStream instanceof FileOutputStream && useChannel) {
FileChannel targetFC = ((FileOutputStream) targetStream).getChannel();
try {
if(sourceStream instanceof FileInputStream && useChannel) {
FileChannel sourceFC = ((FileInputStream) sourceStream).getChannel();
ByteBuffer bb = ByteBuffer.allocateDirect(BULK_BUFFER_SIZE);
bb.clear();
dataSize = 0;
while ((dataSize = sourceFC.read(bb)) > 0) {
bb.flip();
while (bb.hasRemaining()) {
targetFC.write(bb);
}
bb.clear();
}
} else {
ByteBuffer bb = ByteBuffer.allocateDirect(BULK_BUFFER_SIZE);
dataSize = 0;
while ((dataSize = sourceStream.read(readbyte)) > 0) {
if(BULK_BUFFER_SIZE > bb.position() + dataSize) {
bb.put(readbyte, OFFSET, dataSize);
continue;
} else {
bb.flip();
targetFC.write(bb);
bb.clear();
}
bb.put(readbyte, OFFSET, dataSize);
}
if(bb.position() > 0) {
bb.flip();
targetFC.write(bb);
bb.clear();
}
}
} catch(IOException e) {
System.out.println("Got Exception: " + e);
} finally {
if(targetFC != null && targetFC.isOpen()) {
targetFC.close();
targetFC = null;
}
}
} else {
BufferedOutputStream btarget = new BufferedOutputStream(targetStream);
try {
if (sourceStream instanceof FileInputStream && useChannel) {
FileChannel sourceFC = ((FileInputStream) sourceStream).getChannel();
ByteBuffer bb = ByteBuffer.allocateDirect(BULK_BUFFER_SIZE);
bb.clear();
dataSize = 0;
while ((dataSize = sourceFC.read(bb)) > 0) {
bb.position(OFFSET);
bb.limit(dataSize);
while (bb.hasRemaining()) {
dataSize = Math.min(bb.remaining(), READ_BUFFER_SIZE);
bb.get(readbyte, OFFSET, dataSize);
btarget.write(readbyte, OFFSET, dataSize);
}
btarget.flush();
bb.clear();
}
} else {
dataSize = 0;
while ((dataSize = sourceStream.read(readbyte)) > 0) {
btarget.write(readbyte, OFFSET, dataSize);
}
btarget.flush();
}
} catch(IOException e) {
System.out.println("Got Exception: " + e);
} finally {
if(btarget != null) {
btarget.close();
btarget = null;
}
}
}
} catch (IOException e) {
System.out.println("Got Exception: " + e);
}
}
}
and
public class NewProcessExecutor {
public static void main(String ... args) throws IOException, InterruptedException {
NewProcessExecutor pe = new NewProcessExecutor();
pe.startProcessinputfromstreamoutputtostreamintwothread();
}
private void startProcessinputfromstreamoutputtostreamintwothread()
throws IOException, InterruptedException {
String lscriptLocation = "/scratch/demo/RunScript/append.sh";
File inFile = new File("/scratch/demo/Source/inFile");
File outFile = new File("/scratch/demo/Source/outFile");
ProcessBuilder processBuilder = new ProcessBuilder(lscriptLocation);
/*processBuilder.redirectInput(Redirect.PIPE);
processBuilder.redirectOutput(Redirect.PIPE);*/
Process process = processBuilder.start();
if(Redirect.PIPE.file() == null && Redirect.PIPE.type() == Redirect.Type.PIPE) {
System.out.println("IO connected over PIPE");
}
//startStreamRedirector(new FileInputStream(inFile), process.getOutputStream());
startStreamRedirector(process.getInputStream(), new FileOutputStream(outFile));
int exitvalue = process.waitFor();
System.out.println("Exit value: " + exitvalue);
if (exitvalue != 0) {
System.out.println("Script execution failed with error: "
+ readErrorStream(process.getErrorStream()));
return;
} else {
System.out.println("Script executed successfully, please see output file: " + outFile.getAbsolutePath());
}
}
private String readErrorStream(InputStream errorStream) {
StringBuilder sb = new StringBuilder();
try (BufferedReader buffR = new BufferedReader(new InputStreamReader(
errorStream))) {
String line = null;
while ((line = buffR.readLine()) != null) {
sb.append(line);
}
} catch (IOException e) {
System.out.println("Got Exception: " + e);
}
return sb.toString();
}
private void startStreamRedirector(InputStream inputStream, OutputStream outputStream) {
new Thread(new NewRedirector(inputStream, outputStream, true)).start();
}
}
Now the problem is this code sometime runs perfectly but some times it creates zero size file.
Can someone point out what could be the issue?
As per my information default redirect is PIPE so hope I don't need set redirect input and output to PIPE.
processBuilder.redirectInput(Redirect.PIPE)

Program for file transfer in java using sockets

I have written a java code to transfer files from one server to another using the concept of socket programming. I got the codes from another java forum that meet my requirements. The program is said to transfer large sized files (like .mkv , .mprg movies) from one machine to another and can be used to transfer files of all formats. But after running the codes I found that the program is not able to transfer large sized files such as movies and even pdf of sizes 80mb or 111mb. The program has used bytebuffer but still error occurs. The codes are as follows (I got them from this site http://www.coderpanda.com/java-socket-programming-transferring-large-sized-files-through-socket/)
ClientMain.java
import java.io.IOException;
import java.net.Socket;
public class ClientMain
{
private DirectoryTxr transmitter = null;
Socket clientSocket = null;
private boolean connectedStatus = false;
private String ipAddress;
String srcPath = null;
String dstPath = "";
public ClientMain()
{
}
public void setIpAddress(String ip)
{
this.ipAddress = ip;
}
public void setSrcPath(String path)
{
this.srcPath = path;
}
public void setDstPath(String path)
{
this.dstPath = path;
}
private void createConnection()
{
Runnable connectRunnable = new Runnable()
{
public void run()
{
while (!connectedStatus)
{
try
{
clientSocket = new Socket(ipAddress, 22);
connectedStatus = true;
transmitter = new DirectoryTxr(clientSocket, srcPath, dstPath);
}
catch (IOException io)
{
io.printStackTrace();
}
}
}
};
Thread connectionThread = new Thread(connectRunnable);
connectionThread.start();
}
public static void main(String[] args)
{
ClientMain main = new ClientMain();
main.setIpAddress("localHost");
main.setSrcPath("C:/Transfer/");
main.setDstPath("C:/Receive");
main.createConnection();
}
}
DirectoryTxr.java
import java.io.*;
import java.net.Socket;
import java.net.SocketException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
public class DirectoryTxr
{
Socket clientSocket = null;
String srcDir = null;
String dstDir = null;
byte[] readBuffer = new byte[1024];
private InputStream inStream = null;
private OutputStream outStream = null;
int state = 0;
final int permissionReqState = 1;
final int initialState = 0;
final int dirHeaderSendState = 2;
final int fileHeaderSendState = 3;
final int fileSendState = 4;
final int fileFinishedState = 5;
private boolean isLive = false;
private int numFiles = 0;
private int filePointer = 0;
String request = "May I send?";
String respServer = "Yes,You can";
String dirResponse = "Directory created...Please send files";
String fileHeaderRecvd = "File header received ...Send File";
String fileReceived = "File Received";
String dirFailedResponse = "Failed";
File[] opFileList = null;
public DirectoryTxr(Socket clientSocket, String srcDir, String dstDir)
{
try
{
this.clientSocket = clientSocket;
inStream = clientSocket.getInputStream();
outStream = clientSocket.getOutputStream();
isLive = true;
this.srcDir = srcDir;
this.dstDir = dstDir;
state = initialState;
readResponse();
sendMessage(request);
state = permissionReqState;
}
catch (IOException io)
{
io.printStackTrace();
}
}
private void sendMessage(String message)
{
try
{
sendBytes(request.getBytes("UTF-8"));
}
catch (UnsupportedEncodingException e)
{
e.printStackTrace();
}
}
private void readResponse()
{
Runnable readRunnable = new Runnable()
{
public void run()
{
while (isLive)
{
try
{
int num = inStream.read(readBuffer);
if (num > 0)
{
byte[] tempArray = new byte[num];
System.arraycopy(readBuffer, 0, tempArray, 0, num);
processBytes(tempArray);
}
}
catch (SocketException se)
{
System.exit(0);
}
catch (IOException io)
{
io.printStackTrace();
isLive = false;
}
}
}
};
Thread readThread = new Thread(readRunnable);
readThread.start();
}
private void sendDirectoryHeader()
{
File file = new File(srcDir);
if (file.isDirectory())
{
try
{
String[] childFiles = file.list();
numFiles = childFiles.length;
String dirHeader = "$" + dstDir + "#" + numFiles + "&";
sendBytes(dirHeader.getBytes("UTF-8"));
}
catch (UnsupportedEncodingException en)
{
en.printStackTrace();
}
}
else
{
System.out.println(srcDir + " is not a valid directory");
}
}
private void sendFile(String dirName)
{
File file = new File(dirName);
if (!file.isDirectory())
{
try
{
int len = (int) file.length();
int buffSize = len / 8;
RandomAccessFile raf = new RandomAccessFile(file, "rw");
FileChannel channel = raf.getChannel();
int numRead = 0;
while (numRead >= 0)
{
ByteBuffer buf = ByteBuffer.allocate(1024 * 100000);
numRead = channel.read(buf);
if (numRead > 0)
{
byte[] array = new byte[numRead];
System.arraycopy(buf.array(), 0, array, 0, numRead);
sendBytes(array);
}
}
System.out.println("Finished");
}
catch (IOException io)
{
io.printStackTrace();
}
}
}
private void sendHeader(String fileName)
{
try
{
File file = new File(fileName);
if (file.isDirectory())
return;
String header = "&" + fileName + "#" + file.length() + "*";
sendHeader(header);
sendBytes(header.getBytes("UTF-8"));
}
catch (UnsupportedEncodingException e)
{
e.printStackTrace();
}
}
private void sendBytes(byte[] dataBytes)
{
synchronized (clientSocket)
{
if (outStream != null)
{
try
{
outStream.write(dataBytes);
outStream.flush();
}
catch (IOException io)
{
io.printStackTrace();
}
}
}
}
private void processBytes(byte[] data)
{
try
{
String parsedMessage = new String(data, "UTF-8");
System.out.println(parsedMessage);
setResponse(parsedMessage);
}
catch (UnsupportedEncodingException u)
{
u.printStackTrace();
}
}
private void setResponse(String message)
{
if (message.trim().equalsIgnoreCase(respServer) && state == permissionReqState)
{
state = dirHeaderSendState;
sendDirectoryHeader();
}
else if (message.trim().equalsIgnoreCase(dirResponse) && state == dirHeaderSendState)
{
state = fileHeaderSendState;
if (LocateDirectory())
{
createAndSendHeader();
}
else
{
System.out.println("Vacant or invalid directory");
}
}
else if (message.trim().equalsIgnoreCase(fileHeaderRecvd) && state == fileHeaderSendState)
{
state = fileSendState;
sendFile(opFileList[filePointer].toString());
state = fileFinishedState;
filePointer++;
}
else if (message.trim().equalsIgnoreCase(fileReceived) && state == fileFinishedState)
{
if (filePointer < numFiles)
{
createAndSendHeader();
}
System.out.println("Successfully sent");
}
else if (message.trim().equalsIgnoreCase(dirFailedResponse))
{
System.out.println("Going to exit....Error ");
}
else if (message.trim().equalsIgnoreCase("Thanks"))
{
System.out.println("All files were copied");
}
}
private void closeSocket()
{
try
{
clientSocket.close();
}
catch (IOException e)
{
e.printStackTrace();
}
}
private boolean LocateDirectory()
{
boolean status = false;
File file = new File(srcDir);
if (file.isDirectory())
{
opFileList = file.listFiles();
numFiles = opFileList.length;
if (numFiles <= 0)
{
System.out.println("No files found");
}
else
{
status = true;
}
}
return status;
}
private void createAndSendHeader()
{
File opFile = opFileList[filePointer];
String header = "&" + opFile.getName() + "#" + opFile.length() + "*";
try
{
state = fileHeaderSendState;
sendBytes(header.getBytes("UTF-8"));
}
catch (UnsupportedEncodingException e)
{
}
}
private void sendListFiles()
{
createAndSendHeader();
}
}
ServerMain.java
public class ServerMain {
public ServerMain() {
}
public static void main(String[] args) {
DirectoryRcr dirRcr = new DirectoryRcr();
}
}
DirectoryRcr.java
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
import java.net.SocketException;
public class DirectoryRcr
{
String request = "May I send?";
String respServer = "Yes,You can";
String dirResponse = "Directory created...Please send files";
String dirFailedResponse = "Failed";
String fileHeaderRecvd = "File header received ...Send File";
String fileReceived = "File Received";
Socket socket = null;
OutputStream ioStream = null;
InputStream inStream = null;
boolean isLive = false;
int state = 0;
final int initialState = 0;
final int dirHeaderWait = 1;
final int dirWaitState = 2;
final int fileHeaderWaitState = 3;
final int fileContentWaitState = 4;
final int fileReceiveState = 5;
final int fileReceivedState = 6;
final int finalState = 7;
byte[] readBuffer = new byte[1024 * 100000];
long fileSize = 0;
String dir = "";
FileOutputStream foStream = null;
int fileCount = 0;
File dstFile = null;
public DirectoryRcr()
{
acceptConnection();
}
private void acceptConnection()
{
try
{
ServerSocket server = new ServerSocket(22);
socket = server.accept();
isLive = true;
ioStream = socket.getOutputStream();
inStream = socket.getInputStream();
state = initialState;
startReadThread();
}
catch (IOException io)
{
io.printStackTrace();
}
}
private void startReadThread()
{
Thread readRunnable = new Thread()
{
public void run()
{
while (isLive)
{
try
{
int num = inStream.read(readBuffer);
if (num > 0)
{
byte[] tempArray = new byte[num];
System.arraycopy(readBuffer, 0, tempArray, 0, num);
processBytes(tempArray);
}
sleep(100);
} catch (SocketException s)
{
}
catch (IOException e)
{
e.printStackTrace();
}
catch (InterruptedException i)
{
i.printStackTrace();
}
}
}
};
Thread readThread = new Thread(readRunnable);
readThread.start();
}
private void processBytes(byte[] buff) throws InterruptedException
{
if (state == fileReceiveState || state == fileContentWaitState)
{
if (state == fileContentWaitState)
state = fileReceiveState;
fileSize = fileSize - buff.length;
writeToFile(buff);
if (fileSize == 0)
{
state = fileReceivedState;
try
{
foStream.close();
}
catch (IOException io)
{
io.printStackTrace();
}
System.out.println("Received " + dstFile.getName());
sendResponse(fileReceived);
fileCount--;
if (fileCount != 0)
{
state = fileHeaderWaitState;
}
else
{
System.out.println("Finished");
state = finalState;
sendResponse("Thanks");
Thread.sleep(2000);
System.exit(0);
}
System.out.println("Received");
}
}
else
{
parseToUTF(buff);
}
}
private void parseToUTF(byte[] data)
{
try
{
String parsedMessage = new String(data, "UTF-8");
System.out.println(parsedMessage);
setResponse(parsedMessage);
}
catch (UnsupportedEncodingException u) {
u.printStackTrace();
}
}
private void setResponse(String message)
{
if (message.trim().equalsIgnoreCase(request) && state == initialState)
{
sendResponse(respServer);
state = dirHeaderWait;
}
else if (state == dirHeaderWait)
{
if (createDirectory(message))
{
sendResponse(dirResponse);
state = fileHeaderWaitState;
}
else
{
sendResponse(dirFailedResponse);
System.out.println("Error occurred...Going to exit");
System.exit(0);
}
} else if (state == fileHeaderWaitState)
{
createFile(message);
state = fileContentWaitState;
sendResponse(fileHeaderRecvd);
}
else if (message.trim().equalsIgnoreCase(dirFailedResponse))
{
System.out.println("Error occurred ....");
System.exit(0);
}
}
private void sendResponse(String resp)
{
try
{
sendBytes(resp.getBytes("UTF-8"));
}
catch (UnsupportedEncodingException e)
{
e.printStackTrace();
}
}
private boolean createDirectory(String dirName)
{
boolean status = false;
dir = dirName.substring(dirName.indexOf("$") + 1, dirName.indexOf("#"));
fileCount = Integer.parseInt(dirName.substring(dirName.indexOf("#") + 1, dirName.indexOf("&")));
if (new File(dir).mkdir())
{
status = true;
System.out.println("Successfully created directory " + dirName);
}
else if (new File(dir).mkdirs())
{
status = true;
System.out.println("Directories were created " + dirName);
}
else if (new File(dir).exists())
{
status = true;
System.out.println("Directory exists" + dirName);
}
else
{
System.out.println("Could not create directory " + dirName);
status = false;
}
return status;
}
private void createFile(String fileName)
{
String file = fileName.substring(fileName.indexOf("&") + 1, fileName.indexOf("#"));
String lengthFile = fileName.substring(fileName.indexOf("#") + 1, fileName.indexOf("*"));
fileSize = Integer.parseInt(lengthFile);
dstFile = new File(dir + "/" + file);
try
{
foStream = new FileOutputStream(dstFile);
System.out.println("Starting to receive " + dstFile.getName());
}
catch (FileNotFoundException fn)
{
fn.printStackTrace();
}
}
private void writeToFile(byte[] buff)
{
try
{
foStream.write(buff);
}
catch (IOException io)
{
io.printStackTrace();
}
}
private void sendBytes(byte[] dataBytes)
{
synchronized (socket)
{
if (ioStream != null)
{
try
{
ioStream.write(dataBytes);
}
catch (IOException io)
{
io.printStackTrace();
}
}
}
}
}
Note that:-
ClientMain.java and DirectoryTxr.java are the two classes under client application.
ServerMain.java and DirectoryRcr.java are the two classes under Server application.
run the ClientMain.java and ServerMain.java simultaneously
Also specify the source directory, destination directory and host address of the machine in which server is running in the ClientMain.java(as per your computer). Here we are not specifying source file ,instead a source directory or folder is specifying.So the entire files of source directory will be transferred.
I would really appreciate if someone can help me with the problem.
You can check this code. This code can send files till 2GB of file size. Checked and working. Basically what I do here is first I send the file size and file name in an object then I start sending the file. First of all let me show you the class whose object will carry the File's details and send it to Receiver.
FileTransfer Class:
public class FileDetails implements Serializable {
String name;
long size;
public void setDetails(String name, long size) {
this.name = name;
this.size = size;
}
public String getName() {
return name;
}
public long getSize() {
return size;
}
}
Sender:
ServerSocket sendServer = null;
Socket sendSocket;
FileDetails details;
byte data[];
try {
File file = new File("File Url");
// Getting file name and size
if (file.length() > Integer.MAX_VALUE) {
System.out.println("File size exceeds 2 GB");
} else {
sendServer = new ServerSocket(5050);
System.out.println("Waiting for Client...");
sendSocket = sendServer.accept();
// File Object for accesing file Details
System.out.println("Connected to Client...");
data = new byte[2048]; // Here you can increase the size also which will send it faster
details = new FileDetails();
details.setDetails(file.getName(), file.length());
// Sending file details to the client
System.out.println("Sending file details...");
ObjectOutputStream sendDetails = new ObjectOutputStream(sendSocket.getOutputStream());
sendDetails.writeObject(details);
sendDetails.flush();
// Sending File Data
System.out.println("Sending file data...");
FileInputStream fileStream = new FileInputStream(file);
BufferedInputStream fileBuffer = new BufferedInputStream(fileStream);
OutputStream out = sendSocket.getOutputStream();
int count;
while ((count = fileBuffer.read(data)) > 0) {
System.out.println("Data Sent : " + count);
out.write(data, 0, count);
out.flush();
}
out.close();
fileBuffer.close();
fileStream.close();
}
} catch (Exception e) {
System.out.println("Error : " + e.toString());
}
Receiver's code :
int port = 5050;
try {
System.out.println("Connecting to Server...");
Socket receiveSocket = new Socket("IP of Server", port);
System.out.println("Connected to Server...");
// Getting file details
System.out.println("Getting details from Server...");
ObjectInputStream getDetails = new ObjectInputStream(receiveSocket.getInputStream());
FileDetails details = (FileDetails) getDetails.readObject();
System.out.println("Now receiving file...");
// Storing file name and sizes
String fileName = details.getName();
System.out.println("File Name : " + fileName);
byte data[] = new byte[2048]; // Here you can increase the size also which will receive it faster
FileOutputStream fileOut = new FileOutputStream("D:\\" + fileName);
InputStream fileIn = receiveSocket.getInputStream();
BufferedOutputStream fileBuffer = new BufferedOutputStream(fileOut);
int count;
int sum = 0;
while ((count = fileIn.read(data)) > 0) {
sum += count;
fileBuffer.write(data, 0, count);
System.out.println("Data received : " + sum);
fileBuffer.flush();
}
System.out.println("File Received...");
fileBuffer.close();
fileIn.close();
} catch (Exception e) {
System.out.println("Error : " + e.toString());
}
Hope this helps you.
You don't need all this. The canonical way to copy a stream in Java is as follows. It works for any buffer size greater than zero. I generally use 8192. There is certainly no necessity to read entire files into memory. It just wastes time and space.
while ((count = in.read(buffer)) > 0)
{
out.write(buffer, 0, count);
}
If you know the size in advance and need to keep the socket open for another transfer:
while (total < length && (count = in.read(buffer, 0, length-total > buffer.length ? buffer.length : (int)(length-total))) > 0)
{
out.write(buffer, 0, count);
total += count;
}
where total is a long initialized to zero before this loop, and length is the length you know in advance.

Categories

Resources