This is a question concerning setting the size of a byte array for a buffer to write() a local file from a URL stream:
byte[] buffer = new byte[26];
26 seems to be the magic number. If I put anything above 26 (say 1024), it still only writes the first 26 characters into the file. Why? Please note that I'm not trying to write the entire stream (using the while loop) just trying to figure out why the biggest chunk is only 26 characters long.
Here's the full code:
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.MalformedURLException;
import java.net.URL;
public class ReadNetworkFile {
public static void main(String[] args){
File f2 = new File("target3.txt");
try {
OutputStream out = new FileOutputStream(f2);
URL url = new URL("http://services.explorecalifornia.org/rss/tours.php");
InputStream stream = url.openStream();
byte[] buffer = new byte[1024];
int len=stream.read(buffer);
out.write(buffer,0,len);
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
You're missing the magic ingredient: the while-loop.
You need to read from the web and write to the file while len > 0. Then you've finished.
Related
I have written a client-socket "system" that is supposed to upload a file.
Although, when I attempt to upload, content duplicates.
I'm pretty sure that it is because the program doesn't recognise the eof.
I've found something like "Object stream", but I don't fancy importing new classes. I reckon that I don't really require that. But I wanna know how what the problem precisely is and how to hanle it.
package client;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.Socket;
import java.net.UnknownHostException;
public class Client {
private Socket socket;
private DataInputStream in;
private DataOutputStream out;
public static void main(String[] args) {
new Client();
}
public Client()
{
try {
socket = new Socket("127.0.0.1", 5010);
in = new DataInputStream(socket.getInputStream());
out = new DataOutputStream(socket.getOutputStream());
this.sendFile("./some.txt");
in.close();
out.close();
socket.close();
}
catch(UnknownHostException ex)
{
System.out.println("unknown host");
}
catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
private void sendFile(String path)
{
int bytes = 0;
File file = new File(path);
FileInputStream input;
try {
input = new FileInputStream(file);
long size = file.length();
//long size = file.getTotalSpace();
System.out.println(size);
// send a file's size
out.writeLong(size);
byte[] buffer = new byte[1024];
int i = 0, r=0;
//while((bytes = input.read(buffer,0,buffer.length))!=-1)
while(size > 0 && (bytes = input.read(buffer,0,(int)Math.min(buffer.length, size)))!=-1)
{
System.out.println("\n -------------"+(++i));
for (byte b : buffer)
try
{
if ((char)b == '\n' || r == 0)
System.out.print("\n" + (++r));
System.out.print((char)b);
}
catch(NullPointerException ex)
{
}
out.write(buffer, 0, bytes);
out.flush();
size -= bytes;
}
input.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
package server;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;
public class Server {
private ServerSocket ss;
private Socket cs;
private DataInputStream in;
private DataOutputStream out;
public static void main(String[] args) {
new Server();
}
public Server()
{
try {
ss = new ServerSocket(5010);
cs = ss.accept();
in = new DataInputStream(cs.getInputStream());
out = new DataOutputStream(cs.getOutputStream());
this.receiveFile("./uploaded.txt");
in.close();
out.close();
cs.close();
ss.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
private void receiveFile(String path)
{
int bytes = 0;
try {
File file = new File(path);
file.createNewFile();
FileOutputStream output = new FileOutputStream(file);
long size = in.readLong();
byte[] buffer = new byte[1024];
int i = 0;
while(size>0 && (bytes = in.read(buffer, 0, (int)Math.min(buffer.length, size))) != -1)
{
System.out.println("\n -------------"+(++i));
for (byte b : buffer)
try
{
System.out.print((char)b);
}
catch(NullPointerException ex)
{
}
output.write(buffer, 0, bytes);
size -= bytes;
}
output.close();
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
The problem was that I didn't check if the size were 0 on the client side.
That try catch should NOT be in the for loop !!!! It only needs single code use by wrapping. Also use counting metering conventionally the number of bytes with a conventional numeric "for" loop, Not for(byte b : buffer). Note: byte is not strictly numeric, it will only reach to 255 in the for counter! It depends the quantity bytes required iterated over and should be as many as are packaged to it over the connection.
In the read you need to obtain the number of bytes sent andmark that into the write length to take from the array, so it would be better to instantiate the array based on the number of bytes or maximum bytes the the client sender has sent or negotiated as the maximum allowed (see the api docs for the stream.
NB ObjectStream does not apply , it's for RMI)
Of counting into the byte[] buffer array, you should remove it from the method and put it as a global. In the method , instantiate a new "buffer" array on the global variable each iteration of the loop according to the number of bytes read as it's new length.
The code does not appear to be particularly safe or debugable. You might try carefully constructing it starting again from scratch.
My requirement is to invoke a Rest api, the o/p of which is a pdf file & generate a PDF file using a scripting language which is based on java.
We are using readAllBytes() method in our scripts, as we cannot use byte[] in our scripts(restriction). Equivalent java code for our script is below. As using readAllBytes() method is not recommended for reading large streams, is there an alternative for this without using byte[]? .
Please note: We are using 1.8 java and cannot use any third party libraries except Apache Commons IO.
Thank you for your help.
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedWriter;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
public class ExecuteReportFromScript {
public static void main(String[] args) {
// TODO Auto-generated method stub
try {
byte[] buffer = new byte[1024];
URL url = new URL("restEndPoint");
HttpURLConnection con = (HttpURLConnection) url.openConnection();
con.setRequestMethod("GET");
con.setRequestProperty("Cookie", "myCookieData");
con.setRequestProperty("Accept", "application/json");
con.setDoOutput(true);
con.connect();
System.out.println("Response Code:" + con.getResponseCode());
InputStream ip = con.getInputStream();
BufferedInputStream is = new BufferedInputStream(ip);
ByteArrayOutputStream bos = new ByteArrayOutputStream();
bos.write( is.readAllBytes());
/*
* int length; while ((length = is.read(buffer)) > -1 ) { bos.write(buffer, 0,
* length); }
*/
bos.flush();
File file = new File("PathToFile\\FileName.pdf");
try(BufferedOutputStream salida = new BufferedOutputStream(new FileOutputStream(file))){
salida.write(bos.toByteArray());
salida.flush();
}
ip.close();
is.close();
bos.close();
} catch (MalformedURLException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Write the stream directly to a file using Files.copy(InputStream in, Path target, CopyOption... options) (since Java 7).
Path file = Paths.get("PathToFile\\FileName.pdf");
try (InputStream in = new BufferedInputStream(con.getInputStream())) {
Files.copy(in, file);
}
I try to output the content of a text file. But I don't know how to work with the RandomAccessFile. I haven't found good examples at google. I hope for some help.
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.RandomAccessFile;
public class ReadTextFile {
public static void main(String[] args) throws IOException {
File src = new File ("C:/Users/hansbaum/Documents/Ascii.txt");
cat(src);
}
public static void cat(File quelle){
try (RandomAccessFile datei = new RandomAccessFile(quelle, "r")){
// while(datei.length() != -1){
// datei.seek(0); //
// }
} catch (FileNotFoundException fnfe) {
System.out.println("Datei nicht gefunden!");
} catch (IOException ioe) {
System.err.println(ioe);
}
}
}
related from doc
try (RandomAccessFile datei = new RandomAccessFile(quelle, "r")){
String line;
while ( (line = datei.readLine()) != null ) {
System.out.println(line);
}
System.out.println();
} catch (FileNotFoundException fnfe) {
} catch (IOException ioe) {
System.err.println(ioe);
}
What makes you think you need a RandomAccessFile? The easiest way is probably to use nio's convenience methods. With those, reading a file is as close to a one-liner as it gets in Java.
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.charset.StandardCharsets;
import java.util.List;
import java.io.IOException;
class Test {
public static void main(String[] args) throws IOException {
List<String> lines = Files.readAllLines(Paths.get("./Test.java"), StandardCharsets.UTF_8);
for (String l: lines)
System.out.println(l);
}
}
Be aware however that this is not a good idea if you happen to work with very large files as they might not fit into memory.
Try to create Stream from FileChannel to read and write in another file out.txt like this:
try (RandomAccessFile datei = new RandomAccessFile(quelle, "r").getChannel();){
// Construct a stream that reads bytes from the given channel.
InputStream is = Channels.newInputStream(rChannel);
File outFile = new File("out.txt");
// Create a writable file channel
WritableByteChannel wChannel = new RandomAccessFile(outFile,"w").getChannel();
// Construct a stream that writes bytes to the given channel.
OutputStream os = Channels.newOutputStream(wChannel);
// close the channels
is.close();
os.close();
I am trying to read a file that is encoded using Shift_JIS and then convert it into UTF-8. When i use java.nio CharsetDecoder.decode it throws the following error. I am not able to pinpoint the actual cause of this issue.
java.nio.charset.UnmappableCharacterException: Input length = 2
java.nio.charset.UnmappableCharacterException: Input length = 2
at java.nio.charset.CoderResult.throwException(CoderResult.java:278)
at java.nio.charset.CharsetDecoder.decode(CharsetDecoder.java:798)
at CharacterSetConversionUtility.getString(CharacterSetConversionUtility.java:23)
at CharacterSetConversionUtility.convertBetweenEncodings(CharacterSetConversionUtility.java:39)
at CharacterSetConversionUtility.main(CharacterSetConversionUtility.java:94
Below is the code snippet
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.nio.charset.CharacterCodingException;
import java.nio.charset.Charset;
import java.nio.charset.CharsetEncoder;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.charset.CharsetDecoder;
import org.mozilla.universalchardet.UniversalDetector;
public class CharacterSetConversionUtility
{
public static String getString(String charSet, byte[] bytes) throws CharacterCodingException
{
ByteBuffer buffer = ByteBuffer.wrap(bytes);
Charset charset = Charset.forName(charSet);
CharsetDecoder decoder = charset.newDecoder();
CharBuffer output = decoder.decode(buffer);
return output.toString();
}
public static byte[] convertToEncoding(String charSet, String input) throws CharacterCodingException
{
CharBuffer buffer = CharBuffer.wrap(input);
Charset charset = Charset.forName(charSet);
CharsetEncoder encoder = charset.newEncoder();
ByteBuffer output = encoder.encode(buffer);
return output.array();
}
public static byte[] convertBetweenEncodings(byte[] originalBytes, String sourceCharSet, String destCharSet)
throws CharacterCodingException
{
String unicodeString = getString(sourceCharSet, originalBytes);
byte[] output = convertToEncoding(destCharSet, unicodeString);
return output;
}
/** Utility Method to detect character encoding in a byte stream **/
public static String getCharacterEncoding(String fileName){
byte[] buf = new byte[4096];
String encoding = null;
try {
java.io.FileInputStream fis = new java.io.FileInputStream(fileName);
// (1)
UniversalDetector detector = new UniversalDetector(null);
// (2)
int nread;
while ((nread = fis.read(buf)) > 0 && !detector.isDone()) {
detector.handleData(buf, 0, nread);
}
// (3)
detector.dataEnd();
// (4)
encoding = detector.getDetectedCharset();
if (encoding != null) {
System.out.println("Detected encoding = " + encoding);
} else {
System.out.println("No encoding detected.");
}
// (5)
detector.reset();
//
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return encoding;
}
public static void main(String[] args) {
Path path = Paths.get("E:/Encoding Issue/SJISFile");
try {
byte[] inputdata = Files.readAllBytes(path);
//Detect the character encoding of the input data
String inputCharEncoding = getCharacterEncoding("E:/Encoding Issue/SJISFile");
//Perform a character set conversion
byte[] outputdata =convertBetweenEncodings(inputdata,inputCharEncoding,"UTF-8");
FileOutputStream fos = new FileOutputStream("E:/Encoding Issue/convertedutf8.txt");
fos.write(outputdata);
fos.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
So, I don't have a definitive answer to the cause of the failure you experienced, although I suspect it lies somewhere in the conversion between String/char[]/byte[].
That said, I'd like to offer a more simple and compact working solution to the problem at hand, which uses this method instead, i.e. the conversion functionality offered by the String class itself as opposed to using en/decoders.
This will work for the Shift_JIS charset or any other one. Also, nothing wrong with the use of UniversalDetector, but I omitted it for simplicity's sake and hard-coded the source character set instead. Finally this version is JavaSE 1.6 compatible.
Hope it helps :)
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.CharBuffer;
import java.nio.charset.Charset;
public class JapaneseCharsetTest {
public static final int CHAR_LENGTH_TO_READ = 8192;
public static void main(String[] args) {
JapaneseCharsetTest test = new JapaneseCharsetTest();
test.doIt();
}
public void doIt() {
InputStreamReader reader = null;
FileOutputStream fos = null;
try {
FileInputStream stream = new FileInputStream(new File("C:/Path/To/My/ShiftJISFile.txt"));
reader = new InputStreamReader(stream, Charset.forName("Shift_JIS"));
fos = new FileOutputStream("C:/Path/To/My/UTF8TargetFile.txt");
char[] inputdata = new char[CHAR_LENGTH_TO_READ];
int len = 0;
while ((len = reader.read(inputdata)) != -1) {
convert(len, inputdata, fos);
}
}
catch (IOException e) {
e.printStackTrace();
}
finally {
try {if (reader != null) reader.close();} catch (IOException ignored) {}
try {if (fos != null) fos.close();} catch (IOException ignored){}
}
}
private void convert(int len, char[] inputData, FileOutputStream fos) throws IOException {
char[] charsToWrite = inputData;
if (len < CHAR_LENGTH_TO_READ) {
// Last chunk of data - cut it to size
charsToWrite = new char[len];
CharBuffer.wrap(inputData).get(charsToWrite, 0, len);
}
// Convert initial charset (here Shift_JIS) to target (here UTF-8)
byte[] utf8 = new String(charsToWrite).getBytes("UTF-8");
fos.write(utf8);
}
}
I am playing around with saving/loading a text file in Android, works fine. Next step is to encrypt and decrypt with AES.
I can call the encrypt() method within the writetofile method,a dn that works fine. If i call the readfromfile method, I can see the ciphertext is retrieved, great.
But decryption isnt working for me - I have called simplecrypo in a few places - on stringBuffer.toString(), and within the StringBuffer's append() method - but both crash the application.
So does anybody know where I am supposed to decrypt the string within the file?
package com.example.filesdemo;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import android.app.Activity;
import android.os.Bundle;
import android.view.View;
import android.widget.EditText;
import android.widget.Toast;
public class MainActivity extends Activity {
private EditText etInput;
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
etInput = (EditText) findViewById(R.id.etInput);
}
public void writeToFile(View v) throws Exception {
try {
String inputStr = etInput.getText().toString();
//encrypt the string - works!
String encrypted = SimpleCrypto.encrypt("testkey", inputStr);
FileOutputStream fos = openFileOutput("myfile.txt", MODE_PRIVATE);
fos.write(encrypted.getBytes());
fos.flush();
fos.close();
Toast.makeText(this, "File saved!", Toast.LENGTH_SHORT).show();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
public void readFromFile(View v) throws Exception{
try {
FileInputStream fis = openFileInput("myfile.txt");
StringBuffer stringBuffer = new StringBuffer();
BufferedReader bReader = new BufferedReader(new InputStreamReader(
fis));
String strLine = null;
while ((strLine = bReader.readLine()) != null) {
stringBuffer.append(strLine + "\n");
}
bReader.close();
fis.close();
Toast.makeText(this, "File content: \n" +stringBuffer.toString(),
Toast.LENGTH_SHORT).show();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
And the encryption class is publically available here - decrypt class, but i dont think thats the issue.
Thanks!
From the look of things, you are writing the encrypted bytes to the file, and then trying to read them back as text into a Scanner. That won't work, as you have found.
If you want your file to be text, then you need to convert the bytes to Base64 and write as text. On reading the Base64 text, convert back to bytes and decypher. Java 8 has the Base64 class, I an not sure about Android.
If you want the file as raw bytes, then you need to read it as bytes, not as text. Once read as bytes it can be decyphered directly.