I'm learning about Memory Mapped files in java.
I would like to know how to write/read to a MappedByteBuffer.
Here's the code I'm using for writing to MappedByteBuffer.
private static void write(String strFilePath) {
File fl = new File(strFilePath);
FileChannel fChannel = null;
RandomAccessFile rf = null;
MappedByteBuffer mBBuffer = null;
try {
rf = new RandomAccessFile(fl, "rw");
fChannel = rf.getChannel();
mBBuffer = fChannel.map(FileChannel.MapMode.READ_WRITE, 0, 1024 );
for(int i =0;i<10030;i++) {
if(i == mBBuffer.limit()) {
mBBuffer = fChannel.map(FileChannel.MapMode.READ_WRITE,i+1,1024);
}
mBBuffer.put((byte)'a');
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
if(fChannel != null) {
fChannel.close();
}
if(rf != null) {
rf.close();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
I get BufferOverflow Exception while trying to do it.
How do I increase the size of the MappedByteBuffer after reaching the limit while writing, if I do not the know the size of the contents I will write to the file in advance?
For reading, let's take this case,
I create an MappedByteBuffer with an intial buffer size, and if reached the end of that initial size, how do i map a different portion of the file
More generally, I have a file , with byte offsets from one part of the file to another, When I read an offset (y) at a point (x), I would like to jump to (y) from (x). How do I do it?
Thanks in advance for helping me out.
Related
public void writefunct() {
try {
RandomAccessFile raf = new RandomAccessFile("C:/Users/Yugantar/Desktop/out.txt", "rw");
try {
raf.seek(mx);
raf.writeBytes(Integer.toString(imageid));
raf.close();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
} catch (FileNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
}
suppose the number is 3 then the output in file is right text file output is 333333333333 but if the number is 34 the output is 334334334
how to put file pointer after the number using seek method..?
class HelloWorld {
public static void main(String args[]) {
File file = new File("d://1.mp4");
FileInputStream fr = null;
FileOutputStream fw = null;
byte a[] = new byte[(int) file.length()];
try {
fr = new FileInputStream(file);
fw = new FileOutputStream("d://2.mp4");
fr.read(a);
fw.write(a);
fw.write(a);
fw.write(a);
fw.write(a);
fw.write(a);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
try {
fr.close();
fw.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
Here i write fw.write(a) five times, the size of the file increases to 5x but the original 1.mp4 and copy 2.mp4 both have same length i.e. 3:30 minutes ?
Simply duplicating the bytes of certain files does not necessarily mean it simply duplicates things when inspecting them with software. For example, the video player might read the data until some terminal is encountered and not look forward. This terminal would then exist at the end of the first file data block.
You could open the new file with a hex editor and check if you can see the data of the original video file five times in a row.
FileOutputStream fooStream = new FileOutputStream("FilePath", false);
This will overwrite the content and the size of the file created will be same size as of original file.
[Update]Problem is different, nothing about csv file format. Question
would be "While using File Writer to write a csv file last few records
are missing.
In my java application i need to append more than 65535 rows in a csv file. but it only writes 65535 rows in a sheet. I haven't used any libraries. some final records missing. how to resolve this..........
public void writeSubmission(){
try {
writer = new FileWriter("res/sample.csv");
writer.append("PhraseId");
writer.append(',');
writer.append("Sentiment");
writer.append('\n');
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
public void readTestData(){
String path="res/test.tsv";
Calculation cal=new Calculation();
int counter=0;
try {
BufferedReader bReader = new BufferedReader(new FileReader(path));
String line;
writeSubmission();
bReader.readLine();
while ((line = bReader.readLine()) != null) {
String datavalue[] = line.split("\t");
writer.append(datavalue[0]);
writer.append(',');
try {
double value=cal.calculate(datavalue[2]);
System.out.println(value);
String val;
if(value<-0.4)
{
val="0";
}
else if(value>-0.4 && value<-0.1)
{
val="1";
}
else if(value>-0.1 && value<+0.1)
{
val="2";
}
else if(value>+0.1 && value<+0.40)
{
val="3";
}
else{
val="4";
}
counter++;
writer.append(val);
} catch (Exception e) {
// TODO Auto-generated catch block
System.out.println(e);
}
writer.append('\n');
}
System.out.println(counter);
bReader.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
System.out.println(e);
} catch (IOException e){
e.printStackTrace();
System.out.println(e);
}
}
I think your issue is probably that the tool you're opening up the CSVs with on the other end doesn't want more than 65535 rows, not that Java's doing anything wrong. It's a bug on the other end, not your Java code, almost certainly. (FileWriter wouldn't care at all about 65535 lines, for example.)
If you're using Excel 2003, for example, you'd see this issue: How to get around 64k row limit in Excel
Finally i fixed the error.File Writer should be flush after used in the code push existing stream. OR should be close the writer it automatically flush and close
The flush method flushes the output stream and forces any buffered
output bytes to be written out. The general contract of flush is that
calling it is an indication that, if any bytes previously written have
been buffered by the implementation of the output stream, such bytes
should immediately be written to their intended destination.
writer = new FileWriter("res/sample.csv");
writer.flush();
writer.close();
i am getting out of memory error in asynctask which loop to stringbuilder . My target for using this to download image from server and store inside my sd card.My code as below :
HttpClient httpclient = new DefaultHttpClient();
httpclient.getParams().setParameter(CoreProtocolPNames.PROTOCOL_VERSION, HttpVersion.HTTP_1_1);
HttpPost httppost = new HttpPost(severPath);
httppost.setEntity(params[0]);
System.out.println("executing request " + httppost.getRequestLine());
HttpResponse response = null;
try {
response = httpclient.execute(httppost);
} catch (ClientProtocolException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
} catch (IOException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
}
String output;
System.out.println("Output from Server .... \n");
BufferedReader br = null;
try {
br = new BufferedReader(
new InputStreamReader((response.getEntity().getContent())));
} catch (IllegalStateException e5) {
// TODO Auto-generated catch block
e5.printStackTrace();
} catch (IOException e5) {
// TODO Auto-generated catch block
e5.printStackTrace();
}
OutputStreamWriter outputStreamWriter = null;
try {
outputStreamWriter = new OutputStreamWriter(context.openFileOutput("LargeImages.txt", context.MODE_PRIVATE));
} catch (FileNotFoundException e6) {
// TODO Auto-generated catch block
e6.printStackTrace();
}
int i = 0;
StringBuilder builder = new StringBuilder();
String Result = "";
try {
for (String line = null; (line = br.readLine()) != null ; ) {
builder.append(line.toString());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
outputStreamWriter.close();
i am getting out of memory allocation error. please help. i try many method but also not getting the right.
if you are downloading an image, then you should not use Reader/Writer/StringBuilder to store it's content. Because the file is binary content will be scrambled because of the character encoding used by Reader/Writer classes.
Try using InputStream/OutputStream and store the content directly to sdcard without storing it in memory.
Try out the below code:
InputStream in = response.getEntity().getContent();
OutputStream out = context.openFileOutput("LargeImages.txt", context.MODE_PRIVATE);
byte b[] = new byte[4096];
int i;
while ((i = in.read(b)) >= 0) {
out.write(b, 0, i);
}
There may be two problems.
The first - the cycle for (String line = null; (line = br.readLine()) != null ; ) is not terminated properly. Try to find it out by opening a small file(e.g. with 10 lines total).
The second - it's actually a memory insufficient case. Probably it's not the best idea to get image via strings as images may be very heavy and creating a plenty of Strings causes natural memory error. Try to find another approach.
I don't see code that is actually writing to the output stream. Shouldn't there be a line before the close, that is like outputStreamWriter.print(builder)?
About your question. Instead of collecting the whole data in memory in a StringBuilder and than write it at once, you should write directly each line you get within your for-loop. You don't need the StringBuilder at all. Here's a code snippet:
try {
for (String line = br.readLine(); line != null; line = br.readLine()) {
outputStreamWriter.append(line);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
Three more remarks:
When you get an Exception you should also stop the action, e.g. return from your method. Your code above would print the Stacktrace (which is definitely helpful) but would then continue, which would be not so helpful. Just add return after each printstackTrace.
There's still a chance that one line is too long for memory, but the risk is minimized.
Is the data you download binary image or text? You name it image but you download text. Please be aware that there's a difference between bytes and characters (encoded with character set) and stay within what you actually receive.
I'm using MemoryMapped buffer to read a file. Initially I'm getting the channel size and using the same size I"m mapping the file on memory and here the initial position is 0 as I want to map the file from the beginning. Now another 400KB of data is added to that file, now I want to map that 400kb alone. But something is wrong in my code, I'm not able to figure it out and I'm getting this
260java.io.IOException: Channel not open for writing - cannot extend file to required size
at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:812)
at trailreader.main(trailreader.java:55
So here's my code
BufferedWriter bw;
FileInputStream fileinput = null;
try {
fileinput = new FileInputStream("simple.csv");
} catch (FileNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
FileChannel channel = fileinput.getChannel();
MappedByteBuffer ByteBuffer;
try {
ByteBuffer = fileinput.getChannel().map(FileChannel.MapMode.READ_ONLY, 0, channel.size());
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
/*
* Add some 400 bytes to simple.csv. outside of this program...
*/
//following line throw exception.
try {
ByteBuffer = fileinput.getChannel().map(FileChannel.MapMode.READ_ONLY, channel.size(), 400);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
So in my code I'm trying to reread the additional data that has been added but its not working, I know the prob is channel.size(), but I'm not able to rectify it.
channel.size() is always the current end of file. You are attempting to map 400 bytes past it. It isn't there. You need something like:
ByteBuffer = fileinput.getChannel().map(FileChannel.MapMode.READ_ONLY, channel.size()-400, 400);