I want read a large number of BitSet objects from a file (12MB). I used following code but only read first object from file and repeated it. thanks
public static void main(String[] args) {
// TODO code application logic here
ObjectInputStream Input = null;
FileInputStream Database = null;
Object Buffer = null;
BitSet H = null;
try
{
Database = new FileInputStream("BloomFilters.txt");
Input = new ObjectInputStream(Database);
while((Buffer = Input.readObject()) != null)
{
H = (BitSet)Buffer;
System.out.println(H);
System.out.println("Yes" );
}
}
catch(Exception e)
{
System.out.println("Exp = " + e.getMessage());
}
and following code create a file of BitSet objects, I want read objects from this file
public class Main {
public static void main(String[] args) {
BloomFilter Set = new BloomFilter(512, 100);
ObjectOutputStream Output = null;
DataInputStream Input = null;
FileOutputStream DBOut = null;
FileInputStream DBIn = null;
String Sequence = "";
try
{
DBOut = new FileOutputStream("Bloomfilters.txt");
Output = new ObjectOutputStream(DBOut);
DBIn = new FileInputStream("DB.txt");
Input = new DataInputStream(DBIn);
while((Sequence = (String) Input.readLine()) != null)
{
Set.clear();
for(int i = 0; i < Sequence.length() - 1; i++)
Set.add((Sequence.substring(i, i + 2)));
BitSet buffer = Set.getBitSet();
Output.writeObject(buffer);
}
Input.close();
Output.close();
}
catch(Exception e)
{
System.out.println(e.getMessage());
}
}
}
I think you need a Scanner see this code Java Bitset error with large index. It's a different question but the first loop is to read a large file with numbers into a bitset,
Related
I have been writing an updater for my game.
It checks a .version file on drop box and compares it to the local .version file.
If there is any link missing from the local version of the file, it downloads the required link one by one.
This is the error that it shows
Exception in thread "Thread-9" java.lang.OutOfMemoryError: Java heap space
at com.fox.listeners.ButtonListener.readFile(ButtonListener.java:209)
at com.fox.listeners.ButtonListener.readFile(ButtonListener.java:204)
at com.fox.listeners.ButtonListener.UpdateStart(ButtonListener.java:132)
at com.fox.listeners.ButtonListener$1.run(ButtonListener.java:58)
It only shows for some computers though and not all of them this is the readFile method
private byte[] readFile(URL u) throws IOException {
return readFile(u, getFileSize(u));
}
private static byte[] readFile(URL u, int size) throws IOException {
byte[] data = new byte[size];
int index = 0, read = 0;
try {
HttpURLConnection conn = null;
conn = (HttpURLConnection) u.openConnection();
conn.addRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
InputStream is = conn.getInputStream();
progress_a = 0;
progress_b = data.length;
while(index < data.length) {
read = is.read(data, index, size-index);
index += read;
progress_a = index;
}
} catch(Exception e) {
e.printStackTrace();
}
return data;
}
private byte[] readFile(File f) {
byte[] data = null;
try {
data = new byte[(int)f.length()];
#SuppressWarnings("resource")
DataInputStream dis = new DataInputStream(new FileInputStream(f));
dis.readFully(data);
} catch (IOException e) {
e.printStackTrace();
}
return data;
}
This is the main method that is ran
public void UpdateStart() {
System.out.println("Starting Updater..");
if(new File(cache_dir).exists() == false) {
System.out.print("Creating cache dir.. ");
while(new File(cache_dir).mkdir() == false);
System.out.println("Done");
}
try {
version_live = new Version(new URL(version_file_live));
} catch(MalformedURLException e) {
e.printStackTrace();
}
version_local = new Version(new File(version_file_local));
Version updates = version_live.differences(version_local);
System.out.println("Updated");
int i = 1;
try {
byte[] b = null, data = null;
FileOutputStream fos = null;
BufferedWriter bw = null;
for(String s : updates.files) {
if(s.equals(""))
continue;
System.out.println("Reading file "+s);
AppFrame.pbar.setString("Downloading file "+ i + " of "+updates.files.size());
if(progress_b > 0) {
s = s + " " +(progress_a * 1000L / progress_b / 10.0)+"%";
}
b = readFile(new URL(s));
progress_a = 0;
progress_b = b.length;
AppFrame.pbar.setString("Unzipping file "+ i++ +" of "+updates.files.size());
ZipInputStream zipStream = new ZipInputStream(new ByteArrayInputStream(b));
File f = null, parent = null;
ZipEntry entry = null;
int read = 0, entry_read = 0;
long entry_size = 0;
progress_b = 0;
while((entry = zipStream.getNextEntry()) != null)
progress_b += entry.getSize();
zipStream = new ZipInputStream(new ByteArrayInputStream(b));
while((entry = zipStream.getNextEntry()) != null) {
f = new File(cache_dir+entry.getName());
if(entry.isDirectory())
continue;
System.out.println("Making file "+f.toString());
parent = f.getParentFile();
if(parent != null && !parent.exists()) {
System.out.println("Trying to create directory "+parent.getAbsolutePath());
while(parent.mkdirs() == false);
}
entry_read = 0;
entry_size = entry.getSize();
data = new byte[1024];
fos = new FileOutputStream(f);
while(entry_read < entry_size) {
read = zipStream.read(data, 0, (int)Math.min(1024, entry_size-entry_read));
entry_read += read;
progress_a += read;
fos.write(data, 0, read);
}
fos.close();
}
bw = new BufferedWriter(new FileWriter(new File(version_file_local), true));
bw.write(s);
bw.newLine();
bw.close();
}
} catch(Exception e) {
e.printStackTrace();
return;
}
System.out.println(version_live);
System.out.println(version_local);
System.out.println(updates);
CacheUpdated = true;
if(CacheUpdated) {
AppFrame.pbar.setString("All Files are downloaded click Launch to play!");
}
}
I don't get why it is working for some of my players and then some of my other players it does not i have been trying to fix this all day and i am just so stumped at this point but this seems like its the only big issue left for me to fix.
Either increase the memory allocated to your JVM (How can I increase the JVM memory?), or make sure that the file being loaded in memory isn't gigantic (if it is, you'll need to find an alternate solution, or just read chunks of it at a time instead of loading the entire thing in memory).
Do your update in several steps. Here's some pseudo-code with Java 8. It's way shorter than what you wrote because Java has a lot of built-in tools that you re-write much less efficiently.
// Download
Path zipDestination = Paths.get(...);
try (InputStream in = source.openStream()) {
Files.copy(in, zipDestination);
}
// Unzip
try (ZipFile zipFile = new ZipFile(zipDestination.toFile())) {
for (ZipEntry e: Collections.list(zipFile.entries())) {
Path entryDestination = Paths.get(...);
Files.copy(zipFile.getInputStream(e), entryDestination);
}
}
// Done.
I'm trying to read a text file to get a version number but for some reason no matter what I put in the text file it always returns 0 (zero).
The text file is called version.txt and it contains no spaces or letters, just 1 character that is a number. I need it to return that number. Any ideas on why this doesn't work?
static int i;
public static void main(String[] args) {
String strFilePath = "/version.txt";
try
{
FileInputStream fin = new FileInputStream(strFilePath);
DataInputStream din = new DataInputStream(fin);
i = din.readInt();
System.out.println("int : " + i);
din.close();
}
catch(FileNotFoundException fe)
{
System.out.println("FileNotFoundException : " + fe);
}
catch(IOException ioe)
{
System.out.println("IOException : " + ioe);
}
}
private final int VERSION = i;
Here is the default solution that i use whenever i require to read a text file.
public static ArrayList<String> readData(String fileName) throws Exception
{
ArrayList<String> data = new ArrayList<String>();
BufferedReader in = new BufferedReader(new FileReader(fileName));
String temp = in.readLine();
while (temp != null)
{
data.add(temp);
temp = in.readLine();
}
in.close();
return data;
}
Pass the file name to readData method. You can then use for loop to read the only line in the arraylist, and can use the same loop to read multiple lines from different file...I mean do whatever you like with the arraylist.
Please don't use a DataInputStream
Per the linked Javadoc, it lets an application read primitive Java data types from an underlying input stream in a machine-independent way. An application uses a data output stream to write data that can later be read by a data input stream.
You want to read a File (not data from a data output stream).
Please do use try-with-resources
And since you seem to want an ascii integer, I'd suggest you use a Scanner.
public static void main(String[] args) {
String strFilePath = "/version.txt";
File f = new File(strFilePath);
try (Scanner scanner = new Scanner(f)) {
int i = scanner.nextInt();
System.out.println(i);
} catch (Exception e) {
e.printStackTrace();
}
}
Use an initializing block
An initializing block will be copied into the class constructor, in your example remove public static void main(String[] args), something like
private int VERSION = -1; // <-- no more zero!
{
String strFilePath = "/version.txt";
File f = new File(strFilePath);
try (Scanner scanner = new Scanner(f)) {
VERSION = scanner.nextInt(); // <-- hope it's a value
System.out.println("Version = " + VERSION);
} catch (Exception e) {
e.printStackTrace();
}
}
Extract it to a method
private final int VERSION = getVersion("/version.txt");
private static final int getVersion(String strFilePath) {
File f = new File(strFilePath);
try (Scanner scanner = new Scanner(f)) {
VERSION = scanner.nextInt(); // <-- hope it's a value
System.out.println("Version = " + VERSION);
return VERSION;
} catch (Exception e) {
e.printStackTrace();
}
return -1;
}
or even
private final int VERSION = getVersion("/version.txt");
private static final int getVersion(String strFilePath) {
File f = new File(strFilePath);
try (Scanner scanner = new Scanner(f)) {
if (scanner.hasNextInt()) {
return scanner.nextInt();
}
} catch (Exception e) {
e.printStackTrace();
}
return -1;
}
I am reading, from a file, integer values that I should use to calculate the function multiple.
However, after converting to integer, it appears that the integer variable doesn't hold them for further calculation.
Any help please?
import java.io.*;
public class Functions {
int values, mul7, mul11, mul13;
public static void main (String []args) {
Functions go = new Functions ();
go.multiple();
// will call functions here
}
public void multiple () {
int a = 7;
int b = 11;
int c = 13;
try {
File inputFile = new File ("JavaInputData.txt");
FileReader fileReader = new FileReader (inputFile);
BufferedReader reader = new BufferedReader (fileReader);
String line = null;
while ((line = reader.readLine()) !=null)
{
values = Integer.parseInt(line);
System.out.println(values);
}
mul7 = values % a;
mul11 = values %b;
mul13 = values %c;
System.out.println(mul7);
reader.close();
}
catch (Exception ex)
{
ex.printStackTrace();
}
}
}
Perform the calculation and output in the loop body. Something like,
while ((line = reader.readLine()) != null)
{
values = Integer.parseInt(line);
System.out.println(values);
mul7 = values % a;
mul11 = values % b;
mul13 = values % c;
System.out.printf("mul7 = %d, mul11 = %d, mul13 = %d%n", mul7, mul11, mul13);
}
Also, I suggest you use a try-with-resources to close() your Reader;
try (File inputFile = new File ("JavaInputData.txt");
FileReader fileReader = new FileReader (inputFile);
BufferedReader reader = new BufferedReader (fileReader)) {
That way you don't have to call close() explicitly. But, if you're going to call close() explicitly; please do so in a finally block.
First text file
A.txt;
asdfghjklqw12345 qwe3456789
asdfghjklqw12345 qwe3456789
Second text file
B.txt;
|Record 1: Rejected - Error on table AUTHORIZATION_TBL, column AUTH_DATE.ORA-01843: not a valid month|
|Record 2: Rejected - Error on table AUTHORIZATION_TBL, column AUTH_DATE.ORA-01843: not a valid month|
Third text file
C.txt;
asdfghjklqw12345 qwe3456789 |Record 1: Rejected - Error on table AUTHORIZATION_TBL, column AUTH_DATE.ORA-01843: not a valid month|
asdfghjklqw12345 qwe3456789 |Record 2: Rejected - Error on table AUTHORIZATION_TBL, column AUTH_DATE.ORA-01843: not a valid month|
for the above situation where I want to merge two lines from two different text files into one line.My code is below
List<FileInputStream> inputs = new ArrayList<FileInputStream>();
File file1 = new File("C:/Users/dell/Desktop/Test/input1.txt");
File file2 = new File("C:/Users/dell/Desktop/Test/Test.txt");
FileInputStream fis1;
FileInputStream fis2;
try {
fis1 = new FileInputStream(file1);
fis2= new FileInputStream(file2);
inputs.add(fis1);
inputs.add(fis2);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
int total = (int) (file1.length() + file2.length());
System.out.println("total length is " + total);
SequenceInputStream sis = new SequenceInputStream(Collections.enumeration(inputs));
try {
System.out.println("SequenceInputStream.available() = "+ sis.available());
byte[] merge = new byte[total];
int soFar = 0;
do {
soFar += sis.read(merge,total - soFar, soFar);
} while (soFar != total);
DataOutputStream dos = new DataOutputStream(new FileOutputStream("C:/Users/dell/Desktop/Test/C.txt"));
soFar = 0;
dos.write(merge, 0, merge.length);
dos.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Here is code:
public class MergeText {
public static void main(String[] args) throws IOException{
String output="";
try(Scanner sc1=new Scanner((new File("A.txt")));
Scanner sc2=new Scanner((new File("B.txt")))){
while(sc1.hasNext() || sc2.hasNext()){
output+=sc1.next() +" "+ sc2.next();
output+="\n";
}
}
try(PrintWriter pw=new PrintWriter(new File("C.txt"))){
pw.write(output);
}
}
}
You might want to have a look at BufferedReader and BufferedWriter.
Show us what you tried and where you are stuck and we are happy to provide more help.
Merging all txt file from a folder can be done in the following way:
public static void main(String[] args) throws IOException {
ArrayList<String> list = new ArrayList<String>();
//Reading data files
try {
File folder = new File("path/inputFolder");
File[] listOfFiles = folder.listFiles();
for (int i = 0; i < listOfFiles.length; i++) {
File file = listOfFiles[i];
if (file.isFile() && file.getName().endsWith(".txt")) {
BufferedReader t = new BufferedReader (new FileReader (file));
String s = null;
while ((s = t.readLine()) != null) {
list.add(s);
}
t.close();
}
}
}
catch (IOException e) {
e.printStackTrace();
}
//Writing merged data file
BufferedWriter writer=null;
writer = new BufferedWriter(new FileWriter("data.output/merged-output.txt"));
String listWord;
for (int i = 0; i< list.size(); i++)
{
listWord = list.get(i);
writer.write(listWord);
writer.write("\n");
}
System.out.println("complited");
writer.flush();
writer.close();
}
Improved on Masudul's answer to avoid compilation errors:
import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
import java.util.Scanner;
public class MergeText {
public static void main(String[] args) throws IOException {
StringBuilder output = new StringBuilder();
try (Scanner sc1 = new Scanner((new File("C:\\Users\\YourName\\Desktop\\A.txt")));
Scanner sc2 = new Scanner((new File("C:\\Users\\YourName\\Desktop\\B.txt")))) {
while (sc1.hasNext() || sc2.hasNext()) {
String s1 = (sc1.hasNext() ? sc1.next() : "");
String s2 = (sc2.hasNext() ? sc2.next() : "");
output.append(s1).append(" ").append(s2);
output.append("\n");
}
}
try (PrintWriter pw = new PrintWriter(new File("C:\\Users\\mathe\\Desktop\\Fielddata\\RESULT.txt"))) {
pw.write(output.toString());
}
}
}
So I have been working on a java project where the goal is to create a virtual computer. So I am basically done but with one problem. I have created a compiler which translates a txt document with assembly code in it and my compiler has created a new-file with this code written as machine executable ints. But now I need to write a load method that reads these ints and runs the program but I am having difficulty doing this. Any help is much appreciated....also this is not homework if you are thinking this. The project was simply to make a compiler and now I am trying to complete it for my own interest. Thanks.
Here is what I have so far for load:
public void load(String newfile) throws FileNotFoundException
{
try{
File file = new File(newfile);
FileInputStream fs = new FileInputStream(file);
DataInputStream dos = new DataInputStream(fs);
dos.readInt();
dos.close();
}
catch (IOException e)
{
e.printStackTrace();
}
}
Ok here is the part of the Compiler that does the writeInts:
public void SecondPass(SymbolList symbolTable, String filename){
try {
int dc = 99;
//Open file for reading
File file = new File(filename);
Scanner scan = new Scanner(file);
//Make filename of new executable file
String newfile = makeFilename(filename);
//Open Output Stream for writing new file.
FileOutputStream fs = new FileOutputStream(newfile);
DataOutputStream dos = new DataOutputStream(fs);
//Read First line. Split line by Spaces into linearray.
String line = scan.nextLine();
String[] linearray = line.split(" ");
while(line!=null){
if(!linearray[0].equals("REM")){
int inst = 0, opcode, loc;
if(isInstruction(linearray[0])){
opcode = getOpcode(linearray[0]);
loc = symbolTable.searchName(linearray[1]).getMemloc();
inst = (opcode*100)+loc;
} else if(!isInstruction(linearray[0])){
if(isInstruction(linearray[1])){
opcode = getOpcode(linearray[1]);
if(linearray[1].equals("STOP"))
inst=0000;
else {
loc = symbolTable.searchName(linearray[2]).getMemloc();
inst = (opcode*100)+loc;
}
}
if(linearray[1].equals("DC"))
dc--;
}
dos.writeInt(inst);
System.out.println(" inst is being written as:" + inst);
}
try{
line = scan.nextLine();
}
catch(NoSuchElementException e){
line = null;
break;
}
linearray = line.split(" ");
}
scan.close();
for(int i=lc; i<=dc; i++){
dos.writeInt(0);
}
for(int i = dc+1; i < 100; i++)
{
dos.writeInt(symbolTable.searchLocation(i).getValue());
}
dos.close();
fs.close();
}
catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
So what I have done is write a file in txt like:
IN X
In Y
SUB X
STO Y
OUT Y
DC: X 0
DC: Y 0
And I wrote a compiler that has now converted this file into machine code so I have created a file for example called program.txt.ex and it contains a bunch of ####### or machine code and I did this using the SecondPass code above and now I need to write a load method that will allow me to load and run this file.
Here is my Run method
public void run(String filename) throws IOException
{
if (mem == null)
System.out.println("mem null");
if (filename == null)
System.out.println("filename null");
mem.loadFromFile(filename);
cpu.reset();
cpu.setMDR(mem.read(cpu.getMAR()));
cpu.fetch2();
while (!cpu.stop())
{
cpu.decode();
if (cpu.OutFlag())
OutPut.display(mem.read(cpu.getMAR()));
if (cpu.InFlag())
mem.write(cpu.getMDR(),in.getInt());
if (cpu.StoreFlag())
{
mem.write(cpu.getMAR(),in.getInt());
cpu.getMDR();
}
else
{
cpu.setMDR(mem.read(cpu.getMAR()));
cpu.execute();
cpu.fetch();
cpu.setMDR(mem.read(cpu.getMAR()));
cpu.fetch2();
}
}
}
The Run Method:
public void run(int mem)
{
cpu.reset();
cpu.setMDR(mem.read(cpu.getMAR()));
cpu.fetch2();
while (!cpu.stop())
{
cpu.decode();
if (cpu.OutFlag())
OutPut.display(mem.read(cpu.getMAR()));
if (cpu.InFlag())
mem.write(cpu.getMDR(),in.getInt());
if (cpu.StoreFlag())
{
mem.write(cpu.getMAR(),in.getInt());
cpu.getMDR();
}
else
{
cpu.setMDR(mem.read(cpu.getMAR()));
cpu.execute();
cpu.fetch();
cpu.setMDR(mem.read(cpu.getMAR()));
cpu.fetch2();
}
}
}
I notice that your loader does a single
dos.readInt();
...which will read a single integer value from your file. What you probably want to do is create a loop that reads ints until you hit the end-of-file on dos (which might more aptly be named dis, no?). You could add those ints to a dynamic container like an ArrayList, which will grow with every element you stuff into it. Once done loading, you can use toArray to copy all those ints to an array of the appropriate size.
If seems that you need to load the whole file in memory before starting execution, so it would go:
public int[] load(String newfile) throws FileNotFoundException
{
int mem[] = new int[100];
try {
File file = new File(newfile);
FileInputStream fs = new FileInputStream(file);
DataInputStream dis = new DataInputStream(fs);
for (int i = 0; i < mem.length; ++i) {
mem[i] = dis.readInt();
}
dos.readInt();
dos.close();
} catch (IOException e) {
e.printStackTrace();
}
return mem;
}
void run(int mem[]) {
// now execute code
int pc = 0;
loop: while (true) {
int inst = mem[pc++];
int opcode = inst/100;
int loc = inst%100;
switch (opcode) {
case OpCode.STOP:
break loop;
case OpCode.IN:
...
}
}
}