I was trying to make a java program to copy a file with the reader in one thread and the writer in another thread. The program works without showing any errors but the copied file has a different checksum from the original file. I am not able to find the mistake i made in the program. The program copies text files and they dont seem to be having any problems. When i try copying a zip file, copying finishes successfully, but when i try to extract the copied zip, it shows that the zip file is corrupted. Can someone help me in finding the mistake.
import java.io.*;
import java.util.concurrent.*;
import java.util.*;
class qu{
qu(byte[] b,int s){
this.b=b;
this.s=s;
}
byte[] b;
int s;
public String toString(){
return s+"";
}
}
class reader implements Runnable{
public byte[] b = new byte[1048576];
String inf;
int s;
long max;
private BlockingQueue<qu> blockingQueue;
public reader(String inf,BlockingQueue<qu> blockingQueue,long max){
this.inf=inf;
this.max=max;
this.blockingQueue=blockingQueue;
}
public void run(){
RandomAccessFile in=null;
try{
in = new RandomAccessFile(inf,"rw");
while((s=in.read(b))!=-1){
blockingQueue.put(new qu(b,s));
}
}catch(Exception e){System.out.println(e);}
finally{
try{
in.close();
}catch(Exception e){System.out.println(e);}
}
}
}
class writer implements Runnable{
public byte[] b = new byte[1048576];
String outf;
int s=0;
long max=0;
private BlockingQueue<qu> blockingQueue;
public writer(String outf,BlockingQueue<qu> blockingQueue,long max){
this.outf=outf;
this.max=max;
this.blockingQueue=blockingQueue;
}
public void run(){
RandomAccessFile out;
qu asd;
System.out.println("Copying..");
try{
out = new RandomAccessFile(outf,"rw");
while(out.getFilePointer()!=max){
asd=blockingQueue.take();
//System.out.println(new String(asd.b));
out.write(asd.b,0,asd.s);
}
out.close();
}catch(Exception e){System.out.println(e);}
}
}
class mul {
public static void main(String[] args) {
String inf = args[0];
String outf = args[1];
File file =new File(inf);
long max = file.length();
BlockingQueue<qu> blockingQueue = new ArrayBlockingQueue<>(64);
if(file.exists()){
long start_time = System.nanoTime();
Thread t1=new Thread(new reader(inf,blockingQueue,max));
t1.start();
Thread t2=new Thread(new writer(outf,blockingQueue,max));
t2.start();
try{
t1.join();
t2.join();
}catch(Exception ex){System.out.println(ex);}
long end_time = System.nanoTime();
double difference = (end_time - start_time) / 1e6;
System.out.println("Time taken: "+difference+"ms");
System.out.println("Copy Completed..");
}else{System.out.println("File not Found");}
}
}
You need to copy your byte array e.g.
public class Qu {
private final byte[] b;
private final int s;
public Qu(byte[] b,int s) {
this.b = Arrays.copyOf(b, b.length);
this.s = s;
}
public String toString(){
return s+"";
}
}
Related
I was given an assignment to write all ordered contents of given files into a result.txt. At first, the filenames are split into different Arraylists where each file contains a label in a format #n/N where N is the total number of files. e.g.
British explorer James Clark Ross led the first
expedition to reach the north magnetic pole
#001/004
from a file 1831-06-01.txt
The problem with my code is that it has written in order 1,4,2,3 respectively. However, the result must be in order 1,2,3,4. This may be due to a lack of synchronization. Nonetheless, I am still struggling to fix the problem.
This is my code:
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.*;
class PopThread implements Runnable {
ArrayList<String> fileList;
public PopThread(ArrayList<String> fileList) {
this.fileList = fileList;
}
#Override
public void run() {
//System.out.println("running\n");
Thread.currentThread().setPriority(Thread.MIN_PRIORITY);
long startTime = System.nanoTime();
System.out.println("fileList: " + fileList);
ArrayList<String> sortedFileList = sortFiles(fileList);
File resultFile = new File("result.txt");
for (String filename : sortedFileList) {
Writer w1 = new Writer(filename, resultFile);
Thread t = new Thread(w1);
t.setPriority(Thread.MAX_PRIORITY);
t.start();
}
long stopTime = System.nanoTime();
//System.out.println("Total execution time: " + (stopTime - startTime));
}
public ArrayList<String> readFiles(String filename) {
ArrayList<String> list = new ArrayList<String>();
try {
File myObj = new File(filename);
Scanner s = new Scanner(myObj);
while (s.hasNext()) {
list.add(s.next());
}
s.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
}
return list;
}
public int getNumber(String filename) {
String lastLine = "";
String sCurrentLine;
int identifier_integer = -1;
try {
BufferedReader br = new BufferedReader(new FileReader(filename));
while ((sCurrentLine = br.readLine()) != null) {
lastLine = sCurrentLine;
}
String identifier_number = lastLine.substring(1,4);
identifier_integer = Integer.parseInt(identifier_number);
} catch (FileNotFoundException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
return identifier_integer;
}
public ArrayList<String> sortFiles(ArrayList<String> listFileName) {
int i = listFileName.size();
boolean sorted = false;
while ( (i > 1) && (!(sorted)) ) {
sorted = true;
for (int j = 1; j < i; j++) {
if ( getNumber(listFileName.get(j-1)) > getNumber(listFileName.get(j)) ) {
String temp = listFileName.get(j-1);
listFileName.set(j-1, listFileName.get(j));
listFileName.set(j, temp);
sorted = false;
}
}
i--;
}
return listFileName;
}
}
class Writer implements Runnable {
String filename;
File resultFile;
public Writer(String filename, File resultFile) {
this.filename = filename;
this.resultFile = resultFile;
}
#Override
public void run() {
String content;
content = readFromFile(filename);
writeToFile(resultFile, content);
}
private static void writeToFile(File resultFile, String content) {
try {
BufferedWriter writer = new BufferedWriter(new FileWriter(resultFile, true));
writer.write(content);
//writer.write("file content written");
writer.flush();
} catch (IOException e) {
e.printStackTrace();
}
}
static String readFromFile(String filename) {
StringBuffer content = new StringBuffer();
try {
String text;
BufferedReader reader = new BufferedReader(new FileReader(filename));
while ((text = reader.readLine()) != null) {
content.append(text);
content.append("\n");
}
} catch (FileNotFoundException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
return content.toString();
}
}
public class q4 {
public static void main(String[] args) {
ArrayList<String> filesOne = new ArrayList<String>();
filesOne.add("1831-06-01.txt");
filesOne.add("2003-08-27.txt");
ArrayList<String> filesTwo = new ArrayList<String>();
filesTwo.add("1961-04-12.txt");
filesTwo.add("1972-12-11.txt");
PopThread popRunnableOne = new PopThread(filesOne);
PopThread popRunnableTwo = new PopThread(filesTwo);
Thread threadOne = new Thread(popRunnableOne);
Thread threadTwo = new Thread(popRunnableTwo);
threadOne.start();
threadTwo.start();
try {
threadOne.join();
threadTwo.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
( NOTE: The class q4 cannot be altered)
This assignment is horrible. You have my sympathy.
Your two threads will have to communicate with each other. Each thread will have to know, what is the filename that the other thread wants to output next. And, they will have to take turns. Each thread needs to loop:
While the date on my next file is less than or equal to the date on the other thread's next file, output my next file,
Tell the other thread, "it's your turn,"
If I have no more files, then exit (return from the run() method), otherwise, wait for the other thread to tell me it's my turn again,
Go back to step 1.
Having to take turns is the worst part of the assignment. Any time you find yourself needing to make threads take turns doing something—any time you need to make threads do things in a particular order—that's a clear sign that all of the things should be done by a single thread.
The only way threads can communicate is through shared variables. Your instructor has done you a huge disservice by telling you not to modify the q4 class. That prevents you from passing any shared objects in to your PopThread implementation through its constructor.
The only other way your two threads can share any variables is by making the variables static. Forcing you to use static is the second worst part of the assignment. If you go on to study software engineering, you will learn that static is an anti-pattern. Programs that use static variables are brittle (i.e., hard to modify), and they are hard to test.
Forcing you to use static variables also will make your threads do extra work to figure out who is who. Normally, I would do something like this so that each thread would automatically know which state is its own, and which belongs to the other guy:
class SharedState { ... }
class PopThread {
public PopThread(
SharedState myState,
SharedState otherThreadState,
ArrayList<String> fileList
) {
this.myState = myState;
this.otherGuyState = otherThreadState;
this.fileList = fileList;
...initialize this.myState...
}
...
}
class q4 {
public static void main(String[] args) {
SharedState stateOne = new SharedState();
SharedState stateTwo = new SharedState();
PopThread popRunnableOne = new PopThread(stateOne, stateTwo, filesOne);
PopThread popRunnableTwo = new PopThread(stateTwo, stateOne, filesTwo);
...
}
}
The best way I can think of with static variables would be to have an array of two SharedState, and have the threads use an AtomicInteger to each assign themself one of the two array slots:
class PopThread {
static SharedState[] state = new SharedState [2];
static AtomicInteger nextStateIndex = new AtomicInteger(0);
public PopThread(
SharedState myState,
SharedState otherThreadState,
ArrayList<String> fileList
) {
myStateIndex = nextStateIndex.getAndIncrement();
otherGuysStateIndex = myStateIndex ^ 1;
this.fileList = fileList;
...initialize state[myStateIndex]...
}
...
}
I have some problems with the connection between my EV3 and my Android-Device.
I am able to receive Data with my EV3, but I can't send.
I have signed up with my Github account if you like to see the App.
Here is my Code:
public class Bluetooth {
public static int speed = 100;
public static BTConnector connector;
public static NXTConnection connection;
public static void main(String[] args) throws IOException {
openConnection();
Thread moveWithBluetoothThread = new Thread(new moveWithBluetooth());
moveWithBluetoothThread.start();
}
public static void openConnection() {
connector = new BTConnector();
LCD.drawString("Waiting for Connecrion", 3, 1);
connection = connector.waitForConnection(0, NXTConnection.RAW);
LCD.clear();
LCD.drawString("Connected", 3, 5);
}
}
class moveWithBluetooth implements Runnable {
#Override
public void run() {
while (true) {
InputStream is = Bluetooth.connection.openInputStream();
BufferedReader dis = new BufferedReader(new InputStreamReader(is), 1);
OutputStream os = Bluetooth.connection.openOutputStream();
BufferedWriter dos = new BufferedWriter(new OutputStreamWriter(os), 1);
try {
byte[] b;
for (int i = 0; i < 100; i++) {
String n = dis.readLine();
System.out.println(n);
b = n.getBytes();
Bluetooth.connection.write(b, b.length);
Thread.sleep(10);
dos.write(n);
dos.flush();
}
dis.close();
dos.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
I found out, that there was no Problem with my EV3. That means, you can use this code. If you are having the same Problem, here is my solution:
The Problem was the Library I use in my Application. At the "Tssues"-Tab of Github was my Problem. The library waits until there is a complete Transmission, but the EV3 sends a permanent Stream, so I just had to add \r\n at the end of my Transmission. My final Code looked like that:
byte[] b = (n + "\r\n").getBytes();
Bluetooth.connection.write(b, b.length);
I use ObjectInput/Output to initialize the hashmap named temp and it put all entry of the hashmap called map that is initialized to new and then use OutputStream to save it in file formatting is .ser
this work perfectly...
import java.io.*;
import java.util.HashMap;
public class PlayerInfo implements Serializable {
ObjectOutputStream out;
ObjectInputStream in;
File userData =new File("path.ser");
HashMap map ;
HashMap temp;
private Integer ID;
String name ;
boolean isItNull =false;
public static void main(String[] args) {
new PlayerInfo();
}
PlayerInfo(){
try {
initializeHashMap();
}catch (Exception e){
e.printStackTrace();
}
}
#SuppressWarnings("unchecked")
private void initializeHashMap(){
try {
//initialize ObjectInputStream in same method when I use it and close it then
in =new ObjectInputStream(new FileInputStream(userData));
if (isItNull){
temp =new HashMap<Integer,PlayerInfo>();
}else {
map =new HashMap<Integer,PlayerInfo>();
temp = (HashMap<Integer, PlayerInfo>) in.readObject();
in.close();
}
}catch (Exception e){
isItNull =true;
initializeHashMap();
}
}
private void getInfo(){
System.out.println("Ok we are in get info so write your ID:-");
int id = 10;
}
#SuppressWarnings("unchecked")
private void createInfo()throws IOException{
//same here initialize ObjectOutputStreamin same method when I use it and close it then
out =new ObjectOutputStream(new FileOutputStream(userData));
System.out.println("Ok we are in create info so write your ID:-");
ID =10;
String scnS ="Mohammed";
System.out.println("Write your name");
map.put(ID,new PlayerInfo(scnS));
temp.putAll(map);
System.out.println("Saving....");
out.writeObject(temp);
out.close();
}
public PlayerInfo(String name){
this.name =name;
}
}
but this throw EFOException
import java.io.*;
import java.util.HashMap;
public class PlayerInfo implements Serializable {
ObjectOutputStream out;
ObjectInputStream in;
File userData =new File("path.ser");
HashMap map ;
HashMap temp;
private Integer ID;
String name ;
boolean isItNull =false;
public static void main(String[] args) {
new PlayerInfo();
}
PlayerInfo(){
try {
openTheOutPutObjectStreamer();
openTheInPutObjectStreamer();
initializeHashMap();
}catch (Exception e){
e.printStackTrace();
}
}
//here I initialize it in separated method
private void openTheOutPutObjectStreamer()throws IOException{
out =new ObjectOutputStream(new FileOutputStream(userData));
}
//same here I initialize it in separated method
private void openTheInPutObjectStreamer()throws IOException{
in =new ObjectInputStream(new FileInputStream(userData));
}
#SuppressWarnings("unchecked")
private void initializeHashMap(){
try {
if (isItNull){
temp =new HashMap<Integer,PlayerInfo>();
}else {
map =new HashMap<Integer,PlayerInfo>();
temp = (HashMap<Integer, PlayerInfo>) in.readObject();
in.close();
}
}catch (Exception e){
isItNull =true;
initializeHashMap();
}
}
#SuppressWarnings("unchecked")
private void createInfo()throws IOException{
System.out.println("Ok we are in create info so write your ID:-");
ID =10;
String scnS ="Mohammed";
System.out.println("Write your name");
map.put(ID,new PlayerInfo(scnS));
temp.putAll(map);
System.out.println("Saving....");
out.writeObject(temp);
out.close();
}
public PlayerInfo(String name){
this.name =name;
}
}
if you see it the difference is only separate the Object Input/Output to a method and call them
and I am sorry I am a newbie in this website
I don't know a lot about IO but it seems like I cant separate it to methods and call it?
The problem is that in your first code you (correctly) open an input stream uses it and then closes it before doing anything else to the same file but in your second code version you also open the output stream on the same file before having read it and that output stream puts the marker (where to read or write) at the end of the file so when you use your input stream you get an End of file error.
Changing you code to this should work
openTheInPutObjectStreamer();
initializeHashMap();
openTheOutPutObjectStreamer();
I'm writing an small application like IDM in java.
But this has has many Exceptions.
This is the code of Downloader class which implements runnable and I want use it for multithreading.
public class Downloader implements Runnable{
private DataInputStream inputStream;
private byte[][] fileData;
private int index;
private int size;
public Downloader(DataInputStream inputStream, byte[][] fileData, int index, int size) {
this.inputStream = inputStream;
this.fileData = fileData;
this.index = index;
this.size = size;
}
public synchronized void run() {
try{
inputStream.skipBytes(index * size);
for(int i= 0;i<size;i++){
fileData[index][i] = inputStream.readByte();
System.out.println("It works : " + index);
}
}
catch(Exception e){
System.out.println(e.getMessage());
}
}}
and this is my main class
public class Main {
public static void main(String[] args) {
String s;
//Scanner input = new Scanner(System.in);
//System.out.print("Enter file destination : ");
//s = input.nextLine();
s = "http://video.varzesh3.com/video/clip1/92/uclip/fun/gaf_6_borhani.mp4";
URL url;
URLConnection connection;
DataInputStream inputStream;
FileOutputStream outStream;
byte[][] fileData;
try{
url = new URL(s);
connection = url.openConnection();
inputStream = new DataInputStream(connection.getInputStream());
fileData = new byte[8][connection.getContentLength() / 4];
int size = connection.getContentLength() / 4;
Runnable d0 = new Downloader(inputStream, fileData, 0, size);
Runnable d1 = new Downloader(inputStream, fileData, 1, size);
Runnable d2 = new Downloader(inputStream, fileData, 2, size);
Runnable d3 = new Downloader(inputStream, fileData, 3, size);
Thread thread0 = new Thread(d0);
Thread thread1 = new Thread(d1);
Thread thread2 = new Thread(d2);
Thread thread3 = new Thread(d3);
thread0.start();
thread1.start();
thread2.start();
thread3.start();
inputStream.close();
String path = "C:\\Users\\NetTest\\Desktop\\test.mp4";
outStream = new FileOutputStream(new File(path));
outStream.write(fileData[0]);
/*outStream.write(fileData[1]);
outStream.write(fileData[2]);
outStream.write(fileData[3]);
outStream.write(fileData[4]);
outStream.write(fileData[5]);
outStream.write(fileData[6]);
outStream.write(fileData[7]);*/
outStream.close();
}
catch(Exception e){
System.out.println(e.getMessage());
}
}}
but when I run it this happens
It works: 0
null
null
null
null
What should I do now?
There are 2 Problems in your code.
At the current state you close() the InputStream from which all Thread try to read directly after you started them (-> while they are running). To solve this Problem you can call the join() Method of the Thread class. In your case your'd have to call it for all 4 Threads to make sure they are finished.
If I understand it correctly you want to seperate the download File into 4 parts downloading at the same time.
To do this you need 4 independent InputStreams. (Currently you are using ONE [See also: Java Object Copying])
So to change this your code would look something like this:
public class Downloader implements Runnable{
private byte[][] fileData;
private int index;
private int size;
private URL url;
public Downloader(URL url, byte[][] fileData, int index, int size) {
this.fileData = fileData;
this.index = index;
this.size = size;
this.url = url;
}
public synchronized void run() {
try{
URLConnection connection = url.openConnection();
DataInputStream inputStream = new DataInputStream(connection.getInputStream());
inputStream.skipBytes(index * size);
for(int i= 0;i<size;i++){
fileData[index][i] = inputStream.readByte();
System.out.println("It works : " + index);
}
}catch(Exception e){
System.out.println(e.getMessage());
}
}
}
Hello I have a problem wherein I have to read a huge csv file. remove first field from it, then store only unique values to a file. I have written a program using threads which implements producer-consumer pattern.
Class CSVLineStripper does what the name suggests. Takes a line out of csv, removes first field from every line and adds it to a queue. CSVLineProcessor then takes that field stores all one by one in an arraylist and checks if fields are unique so only uniques are stored. Arraylist is only used for reference. every unique field is written to a file.
Now what is happening is that all fields are stripped correctly. I run about 3000 lines it's all correct. When I start the program for all lines, which are around 7,00,000 + lines, i get incomplete records, about 1000 unique are not taken. Every field is enclosed in double-quotes. What is weird is that the last field in the file that is generated is an incomplete word and ending double quote is missing. Why is this happening?
import java.util.*;
import java.io.*;
class CSVData
{
Queue <String> refererHosts = new LinkedList <String> ();
Queue <String> uniqueReferers = new LinkedList <String> (); // final writable queue of unique referers
private int finished = 0;
private int safety = 100;
private String line = "";
public CSVData(){}
public synchronized String getCSVLine() throws InterruptedException{
int i = 0;
while(refererHosts.isEmpty()){
if(i < safety){
wait(10);
}else{
return null;
}
i++;
}
finished = 0;
line = refererHosts.poll();
return line;
}
public synchronized void putCSVLine(String CSVLine){
if(finished == 0){
refererHosts.add(CSVLine);
this.notifyAll();
}
}
}
class CSVLineStripper implements Runnable //Producer
{
private CSVData cd;
private BufferedReader csv;
public CSVLineStripper(CSVData cd, BufferedReader csv){ // CONSTRUCTOR
this.cd = cd;
this.csv = csv;
}
public void run() {
System.out.println("Producer running");
String line = "";
String referer = "";
String [] CSVLineFields;
int limit = 700000;
int lineCount = 1;
try {
while((line = csv.readLine()) != null){
CSVLineFields = line.split(",");
referer = CSVLineFields[0];
cd.putCSVLine(referer);
lineCount++;
if(lineCount >= limit){
break;
}
}
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("<<<<<< PRODUCER FINISHED >>>>>>>");
}
private String printString(String [] str){
String string = "";
for(String s: str){
string = string + " "+s;
}
return string;
}
}
class CSVLineProcessor implements Runnable
{
private CSVData cd;
private FileWriter fw = null;
private BufferedWriter bw = null;
public CSVLineProcessor(CSVData cd, BufferedReader bufferedReader){ // CONSTRUCTOR
this.cd = cd;
try {
this.fw = new FileWriter("unique_referer_dump.txt");
} catch (IOException e) {
e.printStackTrace();
}
this.bw = new BufferedWriter(fw);
}
public void run() {
System.out.println("Consumer Started");
String CSVLine = "";
int safety = 10000;
ArrayList <String> list = new ArrayList <String> ();
while(CSVLine != null || safety <= 10000){
try {
CSVLine = cd.getCSVLine();
if(!list.contains(CSVLine)){
list.add(CSVLine);
this.CSVDataWriter(CSVLine);
}
} catch (Exception e) {
e.printStackTrace();
}
if(CSVLine == null){
break;
}else{
safety++;
}
}
System.out.println("<<<<<< CONSUMER FINISHED >>>>>>>");
System.out.println("Unique referers found in 30000 records "+list.size());
}
private void CSVDataWriter(String referer){
try {
bw.write(referer+"\n");
} catch (Exception e) {
e.printStackTrace();
}
}
}
public class RefererCheck2
{
public static void main(String [] args) throws InterruptedException
{
String pathToCSV = "/home/shantanu/DEV_DOCS/Contextual_Work/excite_domain_kw_site_wise_click_rev2.csv";
CSVResourceHandler csvResHandler = new CSVResourceHandler(pathToCSV);
CSVData cd = new CSVData();
CSVLineProcessor consumer = new CSVLineProcessor(cd, csvResHandler.getCSVFileHandler());
CSVLineStripper producer = new CSVLineStripper(cd, csvResHandler.getCSVFileHandler());
Thread consumerThread = new Thread(consumer);
Thread producerThread = new Thread(producer);
producerThread.start();
consumerThread.start();
}
}
This is how a sample input is:
"xyz.abc.com","4432"."clothing and gifts","true"
"pqr.stu.com","9537"."science and culture","false"
"0.stu.com","542331"."education, studies","false"
"m.dash.com","677665"."technology, gadgets","false"
Producer stores in queue:
"xyz.abc.com"
"pqr.stu.com"
"0.stu.com"
"m.dash.com"
Consumer stores uniques in the file, but after opening file contents one would see
"xyz.abc.com"
"pqr.stu.com"
"0.st
Couple things, you are breaking after 700k, not 7m, also you are not flushing your buffered writer, so the last stuff you could be incomplete, add flush at end and close all your resources. Debugger is a good idea :)