I am currently working on a homework assignment and I am thoroughly stuck. I am on the last question and I just can not figure out how to go about accomplishing the last task. Below is the tasks I had to complete:
The client should
save the file in the "client"
subdirectory of the home directory.
Test your program. Be sure it works
with binary files, not just text
files. Be sure it works when both
programs are on the same machine as
well as when they are separated over
the network.
Thus far when I start the server it asks what port I want to use. Then I start the client and it asks what IP and port to use. The the server immediately sends a list of the files in the home directory "server" folder. I then respond with the client with the file number I wish to download. This is where I get stuck. I can't seem to find any information about how to do this. So as you can see in my code posted below, I am trying to use a FileInputReader to convert the file to an array of bytes. Then I am sending that to the client. I am then trying to FileOutputReader the recieved array of bytes to a file. But I can't seem to find the correct methods to do that, or even if I am doing that correctly.
CLIENT
int i = 0;
while(i < 1000){
String modifiedSentence = inFromServer.readLine();
System.out.println("From Server: " + modifiedSentence);
i++;
}
while(j < 1000) {
int byteString = inFromServer.read();
ArrayList<byte[]> bytes = new ArrayList<byte[]>();
bytes.add(byteString);
}
Integer byteInt = new Integer(byteString);
FileOutputStream fo = new FileOutputStream(System.getProperty("user.home")+ "/client/text.txt");
fo.write(byteInt.byteValue());
}
}
SERVER
byte[] bytes = new byte[1024];
FileInputStream fi = new FileInputStream(file.toString() + fileArray[userChoiceInt]);
fi.read(bytes, 0, 1024);
outToClient.write(bytes, 0, 1024);
}
}
}
If anyone could offer any advice or the correct classes or methods to use I would appreciate it.
Thank you in advance.
Without spoiling the whole thing here's some hints.
This can be easily accomplish by using Socket (Server & Client). Using byte[] for transfering the file(s) will ensure that your program will work with both ascii and binary file(s).
Another approach would be to use the build in Remote Method Invocation (RMI). I haven't transfered file using this approach but I'm sure it's feasible.
And in case you didn't know, getting the user home directory is accomplished with the following call: System.getProperty( "user.home" );
You can take a look at this tutorial by Sun (Oracle). That should give you a basic understanding of sockets.
What I do seem to notice however, in the client side, you iterate a specific amount of times (1000) which is not a good idea since generally, you do not know the size of the file to be sent, the tutorial should show this and how to make the appropriate changes. Also, you keep creating the structures within the loop, so you loose any information that you have received, besides creating new datastructures each and every time, which is inefficient.
What you have to do is to move the initialization of the structures from outside the loop. Also, in the case of the modifiedSentence variable, you might want to change it from string to StringBuilder.
If you can use an IO library for this, I would suggest Netty or Mina. There are some netty examples here: http://jboss.org/netty/documentation.html
Even if you cannot use a library, these may be helpful for learning how things are done.
You probably should not ask how to do homework in the class on Web sites like this. It is not appropriate. Your server does look mostly good. Your program will only read files up to 1024 bytes though. You should look at java.io.File in more detail. There is a length method you can use to find the length of the file, so you know how much to send.
Related
I have studies project. I my teacherd don't want to tell me how to solve problem with receive multiple files. I know I need to use function getInputStream() but I don't know how to split those files in this inputStream object. I need to split this inputStream beacuse I need to save each file in folder.
Thank you for your help and for explaining this problem to me.
The answer is that you probably need a transmission protocol like HTTP or FTP. But if you don't want something that high level, what you can do is tar and then gzip your files, which is what people did on unix back in the day. Tar is still basically a transmission protocol, but maybe not as heavyweight as HTTP or FTP
It sounds like your instructor wants you to create a protocol. The reason you will need a protocol is that if you send multiple files across the same socket you wont know when one file stops and another begins. To simplify the problem I will use a simple chat application as example, but the same will apply to files.
Lets say you have a chat app which has only 2 users (one server to client). Each user can send a message of any length. Lets say User1 wants to send User2 the following messages (each line is one message)
Hello User
How are you doing today?
If you send each of those raw messages across the socket you would likely get
Hello UserHow are you doing today?. Now how do you know where one message started and another stopped?
Simple solution is to send something before each message stating a length of characters in the upcoming message, so your message might be
11Hello User24How are you doing today?
So the end user knows that I read an int which tells me <length>, then read <length> characters to get a full message.
Now thats a pretty basic example and not super great. Lets look at a simple packet format I have seen used in a video game:
Field Name Field Type Notes
Length VarInt Length of packet data + length of the packet ID
Packet ID VarInt
Data Byte Array Depends on the connection state and packet ID, see the sections below
This is the basic format all information between the client and server uses. A length of data to be read, a packet type followed by its data for that packet type.
For your use case you likely need something similar, some sort of meta data about the bytes you are sending. EG: Length of file, file name.
I would start by looking at the DataInputStream class for easily reading primitive data types.
I'm doing a project for the university. The application that I´m doing is a UDP and TCP Client/Server in Java and we have to measure the time that takes send some data. The question that I have, is the following one:
To make the app faster, I would like to know if there is any way to send random data in a buffer, I mean, I do not want to use one of my files in my computer, I just want to send for example 500 bytes of data, but I do not mind what. I know you can to do it from the terminal in Linux directly, or with Iperf, but I do not know how to implement that in my Java app.
I'll answer the first question, having no idea about MTU and MSS.
To create random data and send it, you simply have to use the following:
Random random = new Random();
byte[] data = new byte[500];
random.nextBytes(data); // fill with data
try (OutputStream out = ...) {
out.write(data);
}
My code has to read a portion of data from each file it loads, typically about 100,000 bytes. This works fine for local files but can be really slow over my wifi network, even though my network seems adequate (but not blistering fast)
So I created this simple test:
public void testDataCopySpeed() throws Exception
{
File file = new File("Z:\\TestFile.mp3");
System.out.println("start:"+new Date());
FileChannel fc = new FileInputStream(file).getChannel();
ByteBuffer bb = ByteBuffer.allocate(500000); //1/2 MB
fc.read(bb);
System.out.println("end:"+new Date());
}
Would take less than a second on a local file, but over a minute on a networked file.
So I then tried to test my network speed, I cannot see how to just test the wifi but I tested the internet upload/download speed using http://myspeedtestonline.com/ assuming this would be slower than my actual wifi network. It gave me:
Download Speed:512KB/second
Upload Speed :40KB/second
and I ran the same test on another computer and it gave a similar speed
So how is it I can download 1/2 MB of data in one second but it can take a minute to copy 1/2MB of data from one file in Java, the file is hosted on a NAS. ?
EDIT:So I have a couple of good answers below, what I really want to know is what is the best way to get access to the first 100,000 bytes from a set of files for read only access that will work for local and networked files, or should I have different code depending on whether or not it is not networked. Fixing the network is not really the solution, I may be able to fix my network but this software has to work on any computer, many of my customer may not have optiminal networks and would not have the skill to fix their network issues.
Can you try a memory mapped file?
File file = new File("Z:/TestFile.mp3");
System.out.println("start:"+new Date());
FileChannel fc = new FileInputStream(file).getChannel();
MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
System.out.println("end:"+new Date());
This might only appear faster, or it may help hide how long it takes.
You are comparing apples with oranges here. When you access http://myspeedtestonline.com/, the flash plugin is probably using the HTTP protocol, but certainly not CIFS.
When you address a file on a NAS, it's most probably using the CIFS protocol. This protocol is known to have performance problems, especially when implemented on consumer appliances (Buffalo drives, etc.).
Sometimes the MTU size is too big, causing the packets to be fragmented and resent.
So my guess is that Java is not the right address to blame in this case. In any case however, you cannot analyze the problem with a simple Java program. You should use a network sniffer for this.
I want to execute this command:
/ceplinux_work3/myName/opt/myCompany/ourProduct/bin/EXECUTE_THIS -p cepamd64linux.myCompany.com:19021/ws1/project_name < /ceplinux_work3/myName/stressting/Publisher/uploadable/00000.bin >> /ceplinux_work3/myName/stressting/Publisher/stats/ws1.project_name.19021/2011-07-22T12-45-20_PID-2237/out.up
But it doesn't work because EXECUTE_THIS requires an input file via redirect, and simply passing this command to Runtime.exec doesn't work.
Side note: I searched all over on how to solve this before coming here to ask. There are many questions/articles on the web regarding Runtime.exec and Input/Output redirect. However, I cannot find any that deal with passing a file to a command and outputting the result to another file. Plus, I am totally unfamiliar with Input/Output streams, so I have a hard time putting all the info out there together for my specific situation.
That said, any help is much appreciated.
P.S. If there are multiple ways to do this, I prefer whatever is fastest in terms of throughput.
Edit: As discussed in my last question, I CANNOT change this to a bash call because the program must wait for this process to finish before proceeding.
Unless you are sending a file name to the standard input of the process, there is no distinction of whether the data came from a file or from any other data source.
You need to write to the OutputStream given by Process.getOutputStream(). The data you write to it you can read in from a file using a FileInputStream.
Putting that together might look something like this:
Process proc = Runtime.getRuntime().exec("...");
OutputStream standardInputOfChildProcess = proc.getOutputStream();
InputStream dataFromFile = new FileInputStream("theFileWithTheData.dat");
byte[] buff = new byte[1024];
for ( int count = -1; (count = dataFromFile.read(buff)) != -1; ) {
standardInputOfChildProcess.write(buff, 0, count);
}
I've left out a lot of details, this is just to get the gist of it. You'll want to safely close things, might want to consider buffering and you need to worry about the pitfalls of Runtime.exec().
Edit
Writing the output to a file is similar. Obtain a FileOutputStream pointing to the output file and write the data you read from Process.getInputStream() to that OutputStream. The major caveat here is that you must do this operation in a second thread, since accessing two blocking streams from the same thread will lead to deadlock (see the article above).
i wrote a bit of code that reads download links from a text file and downloads the videos using the copyURLToFile methode from apaches commons-io library and the download is really slow when im in my wlan.
when i put in an internet stick is is about 6 times faster although the stick got 4mbit and my wlan is 8 mbit.
i also tried to do it without the commons-io library but the problem is the same.
normally im downloading 600-700 kb/s in my wlan but with java it only downloads with about 50 kb/s. With the internet stick its about 300 kb/s.
Do you know what the Problem could be?
thanks in advance
//Edit: Here is the code but i dont think it has anything to do with this and what do you mean with network it policies?
FileInputStream fstream = new FileInputStream(linksFile);
DataInputStream in = new DataInputStream(fstream);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
String link;
String name;
while ((link = br.readLine()) != null) {
name = br.readLine();
FileUtils.copyURLToFile(new URL(link), new File("videos/"+name+".flv"));;
System.out.println(link);
}
This isn't likely to be a Java problem.
The code you've posted actually doesn't do any IO over the network - it just determines a URL and passes it to (presumably Apache Commons') FileUtils.copyURLToFile. As usual with popular third-party libraries, if this method had a bug in it that caused slow throughput in all but the most unusual situations, it would already have been identified (and hopefully fixed).
Thus the issue is going to lie elsewhere. Do you get the expected speeds when accessing resource through normal HTTP methods (e.g. in a browser)? If not, then there's a universal problem at the OS level. Otherwise, I'd have a look at the policies on your network.
Two possible causes spring to mind:
The obvious one is some sort of traffic shaping - your network deprioritises the packets that come from your Java app (for an potentially arbitrary reason). You'd need to see hwo this is configured and look at its logs to see if this is the case.
The problem resides with DNS. If Java's using a primary server that's either blocked or incredibly slow, then it could take up to a few seconds to convert that URL to an IP address and begin the actual transfer. I had a similar problem once when a firewall was silently dropping packets to one server and it took three seconds (per lookup!) for the Java process to switch to the secondary server.
In any case, it's almost certainly not the Java code that's at fault.
The FileUtils.copyURLToFile internals uses a buffer to read.
Increasing the value of the buffer could speed up the download, but that seems not possible.