Develop Reference

How can i create a program based on a web page?

I'm trying to create a program based on the web application of WhatsApp. I'm trying to figure out which is the best programming language to start this kind of programs. I've for example tried it with java but with this implementation:
public UrlReader() throws IOException {
try {
URL whatsApp = new URL("https://web.whatsapp.com/");
DataInputStream dis = new DataInputStream(whatsApp.openStream());
String inputLine;
while ((inputLine = dis.readLine()) != null) {
System.out.println(inputLine);
}
dis.close();
} catch (MalformedURLException me) {
System.out.println("MalformedURLException: " + me);
} catch (IOException ioe) {
System.out.println("IOException: " + ioe);
}
}
that is only a basic copy and paste from the oracle website. The output of this program is a site that says me that I have to use browser like chrome. Is there a better way to create programs like this?

You can start with Python to play with web.whatsapp.com. I assume you are trying to send a message on WhatsApp using code.
In Python, you can do it the same way we do it with mobile application
web.open('https://web.whatsapp.com/send?phone='+phone_no+'&text='+message)
This will prepopulate the text for given mobile number(Enter the phone_no as CountryCode and the number eg +918888888888)
Then using pyautogui you can press enter onto whatsapp.web
Working code :
def sendwhatmsg(phone_no, message, time_hour, time_min):
'''Sends whatsapp message to a particulal number at given time'''
if time_hour == 0:
time_hour = 24
callsec = (time_hour*3600)+(time_min*60)
curr = time.localtime()
currhr = curr.tm_hour
currmin = curr.tm_min
currsec = curr.tm_sec
currtotsec = (currhr*3600)+(currmin*60)+(currsec)
lefttm = callsec-currtotsec
if lefttm <= 0:
lefttm = 86400+lefttm
if lefttm < 60:
raise Exception("Call time must be greater than one minute")
else:
sleeptm = lefttm-60
time.sleep(sleeptm)
web.open('https://web.whatsapp.com/send?phone='+phone_no+'&text='+message)
time.sleep(60)
pg.press('enter')
I've taken this from this repository - Github repo

Downloading an image in java

I have to download an image from the nasa website. Problem is, that my code sometimes works, sucessfully downloading an image, while sometimes saves only 186B (don't know why exactly 186).
Problems is for sure connected with the way nasa sahres those photos. For instance, an image from that link https://mars.jpl.nasa.gov/msl-raw-images/msss/00001/mcam/0001ML0000001000I1_DXXX.jpg is saved sucessfully, while from that link https://mars.nasa.gov/mer/gallery/all/2/f/001/2F126468064EDN0000P1001L0M1-BR.JPG fails.
Here is my code
public static void saveImage(String imageUrl, String destinationFile){
URL url;
try {
url = new URL(imageUrl);
System.out.println(url);
InputStream is = url.openStream();
OutputStream os = new FileOutputStream(destinationFile);
byte[] b = new byte[2048];
int length;
while ((length = is.read(b)) != -1) {
os.write(b, 0, length);
}
is.close();
os.close();
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
Does someone have an idea, why is doesn't work?
public boolean downloadPhotosSol(int i) throws JSONException, IOException {
String url0 = "https://api.nasa.gov/mars-photos/api/v1/rovers/spirit/photos?sol=" + this.chosenMarsDate + "&camera=" + this.chosenCamera + "&page=" + i + "&api_key=###";
JSONObject json = JsonReader.readJsonFromUrl(url0);
if(json.getJSONArray("photos").length() == 0) return true;
String workspace = new File(".").getCanonicalPath();
String pathToFolder = workspace+File.separator+this.getManifest().getName() + this.chosenMarsDate + this.chosenCamera +"Strona"+i;
new File(pathToFolder).mkdirs();
for(int j = 0;j<json.getJSONArray("photos").length();j++) {
String url = ((JSONObject) json.getJSONArray("photos").get(j)).getString("img_src");
SaveImage.saveImage(url, pathToFolder+File.separator+"img"+j+".jpg");
}
return false;
}

When you get a 186 byte file, open it with a text editor and see what is inside. It could contain an HTTP error message in HTML format. If instead you see the first 186 bytes of your image file, then something is not working right with your program.
EDIT: From your comments it looks like you are getting an HTTP 301 response, which is a redirect to another location. A web browser handles this automatically without you noticing. However, your Java program is not following the redirect to the new location. You need to use an HTTP Java library that handles redirects.

Best and short way of doing it:
try(InputStream in = new URL("http://example.com/image.jpg").openStream()){
Files.copy(in, Paths.get("C:/File/To/Save/To/image.jpg"));
}

Cannot write output after reading input

I'm writing a program that connects to a servlet thanks to a HttpURLConnection but I stuck while checking the url
public void connect (String method) throws Exception {
server = (HttpURLConnection) url.openConnection ();
server.setDoInput (true);
server.setDoOutput (true);
server.setUseCaches (false);
server.setRequestMethod (method);
server.setRequestProperty ("Content-Type", "application / xml");
server.connect ();
/*if (server.getResponseCode () == 200)
{
System.out.println ("Connection OK at the url:" + url);
System.out.println ("------------------------------------------- ------- ");
}
else
System.out.println ("Connection failed");
}*/
I got the error :
java.net.ProtocolException: Cannot write output after reading input.
if i check the url with the code in comments but it work perfectly without it
unfortunately, I need to check the url so i think the problem comes from the getResponseCode method but i don t know how to resolve it
Thank you very much

The HTTP protocol is based on a request-response pattern: you send your request first and the server responds. Once the server responded, you can't send any more content, it wouldn't make sense. (How could the server give you a response code before it knows what is it you're trying to send?)
So when you call server.getResponseCode(), you effectively tell the server that your request has finished and it can process it. If you want to send more data, you have to start a new request.
Looking at your code you want to check whether the connection itself was successful, but there's no need for that: if the connection isn't successful, an Exception is thrown by server.connect(). But the outcome of a connection attempt isn't the same as the HTTP response code, which always comes after the server processed all your input.

I think the exception is not due toprinting url. There should some piece of code which is trying to write to set the request body after the response is read.
This exception will occur if you are trying to get HttpURLConnection.getOutputStream() after obtaining HttpURLConnection.getInputStream()
Here is the implentation of sun.net.www.protocol.http.HttpURLConnection.getOutputStream:
public synchronized OutputStream getOutputStream() throws IOException {
try {
if (!doOutput) {
throw new ProtocolException("cannot write to a URLConnection"
+ " if doOutput=false - call setDoOutput(true)");
}
if (method.equals("GET")) {
method = "POST"; // Backward compatibility
}
if (!"POST".equals(method) && !"PUT".equals(method) &&
"http".equals(url.getProtocol())) {
throw new ProtocolException("HTTP method " + method +
" doesn't support output");
}
// if there's already an input stream open, throw an exception
if (inputStream != null) {
throw new ProtocolException("Cannot write output after reading
input.");
}
if (!checkReuseConnection())
connect();
/* REMIND: This exists to fix the HttpsURLConnection subclass.
* Hotjava needs to run on JDK.FCS. Do proper fix in subclass
* for . and remove this.
*/
if (streaming() && strOutputStream == null) {
writeRequests();
}
ps = (PrintStream)http.getOutputStream();
if (streaming()) {
if (strOutputStream == null) {
if (fixedContentLength != -) {
strOutputStream =
new StreamingOutputStream (ps, fixedContentLength);
} else if (chunkLength != -) {
strOutputStream = new StreamingOutputStream(
new ChunkedOutputStream (ps, chunkLength), -);
}
}
return strOutputStream;
} else {
if (poster == null) {
poster = new PosterOutputStream();
}
return poster;
}
} catch (RuntimeException e) {
disconnectInternal();
throw e;
} catch (IOException e) {
disconnectInternal();
throw e;
}
}

I have this problem too, what surprises me is that the error is caused by my added code System.out.println(conn.getHeaderFields());
Below is my code:
HttpURLConnection conn=(HttpURLConnection)url.openConnection();
conn.setRequestMethod("POST");
configureConnection(conn);
//System.out.println(conn.getHeaderFields()); //if i comment this code,everything is ok, if not the 'Cannot write output after reading input' error happens
conn.connect();
OutputStream os = conn.getOutputStream();
os.write(paramsContent.getBytes());
os.flush();
os.close();

I had the same problem.
The solution for the problem is that you need to use the sequence
openConnection -> getOutputStream -> write -> getInputStream -> read
That means..:
public String sendReceive(String url, String toSend) {
URL url = new URL(url);
URLConnection conn = url.openConnection();
connection.setDoInput(true);
connection.setDoOutput(true);
connection.sets...
OutputStreamWriter out = new OutputStreamWriter(conn.getOutputStream());
out.write(toSend);
out.close();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String receive = "";
do {
String line = in.readLine();
if (line == null)
break;
receive += line;
} while (true);
in.close();
return receive;
}
String results1 = sendReceive("site.com/update.php", params1);
String results2 = sendReceive("site.com/update.php", params2);
...

Java <-> Flash Socket - Policy File Issue

I know there are many people who already asked this Question, but in all the threads I read I couldn't find 1 solution for my problem (even if others had the same one, it didn't work for me).
As the Title says, I'm trying to connect from a Flash/SWF-Application to a small Java server I wrote via Sockets. It works fine offline (on the same machine), but as soon as I put the .swf on a Webspace and open it from there, Flash requests the Policy file from the server. There's nothing bad with that, but my problem is that Flash disconnects after (hopefully) getting the policy-file but doesn't reconnect again.
My server always opens a new Thread when a client connects, but that's not where the trouble is made, as I already tried it without opening a new Thread.
Here's my code:
while (true) {
connection = providerSocket.accept();
System.out.println("Incoming connection from " +
connection.getInetAddress().getHostName());
BufferedReader in = new BufferedReader(
new InputStreamReader(connection.getInputStream()));
String request = in.readLine();
if (request != null && request.contains("<policy-file-request/>")) {
System.out.println("Authorization request.");
PrintStream out = new PrintStream(connection.getOutputStream(), true);
out.println("<?xml version=\"1.0\"?><cross-domain-policy><!DOCTYPE cross-domain-policy SYSTEM \"http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd\"><allow-access-from domain=\"*\" to-ports=\"3002\" /></cross-domain-policy>\u0000");
out.flush();
System.out.println("AuthData sent.");
connection.close();
System.out.println("Authorization complete.");
connection = providerSocket.accept();
System.out.println("TEST");
RequestProcessor c = new RequestProcessor(connection, connectionCounter++);
Thread t = new Thread(c);
t.start();
} else {
RequestProcessor c = new RequestProcessor(connection, connectionCounter++);
Thread t = new Thread(c);
t.start();
}
}
You will surely notice that I am using "\u0000" at the end instead of "\0", but don't worry, I also tested that case, didn't change anything. :/
I dont even reach the "TEST"-Ouput, because I don't get a new connection. And if I don't close the connection myself, flash automatically disconnects me.
The last thing I tried was just sending the xml without any request (right at the beginning, after the connection is established). I got a "recv failed" - Error for that.
PS: RequestProcessor is my new Thread, in which I would process the Strings/Commands sent from my.swf-File...
Thanks for helping me! :)

I had this problem before, you can not just use in.readLine() to get the policy file request string, because there're zero character.
To make sure you read the whole policy file request:
private String read(BufferedReader in) throws IOException {
StringBuilder builder = new StringBuilder();
int codePoint;
boolean zeroByteRead = false;
System.out.println("Reading...");
do {
codePoint = in.read();
if (codePoint == -1) {
return null;
}
if (codePoint == 0) {
zeroByteRead = true;
} else {
builder.appendCodePoint(codePoint);
}
} while (!zeroByteRead);
return builder.toString();
}
In the calling method:
BufferedReader in = new BufferedReader(new InputStreamReader(
clientSocket.getInputStream()));
String inputLine;
while ((inputLine = read(in)) != null) {
System.out.println("Receive from client: " + inputLine);
if ("<policy-file-request/>".equals(inputLine)) {
// Serve policy file, like the one in your question.
out.println(buildPolicy() +"\u0000");
} else {
// Do your job.
}
}
You can find the policy file project in java which can be downloaded. I myself thank to the guys over there.

java TCP socket message breaks

i have a java client-server app in java, both using the same connection class that contains both send/receive messages.
for some reason, some of the messages i send are received in a malformed order:
here's the code
//set up
_in = new BufferedReader(new InputStreamReader(this._socket.getInputStream()));
_out = new BufferedWriter(new OutputStreamWriter(this._socket.getOutputStream()));
this._socket.setSoTimeout(S_TIMEOUT);
public synchronized boolean send(String message){
try {
_out.write(message);
_out.write(Connection.DELIMITER);
_out.flush();
return true;
} catch (IOException e) {
}
return false;
}
public String receive(){
int c;
try {
String message = "";
System.out.println("Getting message:");
c = _in.read();
while(c != -1 && c != Connection.DELIMITER) {
message += (char) c;
c = _in.read();
}
if (c == -1) {
return null;
}
return message;
} catch (IOException e) { }
return null;
}
some messages, for example "new_order" will might return with "ew_ord".
some characters are lost, others are sent separately. this seems odd as its TCP
could this be an encoding related issue?
Delimiter is (char) 0
socket timeout is 20000 (ie 20 senconds). every 10 seconds i send an empty message to make sure socket does not close
EDIT:
although it was solved using the Scanner, i must say that the original code worked fine for many messages/various machines for a very long time (a few weeks), and then suddenly failed to work with one specific message on one specific machine (other messages went through just fine). i've done socket data transfer in java MANY times and i've written many read/write methods to handle the sockets. it's the first time i ran into this.
although in the original code i set the encoding (in the posted code i didn't), i believe that the problem was encoding related. at one point, the message that was received had every second character missing. afterwards i changed it a bit, and the first/second character of the message were received in a separate message. from my understanding, it's either an encoding issue or some firewall/other security program that was running on the message sender machine, that decided to filter outgoing packets.

Try replacing your receive with a Scanner and let it do the work for you.
// in your setup
Scanner sc = new Scanner(_in).useDelimiter(Connection.DELIMETER);
public String receive() {
try {
return sc.next();
} catch(IOException e) {
return "";
}
}

For starters, I would make sure you're printing exceptions in those catch blocks.
Then, you're using the platform default encoding for converting characters to bytes. If these two processes are running on different machines, it's possible they're using different encodings. I would make sure you're specifying an encoding when you set up the Reader and Writer.

You can use UTF encoding for getting Full String of Message.
U can try this code and I am Sure About this code because i used it in My Chat Application.
String data=" ";
socket = new Socket("localhost",999);
while(true)
{
dos = new DataOutputStream(socket.getOutputStream());
dis = new DataInputStream(socket.getInputStream());
data = dis.readUTF();
jta.append(data +"\n");
}
Where jta is JTextArea.
It's for Client Side
Now For Server Side:
try
{
server = new ServerSocket(999);
Socket soc = server.accept();
while(true)
{
String data="";
try
{
dis = new DataInputStream(soc.getInputStream());
dos = new DataOutputStream(soc.getOutputStream());
data = dis.readUTF();
}
catch(Exception e)
{ }
jta.append(data + "\n");
}
}
catch (IOException e)
{
JOptionPane.showMessageDialog(this, e);
System.exit(-1);
}