I am trying to read the first line of a URL.
Then i want to use that as a string later in the code.
Anyone can help me?
I already tried it with
public static String main(String[] args) {
try {
URL url = new URL("myurlhere");
// read text returned by server
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
String line;
while ((line = in.readLine()) != null) {
return line;
}
in.close();
}
catch (MalformedURLException e) {
System.out.println("Malformed URL: " + e.getMessage());
}
catch (IOException e) {
System.out.println("I/O Error: " + e.getMessage());
}
return null;
}
I just can't get a string out of it.
You can consider using jsoup for your purpose:
try {
Document doc = Jsoup.connect("http://popofibo.com/pop/swaying-views-of-our-past/").get();
Elements paragraphs = doc.select("p");
for(Element p : paragraphs) {
System.out.println(p.text());
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Output:
It is indeed difficult to argue over the mainstream ideas of evolution of human civilizations...
If you want to read from a file on the internet using a URL you should use URLConnection
here is a simple example:
String string = "";
try {
URLConnection connection = new URL(
"http://myurl.org/mypath/myfile")
.openConnection();
Scanner scanner = new Scanner(connection.getInputStream());
while (scanner.hasNext()) {
string += scanner.next() + " ";
}
scanner.close();
} catch (IOException e) {
e.printStackTrace();
}
// Do something with the string.
Related
I need to do soap request to web service. There are only 2 functions, so I decided to use simple HttpURLConnection so speed up development.
Here is the test code(don't be afraid of try/catch, it's just a test).
HttpURLConnection connection = null;
URL url = null;
try {
url = new URL("http://doc.ssau.ru/ssau_biblioteka_test/ws/DspaceIntegration.1cws");
} catch (MalformedURLException e) {
e.printStackTrace();
}
try {
connection = (HttpURLConnection)url.openConnection();
} catch (IOException e) {
e.printStackTrace();
}
connection.setRequestProperty("Authorization", "Basic d2Vic2VydmljZTp3ZWJzZXJ2aWNl");
connection.setRequestProperty("Content-Type", "application/json;charset=UTF-8");
connection.setRequestProperty("Accept-Encoding", "gzip,deflate");
try {
connection.setRequestMethod("POST");
} catch (ProtocolException e) {
e.printStackTrace();
}
connection.setDoOutput(true);
DataOutputStream wr = null;
try {
wr = new DataOutputStream(
connection.getOutputStream());
} catch (IOException e) {
e.printStackTrace();
}
String myString = "RU/НТБ СГАУ/WALL/Х62/С 232-948516";
byte bytes[] = new byte[0];
try {
bytes = myString.getBytes("UTF-8");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
String value = "";
try {
value = new String(bytes, "UTF-8");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
try {
wr.writeBytes("<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:imc=\"http://imc.parus-s.ru\">\n" +
" <soap:Header/>\n" +
" <soap:Body>\n" +
" <imc:GetRecordsInfo>\n" +
" <imc:Codes>RU/НТБ СГАУ/WALL/Х62/С 232-948516</imc:Codes>\n" +
" <imc:Separator>?</imc:Separator>\n" +
" <imc:Type>?</imc:Type>\n" +
" </imc:GetRecordsInfo>\n" +
" </soap:Body>\n" +
"</soap:Envelope>");
} catch (IOException e) {
e.printStackTrace();
}
try {
wr.close();
} catch (IOException e) {
e.printStackTrace();
}
InputStream is = null;
try {
is = connection.getInputStream();
} catch (IOException e) {
e.printStackTrace();
}
BufferedReader rd = new BufferedReader(new InputStreamReader(is));
StringBuilder response = new StringBuilder(); // or StringBuffer if not Java 5+
String line;
try {
while((line = rd.readLine()) != null) {
System.out.println(line);
}
} catch (IOException e) {
e.printStackTrace();
}
try {
rd.close();
} catch (IOException e) {
e.printStackTrace();
}
}
So here is the question:
When I do wirteByes with cyrillyc characters, like "RU/НТБ СГАУ/WALL/Х62/С 232-948516", the web service returns 500. If I use only latin or leave empty, then everthing is ok.
What is the right way to encode cyrillyc?
UPDATE
Problem solved, used this construction:
wr.write(new String("<soap:Envelope xmlns:soap=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:imc=\"http://imc.parus-s.ru\">\n" +
" <soap:Header/>\n" +
" <soap:Body>\n" +
" <imc:GetRecordsInfo>\n" +
" <imc:Codes>RU/НТБ СГАУ/WALL/Х62/С 232-948516</imc:Codes>\n" +
" <imc:Separator>?</imc:Separator>\n" +
" <imc:Type>?</imc:Type>\n" +
" </imc:GetRecordsInfo>\n" +
" </soap:Body>\n" +
"</soap:Envelope>").getBytes(charset));
Not exactly sure that what might be going wrong, there are a couple of things you might try.
1)Try a different encoding. Following code might help.
Charset charset = Charset.forName("UTF-16");
byte[] encodedBytes = myString.getBytes(charset);
2)Another thing you can try is guessing the character set of your string.(Please Note : there is no way to guess the character set 100% right every time)
byte[] thisAppCanBreak = "this app can break"
.getBytes("ISO-8859-1");
CharsetDetector detector = new CharsetDetector();
detector.setText(thisAppCanBreak);
String tableTemplate = "%10s %10s %8s%n";
System.out.format(tableTemplate, "CONFIDENCE",
"CHARSET", "LANGUAGE");
for (CharsetMatch match : detector.detectAll()) {
System.out.format(tableTemplate, match
.getConfidence(), match.getName(), match
.getLanguage());
}
This link might be helpful too
I i am able to read the lines from csv and download the images from url when the url is not having the image it is showing file not found exception in middle of the program i want to continue the program with out terminating.
public static void main(String[] args) throws IOException {
InputStream inputStream = null;
OutputStream outputStream = null;
try {
BufferedReader br = new BufferedReader(new FileReader("D:\\imgdwnld\\file.csv"));
String line = br.readLine();
while ((line = br.readLine()) !=null){
URL url = new URL(line);
inputStream = url.openStream();
outputStream = new FileOutputStream("D:\\imgdwnld\\" +
line.substring(line.lastIndexOf("/")));
byte[] buffer = new byte[2048];
int length;
while ((length = inputStream.read(buffer)) != -1) {
outputStream.write(buffer, 0, length);
}
}
} catch (MalformedURLException e) {
System.out.println("MalformedURLException :- " + e.getMessage());
} catch (FileNotFoundException e) {
System.out.println("FileNotFoundException :- " + e.getMessage());
} catch (IOException e) {
System.out.println("IOException :- " + e.getMessage());
} finally {
try {
inputStream.close();
outputStream.close();
} catch (IOException e) {
System.out.println("Finally IOException :- " + e.getMessage());
}
}
}
This is the relevant outline of your code:
try {
...
while ((line = br.readLine()) !=null) {
... process each CSV line ...
}
} catch (IOException e) {
... handle it ...
}
At the place where you catch the exception you have already broken out of the main loop. Change the code to have this outline:
while ((line = br.readLine()) !=null) {
try {
... process one CSV line ...
} catch (IOException e) {
... handle it, the loop will proceed with the next line
}
}
You need to remove the "return" instruction from the corresponding catch (or maybe from the whole code). In your case, the instruction allows to "exit" the main method so the rest of instruction (those after the return statement) won't be executed.
Another tip is to separate this instructions in blocs. For example, if reading the CSV and the Img are not bound, you may need to encapsulate each treatment in an individual try-catch block. When parsing/reading the CSV file fails, your code may continue fetching the image.
I want to add my Machine Name to the http request header as a custom
variable in JAVA .
Here is my code :
private void sendIssuesToWebService(Issue issue) {
// TODO Auto-generated method stub
System.out.println("\n\n");
//LOG.info("Sending issue : "+issue.getKey());
HttpURLConnection ycSend = null;
BufferedReader in1 = null;
JSONObject j = null ;
try{
URL urlSend = null;
try{
urlSend = new URL(targetURL+"Issue/ImportIssue");
}
catch (MalformedURLException e) {
// TODO Auto-generated catch block
LOG.severe(" URL Malformed Exception while authentication ");
e.printStackTrace();
}
try{
ycSend = (HttpURLConnection) urlSend.openConnection();
}
catch (IOException e) {
// TODO Auto-generated catch block
LOG.severe(" Error in Http URL Connection while authentication");
e.printStackTrace();
}
ycSend.setDoOutput(true);
ycSend.setRequestMethod("POST");
ycSend.setRequestProperty("Content-Type", "application/json");
try{
OutputStreamWriter outSend = null;
outSend = new OutputStreamWriter(ycSend.getOutputStream());
outSend.write(IssueAndHistoryJsonStr);
outSend.flush();
}catch(Exception e)
{
LOG.severe("Cannot write output stream while sending issue ");
}
System.out.println( ycSend.getResponseCode()+"\n\n\n");
if(ycSend.getResponseCode()!=200){
in1 = new BufferedReader(new InputStreamReader(
ycSend.getErrorStream()));
String inputLine;
while ((inputLine = in1.readLine()) != null)
inputResponse=inputResponse+inputLine;
//System.out.println("$$$$$$$"+inputResponse);
try{
j = new JSONObject(inputResponse);
LOG.info(j.get("errors").toString());
}
catch(Exception e){
String errorTitle = inputResponse.substring(inputResponse.indexOf("<title>")+7,inputResponse.indexOf("</title>"));
LOG.severe(errorTitle);
}
LOG.severe(" Error in Sending an Issue to the web service and the issue key is "+issue.getKey());
//LOG.severe("Error is : "+j.get("errors"));
}
else
{
LOG.info("Issue "+issue.getKey()+" Sent successfully" );
countIssues++;
System.out.println("\n\n");
}
//LOG.info("Issue ***** " +issue.getKey()+ " sent with response : "+inputResponse);
}
catch(Exception e)
{
LOG.severe(" Error in Sending an Issue to the web service and the issue key is "+issue.getKey());
//LOG.severe(yc.getResponseMessage());
//LOG.severe("Error is : "+j.get("errors"));
e.printStackTrace();
}
}
I believe you just want to add another requestProperty similar to content type.
ycSend.setRequestProperty("Machine-Name", MachineName);
Maybe this will help. Alternatively you can look at Apache HTTPClient for doing this.
For getting the host name you can do:
try {
InetAddress addr = InetAddress.getLocalHost();
// Get hostname
String hostname = addr.getHostName();
} catch (UnknownHostException e) {
//DO SOMETHING
}
I can pull the user's statuses with no problem with cURL, but when I connect with Java, the xml comes out truncated and my parser wants to cry. I'm testing with small users so it's not choke data or anything.
public void getRuserHx(){
System.out.println("Getting user status history...");
String https_url = "https://twitter.com/statuses/user_timeline/" + idS.rootUser + ".xml?count=100&page=[1-32]";
URL url;
try {
url = new URL(https_url);
HttpsURLConnection con = (HttpsURLConnection)url.openConnection();
con.setRequestMethod("GET");
con.setReadTimeout(15*1000);
//dump all the content into an xml file
print_content(con);
}
catch (MalformedURLException e) {
e.printStackTrace();
}
catch (IOException e) {
e.printStackTrace();
}
System.out.println("Finished downloading user status history.");
}
private void print_content(HttpsURLConnection con){
if(con!=null){
try {
BufferedReader br = new BufferedReader(new InputStreamReader(con.getInputStream()));
File userHx = new File("/" + idS.rootUser + "Hx.xml");
PrintWriter out = new PrintWriter(idS.hoopoeData + userHx);
String input;
while ((input = br.readLine()) != null){
out.println(input);
}
br.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
This request doesn't need auth. Sorry about my ugly code. My professor says input doesn't matter so my I/O is a trainwreck.
You have to flush the output stream when you write the content out. Did you flush or close the output stream?
My program can run with inputStream.toString(); but as you know thats not a good way to convert inputStream to String. So when I try to convert properly it hangs.
My methods are:
private void jButton2ActionPerformed(java.awt.event.ActionEvent evt) {
if(initialized && connected){
try{
sms.findOperator();
jTextArea2.append(sms.logString);
sms.logString = "";
}
catch(Exception e){
JOptionPane.showMessageDialog(null, "Failed to find operator!", "ERROR", JOptionPane.ERROR_MESSAGE);
}
}
else JOptionPane.showMessageDialog(null, "Cannot connect to the port specified!", "ERROR", JOptionPane.ERROR_MESSAGE);
// TODO add your handling code here:
}
This is the findOperator() method:
public void findOperator(){
send("AT+COPS?\r\n");
}
Here is send() method:
public void send(String cmd) {
try {
//Thread.sleep(200);
outputStream.flush();
outputStream.write(cmd.getBytes());
inputStream = serialPort.getInputStream();
//System.out.println(" Input Stream... " + inputStream.toString());
Thread.sleep(300);
logString += inputStreamtoString(inputStream);
}
catch(Exception e){
e.printStackTrace();
}
finally{
//logString += inputStream.toString()+ '\n';
// if(infoType == "msg") return "Input Stream... " + inputStream.toString()+ '\n';
// else return inputStream.toString();
//return logString;
//logString += inputStreamtoString(inputStream);
}
}
And this is the inputStreamtoString() method:
public String inputStreamtoString(InputStream is) throws IOException{
// try {
// return new java.util.Scanner(is).useDelimiter("\\A").next();
// } catch (java.util.NoSuchElementException e) {
// return "";
// }
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
StringBuilder sb = new StringBuilder();
String line = null;
try {
while ((line = reader.readLine()) != null) {
sb.append(line + "\n");
}
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
is.close();
} catch (IOException e) {
e.printStackTrace();
}
}
return sb.toString();
}
If I don't use inputStreamtoString() method and use inputStream.toString() my program runs well, but I don't get proper String. Any suggestion?
Thanks in advance...
Update: My modem uses the port COM3 with a GSM SIM card. I get a String containing huge space, like:
+COPS: <...500 spaces...> 0,0,"Banglalink"
So I hate that spaces. I need a string : +COPS: 0,0,"Banglalink"
You are trying to read exhaustively (that is until all data is read) from a stream that is connected to a serial port. This will hang if there is no data available on the port (waiting for data to come). Even if there is data, you will have an infinite loop.
UPDATE:
You could try something like this instead (adapted from code listed here):
byte[] readBuffer = new byte[200];
try {
while (is.available() > 0) {
int numBytes = is.read(readBuffer);
sb.append(new String(readBuffer, "US-ASCII"));
}
} catch (IOException e) {
// handle exception
}
UPDATE: chaged the string creation to use a specified charset (instead of the system default)