I am trying to read text contents of a webpage ( based on source ), but seem to be unable to make the connection. What could be the issue?
EDIT: updated stack trace
I ran this is debug mode and see the following:
JP_FetchWebPageHeader(Object).<init>()
Source not found
I was able to single setp in spite of the above error. is this a error message an issue?
I'm using eclipse on 64 bit windows and Java 1.7
The code is :
// **** Fetch web page or header
import java.io.*;
import java.net.*;
import java.util.Scanner;
public final class JP_FetchWebPageHeader {
/**
* #param aArgs
* <ul>
* <li> aArgs[0] : an HTTP URL
* <li> aArgs[1] : (header | content)
* </ul>
*/
public static void main(String...aArgs) throws MalformedURLException {
String url = aArgs[0];
String option = aArgs[1];
JP_FetchWebPageHeader fetcher = new JP_FetchWebPageHeader(url);
if (HEADER.equalsIgnoreCase(option)) {
log(fetcher.getPageHeader());
}else if (CONTENT.equalsIgnoreCase(option)) {
log(fetcher.getPageContent());
}else {
log("Unknown option.");
}
}
public JP_FetchWebPageHeader(URL aURL) {
if( ! HTTP.equals(aURL.getProtocol())) {
throw new IllegalArgumentException("URL isnt for HTTP protocol: " + aURL);
}
fURL = aURL;
}
public JP_FetchWebPageHeader(String aUrlName ) throws MalformedURLException {
this(new URL(aUrlName));
}
// Fetch the HTML content of the web page as simple text
public String getPageContent() {
String result = null;
URLConnection connection = null;
try {
connection = fURL.openConnection();
Scanner scanner = new Scanner(connection.getInputStream());
scanner.useDelimiter(END_OF_INPUT);
result = scanner.next();
} catch (IOException ex ) {
log("Cannot open connection to " + fURL.toString());
}
return result;
}
//Fetch HTML headers as simple text
public String getPageHeader() {
return null;
}
//PRIVATE
private URL fURL;
private static final String HTTP = "http";
private static final String HEADER = "header";
private static final String CONTENT = "content";
private static final String END_OF_INPUT = "\\Z";
private static final String NEWLINE = System.getProperty("line.separator");
private static void log(Object aObject){
System.out.println(aObject);
}
}
Arguments: http://www.google.com content
Result :
Cannot open connection to http://www.google.com
java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
null
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at JP_FetchWebPageHeader.getPageContent(JP_FetchWebPageHeader.java:54)
at JP_FetchWebPageHeader.main(JP_FetchWebPageHeader.java:30)
Related
I can't work out why the following program is unable to locate tables in my derby database:
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;
public class NdConnect
{
public final static String SETUP_FILE_PATH = "/AppData/Local/NewdawnTest";
private static final String CONNECTION_URL = "jdbc:derby:" + "C:/Users/" + System.getenv("USERNAME") + SETUP_FILE_PATH + "db" + ";create=true";
private static Connection conn = null;
private final static Properties dbProperties = new Properties();
private static PreparedStatement pstmtSelectTxns;
public static void connect() {
try {
conn = DriverManager.getConnection(CONNECTION_URL + ";create=true", dbProperties);
pstmtSelectTxns = conn.prepareStatement("SELECT * from TXNS");
System.out.println("Connected OK to " + CONNECTION_URL);
}
catch (SQLException sqle) {
System.out.println("SQL exception");
System.out.println("Connected NOK:The connection URL is " + CONNECTION_URL);
Logger.getLogger(NdConnect.class.getName()).log(Level.SEVERE, null, sqle);
}
}
public static void disconnect()
{
try {
if (conn != null) {
conn.close();
conn = null;
System.out.println("0048 NDC OK:DB closed ");
}
} catch (SQLException sqle) {
Logger.getLogger(NdConnect.class.getName()).log(Level.SEVERE, null, sqle);
}
}
public static void main(String[] args)
{
// NdConnect nd = new NdConnect();
NdConnect.connect();
NdConnect.disconnect();
// System.out.println("NdConnect finished");
}
}
The program throws a SQLSyntaxErrorException:
run:
SQL exception
Connected NOK:The connection URL is jdbc:derby:C:/Users/Administrator/AppData/Local/NewdawnTestdb;create=true
Nov 25, 2018 6:58:42 AM NdConnect connect
0048 NDC OK:DB closed
SEVERE: null
java.sql.SQLSyntaxErrorException: Table/View 'TXNS' does not exist.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedPreparedStatement42.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver42.newEmbedPreparedStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.prepareStatement(Unknown Source)
at NdConnect.connect(NdConnect.java:21)
at NdConnect.main(NdConnect.java:47)
Caused by: ERROR 42X05: Table/View 'TXNS' does not exist.
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.impl.sql.compile.FromBaseTable.bindTableDescriptor(Unknown Source)
at org.apache.derby.impl.sql.compile.FromBaseTable.bindNonVTITables(Unknown Source)
at org.apache.derby.impl.sql.compile.FromList.bindTables(Unknown Source)
at org.apache.derby.impl.sql.compile.SelectNode.bindNonVTITables(Unknown Source)
at org.apache.derby.impl.sql.compile.DMLStatementNode.bindTables(Unknown Source)
at org.apache.derby.impl.sql.compile.DMLStatementNode.bind(Unknown Source)
at org.apache.derby.impl.sql.compile.CursorNode.bindStatement(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown Source)
at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown Source)
... 7 more
However the database and table do exist. I have verified this by creating a connection to the database in Netbeans > Services. When I run this command in Services:
SELECT * from TXNS
the desired table is correctly displayed and the output window returns:
Executed successfully in 0 s.
Fetching resultset took 0.016 s.
Line 1, column 1
Execution finished after 0.254 s, no errors occurred.
Looking at the properties of this connection via Netbeans services shows the following attribute values:
Display name NewDawn – TEST DB
Database URL jdbc:derby:C:\Users\Administrator\AppData\Local\NewdawnTest\db
Driver apache_derby_embedded
Driver class org.apache.derby.jdbc.EmbeddedDriver
This seems to correspond to the setup I have in the java code example so I don't understand why the code doesn't work.
The URL is not the same:
In Netbeans: jdbc:derby:C:\Users\Administrator\AppData\Local\NewdawnTest\db
In Java: jdbc:derby:C:/Users/Administrator/AppData/Local/NewdawnTestdb;create=true
Change the CONNECTION_URL line to:
private static final String CONNECTION_URL = "jdbc:derby:" + "C:/Users/" + System.getenv("USERNAME") + SETUP_FILE_PATH + "/db"
I am building a swing application to download multiple files over the internet and save to a windows fileshare. I have used SwingWroker which internally uses the ExecutorService which internally queues them and downloads 10 at a time, but for some reason after downloading say 2 - 3 MB of file it stops and moves to next downloading file, They are downloaded in a batch of 10 as SwingWorker has fixed it in number of Threads for the Executor Service.
I have to write these files in a windows file share and I am using nio.FileChannels to do that. There are files ranging from 50-60 each weighing around 300MB - 500MB. The file links are located on a webpage to where I get to by login in using credentials on a login page(with a post request) over the internet before that I specify CookieHandler.setDefault(new CookieManager()) at the beginning and so it behaves like a browser to me.
Another observation is when I download them locally (not to a windows server share) they do work fine.
This is the code I am using
import java.io.File;
import java.io.FileOutputStream;
import java.net.URL;
import java.net.URLConnection;
import java.nio.channels.Channels;
import java.nio.channels.FileChannel;
import java.nio.channels.ReadableByteChannel;
import javax.swing.SwingWorker;
public class DownloadProcess extends SwingWorker<Boolean, String> {
private String urlPath, filePath;
public DownloadProcess(String urlPath, String filePath){
this.urlPath = urlPath;
this.filePath = filePath;
}
#Override
protected Boolean doInBackground() {
boolean taskState = true;
URLConnection httpConn = null;
ReadableByteChannel readableByteChannel = null;
FileOutputStream fileOutputStream = null;
FileChannel fileOutputChannel = null;
try{
//String filePath = "\\\\fileshare.server\\xyz.txt";
//String urlPath = "http://example.com/anyBigFile.1GB.docx";
File localFile = new File(filePath);//File share
boolean itsThere = localFile!=null && localFile.exists();
long done = itsThere ? localFile.length() : 0;
URL url = new URL(urlPath);
httpConn = url.openConnection();
httpConn.setRequestProperty("Connection", "keep-alive");
if(itsThere) {
httpConn.setRequestProperty("Range","bytes="+done+"-");
}
readableByteChannel = Channels.newChannel(httpConn.getInputStream());
fileOutputStream = itsThere ? new FileOutputStream(filePath) : new FileOutputStream(filePath,true);
fileOutputChannel = fileOutputStream.getChannel();
for (long position = done, size = httpConn.getContentLength(); position < size && !isCancelled(); ) {
position += fileOutputChannel.transferFrom(readableByteChannel, position, 1 << 16);
}
//done
}catch(Exception e){
taskState = false;
e.printStackTrace();
}finally{
//close streams conns etc
}
return taskState;
}
}
This is the error stack trace that I get after 5 - 10 mins of download
/*
javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
at sun.security.ssl.AppInputStream.read(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.MeteredStream.read(Unknown Source)
at java.io.FilterInputStream.read(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown Source)
at java.nio.channels.Channels$ReadableByteChannelImpl.read(Unknown Source)
at com.objects.DownloadByteChannel.read(DownloadByteChannel.java:117)
at sun.nio.ch.FileChannelImpl.transferFromArbitraryChannel(Unknown Source)
at sun.nio.ch.FileChannelImpl.transferFrom(Unknown Source)
at com.core.DownloadTask.doInBackground(DownloadTask.java:154)
at com.core.DownloadTask.doInBackground(DownloadTask.java:59)
at com.util.ZSwingWorker$1.call(ZSwingWorker.java:286)
at java.util.concurrent.FutureTask.run(Unknown Source)
at com.util.ZSwingWorker.run(ZSwingWorker.java:325)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection reset
at sun.security.ssl.Alerts.getSSLException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
at sun.security.ssl.SSLSocketImpl.handleException(Unknown Source)
... 18 more
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at sun.security.ssl.InputRecord.readFully(Unknown Source)
at sun.security.ssl.InputRecord.read(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
... 18 more
*/
Usage:
public static void main(String[] args){
int counter = 1;
for(String url: urls){
new DownloadProcess(url,"\\\\fileshare.server\\xyz"+(counter++)+".txt").execute();
}
}
You are going to have to change your connection timeout serverside. I picked up a few links along the way if they are of any importance:
Modify Session Security settings
Lengthening salesforce session timeout
Hope this helps, good luck and let me know :)
Connection Reset means the remote side is closing the connection with a TCP RST (reset) packet. You need to find out what the remote side isn't liking and fix it.
If the remote side is Apache maybe you are running into the KeepAliveTimeout value. By default that is 5 seconds. It really sounds like you are running into some sort of configured limit on the remote side. When that happens the server is kicking you off with a reset.
I am working a on project which should render SSRS reports in Java.
Below is the procedure that I have followed.
Installed Eclipse
Created a java project By setting Execution Environment Java SE 1.8
Created a proxy using wsimport
wsimport image
Added the references for the generated proxy
added classed
Written this code
import javax.xml.ws.BindingProvider;
import javax.xml.ws.Holder;
import com.microsoft.schemas.sqlserver._2005._06._30.reporting.reportingservices.ArrayOfString;
import com.microsoft.schemas.sqlserver._2005._06._30.reporting.reportingservices.ArrayOfWarning;
import com.microsoft.schemas.sqlserver._2005._06._30.reporting.reportingservices.ExecutionHeader;
import com.microsoft.schemas.sqlserver._2005._06._30.reporting.reportingservices.ExecutionInfo;
import com.microsoft.schemas.sqlserver._2005._06._30.reporting.reportingservices.ReportExecutionService;
import com.microsoft.schemas.sqlserver._2005._06._30.reporting.reportingservices.ReportExecutionServiceSoap;
public class Render {
static {
java.net.Authenticator.setDefault(new java.net.Authenticator() {
#Override
protected java.net.PasswordAuthentication getPasswordAuthentication() {
return new java.net.PasswordAuthentication("username", "pwd".toCharArray());
}
});
}
public static void main(String[] args) {
String reportPath = "/Reporting/Customers" ;
String format = "HTML4.0";
String historyID = null;
String devInfo = "<DeviceInfo><Toolbar>False</Toolbar><HTMLFragment>True</HTMLFragment></DeviceInfo>";
String executionID = null;
Holder<String> extension = null;
Holder<String> mimeType = null;
Holder<String> encoding = null;
Holder<ArrayOfWarning> warnings = null;
Holder<ArrayOfString> streamIDs = null;
Holder<byte[]> result = new Holder<byte[]>();
ReportExecutionService res = new ReportExecutionService();
ReportExecutionServiceSoap ress = res.getReportExecutionServiceSoap();
BindingProvider bp = (BindingProvider)ress;
bp.getRequestContext().put(BindingProvider.SESSION_MAINTAIN_PROPERTY, true);
ExecutionInfo execInfo = new ExecutionInfo();
execInfo = ress.loadReport(reportPath, historyID);
executionID = execInfo.getExecutionID();
bp.getRequestContext().put("sessionID", executionID);
ExecutionHeader eh = new ExecutionHeader();
eh.setExecutionID(executionID);
System.out.println(executionID);
ress.render(format, devInfo, result, extension, mimeType, encoding, warnings, streamIDs);
String resultString = new String(result.value);
}
}
When I am running this code , i am getting the below error.
Exception in thread "main" com.sun.xml.internal.ws.fault.ServerSOAPFaultException: Client received SOAP Fault from server: The session identifier is missing. A session identifier is required for this operation. ---> Microsoft.ReportingServices.Diagnostics.Utilities.MissingSessionIdException: The session identifier is missing. A session identifier is required for this operation. Please see the server log to find more detail regarding exact cause of the failure.
at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(Unknown Source)
at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(Unknown Source)
at com.sun.xml.internal.ws.client.sei.StubHandler.readResponse(Unknown Source)
at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(Unknown Source)
at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(Unknown Source)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(Unknown Source)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(Unknown Source)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(Unknown Source)
at com.sun.proxy.$Proxy35.render(Unknown Source)
at Render.main(Render.java:50)
I am new to java, am I missing something ? I already set session id.
any help is appreciated.
Thanks in advance.
Die Datenbank 'C:\TEMP\derbyDB01' konnte nicht mit dem Klassenladeprogramm sun.misc.Launcher$AppClassLoader#253498 gestartet werden. Details können Sie der nächsten Ausnahme entnehmen.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:579)
at java.sql.DriverManager.getConnection(DriverManager.java:243)
at Importer.getCon(Importer.java:88)
this is the part of the code, causing the exception:
private static final String jdbc_driver = "org.apache.derby.jdbc.EmbeddedDriver";
private static Connection conn = null;
public void importToDB(String DB_URL, String[] header, List<Data> dataList, File csv ) throws SQLException, ClassNotFoundException {
stmt = null;
ResultSet rs = null;
String sql = null;
String tablename = spl.getTableName(csv);
Class.forName(jdbc_driver);
if (conn == null) {
conn = getCon(DB_URL);
}
...
public static Connection getCon(String DB_URL) throws SQLException {
System.out.println("Connecting to a selected database...");
conn = DriverManager.getConnection(DB_URL);
System.out.println("Connected successfully...");
return conn;
}
the String DB_URL is given to the method by taking it from a JTextfield. the URL I use is correct, because the program worked with it, before adding the GUI. it is: "jdbc:derby:C:\TEMP\derbyDB01"
So what is causing so much problems in here?
According to Derby developer guide, you need to reverse the slashes in URL
jdbc:derby:c:/TEMP/derbyDB01
Sorry for taking so long to Update this.
Almost forgot I had it posted here. The problem was that I opened the database connection via an eclipse db plugin. When closing that plugin, it doesn't properly close the connection, which then produces in this exception when the program is trying do gain access again.
try {
{
long startTime = System.currentTimeMillis();
String source="s";
String source1="s";
URL google = new URL("http://google.com/");
HttpURLConnection yc =(HttpURLConnection)google.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
source=source.concat(inputLine);
}
in.close();
yc.disconnect();
}
long endTime1 = System.currentTimeMillis();
System.out.println("Total elapsed time in execution of method callMethod() is :"+ (endTime1-startTime));
}
}
when i tried the above through command prompt
i got
java.net.UnknownHostException: google.com
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at sun.net.NetworkClient.doConnect(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.openServer(Unknown Source)
at sun.net.www.http.HttpClient.<init>(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.http.HttpClient.New(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
at ScagntJavaHttp.httpMakeRequest(ScagntJavaHttp.java:185)
at test.main(test.java:23)
Can any help me in resolving this one?
I believe it's a proxy problem.
Try to see if you have a proxy definition in your browser and then set it:
ProxySelector.setDefault(new ProxySelector() {
#Override
public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
throw new RuntimeException("Proxy connect failed", ioe);
}
#Override
public List select(URI uri) {
return Arrays
.asList(new Proxy(Proxy.Type.HTTP,
new InetSocketAddress(proxyHost,
proxyPort)));
}
});
To see if you have proxy definition in IE, go to Tools - Internet Options -- Connections -- Lan Settings
Try removing http:// from your host url when you get java.net.UnknownHostException and check your internet connection and the host exists (probably safe with google . . .)