I made script for checking all link in one page, but all of the link always give 403 response code, it will opened normally with browser but if checking it by getting response code using selenium java always give 403(Forbidden), i also bring all cookies but it doesn't work, if anyone have solution please let me know.
Thanks in advance.
this is my code to store all cookie and bring them in all link response check
private int getHttpResponseStatusCode(String webPageUrl) {
int ret = -1;
try {
if (webPageUrl != null && !"".equals(webPageUrl.trim())) {
URL urlObj = new URL(webPageUrl.trim());
HttpURLConnection httpConn = (HttpURLConnection) urlObj.openConnection();
Set <Cookie> cookies = driver.manage().getCookies();
String cookieString = "";
for (Cookie cookie: cookies) {
cookieString += cookie.getName() + "=" + cookie.getValue() + ";";
}
httpConn.addRequestProperty("Cookies", cookieString);
httpConn.setRequestMethod("HEAD");
httpConn.connect();
ret = httpConn.getResponseCode();
}
} catch (Exception ex) {
ex.printStackTrace();
} finally {
System.out.println("Http Status Code : " + ret + " , Url : " + webPageUrl);
return ret;
}
};
this is how i get all the link from the page
private List <String> parseOutAllUrlLinksInWebPage(String webPageUrl) {
List <String> retList = new ArrayList <String> ();
if (webPageUrl != null && !"".equals(webPageUrl.trim())) {
/* Get current page belongs domain. */
String urlDomain = this.getPageBelongDomain(webPageUrl);
/* First parse out all a tag href urls. */
By byUrlLink = By.tagName("a");
List <WebElement> aLinkList = driver.findElements(byUrlLink);
if (aLinkList != null) {
int aLinkSize = aLinkList.size();
for (int i = 0; i <aLinkSize; i++) {
WebElement aLink = aLinkList.get(i);
String href = aLink.getAttribute("href");
if (href != null && !"".equals(href.trim())) {
/* Only return same domain page url. */
if (href.toLowerCase().startsWith("http://" + urlDomain) || href.toLowerCase().startsWith("https://" + urlDomain)) {
retList.add(href);
}
}
}
}
/* Second parse out all img tag src urls. */
By byImg = By.tagName("img");
List <WebElement> imgList = driver.findElements(byImg);
if (imgList != null) {
int imgSize = imgList.size();
for (int i = 0; i <imgSize; i++) {
WebElement imgElement = imgList.get(i);
String src = imgElement.getAttribute("src");
if (src != null && !"".equals(src.trim())) {
/* Only return same domain page url. */
if (src.toLowerCase().startsWith("http://" + urlDomain) || src.toLowerCase().startsWith("https://" + urlDomain)) {
retList.add(src);
}
}
}
}
}
System.out.println("Parse out url completed successfully.");
return retList;
};
Related
I need to retrieve google contacts information including contact's profile picture. To do this I use code below and everything works except contact's profile picture: the link that I get leads to nowhere. Is there any other way to get contact's profile picture link?
GoogleCredential gc = new GoogleCredential();
gc.setAccessToken(accessToken);
ContactsService contactsService = new ContactsService("ServiceName");
contactsService.setOAuth2Credentials(gc);
URL url = new URL("https://www.google.com/m8/feeds/contacts/default/full/?max-results=10000");
ContactFeed feed = null;
try {
feed = contactsService.getFeed(url, ContactFeed.class);
} catch (ServiceException e) {
e.printStackTrace();
}
List<SocialContact> contacts = new ArrayList<>();
if (feed != null) {
for (ContactEntry entry : feed.getEntries()) {
SocialContact contact = new SocialContact();
if (entry.hasName()) {
Name name = entry.getName();
if (name.hasFullName()) {
if (name.hasGivenName()) {
String givenName = name.getGivenName().getValue();
if (name.getGivenName().hasYomi()) {
givenName += " (" + name.getGivenName().getYomi() + ")";
}
contact.setFirstName(givenName);
if (name.hasFamilyName()) {
String familyName = name.getFamilyName().getValue();
if (name.getFamilyName().hasYomi()) {
familyName += " (" + name.getFamilyName().getYomi() + ")";
}
contact.setLastName(familyName);
}
}
}
}
for (PhoneNumber number : entry.getPhoneNumbers()) {
contact.setPhone(number.getPhoneNumber());
}
for (Email email : entry.getEmailAddresses()) {
contact.setEmail(email.getAddress());
}
contact.setProfileImageURL(entry.getContactPhotoLink().getHref());
if(contact.getEmail() != null){
contacts.add(contact);
}
}
}
It sounds like you are using the Contacts API but maybe you should use the People API instead. The code below is modified from these docs:
ListConnectionsResponse response = peopleService.people().connections().list("people/me")
.setPersonFields("names,emailAddresses,photos")
.setPageSize(10000)
.execute();
List<Person> connections = response.getConnections();
if (connections != null && connections.size() > 0) {
for (Person person : connections) {
List<Name> names = person.getNames();
if (names != null && names.size() > 0) {
System.out.println("Name: " + person.getNames().get(0)
.getDisplayName());
} else {
System.out.println("No names available for connection.");
}
List<Photo> photos = person.getPhotos();
if (photos != null && photos.size() > 0){
System.out.println("Photo URL: " + person.getPhotos().get(0).getURL());
}
}
} else {
System.out.println("No connections found.");
}
I am calling a external process (git command) from inside of doPost method of my servlet. The post request comes to the servlet via an xhttpRequest() call when user clicks on a button.
In my code I am printing out the response HTML using the writer object of the response. And weirdly when the code reaches the process part it executes fine in the background but I don't see the correct response. Does someone has some insights to this problem? I am using streamgobbler class as described in this link: http://www.javaworld.com/article/2071275/core-java/when-runtime-exec---won-t.html?page=2
Snippet of the process part of the code from the doPost() method is below.
try {
String line;
List<String> lines = new ArrayList<String>();
Process cloneProcess = Runtime.getRuntime().exec("git clone " +
hvafModifiedXMLRepository + " " +
xmlRepoPath.toString());
StreamGobbler errorGlobber = new StreamGobbler(cloneProcess.getErrorStream(), "CMDTOOL-E",resp.getWriter());
StreamGobbler outputGlobber = new StreamGobbler(cloneProcess.getInputStream(), "CMDTOOL-O",resp.getWriter());
errorGlobber.start();
outputGlobber.start();
cloneProcess.waitFor();
if (cloneProcess.exitValue() != 0)
throw new IOException(org.apache.commons.lang.StringUtils.join(lines, "\n"));
System.out.println("Done.");
} catch (IOException ex) {
sendLogHead(resp);
sendLog(resp, "Unable to clone Modified XML repository.", ex);
sendLogTail(resp);
return;
}
Here is my POST request.
function sendSelectedTestCasesToServlet(paramData, projName, uniqueInstanceName, configCapabilities) {
var xhttp;
xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (xhttp.readyState == 4 && xhttp.status == 200)
document.getElementById('content').parentElement.removeChild(document.getElementById('content'));
document.getElementById('footer').parentElement.removeChild(document.getElementById('footer'));
var el = document.createElement('html');
el.innerHTML = xhttp.responseText;
var newBody = document.createElement('div');
newBody.innerHTML = AJS.$(el.getElementsByTagName('body')[0]).html();
document.body.appendChild(newBody);
window.stop();
}
else if (xhttp.readyState == 4 && xhttp.status == 404) {
return;
} else if (xhttp.readyState == 4 && xhttp.status == 500) {
document.getElementById('content').parentElement.removeChild(document.getElementById('content'));
document.getElementById('footer').parentElement.removeChild(document.getElementById('footer'));
var el = document.createElement('html');
el.innerHTML = xhttp.responseText;
var newBody = document.createElement('div');
newBody.innerHTML = AJS.$(el.getElementsByTagName('body')[0]).html();
document.body.appendChild(newBody);
window.stop();
}
};var baseurl = window.location.protocol + '//' + window.location.host + new URI(window.location.href).path()
xhttp.open('POST', baseurl + '?test-cases-for-configs=' + paramData + '&project-name=' + projName + '&job-instance-uniqueId=' + uniqueInstanceName + '&havf-config-capabilities=' + configCapabilities, true);
xhttp.send()
}
Hi all I am working on project in which I am using Google Web Search API for web search. It works in random fashion, it results different set of result for same query string on different searches. What would be the problem.
Please reply as soon as possible. I am filtering the result for suggestions from specific websites only.
Here is my code for WebSearch.
'
public Map getSearchArray(String search_key) throws Exception
{
Reader reader = null;
GoogleResults results = null;
URL url = null;
int index=0;
int result_size=0;
Map search_result_map = new HashMap();
try {
logger.info("Inside GetSearch");
for (int i = 0; i < 20 ; i=i+4) {
String google = "http://ajax.googleapis.com/ajax/services/search/web?v=1.0&start="+i+"&q=";
String charset = "UTF-8";
url = new URL(google + URLEncoder.encode(search_key, charset));
reader = new InputStreamReader(url.openStream(), charset);
results = new Gson().fromJson(reader, GoogleResults.class);
//logger.info("resultSize: "+results.getResponseData().getResults().size());
//System.out.println("total"+t);
if (results.getResponseData().getResults().size()!=0) {
result_size = results.getResponseData().getResults().size();
} else {
result_size = 1;
}
for (int m = 0; m <result_size; m++)
{
String a=results.getResponseData().getResults().get(m).getUrl();
System.out.println(a);
if(a.matches(".*food.Some-Website.*")
||a.matches(".*Some-Website.*")
||a.matches(".*Some-Website.*")
||a.matches(".*Some-Website.*")
{
logger.info("----I----"+i+"----M----"+m);
String title = results.getResponseData().getResults().get(m).getTitle().replaceAll("\\<[^>]*>","");
String url_link = results.getResponseData().getResults().get(m).getUrl();
String embeded_link = createEmbeddedLink(url_link, title, "http");
search_result_map.put("link_"+index, embeded_link);
search_result_map.put("title_"+index, title);
index++;
logger.info("Title "+title);
}
else
{
System.out.println("No Url");
}
}
}
} catch (Exception e)
{
logger.info("Exception Thrown");
}
finally {
return search_result_map;
}
}
`
I'm writing a special permission forms program using MySQL, Javascript, and HTML code, which both respond to. I'm doing all of this using singleton pattern access and facade code(s) in java, and a services code, which responds to the singleton pattern codes.
I'm trying To retrieve a form (AKA 1 Result) by the variables, courseDept & courseNm.
This is a snippet of code I'm using to do all this:
FormDataFacade.java code snippet:
#Path("/specialPermissions/sp")
#GET
#Produces("text/plain")
public Response getSpecialPermissionFormByDeptAndRoomNm(#QueryParam("courseDept") String theDept, #QueryParam("courseNm") String theNm)
throws NamingException, SQLException, ClassNotFoundException
{
//Referenciation to FormDataFacade class.
FormDataFacade iFacade = FormDataFacade.getInstance();
int intNm = 0;
try {
intNm = Integer.parseInt(theNm);
}catch (NumberFormatException FAIL) {
intNm = 1;
}
//Aiming for forms with matching departments & room numbers by calling FormDataFacade
//method, getSpecialPermissionFormByDeptAndRoomNm.
SpecialPermissionForms[] orgaForm = iFacade.getSpecialPermissionFormByDeptAndRoomNm(theDept, intNm);
//Json String Representation...
if (orgaForm != null)
{
Gson neoArcadia = new Gson();
String result = neoArcadia.toJson(orgaForm);
//Json String added to response message body...
ResponseBuilder rb = Response.ok(result, MediaType.TEXT_PLAIN);
rb.status(200); //HTTP Status code has been set!
return rb.build(); //Creating & Returning Response.
}
else
{ //In case of finding no form data for the procedure...
return Response.status(700).build();
}
}
FormDataServices.java code snippet:
public SpecialPermissionForms[] getSpecialPermissionFormByDeptAndRoomNm(String theDept, int theNm) throws SQLException, ClassNotFoundException
{
Connection con = zeon.getConnection();
PreparedStatement pstmt = con.prepareStatement("SELECT formID, studentName, courseDept, courseNm, semester, year, reasonCode FROM spforms WHERE courseDept = ? & courseNm = ?");
pstmt.setString(1, theDept);
pstmt.setInt(2, theNm);
ResultSet rs = pstmt.executeQuery();
SpecialPermissionForms[] neoForm = new SpecialPermissionForms[50];
int current = 0;
while (rs.next())
{
int formID3 = rs.getInt("formID");
String studentName3 = rs.getString("studentName");
String courseDept3 = rs.getString("courseDept");
String courseNm3 = rs.getString("courseNm");
String semester3 = rs.getString("semester");
int year3 = rs.getInt("year");
String reasonCode3 = rs.getString("reasonCode");
SpecialPermissionForms neo = new SpecialPermissionForms(formID3, studentName3, courseDept3, courseNm3, semester3, year3, reasonCode3);
neoForm[current] = neo;
current++;
}
if (current > 0)
{
neoForm = Arrays.copyOf(neoForm, current);
return neoForm;
}
else
{
return null;
}
}
Forms.html code snippet which both pieces of java code respond to:
$("#btnOneName").click(function() {
alert("clicked");
var inputId1=document.getElementById("t_specialFormCourseDept").value;
var inputId2=document.getElementById("t_specialFormCourseNm").value;
var theData = "courseDept=" + inputId1 + "&" + "courseNm=" + inputId2;
alert("Sending: " + theData);
var theUrl = "http://localhost:8080/onlineforms/services/enrollment/specialPermissions/sp?courseDept=&courseNm="+ theData;
$.ajax( {
url: theUrl,
type: "GET",
dataType: "text",
success: function(result) {
alert("success");
var neoForm = JSON.parse(result);
alert(neoForm);
var output="<h3>Current Form Lists:</h3>";
output += "<ul>";
for (var current = 0; current < neoForm.length; current++)
{
output += "<li>" + neoForm[current].formID + ": " + neoForm[current].studentName + " (" + neoForm[current].courseDept + " - " + neoForm[current].courseNm + ") " +
" (" + neoForm[current].semester + " - " + neoForm[current].year + ") " + " - " + neoForm[current].reasonCode + "</li>";
}
output += "</ul>";
alert(output);
$("#p_retrieveOneName").html(output);
},
error:function(xhr) {
alert("error");
$("#p_retrieveOneName").html("Error:"+xhr.status+" "+xhr.statusText);}
} );
});
});
Now, when I go test this code in my webservice after successfully compiling it, it does work, however it retrieves everything, including the specific results I was searching for - I only want to return the results I have specifically searched for and nothing else. What exactly am I doing wrong here in these snippets of code?
Any suggestions or steps in the right direction are highly welcome.
I am having trouble in running my Android app. It is a simple activity meant to get data from a remote database via HTTP.
I am getting these two errors in log cat:
Error parsing to json on getJarrayFromString();
org.json.JSONException: Value Database of type java.lang.String cannot
be converted to JSONArray.
And:
Error at fillproductlist(): java.lang.NullPointerException
I am not very good at Java/Android and this is first time I am asking this question here. I am unable to understand what layer of my app the problem might be in (my Java code, PHP, or database code)?
Code snapshot:
private void fillProductList() {
if (utils.isInternet()
&& CustomHttpClient.isAddressOk(Utils.LINK_PRODUCTS)) {
final int from = bid.size();
final int nr = 5;
long sqliteSize = db.size(mySqlDatabase.TABLE_BOOKS);
if (from == 0) {
// we add the post variables and values for the request
postParameters.add(new BasicNameValuePair("from", String
.valueOf(0).toString()));
postParameters.add(new BasicNameValuePair("nr", String.valueOf(
nr).toString()));
} else {
postParameters.add(new BasicNameValuePair("from", String
.valueOf(from).toString()));
postParameters.add(new BasicNameValuePair("nr", String.valueOf(
nr).toString()));
}
postParameters.add(new BasicNameValuePair("title", titleSearch));
postParameters.add(new BasicNameValuePair("code", codeSearch));
postParameters.add(new BasicNameValuePair("module", moduleSearch));
if (sortOrder != null)
postParameters.add(new BasicNameValuePair("order", sortOrder));
if (sortType != null)
postParameters.add(new BasicNameValuePair("by", sortType));
task = new RequestTask("books", postParameters);
task.setOnTaskCompleted(new OnTaskCompletedListener() {
public void onTaskStarted() {
if (!rlLoading.isShown()) {
rlLoading.startAnimation(fadeIn());
rlLoading.setVisibility(View.VISIBLE);
}
IS_PRODUCTS_TASK = true;
}
public void onTaskCompleted(String result) {
try {
if (result != "") {
// Enter the remote php link
// we convert the response into json array
jarray = utils.getJarrayFromString(result);
int mysqlSize = (jarray.getJSONObject(0)
.getInt("numRows"));
Log.i(DEBUG, "From " + from + " to " + mysqlSize);
if (from <= mysqlSize) {
int rows;
// we check to see if there is 0
if (jarray.length() > 0) {
Log.i(DEBUG,
"From "
+ from
+ " to "
+ Math.floor(mysqlSize / nr)
* nr);
if (from + 5 <= Math.floor(mysqlSize / nr)
* nr) {
rows = jarray.length();
} else {
rows = mysqlSize % nr + 1;
Utils.IS_ENDED_PRODUCT_LIST = true;
}
ArrayList<String> list = new ArrayList<String>();
for (int i = 1; i < rows; i++) {
JSONObject row = jarray
.getJSONObject(i);
bid.add(row.getInt("bid"));
bTitle.add(row.getString("bTitle"));
bCode.add(row.getString("bCode"));
bPrice.add(row.getString("bPrice")
+ "£");
bDescription.add(row
.getString("bDescription"));
bModule.add(row.getString("bModule"));
bImage.add(Utils.PATH
+ row.getString("bImage"));
list.add(row.getString("bImage"));
// we check if this id already exists in the db, if it doesn't exists w create new one
if (!db.hasIDbooks(row.getInt("bid")))
db.createRowOnBooks(
row.getInt("bid"),
row.getString("bTitle"),
row.getString("bCode"),
row.getString("bPrice"),
row.getString("bDescription"),
row.getString("bModule"),
Utils.PATH
+ row.getString("bImage"),
row.getString("bSpecialOffer"),
row.getInt("bSpecialDiscount"),
row.getString("bDateAdded"));
Log.i(DEBUG,
row.getString("bDescription"));
}
new DownloadImages(list, bAdapter)
.execute();
}
}
postParameters.removeAll(postParameters);
} else {
Utils.IS_ENDED_PRODUCT_LIST = true;
if (rlLoading.isShown()) {
rlLoading.startAnimation(fadeOut());
rlLoading.setVisibility(View.INVISIBLE);
}
}
} catch (Exception e) {
Log.e(DEBUG,
"Error at fillProductList(): " + e.toString());
}
}
});
task.execute();
} else {
// if we are not connected on internet or somehow the link would not work, then we will take the rows stored in sqlite db
if (db.size(mySqlDatabase.TABLE_BOOKS) > 0) {
Cursor cursor = db.getBookssRows(mySqlDatabase.TABLE_BOOKS);
cursor.moveToFirst();
while (!cursor.isAfterLast()) {
bid.add(cursor.getInt(cursor
.getColumnIndex(mySqlDatabase.KEY_BID)));
bTitle.add(cursor.getString(cursor
.getColumnIndex(mySqlDatabase.KEY_BTITLE)));
bCode.add(cursor.getString(cursor
.getColumnIndex(mySqlDatabase.KEY_BCODE)));
bPrice.add(cursor.getString(cursor
.getColumnIndex(mySqlDatabase.KEY_BPRICE)) + "£");
bDescription.add(cursor.getString(cursor
.getColumnIndex(mySqlDatabase.KEY_BDESCRIPTION)));
bModule.add(cursor.getString(cursor
.getColumnIndex(mySqlDatabase.KEY_BMODULE)));
bImage.add(cursor.getString(cursor
.getColumnIndex(mySqlDatabase.KEY_BIMAGE)));
cursor.moveToNext();
}
bAdapter.notifyDataSetChanged();
Utils.IS_ENDED_PRODUCT_LIST = true;
}
}
}
In the OnTaskCompleted before
jarray = utils.getJarrayFromString(result);
add a log entry with
Log.i(DEBUG,result);
I am suspecting that you are getting a null value as result
if(result!="")
does not ensure a valid value.
Cheers