Notify when web content change - java

Im new to java and working on a simple application that monitor an url and notify me when a table is updated whit new items. Looking at the entire page will not work as there are commercials that change all the time and they would give false positives.
My thought was to fetch the url line by line looking for the elements. For each element I will check to see if the element is already in an arraylist. If not the element is added to the arraylist and a notification is send.
What I need support with is not the exact code but advice if this would be a good approach and if I should store the elements in an array list or if I should use a file instead as there are 2 lines of text in each element.
Also It would be good to get recomandation on what methods and libs there would be good to look at.
Thanks in advance
Sebastian

To check the site it'd probably be more stable to parse the HTML and work with an object representation of the DOM. I've never had to do this but in a question regarding how to do this another user suggested using JTidy, maybe you could have a look at that.
As for storing the information (what you currently do in your ArrayList): this really depends on what you use your application for. If you only want to be notified of changes that occur during the runtime of your program this is perfectly fine. If you want to have the information persist you should find a way to store the information in the file system or database.

Related

How to make JTree to only display strings and not a file structure

I'm currently working on a client-server project where the client sends a request to the server and the server runs a database statement based on the information from the client. As there's no static length for the output I add all the data to a ArrayList<String> and send it back to the client.
The data that the client requests has a certain hierarchy so that one could also display it using an XML file (however the output to a GUI is wanted here instead of file creation). There's several ways of displaying I came up with. One would be using a simple text area, however using this, I need to do applying the layout myself and the option of folding where I can expand and collapse certain items is not possible.
Another possibility of course would be using a simple JList, however I read on Stackoverflow that using a JList to display hierarchical things is way easier done using a JTree.
When I decided trying to implement JTree I also found the Genealogy.java file provided by Oracle in their documentation of JTree. As to be seen in the first examples of this documentation there is always a filestructure being displayed and hence also a filestructure kind of representation (using folder- and file-symbols) is shown. In contrast to this Genealogy.java only shows the names of the persons so I tried to adapt the implementation from Genealogy.java to my case however I'm completely clueless of how to do this as I don't even know where these symbols come from and where I could possibly disable them and also don't really get what some of the methods I need to implement do.
As I'm currently really clueless of how to possibly accomplish that I really need some detailed help (only telling me to come up with my own implementation of JTree or TreeNode doesn't really help me at this point). Is there any simpler way or is the need to implement any of the before mentioned interfaces inevitable and if so, how would it be done?
EDIT:
This is how it currently would be displayed (example taken from the Oracle documentation, showing folder- and file-symbols in front of the string):
And this is how I want it to be displayed (also from the documentation, this time only displaying a string as node):
The answer almost certainly will be found via the (icon of the) TreeCellRenderer used for the tree nodes. See the File Browser GUI for tips. It shows how to set the icons in the FileTreeCellRenderer. Admittedly the point here is almost exactly opposite what it was there, but it still comes down to the same thing - the icons.
The section of the tutorial that covers it is How to Use Trees: Customizing a Tree's Display.

How to get all commits for a certain release in GitHub?

I know I can get all commits in a project using GET /repos/:owner/:repo/commits
Now I want to get all commits for a certain release of that project.
What should I do?
Judging by your answer to my question, you want the commits made since some tag. This will take a couple steps to complete, first you need to get the SHA for the tag in question. You'll want to use the git references API to get a specific reference. In the specific example that you linked you'll want to do
GET /repos/nasa/mct/git/refs/tags/v1.8b3
And you'll want to get the 'sha' attribute from the object stored in the 'object' attribute of the response object. With the 'sha' attribute, you'll want to use the commits API to list commits starting with that 'sha' so your request will look like this:
GET /repos/nasa/mct/commits?sha=%(sha_from_first_request)s
That will give you 30 commits per-page by default (if I remember correctly), so you should see if adding &per_page=100 to the end helps. I can't tell you exactly how to do this in Java, but I expect you'll be able to use one of the libraries written to interact with the API to make it easier.

Dynamic Content Parsing

I am working with content parsing I executed the sample program for this i have taken a sample link
please visit the below link
http://www.equitymaster.com/stockquotes/sector.asp?sector=0%2CSOFTL&utm_source=top-menu&utm_medium=website&utm_campaign=performance&utm_content=key-sector
or
Click Here
in the above link i parsed the table data and store into java object.
BSE and NSE are not my exact requirement just I am taken sample example. the above link is developed in the tables they are not used id's and classes. in my example I parsed data using XPath
this is my Xpath
/html/body/table[4]/tbody/tr/td/table[2]/tbody/tr[2]/td[2]/font/table[2]
I selected and parsing it is working fine . here is a problem in future if they changed website structure my program will not work for sure. tell me any other way to parse data dynamically and able to store in database. display the results based on the condition even if they changed the webpage structure I used for this JSOUP api for this. Tell me any other ApI's which provide best support for this type of requirement
If you're trying to parse a page without any clear id/class to select your nodes, you have to try and rely on something else. Redefining the whole tree is indeed the weakest way of doing it, if anything is added/changed everything will collapse.
You could try relying on color: //table[#bgcolor="#c9d0e0"], the "GET MORE INFO" field: //table[tr/td//text()="GET MORE INFO"], the "More Info" there is on every line: //table[.//td//text()="&nbspMore Info&nbsp"]...
The idea is to find something ideally unique (if you can't find any unique criteria, table[color condition selecting a few tables][2] is still stronger walking the whole tree), present every time, and use that as an id.

Automatic adding of elements to an array

Hi I'm not particular good at Java so please bear with me. I'm trying to write a very simple android app now and I need help with some coding.
Thing is, I have a server that automatically generates .png files and saves them to a public directory in a numerical order. The update occurs daily and is non-exhaustive.
Is there anyway in which I can assign the dynamic values to an array within my app?
private String[] myRemoteImages = {
"http://hypotheticalurl1.png",
"http://hypotheticalurl2.png",
"http://hypotheticalurl3.png",
"http://hypotheticalurl4.png",
"http://hypotheticalurl5.png",
"http://hypotheticalurl6.png",
"http://hypotheticalurl7.png",
"http://hypotheticalurl8.png",
"http://hypotheticalurl9.png",
"http://hypotheticalurl10.png",
"http://hypotheticalurl11.png",
"http://hypotheticalurl12.png",
//...blah blah blah
// these are all dynamically created so I won't know what is the last number on the list
};
This array will eventually be used to get the images from my server using the app. It works so far but that's only with hardcoded URLs. I would like the URLs to be dynamic, as the number of images will change from day to day.
I'm doubting that regex will work well in Java but then again I'm no expert. Was thinking of perhaps writing a script on the server end that generates a list of existing values and somehow parsing that with the android app.
Can anyone point me in the right direction? Thanks in advance.
Clarification:
The array doesn't have to be dynamically sized while the app is running.
I need a way to read the list of existing images in a remote directory and pass that information to populate the array automatically at runtime.
Resolved
Guys, thanks for the help. Sorry if I wasn't clear enough.
I've found a way to do it. Basically it was rather simple, which was to append an extra line of code to the shell script on the server end to generate a text list of existent image URLs at the same time that it generates the images.
After that, I used a combination of BufferedReader and openStream on the app to parse the remote text file into a String array.
thanks for the help. Sorry if I wasn't clear enough.
I've found a way to do it. Basically it was rather simple, which was to append an extra line of code to the shell script on the server end to generate a text list of existent image URLs at the same time that it generates the images.
After that, I used a combination of BufferedReader and openStream on the app to parse the remote text file into a String array.
With an array you can :
change the elements of the array
but you can't :
add or remove elements. The number of elements if fixed in an array. Some workaround can be found like putting null values and discarding theem when using the values in the array. But that's more troublesome than really useful.
On the other hand, if you want a full dynamic "array" : use a list (java.util.List). An ArrayList would be interesting here, or even a Vector as you will probably need some multihtreading around this array. With a list you can add and remove elements, size can vary and elements can be replaced.
I'd use an ArrayList in this case. You don't have to know the number of elements you want to add then and it's very simple to append elements at the end.
private List<String> list = new ArrayList<String>();
Then simply add elements by
list.add("http://hypotheticalurl1.png");
Regards,
Patrick
instead of using Array of String
use ArrayList<String> It will gives you more flexibility on adding and removing item on runtime refer this link...http://docs.oracle.com/javase/1.4.2/docs/api/java/util/ArrayList.html
hear you can find example on arraylist...http://www.java2s.com/Tutorial/Java/0140_Collections/0160_ArrayList.htm
hope that helps
According to your scenario you need to have the followings:
1- a Web Service which has a method to get you the list of the available image names.
2- You need a web service client for your android application, I suggest you to use KSOAP 2 because it is widely known and easy to implement.
(If you can't figure out how to use the ksoap in your program, I can provide you some example codes)
3- You need to use ArrayList(java.util) to hold your dynamically sized array.
Hey ytou can do it via
ArrayList stringList = new ArrayList();
stringList.add("Item");

What technologies are there for formatted, structured data input and output?

I am working on a project here that ingests internal resumes from people at my company, strips out the skills and relevant content from them and stores it in a database. This was all done using docx4j and Grails. This required the resumes to first be submitted via a template that formatted everything just right so that the ingest tool knew what to look for to strip the data.
The 2nd portion of this, is what if we want to get out a "reduced" resume from the database. In other words, I want to search the uploaded content I now have, and only print out new resumes for people who have Java programming experience lets say. So I can go into my database, find the people who originally had java as a skill, and output a new set of resumes that are also still in a nice templated format, and only have the relevant info in them, instead of ALL the content.
I have been writing some software to do this in Java that will basically use a docx template, overwriting the items in customXML which are bound to the content controls in the doc, so the new data shows up and can eb saved as a new docx with that custom data.
This seems really cumbersome to me, and has some limitations. For one, lets say my template has a place for 3 Skills, and the particular person has 8 skills. There seems to be no good way to add those 5 additional skills to the docx other than painstakingly inserting the data with all of the formatting XML tags and such. This is a real pain, because if the template changes, I dont want to have to go back into my software and edit source code to change that additional data input XML tag to bold instead of italic.
I was doing some reading up on using Infopath to create a form that I could use to get the input, connecting to some sharepoint data source or something to store the stripped out data. However, I can't seem to find out if it is possible using sharepoint to get the data back out, in a nice formatted way. What would the general steps for this be? It seems like I couldnt find very much about this topic with any quick googling.
Thanks
You could set up the skills:
<skills>
<skill>..</skill>
<skill>..</skill>
and use a "repeat" content control pointing to the container. This would handle any number of <skill> entries.

Categories

Resources