So what I need is basically to create a java program that runs from command line and will continue to run until I decide to stop it. The goal of this program is to read email from a particular email address and create JIRA tickets using the contents of the email.
IE: subject of email will be title. Body will be description. Etc...
I am getting confused with how to go about with the design of how to do this. I know I can use JavaMail to gain access to the emails right? Then I just have to parse the email. But other than that I am a little stuck on how I should be making the JIRA Ticket
Thanks!
Your problem is an ideal use case for esb like mule or spring-integration. Basically these eip frameworks provide all building blocks you just need to connect.
First you need to define mail inbound. This component will automatically connect to an mail inbox and fetch all new messages.
Then define a transformation from e-mail message to json object. Finally POST that object using HTTP outbound. You can create ticket in JIRA using /rest/api/2/issue API method.
whole workflow can be implemented almost without coding. Of course you can do everything manually (using javamail and httpclient), but then threading, error handling and retrying is up to you.
For the future - if you're confused what requests are sent and so on - use Google Chrome.
Press Ctrl+Shift+I->Network and make request. If you need to login before etc. it is the same.
For handling HTTP requests (POST, GET, etc.) I recommend to use HttpClient or if you need to use JavaScript HtmlUnit.
So answer is this:
- Track what requests are made when you do certain things via web browser
- implements the same in Java code using HttpClient or HtmlUnit
Related
I have a java (Spring boot) web service which does not have any UI.
I want to send a dynamic Email (created using Thymeleaf and injecting values from a database) using my web service on a daily Cron schedule, using Microsoft Graph and O365 APIs.
Right now I use SMTP to send emails, but I cannot use it anymore as that is no longer going to be supported by the O365 account that I am using.
I found the SendMail APIs on the Graph Documentation which looks pretty straightforward.
But, using the Graph AIPs requires you to create an Azure AD project first and use their Microsoft Identity platform - which I created.
Now, the problem is that most of the flows also require a user to manually login from a login window.
This is where things get complicated.
I do not have a place to show a Login window to any user from my web service, because it is just a backend service there is no UI. I intend to use a service account for sending the emails through the Application.
I found a Daemon support as well, but it seems to only support Python or .Net code.
Migrating my code from Java to either of those platforms just to be able to send emails
does not feel like a good solution.
And, I'm not even sure if they even offer similar capabilities of sending dynamic emails like Java+Thylemeaf do?
Is there a way to be able to continue doing this using my existing code in Java?
If not, then as the worst case scenario, are there any libraries in Python which can allow me to send dynamic emails like thymeleaf does in Java?
As you don't want to manually login from a login window, you can use the client credential flow.
Here is the guide regarding how to access graph api without user.
Reference:
msgraph-sdk-java-auth (You can choose to use Client credential provider)
I have been doing research for a couple of hours and haven't found any viable examples or information on how I could receive PayPal notifications for payments and process them within a Java Application.
What I want to do is:
Person makes payment.
PayPal sends a notification to the java program, indicating that they had made a payment, with email, usernames, amount, package name/ID, etc.
Java program processes the payment.
Perhaps I am looking at it the wrong way. So far I have found out that I need a webhook listener or some kind of IPN serverlet program that can read the POST messages PayPal sends and redirect them? Can't this be done directly within my java program?
Thank you!
As you've found, you really need to use Webhook (which is newer in PayPal) or IPN. This technologies let you to configure PayPal to send (as POST requests) certain events on a certain URLs. All you need to do in your program is to handle this events (requests) in Serlvet, RestHandler or whatever else.
I know what I am asking is somehow weird. There is a web application (which we don't have access to its source code), and we want to expose a few of its features as web services.
I was thinking to use something like Selenium WebDriver, so I simulate web clicks on the application according to the web service request.
I want to know whether this is a better solution or pattern to do this.
I shall mention that the application is written using Java, Spring MVC (it is not SPA) and Spring Security. And there is a CAS server providing SSO.
There are multiple ways to implement it. In my opinion Selenium/PhantomJS is not the best option as if the web is properly designed, you can interact with it only using the provided HTML or even some API rather than needing all the CSS, and execute the javascript async requests. As your page is not SPA it's quite likely that an "API" already exists in form of GET/POST requests and you might be lucky enough that there's no CSRF protection.
First of all, you need to solve the authentication against the CAS. There are multiple types of authentication in oAuth, but you should get an API token that enables you access to the application. This token should be added in form of HTTP Header or Cookie in every single request. Ideally this token shouldn't expire, otherwise you'll need to implement a re-authentication logic in your app.
Once the authentication part is resolved, you'll need quite a lot of patience, open the target website with the web inspector of your preferred web browser and go to the Network panel and execute the actions that you want to run programmatically. There you'll find your request with all the headers and content and the response.
That's what you need to code. There are plenty of libraries to achieve that in Java. You can have a look at Jsop if you need to parse HTML, but to run plain GET/POST requests, go for RestTemplate (in Spring) or JAX-RS/Jersey 2 Client.
You might consider implementing a cache layer to increase performance if the result of the query is maintained over the time, or you can assume that in, let's say 5 minutes, the response will be the same to the same query.
You can create your app in your favourite language/framework. I'd recommend to start with SpringBoot + MVC + DevTools. That'd contain all you need + Jsoup if you need to parse some HTML. Later on you can add the cache provider if needed.
We do something similar to access web banking on behalf of a user, scrape his account data and obtain a credit score. In most cases, we have managed to reverse-engineer mobile apps and sniff traffic to use undocumented APIs. In others, we have to fall back to web scraping.
You can have two other types of applications to scrape:
Data is essentially the same for any user, like product listings in Amazon
Data is specific to each user, like in a banking app.
In the firs case, you could have your scraper running and populating a local database and use your local data to provide the web service. In the later case, you cannot do that and you need to scrape the site on user's request.
I understand from your explanation that you are in this later case.
When web scraping you can find really difficult web apps:
Some may require you to send data from previous requests to the next
Others render most data on the client with JavaScript
If any of these two is your case, Selenium will make your implementation easier though not performant.
Implementing the first without selenium will require you to do lots of trial an error to get the thing working because you will be simulating the requests and you will need to know what data is expected from the client. Whereas if you use selenium you will be executing the same interactions that you do with the browser and hence sending the expected data.
Implementing the second case requires your scraper to support JavaScript. AFAIK best support is provided by selenium. HtmlUnit claims to provide fair support, and I think JSoup provides no support to JavaScript.
Finally, if your solution takes too much time you can mitigate the problem providing your web service with a notification mechanism, similar to Webhooks or Resthooks:
A client of your web service would make a request for data providing a URI they would like to get notified when the results are ready.
Your service would respond immediatly with an id of the request and start scraping the necessary info in the background.
If you use skinny payload model, when the scraping is done, you store the response in your data store with an id identifying the original request. This response will be exposed as a resource.
You would execute an HTTPPOST on the URI provided by the client. In the body of the request you would add the URI of the response resource.
The client can now GET the response resource and because the request and response have the same id, the client can correlate both.
Selenium isn't a best way to consume webservices. Selenium is preferably an automation tool largely used for testing the applications.
Assuming the services are already developed, the first thing we need to do is authenticate user request.
This can be done by adding a HttpHeader with key as "Authorization" and value as "Basic "+ Base64Encode(username+":"+password)
If the user is valid (Users login credentials match with credentials in server) then generate a unique token, store the token in server by mapping with the user Id and
set the same token in the response header or create a cookie containing token.
By doing this we can avoid validating credentials for the following requests form the same user by just looking for the token in the response header or cookie.
If the services are designed to chcek login every time the "Authorization" header needs to be set in request every time when the request is made.
I think it is a lot of overhead using a webdriver but it depends on what you really want to achieve. With the info you provided I would rather go with a restTemplate implementation sending the appropriate http messages to the existing webapp, wrap it with a nice #service layer and build your web service (rest or soap) on top of it.
The authentication is a matter of configuration, you can pack this in a microservice with #EnableOAuth2Sso and your restTemplate bean, thanks to spring boot, will handle the underlining auth part for you.
May be overkill..... But RPA? http://windowsitpro.com/scripting/review-automation-anywhere-enterprise
So, I'm currently developing an app for a service which has a json-based (unfortunately) read only API. Retrieving content is no problem at all, however the only way to post content is using a form on their site which location is a PHP script. The service is open source so I know which fields the form expects, but whatever I send, it always results in a BAD REQUEST.
I captured the network traffic inside my browser and as far as I can see, the browser constructs a multipart form request, however when I copy the request and invoke it again using a REST client, a BAD REQUEST gets returned.
Is there a way to construct a http request in Android that simulates a form post?
If it's readonly I think you wouldn't be able to make requests with POST (it's assume for editing or adding things).
If you let me make you an advise, I recommend you using this project as a Library.
https://github.com/matessoftwaresolutions/AndroidHttpRestService
It makes you easy deal with apis, control network problems etc.
You can find a sample of use there.
You only have to:
Build your URL
Tell the component to execute in POST mode
Build your JSON
As I told you, I don't know even if it will work.
I hope it helps!!!
I'm new to java and I'm trying to understand the way we identify users who uses webservices.
The program will be downloaded from my website. It needs to make a connection to my server side web service program.
I think there are 2 options for identifying the user:
Register on website and download web service. A single user id key is then generated when downloading the program. I don't know if this is possible + verification of registration can only be done by email: not 100% sure of user identity.
Download web service and log in into it.
This seems a better way, but I'm not sure this is the way to do it...
Most services use HTTP authentication because the surrounding HTTP protocol already brings all the necessary features. Actually, your web service framework comes with all the plumbing necessary to easily set this up.
Another solution is to have a method which is called login() that takes a user name and a password. All other methods return errors until login() has been called successfully once.
Note that you must use HTTPS as protocol, otherwise passwords will be transmitted either as plain text or with a trivial encryption that is easy to break. Or to put it another way: Without HTTPS anyone willing to invest a couple of minutes of time will be able to use your service.