Develop Reference

How to get video src with jsoup (twitch)?

I am getting null pointer, is possible to get this link?
Element element = document.select("div.tw-absolute.tw-bottom-0.tw-left-0.tw-overflow-hidden.tw-right-0.tw-top-0.video-player__container").first();
System.out.println(element.absUrl("src"));
Tried this too
nullpointer as well
Element video = document.select("video").first();
String absSrc = video.absUrl("src");
System.out.println(absSrc);
html part
<div class = "tw-absolute tw-bottom-0 tw-left-0 tw-overflow-hidden tw-right-0 tw-top-0 video-player__container" data-test-selector="video-player__video-container">
<video playsinline="" webkit-playsinline="" src="https://clips-media-assets2.twitch.tv/40487770748-offset-9048.mp4?token=%7B%22authorization%22:%7B%22forbidden%22:false,%22reason%22:%22%22%7D,%22chansub%22:%7B%22restricted_bitrates%22:%5B%5D%7D,%22device_id%22:%226518a1542e035018%22,%22expires%22:1609419047,%22https_required%22:true,%22privileged%22:false,%22user_id%22:500437676,%22version%22:2,%22vod_id%22:850278065%7D&sig=5e17731db577b99e535c4aad3eacc70c0cc34521"></video>
link: https://www.twitch.tv/scream/clip/BrightOilyAppleMcaT

Looks like this one will require again, a lot of work to unpick.
Here's what I can tell you just from a quick look:
when you make the initial request, it does not contain the result you're looking for in the HTML. Therefore it must be coming from a subsequent HTTP request that is fired off once the page is loaded... i.e. there's javascript communicating with back end servers to get JSON payloads. In one of those payloads you'll find ".mp4".
If you use Chrome developer tools, you can flick over to the "Network" tab, click on each request following the first one, and check the "Preview" tab. You will find some requests contain JSON responses, others are just .css, .png, etc. ignore these. In the JSON responses, check the results for the occurrence of some generic value you're interested in like ".mp4". Once you've found it:
.. you then need to try to recreate the headers, the request body (as its not empty), the type of HTTP request (POST), and pass any relevant cookies (in the headers).
You're going to have to make anywhere between 1 and 5 HTTP requests to get what you need to get this JSON payload. Once you have it you can then parse it back.
This is another one of those jobs that's so big I'm not going to begin to try to do it for you.
If it were me doing the job, I'd check the Twitch API docs https://dev.twitch.tv/docs/api/ to see if there's a better/easier way that's just 1-2 requests.

You can change the CSS query as below.
Element element = document.select("div.tw-absolute.tw-bottom-0.tw-left-0.tw-overflow-hidden.tw-right-0.tw-top-0.video-player__container > video").first();
String src = element.attr("src");
System.out.println(src);

Document.select("a[href]") not getting all the href

I am using JSOUP to fetch the documents from a website.
Below is my code
webPageUrl = https://mwcc.ms.gov/#/electronicDataInterchange
Document doc = Jsoup.connect(webPageUrl).get();
Elements links = doc.getElementsByAttribute("a[href]");
Below line of code is not working. It is supposed to return an element but doesn't:
doc.getElementsByAttribute("a[href]")
Can someone please point out the mistake in my code?

That page seems to be an Angular application, which means it loads some (probably all or most) of its content via JavaScript scripts.
The fact that the URL contains the fragment separator # is already a strong indicator of that fact, because if you do a HTTP request, then everything after that indicator is cut off (i.e. not sent to the server), so the actual request will just be of https://mwcc.ms.gov/.
As far as I know JSoup does not support running JavaScript, so you might need to look into a more involved scraping tool (possibly running a full browser engine).

Microstrategy URL API

Reading from this link, link, link, link, link and this SO answer about Microstrategy URL API, I want to pass the answer to one or more prompts in the request to that the report document is executed automatically without requiring the user to enter information.
When I tried to use valuePromptAnswer method for multiple prompt, Microstrategy only take the first argument. The other argument doensn't works. Here are the URL
http://111.222.333.444/MicroStrategy/servlet/mstrWeb?Server=******&Project=*****&Port=0&evt=4001&src=mstrWeb.4001&reportID=226F3C76422512B0E56995939C230DF1&visMode=0&reportViewMode=1&valuePromptAnswers=12^11
The microstrategy only takes the first argument.
Using elementsPromptAnswers generate the same result, only the first prompt receive the value
http://111.222.333.444/MicroStrategy/servlet/mstrWeb?Server=****&Project=***&Port=0&evt=4001&src=mstrWeb.4001&reportID=226F3C76422512B0E56995939C230DF1&visMode=0&reportViewMode=1&elementsPromptAnswers=F9365CA24CFDFD394F3D01ACB4EA5F21:1;474209F94C5C6140979F31848D01E40F:2
Using objectPromptAnswer generate the same result
http://111.222.333.444/MicroStrategy/servlet/mstrWeb?Server=****&Project=****&Port=0&evt=4001&src=mstrWeb.4001&reportID=226F3C76422512B0E56995939C230DF1&visMode=0&reportViewMode=1&objectsPromptAnswers=474209F94C5C6140979F31848D01E40F~12~DistChl
I've read the documentation carefully for each step by step. What I'm missed out? Is there any configuration on server side to accept all argument / parameters?
Thanks in advance.

We had did the thing you talk about using the next url:
mstrWeb?port=0&evt=2048001&src=mstrWeb.2048001&documentID=YOURDOCUMENTID&valuePromptAnswers={&Prompt1&}^{&Prompt2&}^{&Prompt3&}^{&Prompt4&}^{&Prompt5&}^{&Prompt6&}&promptAnswerMode=2
In the previous example, {&Prompt1&}, {&Prompt2&}, etc. are responses of a previous prompt. In your case, the url to create should be the next url:
mstrWeb?port=0&evt=2048001&src=mstrWeb.2048001&documentID=226F3C76422512B0E56995939C230DF1&valuePromptAnswers=12^11&promptAnswerMode=2
I think that your problem could be on the evt or promptAnswerMode parameters, because if these are not correctly configured it can get in troubles.
Hope that this answer solves your problem.
Greetings.

How to hit the rest endpoint that has `:.*` as a part of the path param field

I am using the Stash's REST API in my project. My task is to get the tag details for a specific tag. After checking the Stash's REST API documentation, I found the correct endpoint that I should be using. It is
/rest/api/1.0/projects/{projectKey}/repos/{repositorySlug}/tags/{name:.*}
Please see this link for the Stash's REST API documentation.
There is one more endpoint /rest/api/1.0/projects/{projectKey}/repos/{repositorySlug}/tags
With this endpoint I am able to retrieve all the tags. The StashTag object looks something like this.
{
"id": "refs/tags/v4.0.0",
"displayId": "v4.0.0",
"latestChangeset": "234dadf41742cfc2a10cadc7c2364438bd8891c5",
"latestCommit": "234dadf41742cfc2a10cadc7c2278658bd8891c5"
"hash" : "null"
}
My first problem is, I don't know which field to use as the parameter for {name:.*}. Should it be the displayId or Id or anything else.
The second problem is, I don't understand what it means to have : (colon) followed by a . (dot) in the endpoint /rest/api/1.0/projects/{projectKey}/repos/{repositorySlug}/tags/{name:.*}.
Can someone explain me what is the purpose of :. in the path param and how to hit this kind of an endpoint. Also an example of the complete endpoint would be nice.
So far I have tried hitting
https://stashtest.abc.com/rest/api/1.0/projects/KARTIK/repos/kartiks-test-repository/tags/v4.0.0
https://stashtest.abc.com/rest/api/1.0/projects/KARTIK/repos/kartiks-test-repository/tags/refs/tags/v4.0.0
None of these endpoints work.
Any help is appreciated.

The {name:.*} is really just saying that the field name can be anything. Chalk this one up to poor documentation on their part. Think of it like Regex field, because that's exactly what it is. I'm sure at one point they had something like ^[0-9] then went back and changed it when they realized using only tag numbers would omit anyone using their lightweight tag features.
Remove the v from your tag version and see if that helps. If it does not, I would also recommend creating a lightweight tag (something like mytag) and seeing if you can hit it that way (i.e., /kartiks-test-repository/tags/mytag).
But looking at that documentation is telling me that your v in your tag name is throwing off the REST call.

Redirect to another servlet in java

I am having troubles of directing to another servlet in a servlet file. I have a servlet file called NewDreamServlet.java and I want to redirect it to MyDreamsServlet.java.
This is what I currently have in the NewDreamServlet.java for redirecting.
request.getRequestDispatcher("/MyDreamsServlet").forward(request, response);
When I call this it ends up going to a blank page,
http://localhost:8080/ps10-austint/NewDreamServlet
How exactly would I accomplish this? Please let me know if there is any misunderstanding.

Did you try: response.sendRedirect("/YourApp/MyDreamsServlet")

Please try response.sendRedirect("/MyDreamsServlet"). Also, please note that you might have to add an return statement. The following post discusses this in more details java.lang.IllegalStateException: Cannot (forward | sendRedirect | create session) after response has been committed

All these answers to your question are wrong.
1. if you like to use RD().forward, which is more used for with in application calls, all you need to do is go to your web.xml file and for the url part of your 2nd servlet give it any name you would like eg. /fireServletTwo....
Now come back to your 1st servlet and in the getRqstDispatcher braces, write("/fireServletTwo"); this will tell the xml file to look for a servlet mapping with that name and run that servlet.
2. if you would like to use send.Redirect(); which takes a URL and is used to mostly pass controls outside of the application to another domain, its simple.. DO NOT USE A SLASH /.... just write the name of your servlet2 inside "";
Hope that helps

This one works for me but it usually better to have the context path:
response.sendRedirect(request.getContextPath() + "/home.jsp");