Develop Reference

Jsoup css selector "not", not return anything

I'm trying to ignore an item and not parse it on Jsoup
But css selector "not", not working !!
I don't understand what is wrong ??
my code:
MangaList list = new MangaList();
Document document = getPage("https://3asq.org/");
MangaInfo manga;
for (Element o : document.select("div.page-item-detail:not(.item-thumb#manga-item-5520)")) {
manga = new MangaInfo();
manga.name = o.select("h3").first().select("a").last().text();
manga.path = o.select("a").first().attr("href");
try {
manga.preview = o.select("img").first().attr("src");
} catch (Exception e) {
manga.preview = "";
}
list.add(manga);
}
return list;
html code:
<div class="col-12 col-md-6 badge-pos-1">
<div class="page-item-detail manga">
<div id="manga-item-5520" class="item-thumb hover-details c-image-hover" data-post-id="5520">
<a href="https://3asq.org/manga/gosu/" title="Gosu">
<img width="110" height="150" src="https://3asq.org/wp-content/uploads/2020/03/IMG_4497-110x150.jpg" srcset="https://3asq.org/wp-content/uploads/2020/03/IMG_4497-110x150.jpg 110w, https://3asq.org/wp-content/uploads/2020/03/IMG_4497-175x238.jpg 175w" sizes="(max-width: 110px) 100vw, 110px" class="img-responsive" style="" alt="IMG_4497"/> </a>
</div>
<div class="item-summary">
<div class="post-title font-title">
<h3 class="h5">
<span class="manga-title-badges custom noal-manga">Noal-Manga</span> Gosu
</h3>

If I debug your code and extract the HTML for:
System.out.println(document.select("div.page-item-detail").get(0)) (hint use the expression evaluator in IntelliJ IDEA (Alt+F8 - for in-session, real-time debugging)
I get:
<div class="page-item-detail manga">
<div id="manga-item-2003" class="item-thumb hover-details c-image-hover" data-post-id="2003">
<a href="http...
...
</div>
</div>
</div>
It looks like you want to extract the next div tag down with class containing item-thumb ... but only if the id isn't manga-item-5520.
So here's what I did to remove that one item
document.select("div.page-item-detail div[class*=item-thumb][id!=manga-item-5520]")
Result size: 19
With the element included:
document.select("div.page-item-detail div[class*=item-thumb]")
Result size: 20
You can also try the following if you want to remain based at the outer div tag rather than the inner div tag.
document.select("div.page-item-detail:has(div[class*=item-thumb][id!=manga-item-5520])")

Java and Selenium: Trouble getting contents of input field

I'm having problems getting the text contents of an input field. I seem to only be getting the things around it with the method I'm using.
Snippet from the page:
(It's a list of itemsincluding an input field in each row.)
The markup:
<ul class="budsjett budsjett--kompakt" id="sifobudsjett">
<li class="budsjett-post ng-isolate-scope ng-valid" id="SIFO_mat">
<div class="felt" >
<div class="felt-indre">
<div id="SIFO_mat-farge" class="sifo-farge farge-graa"></div>
<span class="budsjett-post-beskrivelse" >
<span tabindex="0" title="Vis hjelpetekst" role="button">
<span class="hjelpetekst-label" >Mat og drikke</span>
</span>
<span class="sifo-hjelp" aria-hidden="true"></span>
</span>
</span>
<span class="budsjett-post-verdi">
<span class="budsjett-post-verdi-endret" ng-show="!skrivebeskyttet" aria-hidden="false" style="">
<input id="SIFO_mat-input" name="SIFO_mat" type="number">
<span class="felt-enhet"><abbr id="SIFO_mat-enhet" title="kroner" translate=""><span class="ng-scope">kr</span></abbr></span>
</span>
</span>
</div>
</div>
</li>
The code:
List<WebElement> sifoliste = driver.findElement(By.id("sifobudsjett")).findElements(By.tagName("li"));
Result of first element: "Mat og drikke".
List<WebElement> sifoliste = driver.findElement(By.id("sifobudsjett")).findElements(By.tagName("input"));
Result of first element: ""
List<WebElement> sifoliste = driver.findElement(By.id("sifobudsjett")).findElements(By.className("budsjett-post-verdi-endret"));
Result of first element: "kr"
Any ideas?

The <input> tag doesn't have text, what you see in the UI is kept in the value attribute. It exists even if you can't see it in the html
driver.findElement(By.id("SIFO_mat-input")).getAttribute("value");
For all the <input>s
List<WebElement> sifoliste = driver.findElement(By.id("sifobudsjett")).findElements(By.tagName("input"));
String text = sifoliste.get(0).getAttribute("value"); // 2790

Try
String inputValue = driver.findElement(By.tagName("input")).getAttribute("value");

Iterate a list of web elements and obtain two or more values

I need to iterate to the list of web elements to get the text of the name and title
example:
<ul id="GalleryViewInner" class="gv-ic">
<li id="item3ad73f1239" class="sresult gvresult">
<div id="1"
<span id="span1">5,000USD</span>
</div>
<div id="2"
<td id="td1">TITLE</td>
</div>
<li id="item3ad73f1239" class="sresult gvresult">
<li id="item3ad73f1239" class="sresult gvresult">
<li id="item3ad73f1239" class="sresult gvresult">
<li id="item3ad73f1239" class="sresult gvresult">
</ul>
iterate ul list:
List<WebElement> allElements = driver.findElements(By.xpath("//ul[#id='GalleryViewInner']/li"));
Iterator<WebElement> iter = allElements.iterator();
while (iter.hasNext()) {
WebElement PRICE = iter.next();
PRICE.getTxt();
TITLE.getText();
}
in each iteration I need to get two or more elements from each "li"
I need to get price and name of all the li elements
Java
Selenium Webdriver

Asumming your web elements like you showed us above, and other <li></li> are the same
<ul id="GalleryViewInner" class="gv-ic">
<li id="item3ad73f1239" class="sresult gvresult">
<div id="1"
<span id="span1">5,000USD</span>
</div>
<div id="2"
<td id="td1">TITLE</td>
</div>
</li>
<li></li>
</ul>
Asumming price is <span></span> with id "span1" and name is <td></td> with id "td1" I'll go with this approach
List<WebElement> liElements = driver.findElements(By.xpath("//ul[#id='GalleryViewInner']/li"));
for(WebElement li : liElements){
WebElement spanPrice = li.findElement(By.id("span1"));
String price = spanPrice.getText();
WebElement tdName = li.findElement(By.id("td1"));
String name = tdName.getText();
}

As per the HTML you have provided to get the text of the name and title you can use the following code block :
List<WebElement> all_span_elements = driver.findElements(By.xpath("//ul[#id='GalleryViewInner']//li/div/span"));
List<WebElement> all_td_elements = driver.findElements(By.xpath("//ul[#id='GalleryViewInner']//li//following::div[2]/td"));
List<String> names = new ArrayList<>();
List<String> titles = new ArrayList<>();
for(WebElement ele1:all_span_elements)
names.add(ele1.getAttribute("innerHTML"));
for(WebElement ele2:all_td_elements)
titles.add(ele2.getAttribute("innerHTML"));
for(int i=0; i<all_span_elements.size(); i++)
System.out.println("Medicine Name is : " + names.get(i) + "and Title is : " + titles.get(i));

Iterate through a list?

I have the following markup.
Edit: Added full markup
<div id="SelectList">
<div class="select-area-left"></div>
<div class="select-area-right"></div>
<div id="SelectedOption">Option0</div>
<ul id="ShowOptions">
<li id="ShowOption0">Option0</li>
<li id="ShowOption1">Option1</li>
<li id="ShowOption2">Option2</li>
<li id="ShowOption3">Option3</li>
<li id="ShowOption4">Option4</li>
<li id="ShowOption5">Option5</li>
<li id="ShowOption6">Option6</li>
<li id="ShowOption7">Option7</li>
<li id="ShowOption8">Option8</li>
<li id="ShowOption9">Option9</li>
<li id="ShowOption10">Option10</li>
<li id="ShowOption11">Option11</li>
<li id="ShowOption12">Option12</li></ul></div>
And i'm trying to use the following code to print out each list elements text.
List<WebElement> allElements = driver.findElements(By.xpath("//div[#id='SelectList']/ul"));
for (WebElement element: allElements) {
System.out.println(element.getText());
}
But its just given a blank output. Is there something i'm missing?

Try following code :
List<WebElement> allElements = driver.findElements(By.xpath("//div[#id='SelectList']/ul/li"));
for (WebElement element: allElements) {
System.out.println(element.getText());
}
Basically you missed to point li element in the xpath used. Above code should work for you.

How to find elements whose sibling index is less than x and greater than y

I have some Element eNews. After finding indexes by CssQuery I have to select sibling elements with index less than y and greater than x;
Elements lines = eNews.select("div.clear");
int x = lines.get(0).elementSiblingIndex();
int y = lines.get(1).elementSiblingIndex();
Elements tNews = eNews.getElementsByIndexGreaterThan(x)
?AND?
eNews.getElementsByIndexLessThan(y)
This is some sample code. I want to extract text from html tags between first and second <div class="clear></div>
<div class="aktualnosci">
<div class="zd">
<a href="/Data/Thumbs/ODAweDYwMA,dsc_0458.jpg" title="" rel="lightbox">
<img src="/Data/Thumbs/dsc_0458.jpg"/>
</a>
<p class="show"></p>
</div>
<h3>Awanse</h3>
<div class="data">
<img alt="" src="/Themes/kalendarz-ico.gif">
2013-11-18 12:26
</div>
<!--Start tag-->
<div class="clear"></div>
<!--Tags to extract-->
<p class="gr">W związku z Narodowym Świętem Niepodległości ....</p>
<p style="text-align: justify">W zeszły p....</p>
<p style="text-align: justify">OISW Kraków</p>
<!--End tag-->
<div class="clear"></div>
<div class="slider">
<span class="slide-left"></span>
<span class="slide-right"></span>
</div>
</div>

You can use a selector like div.clear ~ :gt(1):lt(4)
E.g.:
Elements tNews = eNews.select("div.clear ~ :gt(1):lt(4)");
See this example and the selector docs. (It's a bit hard to validate this does what you're trying to achieve without knowing your input HTML and the data you're trying to extract.)
Update based on your edit: there are a couple ways to do this if you can't know the indexes in advance. Below I get the first div, then accumulate sibling elements until we hit the next div.clear. (I'll have a think if I can generify this pattern and add it to jsoup.)
Document doc = Jsoup.parse(h);
Element firstDiv = doc.select("div.clear").first();
Elements news = new Elements();
Element item = firstDiv.nextElementSibling();
while (item != null && !(item.tagName().equals("div") && item.className().equals("clear"))) {
news.add(item);
item = item.nextElementSibling();
}
System.out.println(String.format("Found %s items", news.size()));
for (Element element : news) {
System.out.println(element.text());
}
Outputs:
Found 3 items
W związku z Narodowym Świętem Niepodległości ....
W zeszły p....
OISW Kraków