Python selenium: nested element finding

Question

I'm trying to write a function, which returns a list of elements from any level of nesting from the html, containing the word 'products' in its text, case insensitive.

here is my function:

def __find_elements(driver):
    return driver.find_elements(By.XPATH,
                                "//*[contains(translate(descendant::text(), "
                                "'ABCDEFGHIJKLMNOPQRSTUVWXYZ', "
                                "'abcdefghijklmnopqrstuvwxyz'), "
                                "'products')]")

; for some reason, processing this part of HTML

<a class="nav-link main-navigation-link" href="https://www.leuze.com/en-int/products" itemprop="url" data-flyout-menu-trigger="e3e2319128d6464d8a025ac00823ce4e" title="Products">
    <div class="main-navigation-link-text"\>
        <span itemprop="name"\>Products\</span\>
    </div\>
</a\>

, the function returns [] instead of the list, containing the following span element:

<span itemprop="name">Products</span>

QUESTION:
Could you please clarify, how to modify the function to return a list with the mentioned above span element, while processing the part of HTML?

Expected: list with the element

<span itemprop="name">Products</span>

Actually happened: empty list

JeffC · Accepted Answer · 2024-03-05 18:15:25Z

1

If it were me, I would change the XPath to

//*[text()='Products']

That will get you any element that contains the text, "Products". What other capitalization are you expecting? All caps or all lower case? If those are truly alternatives, I would change it to

//*[text()='products'] | //*[text()='Products'] | //*[text()='PRODUCTS']

or some variation only containing the spellings I was expecting. I think it's much more readable but that's just my opinion.

If those still don't work, you are likely running into a timing issue and you need to add a WebDriverWait or it may be that some/all of the found elements are not visible so their text value will be returned as empty string.

answered Mar 5, 2024 at 18:15

JeffC

26.5k5 gold badges35 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

antonio_oreany Over a year ago

both XPaths: still return [], WebDriverWait(driver, SELENIUM_WAIT_TIMEOUT).until(EC.presence_of_element_located((By.TAG_NAME, 'body'))) was/is in the code. QUESTION: Could you please clarify, how to check, whether some/all of the found elements are not visible so their text value?

JeffC Over a year ago

You can test your locators in the devtools console. Press F12 to open it and then use $$() for CSS selectors and $x() for XPaths, e.g. $x("//*[text()='Products']") for your XPath. It will show how many matching elements were found. You can expand the search results and then click each element to see it in the DOM and/or hover it to highlight the element on the page. You'll have to visually inspect the element to see if it's visible or not. You could write some code using .is_displayed() and print the result from looping through the elements, if you wanted to do it that way.

mario adisson · Accepted Answer · 2024-03-05 17:56:49Z

-1

Try this :

elements = driver.find_elements_by_xpath("//span[@itemprop='name']")

this will get all the elements within a span that have itemprop='name'

answered Mar 5, 2024 at 17:56

mario adisson

12 bronze badges

3 Comments

JeffC Over a year ago

.find_elements_by_*() has been deprecated for a while now

JeffC Over a year ago

This is also not an answer to the question asked

antonio_oreany Over a year ago

still return [] QUESTION: What could be the issue?

Collectives™ on Stack Overflow

Python selenium: nested element finding

2 Answers 2

2 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related