url(public = True) is broken #309

Activity

1. sparkiegeek

   added a commit that references this issue on Oct 23, 2023

   fix(base_parser): validators.url no longer has a public=True kwarg

   ...

   576b689
2. nandgator commented on Oct 26, 2023

   

   nandgator

   on Oct 26, 2023

   Collaborator

   More actions

   Hi @haavardw, can you please provide a couple of use cases to consider?
3. nandgator

   added

   waitingIssue/PR: Wating for replyIssue/PR: Wating for reply

   on Oct 26, 2023
4. haavardw commented on Oct 27, 2023

   

   haavardw

   on Oct 27, 2023

   Author

   More actions

   We have a system where you can submit a URL and get the web-site analyzed and scanned for phishing, malware etc. The argument public=True is used to prevent scanning of local resources, e.g. file:///etc/passwd or local cloud resources, which would open up for an attack on our servers. We are currently using 0.18.2 but would like to upgrade.

   👍React with 👍1Reacted by nandgator
5. nandgator

   added

   enhancementIssue/PR: A new featureIssue/PR: A new feature

   and removed

   waitingIssue/PR: Wating for replyIssue/PR: Wating for reply

   on Oct 31, 2023
6. stratigos commented on Jan 12, 2024

   

   stratigos

   on Jan 12, 2024

   Last edited by stratigos

   More actions

   This article from Snyk is also apparently SEO optimized to appear for search results regarding python and URL validation: https://snyk.io/blog/secure-python-url-validation/

   The code presented is essentially flawed as validators believes the local URL is valid, and only fails due to an invalid kwarg, public.

   validation = validators.url("https://10.0.0.1", public=True)
   
   if validation:
   
   print("URL is valid")
   
   else:
   
   print("URL is invalid")

   I read the source, and a comment in the source code suggests that public is a parameter, though the arg does not appear in the code.

   I came to GitHub to search for a commit or perhaps issues that describe when/why public was removed.

   Just mentioning this in case others find themselves here for the same reasons.

   @haavardw you may be able to limit protocols or first check the url uses http | https explicitly before validation. Additionally it appears like file:///etc/passwd would be seen as an invalid url by validators.
7. nandgator

   added

   outdatedIssue/PR: Open for more than 3 monthsIssue/PR: Open for more than 3 months

   on Jan 30, 2024
8. nandgator commented on Mar 19, 2024

   

   nandgator

   on Mar 19, 2024

   Collaborator

   More actions

   precedes #325
9. nicolafio

   added a commit that references this issue on Mar 27, 2024

   Fix URL validation

   ...

   6e745ee
10. nandgator

    closed this as completedin #356on Apr 3, 2024