0

I'm trying to write a web parser script using requests module. Here is my current code:

import requests
import subprocess
import json
import sys
import threading
import time
from Queue import Queue

numberOfViewers = int(sys.argv[1])
builderThreads = int(sys.argv[2])
startTime = time.time()
numberOfSockets = 0
concurrent = 25
urls = []
urlsUsed = []

def getURL(): # Get tokens
  output = subprocess.Popen(["livestreamer", "twitch.tv/CHANNEL_NAME", "-j"],
  stdout=subprocess.PIPE).communicate()[0]
  return json.loads(output)['streams']['worst']['url'] # Parse json and return the URL parameter

def build(): # Builds a set of tokens, aka viewers
  global numberOfSockets
  global numberOfViewers
  while True:
    if numberOfSockets < numberOfViewers:
      numberOfSockets += 1
      print ("Building viewers " + str(numberOfSockets) + "/" + str(numberOfViewers))
      urls.append(getURL())

def view(): # Opens connections to send views
  global numberOfSockets
  while True:
    url=q.get()
    requests.head(url)
    if (url in urlsUsed):
      urls.remove(url)
      urlsUsed.remove(url)
      numberOfSockets -= 1
    else:
      urlsUsed.append(url)
      q.task_done()

      if __name__ == '__main__':
        for i in range(0, builderThreads):
          threading.Thread(target = build).start()

          while True:
            while (numberOfViewers != numberOfSockets): # Wait until sockets are built
              time.sleep(1)

              q=Queue(concurrent*2)
              for i in range(concurrent):
                try:
                  t=threading.Thread(target=view)
                  t.daemon=True
                  t.start()
                except:
                  print ('thread error')
                  try:
                    for url in urls:
                      print (url)
                      q.put(url.strip())
                      q.join()
                  except KeyboardInterrupt:
                    sys.exit(1)

But when I run the code, it says:

Traceback (most recent call last):
  File "C:\Users\flamelier\Desktop\Twitch.py", line 1, in <module>
    import requests
ImportError: No module named 'requests'

Why am I getting this error? How do I install this module? Will this error keep repeating for all the scripts henceforth? How can I prevent such similar errors in the future?

5
  • 1
    Did you install requests? It doesn't come with the default Python installation. Commented Jul 8, 2017 at 20:26
  • pip3 install requests Commented Jul 8, 2017 at 20:29
  • Don't run code you haven't written unless you know what it is doing. Commented Jul 8, 2017 at 20:29
  • 1
    Possible duplicate of Python module not found Commented Jul 8, 2017 at 20:57
  • Possible duplicate of ImportError: No module named requests Commented Jul 9, 2017 at 2:52

3 Answers 3

1

Requests is a 3rd party module. You should first install it to Python using PIP or easy_install.

Sign up to request clarification or add additional context in comments.

Comments

0

You have to run pip3 install requests as requests doesn't come with Python by default, as it is a third party library.

Comments

0

Even after you have pip3-installed requests, the code shown won't do anything. The

    if __name__ == "__main__"

test and everything after it is part of an else block in the view function. Back this line and the block that follows out to the left margin.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.