Concurrent Programming in Python

Concurrent Programming in Python

• 1.
  @adam_englander Concurrent Programming in Python Programmingthe multi-verse
• 2.
  @adam_englander Threading • Thread-Local canbe used for global data only accessible by a single thread. • Semaphores and locks prevent simultaneous access of memory. Python’s with statement makes it a lot cleaner than many languages. • Events and Barriers allow for easy interprocess coordination.
• 3.
  @adam_englander Processes • Process classallows for spawning and forking of processes from a parent process. • Popular for multi-tasking in scripts as you are simply executing a command rather Threading which runs code. • Provides pipes, queues, shared memory, and server processes for interprocess communication. • Allows for worker pools to manage resource utilization.
• 4.
  @adam_englander Subprocesses • Run otherscripts from a main script • Uses popen to open other scripts • Allows for better error handling of subscripts • Allows for waiting on the subscript to complete
• 5.
  @adam_englander Concurrent • Uses Futures •Can use Thread Pools or Process Pools • Allows for asynchronous function programming • Works just like any other language with Futures/Promises
• 6.
  @adam_englander Suggestions • For scripts,uses Subprocess • For all but the simplest of applications, use a concurrent framework like curio, Tornado, Twisted, etc.
• 7.
  @adam_englander Process Example from multiprocessingimport Process
 import os
 from random import randint
 from time import sleep
 
 
 def start():
 print("Hello from pid", os.getpid())
 sleep(randint(1, 3))
 print("Goodbye from pid", os.getpid())
 exit()
 
 print("Hello from master pid", os.getpid())
 for i in range(1, 5):
 p = Process(target=start)
 p.start()
 print("Goodbye from master pid", os.getpid())
• 8.
  @adam_englander Subprocess Example import subprocess
 
 print("nnManagedexec error")
 p = subprocess.Popen( [“/not/a/valid/script"], shell=True, stderr=subprocess.PIPE)
 p.wait()
 print(p.returncode, p.stderr.read())
• 9.
  @adam_englander Curio • High concurrencyframework with no asynchronous I/O • Utilizes core asynchronous and concurrency libraries available in Python 3.5+ • Simplifies concurrent programming • Highly performant
• 10.
  @adam_englander Demos!
• 11.
  @adam_englander Resources • http://curio.readthedocs.io/ • https://docs.python.org/3/library/concurrency.html