netkiller.github.io/python/library/process.html at master · netkiller/netkiller.github.io

596 lines (449 loc) · 16.9 KB
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>2.3. 进程与线程</title><link rel="stylesheet" type="text/css" href="../docbook.css" /><meta name="generator" content="DocBook XSL Stylesheets Vsnapshot" /><meta name="keywords" content="php,pear,pecl,phar, python, , " /><link rel="home" href="../index.html" title="Netkiller Python 手札" /><link rel="up" href="index.html" title="第 2 章 Library" /><link rel="prev" href="random.html" title="2.2. 随机数" /><link rel="next" href="python.log.html" title="2.4. 日志" /></head><body><a xmlns="" href="//www.netkiller.cn/">Home</a> | <a xmlns="" href="//netkiller.github.io/">简体中文</a> | <a xmlns="" href="http://netkiller.sourceforge.net/">繁体中文</a> | <a xmlns="" href="/journal/index.html">杂文</a>
		| <a xmlns="" href="https://github.com/netkiller">Github</a> | <a xmlns="" href="https://zhuanlan.zhihu.com/netkiller">知乎专栏</a> | <a xmlns="" href="https://www.facebook.com/bg7nyt">Facebook</a> | <a xmlns="" href="http://cn.linkedin.com/in/netkiller/">Linkedin</a> | <a xmlns="" href="https://www.youtube.com/user/bg7nyt/videos">Youtube</a> | <a xmlns="" href="//www.netkiller.cn/home/donations.html">打赏(Donations)</a> | <a xmlns="" href="//www.netkiller.cn/home/about.html">About</a><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">2.3. 进程与线程</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="random.html">上一页</a> </td><th width="60%" align="center">第 2 章 Library</th><td width="20%" align="right"> <a accesskey="n" href="python.log.html">下一页</a></td></tr></table><hr /></div><table xmlns=""><tr><td><iframe src="//ghbtns.com/github-btn.html?user=netkiller&amp;repo=netkiller.github.io&amp;type=watch&amp;count=true&amp;size=large" height="30" width="170" frameborder="0" scrolling="0" style="width:170px; height: 30px;" allowTransparency="true"></iframe></td><td><iframe src="//ghbtns.com/github-btn.html?user=netkiller&amp;repo=netkiller.github.io&amp;type=fork&amp;count=true&amp;size=large" height="30" width="170" frameborder="0" scrolling="0" style="width:170px; height: 30px;" allowTransparency="true"></iframe></td><td><iframe src="//ghbtns.com/github-btn.html?user=netkiller&amp;type=follow&amp;count=true&amp;size=large" height="30" width="240" frameborder="0" scrolling="0" style="width:240px; height: 30px;" allowTransparency="true"></iframe></td><td></td><td><a href="https://zhuanlan.zhihu.com/netkiller"><img src="/images/logo/zhihu-card-default.svg" height="25" /></a></td><td valign="middle"><a href="https://zhuanlan.zhihu.com/netkiller">知乎专栏</a></td><td></td><td></td><td></td><td></td></tr></table><div class="section"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="process"></a>2.3. 进程与线程</h2></div></div></div>
	<div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="subprocess"></a>2.3.1. subprocess</h3></div></div></div>
		<div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="id935"></a>2.3.1.1. check_output</h4></div></div></div>
			<pre class="programlisting">
sp = subprocess.check_output(cmd)
text = sp.decode('UTF8')
print(text)
			<p>获取IP地址</p>
			<pre class="programlisting">
import subprocess
command = "/usr/bin/ip addr show eth0 | grep 'inet ' | awk '{print $2}' | cut -d/ -f1"
screen = subprocess.check_output(command, shell=True)
print(screen.decode().replace("\n", ""))		
			<p>制定运行目录</p>
			<pre class="programlisting">
#!/usr/bin/python
# -*-coding：utf-8-*-
import subprocess
output = subprocess.check_output("ls", cwd="/")
print(output.decode())
output = subprocess.check_output("/usr/bin/git pull", cwd="/opt/netkiller", shell=True)
print(output.decode())		
	<div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="threading"></a>2.3.2. Python 多线程</h3></div></div></div>
		<div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="id936"></a>2.3.2.1. threading 高级线程接口</h4></div><div><h5 class="subtitle">threading — Higher-level threading interface</h5></div></div></div>
			<p>创建线程</p>
			<pre class="programlisting">
from threading import Thread
import time
def fun1():
    print("fun1 begin")
    time.sleep(2)
    print("fun1 end")
def fun2():
    print("fun2 begin")
    time.sleep(6)
    print("fun2 end")
threads = []
threads.append(Thread(target=fun1))
threads.append(Thread(target=fun2))
print(threads)
if __name__ == "__main__":
    for t in threads:
        print(t)
        t.start()
    print("Done")		
			<pre class="screen">
import threading
class MyThread(threading.Thread):
	def __init__(self, name=None):
		threading.Thread.__init__(self)
		self.name = name
	def run(self):
		print self.name
	def test():
		for i in range(0, 100):
			t = MyThread("thread_" + str(i))
			t.start()
	if __name__=='__main__':
		<div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="threading.lock"></a>2.3.2.2. Lock 线程锁</h4></div></div></div>
			<p>这里实现了一个计数器 count 这个全局变量会被多个线程同时操作，使其能够被顺序相加，需要靠线程锁的帮助。</p>
			<pre class="programlisting">
#-*- encoding: utf-8 -*-
import threading
import time
class Test(threading.Thread):
    def __init__(self, num):
        threading.Thread.__init__(self)
        self._run_num = num
    def run(self):
        global count, mutex
        threadname = threading.currentThread().getName()
        for x in range(int(self._run_num)):
            mutex.acquire()
            count = count + 1
            mutex.release()
            print (threadname, x, count)
            time.sleep(1)
if __name__ == '__main__':
    global count, mutex
    threads = []
    num = 5
    count = 0
    mutex = threading.Lock()
    # 创建线程对象
    for x in range(num):
        threads.append(Test(10))
    for t in threads:
        t.start()
    # 等待子线程结束
    for t in threads:
        t.join()
			<pre class="programlisting">
import threading
lock = threading.Lock()
    global count
    for _ in range(100000):
        lock.acquire()  # 加锁
        count += 1
        lock.release()  # 释放锁
# 两个线程同时修改共享变量
t1 = threading.Thread(target=add)
t2 = threading.Thread(target=add)
print("最终结果：", count)  # 正确结果 200000			
			<p>用 with 加锁</p>
			<pre class="programlisting">
import threading
lock = threading.Lock()
    global count
    for _ in range(100000):
        with lock:
            count += 1
# 两个线程同时修改共享变量
t1 = threading.Thread(target=add)
t2 = threading.Thread(target=add)
print("最终结果：", count)  # 正确结果 200000		
		<div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="threading.queue"></a>2.3.2.3. Queue 队列</h4></div></div></div>
				http://www.ibm.com/developerworks/aix/library/au-threadingpython/
			<pre class="programlisting">
 #!/usr/bin/env python
import Queue
import threading
import urllib2
import time
hosts = ["http://yahoo.com", "http://google.com", "http://amazon.com",
"http://ibm.com", "http://apple.com"]
queue = Queue.Queue()
class ThreadUrl(threading.Thread):
	"""Threaded Url Grab"""
	def __init__(self, queue):
	  threading.Thread.__init__(self)
	  self.queue = queue
	def run(self):
	  while True:
		#grabs host from queue
		host = self.queue.get()
		#grabs urls of hosts and prints first 1024 bytes of page
		url = urllib2.urlopen(host)
		print url.read(1024)
		#signals to queue job is done
		self.queue.task_done()
start = time.time()
def main():
#spawn a pool of threads, and pass them queue instance 
for i in range(5):
  t = ThreadUrl(queue)
  t.setDaemon(True)
  t.start()
#populate queue with data   
  for host in hosts:
    queue.put(host)
#wait on the queue until everything has been processed     
queue.join()
print "Elapsed Time: %s" % (time.time() - start)
			<pre class="programlisting">
import threading
import queue
q = queue.Queue()
# 生产者线程：往队列放数据
def producer():
    for i in range(5):
        q.put(f"数据{i}")
        print(f"生产：数据{i}")
# 消费者线程：从队列取数据
def consumer():
    while True:
        data = q.get()  # 阻塞等待数据
        print(f"消费：{data}")
        q.task_done()  # 标记处理完成
t1 = threading.Thread(target=producer)
t2 = threading.Thread(target=consumer, daemon=True)  # 守护线程
# 等待生产者结束 + 队列所有数据处理完
print("所有通信完成")			
		<div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="threading.event"></a>2.3.2.4. 线程事件</h4></div></div></div>
			<pre class="programlisting">
import threading
import time
event = threading.Event()
def waiter():
    print("等待信号...")
    event.wait()  # 阻塞等待
    print("收到信号！")
def sender():
    time.sleep(2)
    print("发送信号！")
    event.set()
t1 = threading.Thread(target=waiter)
t2 = threading.Thread(target=sender)
t2.start()			
			<pre class="programlisting">
from datetime import datetime
import threading
import time
event = threading.Event()
def waiter():
    while(True):
        print("等待信号...")
        event.wait()  # 阻塞等待
        print("收到信号！")
        event.clear()
        print("-"*20, datetime.now().strftime("%Y-%m-%d %H:%M:%S"), "-"*20)
def sender():
    while(True):
        time.sleep(2)
        print("发送信号！")
        event.set()
t1 = threading.Thread(target=waiter)
t2 = threading.Thread(target=sender)
t2.start()			
			<pre class="programlisting">
from datetime import datetime
import threading
import time
event = threading.Event()
def waiter():
    print("等待信号...")
    while(event.wait()): # 阻塞等待
        print("收到信号！")
        event.clear()
        print("-"*20, datetime.now().strftime("%Y-%m-%d %H:%M:%S"), "-"*20)
def sender():
    while(True):
        time.sleep(2)
        print("发送信号！")
        event.set()
t1 = threading.Thread(target=waiter)
t2 = threading.Thread(target=sender)
t2.start()		
		<div class="section"><div class="titlepage"><div><div><h4 class="title"><a id="threading.Condition"></a>2.3.2.5. 线程通知</h4></div></div></div>
			<pre class="programlisting">
import threading
import time
cond = threading.Condition()
products = []
def producer():
    with cond:
        products.append("商品")
        print("生产完成，通知消费者")
        cond.notify()  # 唤醒消费者
def consumer():
    with cond:
        cond.wait()  # 等待生产
        print("消费者拿到：", products[0])
t1 = threading.Thread(target=consumer)
t2 = threading.Thread(target=producer)
t2.start()			
			<pre class="programlisting">
import threading
import time
cond = threading.Condition()
def producer():
    while True:
        with cond:
            # while True:
            time.sleep(2)
            cond.notify()  
def consumer():
    global count
    print("开始计数器")
    while True:
        with cond:
            cond.wait()
            count += 1
            print(count)
t1 = threading.Thread(target=consumer)
t2 = threading.Thread(target=producer)
t2.start()			
			<pre class="programlisting">
import threading
import time
cond = threading.Condition()
data = None
def consumer():
    global data
    # 手动加锁（代替 with）
    cond.acquire()
    print("消费者：等待数据...")
    cond.wait()  # 自动释放锁 → 阻塞 → 唤醒后自动重新拿锁
    print(f"消费者：拿到数据 -&gt; {data}")
    # 手动释放锁
    cond.release()
def producer():
    global data
    cond.acquire()
    print("生产者：正在生产数据...")
    time.sleep(2)
    data = "Hello Threading!"
    print("生产者：数据生产完成，通知消费者")
    cond.notify()  # 唤醒，但不释放锁
    cond.release()  # 这里释放锁，消费者才能真正运行
t1 = threading.Thread(target=consumer)
t2 = threading.Thread(target=producer)
time.sleep(0.1)
t2.join()			
	<div class="section"><div class="titlepage"><div><div><h3 class="title"><a id="daemon"></a>2.3.3. 守护进程（Daemon）</h3></div></div></div>
		<p>http://www.myelin.co.nz/post/2003/3/13/#200303135</p>
		<pre class="programlisting">
#!/usr/bin/env python
import os, sys
print "I'm going to fork now - the child will write something to a pipe, and the parent will read it back"
r, w = os.pipe()           # r,w是文件描述符, 不是文件对象
pid = os.fork()
    os.close(w)           # 关闭一个文件描述符
    r = os.fdopen(r)      # 将r转化为文件对象
    print "parent: reading"
    txt = r.read()
    os.waitpid(pid, 0)   # 确保子进程被撤销
    os.close(r)
    w = os.fdopen(w, 'w')
    print "child: writing"
    w.write("here's some text from the child")
    w.close()
    print "child: closing"
    sys.exit(0)
print "parent: got it; text =", txt
		<pre class="programlisting">
import sys, os
if __name__ == "__main__":
    # do the UNIX double-fork magic, see Stevens' "Advanced
    # Programming in the UNIX Environment" for details (ISBN 0201563177)
        pid = os.fork()
        if pid &gt; 0:
            # exit first parent
            sys.exit(0)
    except OSError, e:
        print &gt;&gt;sys.stderr, "fork #1 failed: %d (%s)" % (e.errno, e.strerror)
        sys.exit(1)
    # decouple from parent environment
    os.chdir("/")
    os.setsid()
    os.umask(0)
    # do second fork
        pid = os.fork()
        if pid &gt; 0:
            # exit from second parent, print eventual PID before
            print "Daemon PID %d" % pid
            sys.exit(0)
    except OSError, e:
        print &gt;&gt;sys.stderr, "fork #2 failed: %d (%s)" % (e.errno, e.strerror)
        sys.exit(1)
    # start the daemon main loop
		<pre class="programlisting">
# Redirect standard file descriptors
sys.stdin = open('/dev/null', 'r')
sys.stdout = open('/dev/null', 'w')
sys.stderr = open('/dev/null', 'w')
</div><script xmlns="" type="text/javascript" id="clustrmaps" src="//cdn.clustrmaps.com/map_v2.js?u=r5HG&amp;d=9mi5r_kkDC8uxG8HuY3p4-2qgeeVypAK9vMD-2P6BYM"></script><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="random.html">上一页</a> </td><td width="20%" align="center"><a accesskey="u" href="index.html">上一级</a></td><td width="40%" align="right"> <a accesskey="n" href="python.log.html">下一页</a></td></tr><tr><td width="40%" align="left" valign="top">2.2. 随机数 </td><td width="20%" align="center"><a accesskey="h" href="../index.html">起始页</a></td><td width="40%" align="right" valign="top"> 2.4. 日志</td></tr></table></div><script xmlns="">
			(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
		(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
		m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
		})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
			ga('create', 'UA-11694057-1', 'auto');
			ga('send', 'pageview');
		</script><script xmlns="" async="async">
			var _hmt = _hmt || [];
			(function() {
			var hm = document.createElement("script");
			hm.src = "https://hm.baidu.com/hm.js?93967759a51cda79e49bf4e34d0b0f2c";
			var s = document.getElementsByTagName("script")[0];
			s.parentNode.insertBefore(hm, s);
</script><script xmlns="" async="async">
			(function(){
			var bp = document.createElement('script');
			var curProtocol = window.location.protocol.split(':')[0];
			if (curProtocol === 'https') {
			bp.src = 'https://zz.bdstatic.com/linksubmit/push.js';
			bp.src = 'http://push.zhanzhang.baidu.com/push.js';
			var s = document.getElementsByTagName("script")[0];
			s.parentNode.insertBefore(bp, s);
</script></body></html>
FilesExpand file tree

process.html

Latest commit

History

process.html

File metadata and controls
