forked from yasoob/intermediatePython
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathcontext_managers.html
More file actions
200 lines (187 loc) · 16.3 KB
/
Copy pathcontext_managers.html
File metadata and controls
200 lines (187 loc) · 16.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Context managers</title>
<link rel="stylesheet" href="_static/epub.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="function_caching.html" title="Function caching"
accesskey="P">previous</a></li>
<li class="nav-item nav-item-0"><a href="index.html">Python Tips 0.1 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="body" role="main">
<div class="section" id="context-managers">
<h1>Context managers</h1>
<p>Context managers allow you to allocate and release resources precisely
when you want to. The most widely used example of context managers is
the <code class="docutils literal"><span class="pre">with</span></code> statement. Suppose you have two related operations which
you’d like to execute as a pair, with a block of code in between.
Context managers allow you to do specifically that. For example:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s">'some_file'</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">opened_file</span><span class="p">:</span>
<span class="n">opened_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">'Hola!'</span><span class="p">)</span>
</pre></div>
</div>
<p>The above code opens the file, writes some data to it and then closes
it. If an error occurs while writing the data to the file, it tries to
close it. The above code is equivalent to:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="nb">file</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="s">'some_file'</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span>
<span class="k">try</span><span class="p">:</span>
<span class="nb">file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">'Hola!'</span><span class="p">)</span>
<span class="k">finally</span><span class="p">:</span>
<span class="nb">file</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>While comparing it to the first example we can see that a lot of
boilerplate code is eliminated just by using <code class="docutils literal"><span class="pre">with</span></code>. The main
advantage of using a <code class="docutils literal"><span class="pre">with</span></code> statement is that it makes sure our file
is closed without paying attention to how the nested block exits.</p>
<p>A common use case of context managers is locking and unlocking resources
and closing opened files (as I have already showed you).</p>
<p>Let’s see how we can implement our own Context Manager. This would allow
us to understand exactly what’s going on behind the scenes.</p>
<div class="section" id="implementing-context-manager-as-a-class">
<h2>Implementing Context Manager as a Class:</h2>
<p>At the very least a context manager has an <code class="docutils literal"><span class="pre">__enter__</span></code> and
<code class="docutils literal"><span class="pre">__exit__</span></code> methods defined. Let’s make our own file opening Context
Manager and learn the basics.</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">class</span> <span class="nc">File</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">file_name</span><span class="p">,</span> <span class="n">method</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">file_obj</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">file_name</span><span class="p">,</span> <span class="n">method</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">__enter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">file_obj</span>
<span class="k">def</span> <span class="nf">__exit__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">type</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">trace_back</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">file_obj</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>Just by defining <code class="docutils literal"><span class="pre">__enter__</span></code> and <code class="docutils literal"><span class="pre">__exit__</span></code> methods we can use it in
a <code class="docutils literal"><span class="pre">with</span></code> statement. Let’s try:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">with</span> <span class="n">File</span><span class="p">(</span><span class="s">'demo.txt'</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">opened_file</span><span class="p">:</span>
<span class="n">opened_file</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">'Hola!'</span><span class="p">)</span>
</pre></div>
</div>
<p>Our <code class="docutils literal"><span class="pre">__exit__</span></code> function accepts three arguments. They are required by
every <code class="docutils literal"><span class="pre">__exit__</span></code> method which is a part of a Context Manager class.
Let’s talk about what happens under-the-hood.</p>
<ol class="arabic simple">
<li>The <code class="docutils literal"><span class="pre">with</span></code> statement stores the <code class="docutils literal"><span class="pre">__exit__</span></code> method of <code class="docutils literal"><span class="pre">File</span></code>
class.</li>
<li>It calls the <code class="docutils literal"><span class="pre">__enter__</span></code> method of <code class="docutils literal"><span class="pre">File</span></code> class.</li>
<li><code class="docutils literal"><span class="pre">__enter__</span></code> method opens the file and returns it.</li>
<li>the opened file handle is passed to <code class="docutils literal"><span class="pre">opened_file</span></code>.</li>
<li>we write to the file using <code class="docutils literal"><span class="pre">.write()</span></code></li>
<li><code class="docutils literal"><span class="pre">with</span></code> statement calls the stored <code class="docutils literal"><span class="pre">__exit__</span></code> method.</li>
<li>the <code class="docutils literal"><span class="pre">__exit__</span></code> method closes the file.</li>
</ol>
</div>
<div class="section" id="handling-exceptions">
<h2>Handling exceptions</h2>
<p>We did not talk about the <code class="docutils literal"><span class="pre">type</span></code>, <code class="docutils literal"><span class="pre">value</span></code> and <code class="docutils literal"><span class="pre">trace_back</span></code>
arguments of the <code class="docutils literal"><span class="pre">__exit__</span></code> method. Between the 4th and 6th step, if
an exception occurs, Python passes the type, value and traceback of the
exception to the <code class="docutils literal"><span class="pre">__exit__</span></code> method. It allows the <code class="docutils literal"><span class="pre">__exit__</span></code> method
to decide how to close the file and if any further steps are required.
In our case we are not paying any attention to them.</p>
<p>What if our file object raises an exception? We might be trying to
access a method on the file object which it does not supports. For
instance:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">with</span> <span class="n">File</span><span class="p">(</span><span class="s">'demo.txt'</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">opened_file</span><span class="p">:</span>
<span class="n">opened_file</span><span class="o">.</span><span class="n">fuck</span><span class="p">(</span><span class="s">'Hola!'</span><span class="p">)</span>
</pre></div>
</div>
<p>Let’s list down the steps which are taken by the <code class="docutils literal"><span class="pre">with</span></code> statement when
an error is encountered.</p>
<ol class="arabic simple">
<li>It passes the type, value and traceback of the error to the
<code class="docutils literal"><span class="pre">__exit__</span></code> method.</li>
<li>It allows the <code class="docutils literal"><span class="pre">__exit__</span></code> method to handle the exception.</li>
<li>If <code class="docutils literal"><span class="pre">__exit__</span></code> returns True then the exception was gracefully
handled.</li>
<li>If anything else than True is returned by <code class="docutils literal"><span class="pre">__exit__</span></code> method then
the exception is raised by <code class="docutils literal"><span class="pre">with</span></code> statement.</li>
</ol>
<p>In our case the <code class="docutils literal"><span class="pre">__exit__</span></code> method returns <code class="docutils literal"><span class="pre">None</span></code> (when no return
statement is encountered then the method returns <code class="docutils literal"><span class="pre">None</span></code>). Therefore,
<code class="docutils literal"><span class="pre">with</span></code> statement raises the exception.</p>
<div class="highlight-python"><div class="highlight"><pre>Traceback (most recent call last):
File "<stdin>", line 2, in <module>
AttributeError: 'file' object has no attribute 'fuck'
</pre></div>
</div>
<p>Let’s try handling the exception in the <code class="docutils literal"><span class="pre">__exit__</span></code> method:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">class</span> <span class="nc">File</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
<span class="k">def</span> <span class="nf">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">file_name</span><span class="p">,</span> <span class="n">method</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">file_obj</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">file_name</span><span class="p">,</span> <span class="n">method</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">__enter__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">file_obj</span>
<span class="k">def</span> <span class="nf">__exit__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="nb">type</span><span class="p">,</span> <span class="n">value</span><span class="p">,</span> <span class="n">trace_back</span><span class="p">):</span>
<span class="k">print</span> <span class="s">"Exception has been handled"</span>
<span class="bp">self</span><span class="o">.</span><span class="n">file_obj</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
<span class="k">return</span> <span class="bp">True</span>
<span class="k">with</span> <span class="n">File</span><span class="p">(</span><span class="s">'demo.txt'</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span> <span class="k">as</span> <span class="n">opened_file</span><span class="p">:</span>
<span class="n">opened_file</span><span class="o">.</span><span class="n">fuck</span><span class="p">()</span>
<span class="c"># Output: Exception has been handled</span>
</pre></div>
</div>
<p>Our <code class="docutils literal"><span class="pre">__exit__</span></code> method returned True, therefore no exception was raised
by the <code class="docutils literal"><span class="pre">with</span></code> statement.</p>
<p>This is not the only way to implement context managers. There is another
way and we will be looking at it in this next section.</p>
</div>
<div class="section" id="implementing-a-context-manager-as-a-generator">
<h2>Implementing a Context Manager as a Generator</h2>
<p>We can also implement Context Managers using decorators and generators.
Python has a contextlib module for this very purpose. Instead of a
class, we can implement a Context Manager using a generator function.
Let’s see a basic, useless example:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="kn">from</span> <span class="nn">contextlib</span> <span class="kn">import</span> <span class="n">contextmanager</span>
<span class="nd">@contextmanager</span>
<span class="k">def</span> <span class="nf">open_file</span><span class="p">(</span><span class="n">name</span><span class="p">):</span>
<span class="n">f</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">name</span><span class="p">,</span> <span class="s">'wb'</span><span class="p">)</span>
<span class="k">yield</span> <span class="n">f</span>
<span class="n">f</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
</pre></div>
</div>
<p>Okay! This way of implementing Context Managers appear to be more
intuitive and easy. However, this method requires some knowledge about
generators, yield and decorators. In this example we have not caught any
exceptions which might occur. It works in mostly the same way as the
previous method.</p>
<p>Let’s disect this method a little.</p>
<ol class="arabic simple">
<li>Python encounters the <code class="docutils literal"><span class="pre">yield</span></code> keyword. Due to this it creates a
generator instead of a normal function.</li>
<li>Due to the decoration, contextmanager is called with the function
name (open_file) as it’s argument.</li>
<li>The <code class="docutils literal"><span class="pre">contextmanager</span></code> function returns the generator wrapped by the
<code class="docutils literal"><span class="pre">GeneratorContextManager</span></code> object.</li>
<li>The <code class="docutils literal"><span class="pre">GeneratorContextManager</span></code> is assigned to the <code class="docutils literal"><span class="pre">open_file</span></code>
function. Therefore, when we later call <code class="docutils literal"><span class="pre">open_file</span></code> function, we
are actually calling the <code class="docutils literal"><span class="pre">GeneratorContextManager</span></code> object.</li>
</ol>
<p>So now that we know all this, we can use the newly generated Context
Manager like this:</p>
<div class="highlight-python"><div class="highlight"><pre><span class="k">with</span> <span class="n">open_file</span><span class="p">(</span><span class="s">'some_file'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s">'hola!'</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer" role="contentinfo">
© Copyright 2015, Muhammad Yasoob Ullah Khalid.
</div>
</body>
</html>