forked from yasoob/intermediatePython
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathgenerators.html
More file actions
194 lines (181 loc) · 12.7 KB
/
Copy pathgenerators.html
File metadata and controls
194 lines (181 loc) · 12.7 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
"http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Generators</title>
<link rel="stylesheet" href="_static/epub.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
</head>
<body role="document">
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="map_&_filter.html" title="Map & Filter"
accesskey="N">next</a></li>
<li class="right" >
<a href="debugging.html" title="Debugging"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="index.html">Python Tips 0.1 documentation</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="body" role="main">
<div class="section" id="generators">
<h1>Generators</h1>
<p>First lets understand iterators. According to Wikipedia, an iterator is
an object that enables a programmer to traverse a container,
particularly lists. However, an iterator performs traversal and gives
access to data elements in a container, but does not perform iteration.
You might be confused so lets take it a bit slow. There are three parts
namely:</p>
<ul class="simple">
<li>Iterable</li>
<li>Iterator</li>
<li>Iteration</li>
</ul>
<p>All of these parts are linked to each other. We will discuss them one by
one and later talk about generators.</p>
<div class="section" id="iterable">
<h2>Iterable</h2>
<p>An <code class="docutils literal"><span class="pre">iterable</span></code> is any object in Python which has an <code class="docutils literal"><span class="pre">__iter__</span></code> or a
<code class="docutils literal"><span class="pre">__getitem__</span></code> method defined which returns an <strong>iterator</strong> or can take
indexes (Both of these dunder methods are fully explained in a previous
chapter). In short an <code class="docutils literal"><span class="pre">iterable</span></code> is any object which can provide us
with an <strong>iterator</strong>. So what is an <strong>iterator</strong>?</p>
</div>
<div class="section" id="iterator">
<h2>Iterator</h2>
<p>An iterator is any object in Python which has a <code class="docutils literal"><span class="pre">next</span></code> (Python2) or
<code class="docutils literal"><span class="pre">__next__</span></code> method defined. That’s it. That’s an iterator. Now let’s
understand <strong>iteration</strong>.</p>
</div>
<div class="section" id="iteration">
<h2>Iteration</h2>
<p>In simple words it is the process of taking an item from something e.g a
list. When we use a loop to loop over something it is called iteration.
It is the name given to the process itself. Now as we have a basic
understanding of these terms let’s understand <strong>generators</strong>.</p>
</div>
<div class="section" id="id1">
<h2>Generators</h2>
<p>Generators are iterators, but you can only iterate over them once. It’s
because they do not store all the values in memory, they generate the
values on the fly. You use them by iterating over them, either with a
‘for’ loop or by passing them to any function or construct that
iterates. Most of the time <code class="docutils literal"><span class="pre">generators</span></code> are implemented as functions.
However, they do not <code class="docutils literal"><span class="pre">return</span></code> a value, they <code class="docutils literal"><span class="pre">yield</span></code> it. Here is a
simple example of a <code class="docutils literal"><span class="pre">generator</span></code> function:</p>
<div class="code python highlight-python"><div class="highlight"><pre><span class="k">def</span> <span class="nf">generator_function</span><span class="p">():</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">):</span>
<span class="k">yield</span> <span class="n">i</span>
<span class="k">for</span> <span class="n">item</span> <span class="ow">in</span> <span class="n">generator_function</span><span class="p">():</span>
<span class="k">print</span><span class="p">(</span><span class="n">item</span><span class="p">)</span>
<span class="c"># Output: 0</span>
<span class="c"># 1</span>
<span class="c"># 2</span>
<span class="c"># 3</span>
<span class="c"># 4</span>
<span class="c"># 5</span>
<span class="c"># 6</span>
<span class="c"># 7</span>
<span class="c"># 8</span>
<span class="c"># 9</span>
</pre></div>
</div>
<p>It is not really useful in this case. Generators are best for
calculating large sets of results (particularly calculations involving
loops themselves) where you don’t want to allocate the memory for all
results at the same time. Python modified a lot of Python 2 functions
which returned <code class="docutils literal"><span class="pre">lists</span></code> to return <code class="docutils literal"><span class="pre">generators</span></code> in Python 3. It is
because <code class="docutils literal"><span class="pre">generators</span></code> are not resource intensive. Here is an example
which calculates fibonacci numbers:</p>
<div class="code python highlight-python"><div class="highlight"><pre><span class="c"># generator version</span>
<span class="k">def</span> <span class="nf">fibon</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="n">a</span> <span class="o">=</span> <span class="n">b</span> <span class="o">=</span> <span class="mi">1</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="k">yield</span> <span class="n">a</span>
<span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">b</span><span class="p">,</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span>
</pre></div>
</div>
<p>Now we can use it like this:</p>
<div class="code python highlight-python"><div class="highlight"><pre><span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">fibon</span><span class="p">(</span><span class="mi">1000000</span><span class="p">):</span>
<span class="k">print</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
</pre></div>
</div>
<p>This way we would not have to worry about it using a lot of resources.
However, if we would have implemented it like this:</p>
<div class="code python highlight-python"><div class="highlight"><pre><span class="k">def</span> <span class="nf">fibon</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="n">a</span> <span class="o">=</span> <span class="n">b</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">result</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">xrange</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="n">result</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="n">a</span><span class="p">,</span> <span class="n">b</span> <span class="o">=</span> <span class="n">b</span><span class="p">,</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span>
<span class="k">return</span> <span class="n">result</span>
</pre></div>
</div>
<p>It would have used up all our resources while calculating a large input.
We have discussed that we can iterate over <code class="docutils literal"><span class="pre">generators</span></code> only once but
we haven’t tested it. Before testing it you need to know about one more
built-in function of Python, <code class="docutils literal"><span class="pre">next()</span></code>. It allows us to access the next
element of a sequence. So let’s test out our understanding:</p>
<div class="code python highlight-python"><div class="highlight"><pre><span class="k">def</span> <span class="nf">generator_function</span><span class="p">():</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">3</span><span class="p">):</span>
<span class="k">yield</span> <span class="n">i</span>
<span class="n">gen</span> <span class="o">=</span> <span class="n">generator_function</span><span class="p">()</span>
<span class="k">print</span><span class="p">(</span><span class="nb">next</span><span class="p">(</span><span class="n">gen</span><span class="p">))</span>
<span class="c"># Output: 0</span>
<span class="k">print</span><span class="p">(</span><span class="nb">next</span><span class="p">(</span><span class="n">gen</span><span class="p">))</span>
<span class="c"># Output: 1</span>
<span class="k">print</span><span class="p">(</span><span class="nb">next</span><span class="p">(</span><span class="n">gen</span><span class="p">))</span>
<span class="c"># Output: 2</span>
<span class="k">print</span><span class="p">(</span><span class="nb">next</span><span class="p">(</span><span class="n">gen</span><span class="p">))</span>
<span class="c"># Output: Traceback (most recent call last):</span>
<span class="c"># File "<stdin>", line 1, in <module></span>
<span class="c"># StopIteration</span>
</pre></div>
</div>
<p>As we can see that after yielding all the values <code class="docutils literal"><span class="pre">next()</span></code> caused a
<code class="docutils literal"><span class="pre">StopIteration</span></code> error. Basically this error informs us that all the
values have been yielded. You might be wondering that why don’t we get
this error while using a <code class="docutils literal"><span class="pre">for</span></code> loop? Well the answer is simple. The
<code class="docutils literal"><span class="pre">for</span></code> loop automatically catches this error and stops calling
<code class="docutils literal"><span class="pre">next</span></code>. Do you know that a few built-in data types in Python also
support iteration? Let’s check it out:</p>
<div class="code python highlight-python"><div class="highlight"><pre><span class="n">my_string</span> <span class="o">=</span> <span class="s">"Yasoob"</span>
<span class="nb">next</span><span class="p">(</span><span class="n">my_string</span><span class="p">)</span>
<span class="c"># Output: Traceback (most recent call last):</span>
<span class="c"># File "<stdin>", line 1, in <module></span>
<span class="c"># TypeError: str object is not an iterator</span>
</pre></div>
</div>
<p>Well that’s not what we expected. The error says that <code class="docutils literal"><span class="pre">str</span></code> is not an
iterator. Well it is right! It is an iterable but not an iterator. This
means that it supports iteration but we can not directly iterate over
it. How can we then iterate over it? It’s time to learn about one more
built-in function, <code class="docutils literal"><span class="pre">iter</span></code>. It returns an <code class="docutils literal"><span class="pre">iterator</span></code> object from an
iterable. Here is how we can use it:</p>
<div class="code python highlight-python"><div class="highlight"><pre><span class="n">my_string</span> <span class="o">=</span> <span class="s">"Yasoob"</span>
<span class="n">my_iter</span> <span class="o">=</span> <span class="nb">iter</span><span class="p">(</span><span class="n">my_string</span><span class="p">)</span>
<span class="nb">next</span><span class="p">(</span><span class="n">my_iter</span><span class="p">)</span>
<span class="c"># Output: 'Y'</span>
</pre></div>
</div>
<p>Now that is much better. I am sure that you loved learning about
generators. Do bear it in mind that you can fully grasp this concept
only when you use it. Make sure that you follow this pattern and use
<code class="docutils literal"><span class="pre">generators</span></code> whenever they make sense to you. You wont be
disappointed!</p>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="footer" role="contentinfo">
© Copyright 2015, Muhammad Yasoob Ullah Khalid.
</div>
</body>
</html>