Python Functional Programming (Part 1) Concepts and itertools

Python Practical Tutorial_Spiritx's Blog - CSDN Blog.

Python functional programming is a programming paradigm that emphasizes the use of pure functions to process data. Functions are the fundamental building blocks of a program, and strive to avoid or minimize mutable states and side effects as much as possible. In functional programming, functions are considered first-class citizens and can be passed and stored like values.

Concepts of Functional Programming

Programming languages support deconstructing specific problems through the following methods:

• Most programming languages are Procedural A program is a series of instructions that tell a computer how to process program input. C. Pascal and even Unix shells are procedural languages.
• In the Declarative In language, you write an explanation to describe the problem to be solved, and the specific implementation of this language will indicate how to efficiently perform calculations. SQL may be your most familiar declarative language. A SQL query statement describes the dataset you want to retrieve, and the SQL engine will decide whether to scan the entire table or use indexes, which clauses should be executed first, and so on.
• Object oriented The program operates on a set of objects. The object has an internal state and can support methods to request and modify this internal state in some way. Smalltalk and Java are both object-oriented languages. C++ Python supports object-oriented programming, but does not mandate the use of object-oriented features.
• Functional expression Programming decomposes a problem into a series of functions. In an ideal situation, the function only accepts input and outputs the result, without affecting the internal state of the output for a given input. Famous functional languages include the ML family (Standard ML, Ocaml, and other variants) and Haskell.

Some language designers choose to emphasize a specific programming approach. This often makes it difficult to write programs in different ways. Other multi paradigm languages support several different programming approaches. Lisp, C++ Both Python and Python are multi paradigm languages; Using these languages, you can write programs and libraries that are primarily procedural, object-oriented, or functional. In large programs, different parts may be written in different ways; For example, GUI may be object-oriented while processing logic may be procedural or functional.

In a functional program, input flows through a series of functions. Each function takes input and outputs the result. The functional style opposes the use of functions with side effects that can modify internal states or cause changes that cannot be reflected in the return value of the function. A function that does not produce any side effects is called a "pure function". Eliminating side effects means that data structures that update with program execution cannot be used; The output of each function must depend only on the input
Some languages have very strict requirements for purity, to the extent that they do not have the same level of purity as a= 3 or c; A+ It is very difficult to completely eliminate side effects with assignment expressions like b. For example, displaying on the screen or writing to disk files are both side effects. For example, in Python, calling functions print() or time. sleep() does not return useful results; Their purpose is only for side effects, sending a paragraph of text to the screen or pausing for one second
Functional style Python programs are not extreme enough to eliminate all I/O or assignments; On the contrary, they will provide interfaces that are functional, but will use non functional features internally. For example, the implementation of a function still uses local variables, but does not modify global variables or have other side effects
Functional programming can be considered as the opposite of object-oriented programming. An object is like a small capsule, wrapped around an internal state and a set of call methods that allow you to modify this internal state, as well as a program composed of correct state changes. Functional programming aims to eliminate state changes as much as possible and only deal with data flowing through the function. In Python, you can combine two programming methods and write functions that accept and return object instances in your application (email messages, transaction processing).

Functional design may seem like a strange constraint in work. Why do you want to eliminate objects and side effects? However, the functional style has its theoretical and practical advantages:

• Formal proof.
• Modularization.
• Combination.
• Easy to debug and test.

Formal proof

A theoretical advantage is that it is relatively easier to construct mathematical proofs to prove that functional programs are correct.

For a long time, researchers have been interested in finding mathematical methods to prove the correctness of programs. This is different from testing with a large amount of input and finding that the program's output is basically correct, or reading the source code of a program and finding that the code looks fine; On the contrary, the goal here is a rigorous proof that the program can provide correct results for all possible inputs.

The technique used to prove the correctness of a program is to write invariant That is to say, the property of always being true for input data and variables in the program. Then for each line of code, you explain that the invariants X and Y before execution and the slightly different invariants X 'and Y' after execution are true. Until the end of the program, at this point, the invariant should be consistent with the expected state in the output of the program.

The reason why functional programming eliminates assignment is because assignment is difficult to handle in this technique; Assignment may break the true invariants before assignment, but it does not produce any new invariants that can be passed down.

Unfortunately, proving the correctness of a program is largely empirical and independent of Python software. Even insignificant programs require several pages of proof; The correctness proof of a moderately complex program can be very extensive, and there are very few or even no programs (Python interpreters, XML parsers, browsers) that you use in your daily life that can be proven to be correct. Even if you write or generate a proof, verifying the proof can still be a problem; There may have been an error inside, and you mistakenly believed that you proved the correctness of the program.

Modular

A more practical advantage of functional programming is that it forces you to break down problems into smaller aspects. Therefore, the program will be more modular. Compared to a large function that undergoes complex transformations, a small function is more explicit, easier to write, and easier to read and check for errors.

Easy to debug and test

Testing and debugging functional programs is relatively easier.

Debugging is simple because functions are usually small and clear. When the program fails to work, each function is an access point that can check if the data is correct. You can quickly find the function that went wrong by looking at the intermediate input and output.

Testing is easier because each function is a potential target for unit testing. Before executing the test, the function does not depend on the system state that needs to be reproduced; On the contrary, you only need to provide the correct input and then check if the output matches the expected result.

Combination

When writing functional style programs, you will write many functions with different inputs and outputs. Some of them are inevitably limited to specific applications, while others can be widely used in programs. For example, a function that accepts folder directories and returns XML files from all folders; Or a function that accepts the file name and then returns the file content can be applied in many different situations.

Over time, you will form a personal tool library. Usually, you can reorganize existing functions to form a new program, and then write some special functions for the current work.

Iterator

The iterator is the foundation of Python functional programming. Previously, we have repeatedly studied iterators and reviewed their core knowledge:

• An iterator is an object that represents a data stream; This object only returns one element at a time.
• The iterator must support__Next__() Method; This method does not accept parameters and always returns the next element in the data stream.
• If there are no elements in the data flow__Next__() will throw a StopIteration exception. Iterators may not necessarily be finite; It is entirely reasonable to construct an iterator that outputs an infinite stream of data.
• The built-in iter() function takes any object and attempts to return an iterator to output the content or elements of the object, and throws a TypeError exception when the object does not support iteration.
• Python has several built-in data types that support iteration, the most common of which are lists and dictionaries.
• If an object can generate an iterator, it will be called iterable.
• There are many scenarios in Python that use iterators, among which the most common is in for expressions, such as max(), min(), and some advanced functions learned earlier, which all support iterators.

Note that you can only advance sequentially in the iterator; There is no method to retrieve the previous element unless the iterator is reset or a new copy is made. The iterator object can provide these additional features, but the iterator protocol only specifies__Next__() Method. The function may exhaust the output of the iterator as a result, and if you want to perform different operations on the same data stream, you must create a new iterator.

We already know that types such as lists, tuples, strings, dictionaries, etc. support iterators, and custom classes can also be used to implement__Iter__() Method and__Next__Implement your own iterator using the () method.

The iterator also supports generator expressions and list derivations, which make operations simpler and clearer.

List derivation formula:

[ expression for expr in sequence1
       if condition1
       for expr2 in sequence2
       if condition2
       for expr3 in sequence3 ...
       if condition3
       for exprN in sequenceN
       if conditionN ]

Equivalent to:

for expr1 in sequence1:
if not (condition1):
  continue   # Skip this element
for expr2 in sequence2:
  if not (condition2):
      continue   # Skip this element
  ...
  for exprN in sequenceN:
      if not (conditionN):
          continue   # Skip this element
      # Output the value of
      # the expression.

The generator expression only needs to replace the [] above with ().

High order functions such as map(), reduce(), and filter() actually operate in a functional programming manner. In order to support more functional programming, the itertools module supports more commonly used iterators and functions for combining iterators.

Itertools module

This module implements a series of iterators inspired by functional programming languages such as APL, Haskell, and SML. In order to be applicable to Python, they have all been rewritten.

This module standardizes a core toolset for fast and efficient memory utilization, both in themselves and in combination. They together form an "iterator algebra", which makes it possible to create concise and efficient specialized tools in pure Python.

For example, SML has a tabulate tool: tabulate (f), which can generate a sequence f (0), f (1). In Python, you can combine map() and count() to implement: map (f, count()).

These tools can also provide good performance and integrate with the functionality of the operator module.

Iterator

In addition to using the original general iterators, itertools also provide many extended iterators.

Infinite iterator

The biggest feature of an infinite iterator is that it can iterate elements infinitely, so when using it, it is important to have constraints to control the stop of the iterator, otherwise it will cause a dead loop.

Number iterator count()

Itertools. count ( start= 0 , step= 1 ).

Create an iterator that starts with the start value and returns evenly spaced values. Commonly used as arguments in map() to generate continuous data points. In addition, it is also used for zip() to add a serial number.

Parameter Description:

• Start: Starting value
• Step: Interval value

Roughly equivalent to:

def count(start=0, step=1):
# count(10) --> 10 11 12 13 14 ...
# count(2.5, 0.5) --> 2.5 3.0 3.5 ...
n = start
while True:
  yield n
  n += step

When counting floating-point numbers, sometimes replacing them with multiplication code can result in better accuracy, such as (start + step * i for i in count()).

import itertools
Fruits = ['pear', 'peach', 'apple', 'grape', 'banana', 'cherry', 'strawberry', 'watermelon']
fruitList = list(zip(itertools.count(start=1), Fruits))
print(fruitList)
'’'
[(1, 'pear'), (2, 'peach'), (3, 'apple'), (4, 'grape'), (5, 'banana'), (6, 'cherry'), (7, 'strawberry'), (8, 'watermelon')]
'''

Loop iterator cycle()

Itertools. cycle ( iterable ).

Create an iterator that returns all elements in iterable and saves a copy. When all elements in iterable are retrieved, all elements in the replica are returned. Infinite repetition. Roughly equivalent to:

def cycle(iterable):
# cycle('ABCD') --> A B C D A B C D A B C D ...
saved = []
for element in iterable:
  yield element
  saved.append(element)
while saved:
  for element in saved:
        yield element

import itertools
import datetime
Days =['monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday']
BaseDate = datetime.date(2023, 9, 4)
datelist = [BaseDate + datetime.timedelta(days=d) for d in range(10)]
print(datelist)
daylist = list(zip(datelist, itertools.cycle(Days)))
print(daylist)
'’'
[datetime.date(2023, 9, 4), datetime.date(2023, 9, 5), datetime.date(2023, 9, 6), datetime.date(2023, 9, 7), datetime.date(2023, 9, 8), datetime.date(2023, 9, 9), datetime.date(2023, 9, 10), datetime.date(2023, 9, 11), datetime.date(2023, 9, 12), datetime.date(2023, 9, 13)]
[(datetime.date(2023, 9, 4), 'monday'), (datetime.date(2023, 9, 5), 'tuesday'), (datetime.date(2023, 9, 6), 'wednesday'), (datetime.date(2023, 9, 7), 'thursday'), (datetime.date(2023, 9, 8), 'friday'), (datetime.date(2023, 9, 9), 'saturday'), (datetime.date(2023, 9, 10), 'sunday'), (datetime.date(2023, 9, 11), 'monday'), (datetime.date(2023, 9, 12), 'tuesday'), (datetime.date(2023, 9, 13), 'wednesday')]
'''

Repeater iteratorrepet()

Itertools. repeat ( object [, times ]).

Repeatedly provide times for objects, if times are not set, provide an infinite number.

import itertools
for it in itertools.repeat('fruits', times=3):
print(it)
’’’
fruits
fruits
fruits
'’'

Short sequence iterator

Iterator stopped based on the shortest input sequence length .

Cumulative iterator Accumulate

Itertools. acumulate ( iterable [, fun , * , initial= None ]).

Create an iterator that returns the cumulative sum or the cumulative result value of other binary operation functions (specified by the optional function parameter).

For example, a sequence is [1,2,3,4,5].

• The first value of the iterator is equal to the first value of the sequence, which is 1
• The second value of the iterator is the first value of the iterator plus sequence The second value of 1; 2 is 3
• The third value of the iterator is the second value of the iterator plus the third value of the sequence, 3; 3 is 6
• And so on

import itertools
nums = [1,2,3,4,5]
anums = [it for it in itertools.accumulate(nums)]
print(anums)
'’'
[1, 3, 6, 10, 15]
'''

If fun is provided, it should be a function with two parameters. The input element for iterable can be of any type that Func can accept as a parameter. (For example, for default addition operations, elements can be of any additive type including Decimal or Fraction.).

For example, if we change to continuous multiplication:

import itertools
nums = [1,2,3,4,5]
anums = [it for it in itertools.accumulate(nums, func=lambda a,b:a*b)]
print(anums)
'’'
[1, 2, 6, 24, 120]
'''

Usually, the number of elements output is consistent with the number of iterable objects input. However, if the keyword parameter initial is provided, the accumulation will start with the initial value, resulting in an additional element in the output compared to the input iterable object.

Sequence connected iterator chain

Itertools. chain ( * iterables ).

Create an iterator that first returns all elements in the first iterable object, then returns all elements in the next iterable object until all elements in the iterable object are exhausted. Multiple sequences can be processed into a single sequence.

import itertools
fruits = ['mango', 'pear', 'peach']
nums = [1,2,3,4,5]
anums = list(itertools.chain(fruits, nums))
print(anums)
'’'
['mango', 'pear', 'peach', 1, 2, 3, 4, 5]
'''

Expand iterator chain. from_Iterable

Classmethod chain. from_Iterable.

A nested list can be expanded into a one-dimensional iterator.

import itertools
fruits = ['mango', 'pear', 'peach']
nums = [1,2,3,4,5]
anums = list(itertools.chain.from_iterable([fruits, nums, 'ABCDE']))
print(anums)
alist = [['mango', 'pear', 'peach'], ['grape', 'banana', 'cherry'], (['a', 'b', 'c'], ('n', 'l', 1)), {'k1':'v1', 'k2':'v2'}]
nlist = list(itertools.chain.from_iterable(alist))
print(nlist)
blist = list(itertools.chain(alist))
print(blist)
'’'
['mango', 'pear', 'peach', 1, 2, 3, 4, 5, 'A', 'B', 'C', 'D', 'E']
['mango', 'pear', 'peach', 'grape', 'banana', 'cherry', ['a', 'b', 'c'], ('n', 'l', 1), 'k1', 'k2']
[['mango', 'pear', 'peach'], ['grape', 'banana', 'cherry'], (['a', 'b', 'c'], ('n', 'l', 1)), {'k1': 'v1', 'k2': 'v2'}]'''

From the above example, it can be seen that expanding only once will not delve deeper.

Simple filter iterator compress

Itertools. compress ( data , selectors ).

Create an iterator that returns the elements in data (iterable object) that have been tested as True by selectors truth value. The iterator stops at the shorter length of both.

print(list(itertools.compress(['mango', 'pear', 'peach', 'grape', 'banana'], [1,0,0,1])))
#['mango', 'grape']

Skip opening iterator dropwhile

Itertools. dropwhile ( predict , iterable ).

Predicate is a function object that takes a parameter, which is an element of iterable.

Create an iterator that discards the preceding elements if their predict values are consecutive true. If the first false element appears, the iterator starts returning the following elements (no longer checking if their predict values are true) . Note that the iterator will not produce any output (skip directly) until predict is first false, so it may require a certain amount of startup time. Roughly equivalent to:

def dropwhile(predicate, iterable):
# dropwhile(lambda x: x<5, [1,4,6,4,1]) --> 6 4 1
iterable = iter(iterable)
for x in iterable:
  if not predicate(x):
      yield x
      break
for x in iterable:
  yield x

Example:

import itertools
import datetime
Days =['monday', 'tuesday', 'wednesday', 'thursday', 'friday', 'saturday', 'sunday']
BaseDate = datetime.date(2023, 9, 1)
def checkMonday(d:datetime.date):
print(f'{d}, {d.weekday()}')
if d.weekday() == 0:
  return False
else:
  return True
datelist = [BaseDate + datetime.timedelta(days=d) for d in range(14)]
print(datelist)
daylist = list(zip(Days, itertools.dropwhile(checkMonday,datelist)))
print(daylist)
'’’
[datetime.date(2023, 9, 1), datetime.date(2023, 9, 2), datetime.date(2023, 9, 3), datetime.date(2023, 9, 4), datetime.date(2023, 9, 5), datetime.date(2023, 9, 6), datetime.date(2023, 9, 7), datetime.date(2023, 9, 8), datetime.date(2023, 9, 9), datetime.date(2023, 9, 10), datetime.date(2023, 9, 11), datetime.date(2023, 9, 12), datetime.date(2023, 9, 13), datetime.date(2023, 9, 14)]
2023-09-01, 4
2023-09-02, 5
2023-09-03, 6
2023-09-04, 0
[('monday', datetime.date(2023, 9, 4)), ('tuesday', datetime.date(2023, 9, 5)), ('wednesday', datetime.date(2023, 9, 6)), ('thursday', datetime.date(2023, 9, 7)), ('friday', datetime.date(2023, 9, 8)), ('saturday', datetime.date(2023, 9, 9)), ('sunday', datetime.date(2023, 9, 10))]
’'’

Skip tail iterator takewhile

Itertools. takewhile ( predict , iterable ).

Create an iterator that returns an element from an iterable object as long as the predicate of the preceding element is continuous and true. If any element's predicate is false, terminate immediately (no longer checking the predicate of subsequent elements) .

Predicate is a function object that takes a parameter, which is an element of iterable.

alist = list(itertools.takewhile(lambda x:x<10, [1,3,5,6,7,11,3,2,1]))
print(alist) #[1, 3, 5, 6, 7]

tee iterator

Itertools. tee ( iterable , n= 2 ).

Returns n independent iterators from an iterable object.

For example, changing [1,2,3,4,5] to [1,2,3,4,5], [1,2,3,4,5].

its = itertools.tee([1,2,3,4,5,6,7], 3)
for it in its:
print(list(it))
'’'
[1, 2, 3, 4, 5, 6, 7]
[1, 2, 3, 4, 5, 6, 7]
[1, 2, 3, 4, 5, 6, 7]
'''

Zip completion iterator

Itertools.zip_Longest ( * iterables , fillvalue= None ).

Zip will compress the iterator according to the shortest iteration object, zip_Longest() compresses according to the longest, and if an iterator is not long enough, use fillvalue to fill in.

alist = [1,2,3,4,5,6,7,8]
blist = ['mango', 'pear', 'peach', 'grape', 'banana', 'cherry','apricot', 'persimmon',
     'medlar', 'watermelon', 'apple', 'pomegranate', 'currant', 'blackberry', 'avocado']
zip1 = zip(alist, blist)
print(list(zip1))
#[(1, 'mango'), (2, 'pear'), (3, 'peach'), (4, 'grape'), (5, 'banana'), (6, 'cherry'), (7, 'apricot'), (8, 'persimmon')]
zip2 = itertools.zip_longest(alist, blist, fillvalue=0)
print(list(zip2))
#[(1, 'mango'), (2, 'pear'), (3, 'peach'), (4, 'grape'), (5, 'banana'), (6, 'cherry'), (7, 'apricot'), (8, 'persimmon'), (0, 'medlar'), (0, 'watermelon'), (0, 'apple'), (0, 'pomegranate'), (0, 'currant'), (0, 'blackberry'), (0, 'avocado')]

Negative filter iterator filterfalse

Itertools. filterfalse ( predict , iterable ).

Create an iterator. If predict is true, the iterator discards these elements and returns the ones that are followed by False. If predict is not None, the iterator directly judges the authenticity of the elements, which is roughly equivalent to: .

def filterfalse(predicate, iterable):
# filterfalse(lambda x: x%2, range(10)) --> 0 2 4 6 8
if predicate is None:
  predicate = bool
for x in iterable:
  if not predicate(x):
      yield x

Example:

import itertools
#odd infinite sequences 
odd = itertools.filterfalse(lambda x:x%2 == 0, itertools.count(start=1))
for num in odd:
print(num)
if num == 11:
  break
'’’
1
3
5
7
9
11
’'’

Group iterator groupby

Itertools. groupby ( iterable , key= None ).

Create an iterator that returns adjacent duplicate keys grouped together in an iterable. Key is a function that calculates the key value of an element. If not specified or set to None, the key defaults to the identity function and returns the element unchanged. Generally speaking, iterable needs to be pre sorted using the same key value function.

The groupby() operation is similar to uniq in Unix. When the key value generated by the key function changes each time, the iterator will group or generate a new group (which is why it is usually necessary to use the same key value function to sort the data first). This behavior is different from SQL's GROUP BY operation, which ignores the order of input and groups elements with the same key value into the same group.

The returned group itself is also an iterator, which shares the underlying iterable object with groupby(). Because the source is shared, when the groupby() object iterates backwards, the previous group will disappear. Therefore, if you need to return the results later, you can save them as a list:

groups = []
uniquekeys = []
data = sorted(data, key=keyfunc)
for k, g in groupby(data, keyfunc):
groups.append(list(g))      # Store group iterator as a list
uniquekeys.append(k)

Roughly equivalent to:

class groupby:
# [k for k, g in groupby('AAAABBBCCDAABBB')] --> A B C D A B
# [list(g) for k, g in groupby('AAAABBBCCD')] --> AAAA BBB CC D
def __init__(self, iterable, key=None):
  if key is None:
      key = lambda x: x
  self.keyfunc = key
  self.it = iter(iterable)
  self.tgtkey = self.currkey = self.currvalue = object()
def __iter__(self):
  return self
def __next__(self):
  self.id = object()
  while self.currkey == self.tgtkey:
      self.currvalue = next(self.it)    # Exit on StopIteration
      self.currkey = self.keyfunc(self.currvalue)
  self.tgtkey = self.currkey
  return (self.currkey, self._grouper(self.tgtkey, self.id))
def _grouper(self, tgtkey, id):
  while self.id is id and self.currkey == tgtkey:
      yield self.currvalue
      try:
          self.currvalue = next(self.it)
      except StopIteration:
          return
      self.currkey = self.keyfunc(self.currvalue)

Example 1:

import itertools
fruits = ['hippopotamus', 'pear', 'peach', 'grape', 'banana', 'cherry','mulberry', 'persimmon',
     'strawberry', 'watermelon', 'apple', 'pomegranate']
newfruits = itertools.groupby(fruits, key=len)
# newfruits = itertools.groupby(fruits, key=lambda x:x[0])
for k,v in newfruits:
print(f'---{k}---')
for vit in v:
  print(vit)
’’’
---12---
hippopotamus
---4---
pear
---5---
peach
grape
---6---
banana
cherry
---8---
mulberry
---9---
persimmon
---10---
strawberry
watermelon
---5---
apple
---11---
pomegranate
'’'

From the above example, there are two groups of 5.

Example 2:

import itertools
fruits = ['hippopotamus', 'pear', 'peach', 'grape', 'banana', 'cherry','mulberry', 'persimmon',
     'strawberry', 'watermelon', 'apple', 'pomegranate']
# newfruits = itertools.groupby(fruits, key=len)
newfruits = itertools.groupby(fruits, key=lambda x:x[0])
for k,v in newfruits:
print(f'---{k}---')
for vit in v:
  print(vit)
’’’
---h---
hippopotamus
---p---
pear
peach
---g---
grape
---b---
banana
---c---
cherry
---m---
mulberry
---p---
persimmon
---s---
strawberry
---w---
watermelon
---a---
apple
---p---
pomegranate
'’'

There are groups of 3 p. The group is not sorted and requires us to sort it in advance.

import itertools
fruits = ['hippopotamus', 'pear', 'peach', 'grape', 'banana', 'cherry','mulberry', 'persimmon',
     'strawberry', 'watermelon', 'apple', 'pomegranate']
fruitsnew = sorted(fruits, key=lambda x:x[0])
# newfruits = itertools.groupby(fruits, key=len)
newfruits = itertools.groupby(fruitsnew, key=lambda x:x[0])
for k,v in newfruits:
print(f'---{k}---')
for vit in v:
  print(vit)
'’'
---a---
apple
---b---
banana
---c---
cherry
---g---
grape
---h---
hippopotamus
---m---
mulberry
---p---
pear
peach
persimmon
pomegranate
---s---
strawberry
---w---
watermelon
'''

Slice iterator

Itertools. ice ( iterable , stop ).

Itertools. ice ( iterable , start , stop [, step ]).

Get a slice of iterable, where start is the starting position, stop is the ending position, and step is the step size.

Equivalent to iterable [start: stop: step] .

import itertools
fruits = ['mango', 'pear', 'peach', 'grape', 'banana', 'cherry','apricot', 'persimmon',
     'medlar', 'watermelon', 'apple', 'pomegranate', 'currant', 'blackberry', 'avocado',
    'walnut', 'walnut', 'coconut', 'bilberry', 'plum']
for it in itertools.islice(fruits, 3, 10, 2):
print(it)
'’'
grape
cherry
persimmon
watermelon
'''

Overlap pair iterator pairwise

Itertools. pairwise ( iterable ).

Returns the consecutive overlapping pairs obtained from the input iterable .

The number of 2-tuples in the output iterator will be one less than the number of inputs. If there are less than two values in the input iterable object, it will be empty.

import itertools
fruits = ['mango', 'pear', 'peach', 'grape', 'banana']
for it in itertools.pairwise(fruits):
print(it)
for it in itertools.pairwise('ABCDE'):
print(it)
'’'
('mango', 'pear')
('pear', 'peach')
('peach', 'grape')
('grape', 'banana')
('A', 'B')
('B', 'C')
('C', 'D')
('D', 'E')
'''

Multi parameter map starmap

Itertools. starmap ( function , iterable ).

The map function can only accept a single parameter, while startmap can accept multiple parameters. It requires iterable to be an iterative object of an element (similar to: [(a1, a2.), (b1, b2.).], where each tuple is a set of parameters for the function.

import itertools
datalist = [(2,4), (3,7), (19,14), (20,31)]
sumlist = list(itertools.starmap(lambda x,y:x+y, datalist))
print(sumlist) #[6, 10, 33, 51] , similar to 2 +4,3+7,19+14,20+31
multilist = list(itertools.starmap(lambda x,y:x*y, datalist))
print(multilist) #[8, 21, 266, 620]
powlist = list(itertools.starmap(lambda x,y:x**y, datalist))
print(powlist) #[16, 2187, 799006685782884121, 21474836480000000000000000000000000000000]

permutation combination iterator

Permutations

Itertools. permutations ( iterable , r= None ).

Continuously returns an arrangement of length r generated by the iterable element.

If r is not specified or is None, r defaults to the length of iterable , in which case all full length arrangements are generated.

import itertools
fruits = ['mango', 'pear', 'peach']
for it in itertools.permutations(fruits):
print(it)
’’’
('mango', 'pear', 'peach')
('mango', 'peach', 'pear')
('pear', 'mango', 'peach')
('pear', 'peach', 'mango')
('peach', 'mango', 'pear')
('peach', 'pear', 'mango')
'’'

If r is set, selecting r elements from iterable for full arrangement will result in a relatively large amount of data .

import itertools
fruits = ['mango', 'pear', 'peach', 'grape', 'banana']
for it in itertools.permutations(fruits, 3):
print(it)
’’’
('mango', 'pear', 'peach')
('mango', 'pear', 'grape')
('mango', 'pear', 'banana')
('mango', 'peach', 'pear')
('mango', 'peach', 'grape')
('mango', 'peach', 'banana')
('mango', 'grape', 'pear')
('mango', 'grape', 'peach')
('mango', 'grape', 'banana')
('mango', 'banana', 'pear')
('mango', 'banana', 'peach')
('mango', 'banana', 'grape')
('pear', 'mango', 'peach')
('pear', 'mango', 'grape')
('pear', 'mango', 'banana')
('pear', 'peach', 'mango')
('pear', 'peach', 'grape')
('pear', 'peach', 'banana')
('pear', 'grape', 'mango')
('pear', 'grape', 'peach')
('pear', 'grape', 'banana')
('pear', 'banana', 'mango')
('pear', 'banana', 'peach')
('pear', 'banana', 'grape')
('peach', 'mango', 'pear')
('peach', 'mango', 'grape')
('peach', 'mango', 'banana')
('peach', 'pear', 'mango')
('peach', 'pear', 'grape')
('peach', 'pear', 'banana')
('peach', 'grape', 'mango')
('peach', 'grape', 'pear')
('peach', 'grape', 'banana')
('peach', 'banana', 'mango')
('peach', 'banana', 'pear')
('peach', 'banana', 'grape')
('grape', 'mango', 'pear')
('grape', 'mango', 'peach')
('grape', 'mango', 'banana')
('grape', 'pear', 'mango')
('grape', 'pear', 'peach')
('grape', 'pear', 'banana')
('grape', 'peach', 'mango')
('grape', 'peach', 'pear')
('grape', 'peach', 'banana')
('grape', 'banana', 'mango')
('grape', 'banana', 'pear')
('grape', 'banana', 'peach')
('banana', 'mango', 'pear')
('banana', 'mango', 'peach')
('banana', 'mango', 'grape')
('banana', 'pear', 'mango')
('banana', 'pear', 'peach')
('banana', 'pear', 'grape')
('banana', 'peach', 'mango')
('banana', 'peach', 'pear')
('banana', 'peach', 'grape')
('banana', 'grape', 'mango')
('banana', 'grape', 'pear')
('banana', 'grape', 'peach')
'’'

Combining iterators

Itertools combinations ( iterable , r ).

Returns a subsequence of length r composed of elements from input iterable , and a combination sequence of r elements selected from iterable .

alist = itertools.combinations('ABCDEFG', 3)
print(list(alist))
#[('A', 'B', 'C'), ('A', 'B', 'D'), ('A', 'B', 'E'), ('A', 'B', 'F'), ('A', 'B', 'G'), ('A', 'C', 'D'), ('A', 'C', 'E'), ('A', 'C', 'F'), ('A', 'C', 'G'), ('A', 'D', 'E'), ('A', 'D', 'F'), ('A', 'D', 'G'), ('A', 'E', 'F'), ('A', 'E', 'G'), ('A', 'F', 'G'), ('B', 'C', 'D'), ('B', 'C', 'E'), ('B', 'C', 'F'), ('B', 'C', 'G'), ('B', 'D', 'E'), ('B', 'D', 'F'), ('B', 'D', 'G'), ('B', 'E', 'F'), ('B', 'E', 'G'), ('B', 'F', 'G'), ('C', 'D', 'E'), ('C', 'D', 'F'), ('C', 'D', 'G'), ('C', 'E', 'F'), ('C', 'E', 'G'), ('C', 'F', 'G'), ('D', 'E', 'F'), ('D', 'E', 'G'), ('D', 'F', 'G'), ('E', 'F', 'G')]

Repeatable combination iterators combinations_With_Replacement

Itertools combinations_With_Replacement ( iterable , r ).

The elements selected by the combination iterator above are all different, so you can choose yourself.

alist = itertools.combinations_with_replacement('ABCDEFG', 3)
print(list(alist))
#[('A', 'A', 'A'), ('A', 'A', 'B'), ('A', 'A', 'C'), ('A', 'A', 'D'), ('A', 'A', 'E'), ('A', 'A', 'F'), ('A', 'A', 'G'), ('A', 'B', 'B'), ('A', 'B', 'C'), ('A', 'B', 'D'), ('A', 'B', 'E'), ('A', 'B', 'F'), ('A', 'B', 'G'), ('A', 'C', 'C'), ('A', 'C', 'D'), ('A', 'C', 'E'), ('A', 'C', 'F'), ('A', 'C', 'G'), ('A', 'D', 'D'), ('A', 'D', 'E'), ('A', 'D', 'F'), ('A', 'D', 'G'), ('A', 'E', 'E'), ('A', 'E', 'F'), ('A', 'E', 'G'), ('A', 'F', 'F'), ('A', 'F', 'G'), ('A', 'G', 'G'), ('B', 'B', 'B'), ('B', 'B', 'C'), ('B', 'B', 'D'), ('B', 'B', 'E'), ('B', 'B', 'F'), ('B', 'B', 'G'), ('B', 'C', 'C'), ('B', 'C', 'D'), ('B', 'C', 'E'), ('B', 'C', 'F'), ('B', 'C', 'G'), ('B', 'D', 'D'), ('B', 'D', 'E'), ('B', 'D', 'F'), ('B', 'D', 'G'), ('B', 'E', 'E'), ('B', 'E', 'F'), ('B', 'E', 'G'), ('B', 'F', 'F'), ('B', 'F', 'G'), ('B', 'G', 'G'), ('C', 'C', 'C'), ('C', 'C', 'D'), ('C', 'C', 'E'), ('C', 'C', 'F'), ('C', 'C', 'G'), ('C', 'D', 'D'), ('C', 'D', 'E'), ('C', 'D', 'F'), ('C', 'D', 'G'), ('C', 'E', 'E'), ('C', 'E', 'F'), ('C', 'E', 'G'), ('C', 'F', 'F'), ('C', 'F', 'G'), ('C', 'G', 'G'), ('D', 'D', 'D'), ('D', 'D', 'E'), ('D', 'D', 'F'), ('D', 'D', 'G'), ('D', 'E', 'E'), ('D', 'E', 'F'), ('D', 'E', 'G'), ('D', 'F', 'F'), ('D', 'F', 'G'), ('D', 'G', 'G'), ('E', 'E', 'E'), ('E', 'E', 'F'), ('E', 'E', 'G'), ('E', 'F', 'F'), ('E', 'F', 'G'), ('E', 'G', 'G'), ('F', 'F', 'F'), ('F', 'F', 'G'), ('F', 'G', 'G'), ('G', 'G', 'G')]

Cartesian set iterator product

Itertools. product ( * iterables , repeat= 1 ).

The Cartesian product of iterable object inputs.

Roughly equivalent to nested loops in generator expressions. For example, product(A, B) and ((x,y) for x in A for y in B) return the same result.

Nested loops loop like odometers, iterating backwards on the rightmost element during each iteration. This pattern forms a lexicographic order, so if the input iterable object is sorted, Cartesian product tuples are sequentially emitted.

To calculate the Cartesian product of the iterable object itself, set the optional parameter repeat to the number of times to repeat. For example, product(A, repeat=4) and product(A, A, A, A) are the same.

import itertools
fruits = ['mango', 'pear', 'peach']
ids =['1', '2']
for it in itertools.product(fruits, ids):
print(it)
'’'
('mango', '1')
('mango', '2')
('pear', '1')
('pear', '2')
('peach', '1')
('peach', '2')
'''

import itertools
fruits = ['mango', 'pear', 'peach']
ids =['1', '2']
for it in itertools.product(fruits, repeat=2):
print(it)
print('----')
for it in itertools.product(fruits, fruits):
print(it)
'’'
('mango', 'mango')
('mango', 'pear')
('mango', 'peach')
('pear', 'mango')
('pear', 'pear')
('pear', 'peach')
('peach', 'mango')
('peach', 'pear')
('peach', 'peach')
----
('mango', 'mango')
('mango', 'pear')
('mango', 'peach')
('pear', 'mango')
('pear', 'pear')
('pear', 'peach')
('peach', 'mango')
('peach', 'pear')
('peach', 'peach')
'''