Thinking in Functions: Functional Programming in Python

Thinking in Functions
atmb4u
CTO @ Profoundis Inc.

Agenda
★ Basics of Functional Programming
★ Functional Programming and Python
★ Everyday Functional Programming
★ Scaling with Functional Programming
★ Performance Implications
★ Data Explorations

House rules
★ Show of hands please
★ Ask anytime - Raise your hands
★ Be hands on
★ Free to use online resources
★ One step at a time
★ No-one left out policy

About Functional
Programming ★ Formal Provability

About Functional
Programming ★ Modularity

About Functional
Programming
★ Ease of Debugging and
Testing

About Functional
Programming ★ Composability

Basics of Functional
Programming
★ Function as the atomic entity
○ First-class functions
○ Higher-order functions
○ Pure functions

Programming
★ Side Effects
○ Avoid state
○ Immutability

Programming ★ Lazy Evaluation

Programming ★ Type Systems

Programming
★ Recursion, Tail Recursion, Iterators,
Currying, Sequences, pattern matching,
monads....

Myths of Functional
Programming
★ It requires immutability/pureness
★ It requires an advanced type system
★ It is significantly less efficient
★ It makes you learn advanced math
★ You must give up all your imperative
programming notions
★ Object orientation and functional paradigms
are incompatible
★ Functional programs are easier to debug
★ Dynamic types are better than Static types

Functional methodology in Python
★ itertools and functools
★ Decorators and Generators
★ What python CAN do
○ lazy eval, lambda, map, reduce, filter
★ What python CANNOT do (pragmatically)
○ Eg: Tail Recursion, Pure immutability, Pure Functions

Functional Programming:
Done everyday
★ map
★ reduce
★ filter
★ enumerate
★ iter
★ list comprehension
★ Generator Expressions
★ Generators
★ itertools
★ functools
★ operator
★ decorator
★ Lambda Functions

List Comprehension
data = [1,2,3,4,5]
[item * 2 for item in data]

Generator Expression
data = [1,2,3,4,5]
generator_exp = (i*i for i in data)
sum(generator_exp)

operator module
import operator
operator.add(40,2)
operator.sub(45,3)
data = {'company': 'Acme Corp.'}
getCompany = operator.itemgetter('company')
getCompany(data)

enumerate
for item in enumerate(['subject', 'verb', 'object']):
print item

map
def upper(s):
return s.upper()
data = [“pycon”,”india”]
map(upper, data)

Map - Explained
from urllib import urlopen
urls = ['http://www.google.com',
'http://www.wikipedia.com',
'http://www.apple.com',
'http://www.python.org'
]
result = []
for item in urls:
result.append(urlopen(item))
return result
def fib(n):
a, b = 0, 1
for i in range(n):
a, b = b, a + b
....
integers = [1, 2, 3, 4, 5]
result = []
for item in integers:
result.append(fib(item))
return result
?

Map - Explained
from urllib import urlopen
urls = ['http://www.google.com',
'http://www.wikipedia.com',
'http://www.apple.com',
'http://www.python.org'
]
result = []
for item in urls:
result.append(urlopen(item))
return result
def fib(n):
a, b = 0, 1
for i in range(n):
a, b = b, a + b
...
integers = [1, 2, 3, 4, 5]
result = []
for item in integers:
result.append(fib(item))
return result
def map(function, sequence):
result = []
for item in sequence:
result.append(function(item))
return result
html_texts = map(urlopen, urls)
fib_integers = map(fib, integers)

Lambda
count_lambda =lambda w: len(w)
print map(count_lambda, 'It is raining cats and dogs'.split())
#conditions in lambda
lambda x: True if x % 2 == 0 else False

reduce
import operator
data = ["PyCon", "India", "2015"]
reduce(operator.concat, data)

Reduce - Explained
# Sum of a list of numbers
def add(x, y):
return x + y
def sum(data):
result = 0
for x in data:
result = add(result, x)
return result sum([5, 2, 3])
# Smallest in a list
def lesser(x, y):
if x < y:
return x
else:
return y
def min(data):
result = 999999999999
for x in data:
result = lesser(result, x)
return result min([5, 2, 3])
?

Reduce - Explained
# Sum of a list of numbers
def add(x, y):
return x + y
def sum(data):
result = 0
for x in data:
result = add(result, x)
return result sum([5, 2, 3])
# Smallest in a list
def lesser(x, y):
if x < y:
return x
else:
return y
def min(data):
result = 999999999999
for x in data:
result = lesser(result, x)
return result min([5, 2, 3])
# Sum
result = sum(data)
result = reduce(add, data, 0)
# Min
result = min(data)
result = reduce(lesser, data, 9999999999)

filter
def is_even(x):
return (x % 2) == 0
filter(is_even, range(10))
#even better
from itertools import ifilter
filter_iterator = ifilter(is_even,data)
filter_iterator.next()

iter function
numbers = [1,2,3]
it = iter(numbers)
# using while and StopIteration Exception
try:
while True:
print it.next()
except StopIteration:
print "Complete"
# as iterator in for loop
it = iter(numbers)
for value in it:
print value

import itertools
itertools.count(100)
itertools.cycle([1,2,3,4,5])
itertools.repeat("Hello", 5)
itertools - count, cycle, repeat

itertools - chain
it1 = iter([1, 2, 3])
it2 = iter([4, 5, 6])
itertools.chain(it1, it2)

itertools - groupby
city_list = [('Cochin', 'KL'),
('Bengaluru', 'KA'),
('Chennai', 'TN'),
('Mumbai', 'MH'),
('Trivandrum', 'KL'),
('Salem', 'TN'),
('Pune', 'MH')]
for city, city_iterator in groupby(city_list, lambda x: x[0]):
for city_state in city_iterator:
print "%s is in %s." % (city, city_state[1])

itertools - combinations, permutations
import itertools
itertools.permutations([1, 2, 3])
itertools.combinations([1, 2, 3, 4, 5], 2)

itertools - izip
dict(itertools.izip("ABCD", [1,2,3,4]))
#OR
zip("ABCD", [1,2,3,4])

functools - partial
import functools
def log (message, subsystem):
print '%s: %s' % (subsystem, message)
server_log = functools.partial(log,
subsystem='server')
server_log('Unable to open socket')

Decorators
def p_decorate(func):
def func_wrapper(name):
return "Super %s" func
(name)
return func_wrapper
@p_decorate
def get_text(name):
return "Hello %s" % name
print get_text("John")

Yield - creating custom generators
data_list = ["Orange", "Microsoft", "Apple"]
def one_at_a_time(data_list):
for item in data_list:
yield item
one_at_a_time.next()

Don't do unnecessary Classes
class Greeting(object):
def __init__(self, greeting="hello"):
self.greeting = greeting
def greet(self, name):
return "{greeting} {name}!".format
(greeting=greeting, name=name)
hola = Greeting("hola")
print hola.greet("bob")
or
?

Don't do unnecessary Classes
class Greeting(object):
def __init__(self, greeting="hello"):
self.greeting = greeting
def greet(self, name):
return "{greeting} {name}!".format
(greeting=greeting, name=name)
hola = Greeting("hola")
print hola.greet("bob")
def greet(greeting, name):
return "{greeting} {name}!".format(greeting=greeting,
name=name)
hello = functools.partial(greet, "Hello")
hello("Dude")
or

The Bad Parts
★ Memory Inefficiencies
★ Purity
★ No Tail Recursion
★ Innately imperative (Guido)
★ Class based type system
★ Only imperative Error Handling
(Exception)
★ Function Overloading
★ Mutable variables
Python vs Functional

Thinking about Scalability with Functions
★ map-reduce-filter - recipe for distributed computing
★ shared states- to be or not to be
★ immutable 'variables'
★ independent functions
★ Execution Pipelines - chained map-reduce

Performance v/s Scalability
★ Functional Programs vs Object Oriented Programs
★ CPU intensive processes vs I/O intensive processes
★ The curse of GIL - workarounds
○ multiprocessing
★ Benchmarking
○ %timeit

multiprocessing.Pool
import multiprocessing
def worker():
print 'Execute your function here'
return
if __name__ == '__main__':
jobs = []
for i in range(5):
p = multiprocessing.Process(target=worker)
jobs.append(p)
p.start()

// Why should I think in functions?

why!
★ no side effects -no state, no deadlocks, no semaphores
★ automatic parallelization - unlimited scalability
★ composability - break down into smaller functions
★ Testing -independent functions; well defined arguments and return values
★ partial evaluation - pass around half baked functions instead of objects
★ elegant code -forces to write logically correct programs

Hands On: Let’s do some Data Wrangling
Blockbuster Database (http://www.crowdflower.com/data-for-everyone)
➢ Which genre has most movies?
➢ Which movie studio gross the most?
➢ Sort by most grossed and Highly rated movies
demo code here

Questions
code samples in this presentation here
atm@profoundis.com
@atmb4u

Thinking in Functions: Functional Programming in Python

More Related Content

What's hot

Viewers also liked

Similar to Thinking in Functions: Functional Programming in Python

More from Anoop Thomas Mathew

Recently uploaded

Thinking in Functions: Functional Programming in Python