Functions as First-Class Objects

First-class-objects is a program entity.

• created at runtime
• assigned a variable or element in a data structure
• passed as an argument to a function
• returned as the result of a function

Treating a Function Like an Object

def factorial(n):
  """return n!"""
  return 1 if n < 2 else n * factorial(n-1)
factorial(42) # 140500...
factorial.__doc__   # returns doc string
type(factorial) # <class 'function'>

fact = factorial    # <function factorial at 0x..>
fact(5) # 120
map(factorial, range(11))   # <map object at 0x..>
list(map(factorial, range(11))) # [1, 1, 2, 6, ....]

Higher Order Functions

A function that takes a function as an argument or returns a function as the result is a higher-order function. One example is map, another is sorted the optional key argument lets you provide a function to be applied to each item for sorting. sorted(fruits, key = len)

Some of the best know higher-order functions are map, filter, reduce and apply. apply was deprecated in python2.3 and removed in python3. As we can do fn(*args, **kwargs) instead or apply(fn, args, kwargs)

Modern Replacements for map, filter and reduce

• map and filter functions are still built-ins in python3 but since introduction of listcomp and genexp they are not as important.

list(map(factorial, range(6)))
[factorial(n) for n in range(6)]
# another
list(map(factorial, filter(lambda n : n%2, range(6))))
[ factorial(n) for n in range(6) if n%2 ]

• reduce was demoted from built-in in python2 to functools module in python3. Its most common use case, summation is better server by sum built-in

from functools import reduce
from operator import add
reduce(add, range(100))
sum(range(100))

• Other reducing built-ins are all or any
  • all(iterable) : return True if there are no falsy elements in iterable
  • any(iterable) : returns True if any element of the iterable is truthy.

Anonymous Functions

The lambda keyword creates an anonymous function within a Python expression.

However, the simple syntax of Python limits the body of lambda functions to be pure expressions. In other words, the body cannot contain other Python statements such as while, try, etc. Assignment with = is also a statement, so it cannot occur in a lambda.

The new assignment expression syntax using := can be used—but if you need it, your lambda is probably too complicated and hard to read, and it should be refactored into a regular function using def.

fruits = ['strawberry', 'fig', 'apple', 'cherry', 'raspberry', 'banana']
sorted(fruits, key=lambda word: word[::-1])

The Nine Flavors of Callable Types

• User-defined functions
  • created with def statements or lambda expressions
• Built-in functions
  • A function implemented in C(for CPython), like len or time.strftime
• Built-in methods
  • Methods implemented in C, like dict.get
• Methods
  • Functions defined in the body of a class
• Classes
  • when invoked ad class runs its __new__ method to create an instance then __init__ to initialize it, and finally instance is returned to caller. There is no new operator in python
• Class instances
  • If a class defines __call__ method, then its instances may be invoked as functions

• Generator Functions
  • Functions or methods that use the yield keyword in their body. When called, they return a generator object.
• Native coroutine functions
  • Functions or methods defined with async def. When called, they return a coroutine object. Added in Python 3.5.
• Asynchronous generator functions
  • Functions or methods defined with async def that have yield in their body. When called, they return an asynchronous generator for use with async for. Added in Python 3.6.

Given the variety of existing callable types in Python, the safest way to determine whether an object is callable is to use the callable()

User Defined Callable Types

import random

class BingoCage:

    def __init__(self, items):
        self._items = list(items)   # build local copy to avoid mutating items
        random.shuffle(self._items)

    def pick(self):
        try:
            return self._items.pop()
        except IndexError:
            raise LookupError('pick from empty BingoCage')

    def __call__(self): # shortcut to bingo.pick() : bingo()
        return self.pick()

From Positional to Keyword-only Parameters

One of the best features of Python functions is the extremely flexible parameter handling mechanism. Closely related are the use of * and ** to unpack iterables and mappings into separate arguments when we call a function.

def tag(name, *content, class_=None, **attrs):
    """Generate one or more HTML tags"""
    if class_ is not None:
        attrs['class'] = class_
    attr_pairs = (f' {attr}="{value}"' for attr, value
                    in sorted(attrs.items()))
    attr_str = ''.join(attr_pairs)
    if content:
        elements = (f'<{name}{attr_str}>{c}</{name}>'
                    for c in content)
        return '\n'.join(elements)
    else:
        return f'<{name}{attr_str} />'

# we can invoke it in multiple ways
tag('br') # '<br />'
tag('p', 'hello') # '<p>hello</p>'
print(tag('p', 'hello', 'world'))
# <p>hello</p>
# <p>world</p>
tag('p', 'hello', id=33)  # <p id="33">hello</p>'
print(tag('p', 'hello', 'world', class_='sidebar'))  4
# <p class="sidebar">hello</p>
# <p class="sidebar">world</p>
tag(content='testing', name="img")  # '<img content="testing" />'
my_tag = {'name': 'img', 'title': 'Sunset Boulevard', 'src': 'sunset.jpg', 'class': 'framed'}
tag(**my_tag) #'<img class="framed" src="sunset.jpg" title="Sunset Boulevard" />'

Keyword-only arguments are a feature of Python 3. In Example above, the class_ parameter can only be given as a keyword argument—it will never capture unnamed positional arguments. To specify keyword-only arguments when defining a function, name them after the argument prefixed with *. If you don’t want to support variable positional arguments but still want keyword-only arguments, put a * by itself in the signature, like this:

def f(a, *, b):
  return a, b

# now we cannot call b without keyword args
f(1, b = 2) # (1, 2)
f(1, 2) # raises exception

Positional Only Parameters

Since Python 3.8, user-defined function signatures may specify positional-only parameters. This feature always existed for built-in functions, such as divmod(a, b), which can only be called with positional parameters, and not as divmod(a=10, b=4).

To define a function requiring positional-only parameters, use / in the parameter list.

def divmod(a, b, /):
  return (a // b, a%b)

Packages for Functional Programming

The operator Module

Often in functional programming it is convenient to use an arithmetic operator as a function. For example, suppose you want to multiply a sequence of numbers to calculate factorials without using recursion. To perform summation, you can use sum, but there is no equivalent function for multiplication.

from functools import reduce

def factorial(n):
    return reduce(lambda a, b: a*b, range(1, n+1))

from functools import reduce
from operator import mul

def factorial(n):
    return reduce(mul, range(1, n+1))

# itemgetter
metro_data = [
    ('Tokyo', 'JP', 36.933, (35.689722, 139.691667)),
    ('Delhi NCR', 'IN', 21.935, (28.613889, 77.208889)),
    ('Mexico City', 'MX', 20.142, (19.433333, -99.133333)),
    ('New York-Newark', 'US', 20.104, (40.808611, -74.020386)),
    ('São Paulo', 'BR', 19.649, (-23.547778, -46.635833)),
]

from operator import itemgetter

for city in sorted(metro_data, key=itemgetter(1)):
    print(city)

Partial list of functions defined in operator:

>>> [name for name in dir(operator) if not name.startswith('_')]
['abs', 'add', 'and_', 'attrgetter', 'concat', 'contains',
'countOf', 'delitem', 'eq', 'floordiv', 'ge', 'getitem', 'gt',
'iadd', 'iand', 'iconcat', 'ifloordiv', 'ilshift', 'imatmul',
'imod', 'imul', 'index', 'indexOf', 'inv', 'invert', 'ior',
'ipow', 'irshift', 'is_', 'is_not', 'isub', 'itemgetter',
'itruediv', 'ixor', 'le', 'length_hint', 'lshift', 'lt', 'matmul',
'methodcaller', 'mod', 'mul', 'ne', 'neg', 'not_', 'or_', 'pos',
'pow', 'rshift', 'setitem', 'sub', 'truediv', 'truth', 'xor']

Method caller

from operator import methodcaller

s = 'The time has come'

upcase = methodcaller('upper')
print(upcase(s))  # Output: 'THE TIME HAS COME'

hyphenate = methodcaller('replace', ' ', '-')
print(hyphenate(s))  # Output: 'The-time-has-come'

Freezing Arguments with functools.partial

partial: given a callable, it produces a new callable with some of the arguments of the original callable bound to predetermined values. This is useful to adapt a function that takes one or more arguments to an API that requires a callback with fewer arguments.

from operator import mul
from functools import partial
triple = partial(mul, 3)    # how 3 is bound as first positional argument
triple(7)  # 21
list(map(triple, range(1, 10)))  # [3, 6, 9, 12, 15, 18, 21, 24, 27]