Operator Overloading🔗

Operator overloading is necessary to support infix operator notation with user-defined or extension types, such as NumPy arrays. Having operator overloading in a high-level, easy-to-use language was probably a key reason for the huge success of Python in data science, including financial and scientific applications.

Operator Overloading 101🔗

allows user-defined objects to interoperate with infix operator like + and | or unary operators like - and ~. More generally function invocation (()), attribute access (.), and items-access/slicing [] are also operators in python.
This chapter deals with unary and infix operators
Python strikes a good balance among flexibility, usability, and safety by imposing some limitations:
- We cannot change the meaning of the operators for the built-in types.
- We cannot create new operators, only overload existing ones.
- A few operators can’t be overloaded: is, and, or, not (but the bitwise &, |, ~, can).

Unary Operators🔗

-, implemented by __neg__
+, implemented by __add__
~, implemented by __invert__

The “Data Model” chapter of The Python Language Reference also lists the abs() built-in function as a unary operator. The associated special method is __abs__, as we’ve seen before.

its simple to support Unary Operators just implement the supporting function
Follow only one rule, always return a new objects of suitable types, don’t modify the receiver self.

For +/- usually returned object would be of same type but for ~ its depends on context. abs will return scalar always.

    def __abs__(self):
        return math.hypot(*self)

    def __neg__(self):
        return Vector(-x for x in self)

    def __pos__(self):
        return Vector(self)

Everybody expects that x == +x, and that is true almost all the time in Python, but there are two cases in the standard library where x != +x.

The first case involves the decimal.Decimal class. You can have x != +x if x is a Decimal instance created in an arithmetic context and +x is then evaluated in a context with different settings.

You can find the second case where x != +x in the collections.Counter documentation. The Counter class implements several arithmetic operators, including infix + to add the tallies from two Counter instances.

However, for practical reasons, Counter addition discards from the result any item with a negative or zero count. And the prefix + is a shortcut for adding an empty Counter, therefore it produces a new Counter, preserving only the tallies that are greater than zero.

ct = Counter('abracadabra')
ct # Counter({'a': 5, 'r': 2, 'b': 2, 'd': 1, 'c': 1})
ct['r'] = -3
ct['d'] = 0
ct # Counter({'a': 5, 'b': 2, 'c': 1, 'd': 0, 'r': -3})
+ct # Counter({'a': 5, 'b': 2, 'c': 1})

Overloading + for Vector Addition🔗

Adding two Euclidean vectors results in a new vector in which the components are the pairwise additions of the components of the operands.

What happens if we try to add two Vector instances of different lengths? We could raise an error, but considering practical applications (such as information retrieval), it’s better to fill out the shortest Vector with zeros.

    # inside the Vector class

    def __add__(self, other):
        pairs = itertools.zip_longest(self, other, fillvalue=0.0) # generator that produces tuples (a, b) where a is from self, and b. If self and other has different lenghts, fillvalue supplies the missing values for the shortes iterable.
        return Vector(a + b for a, b in pairs)  # built from a generator expression, producing one addition for each (a,b) from pairs

v1 = Vector([3, 4, 5])
v1 + (10, 20, 30)   # works because zip_longest consumes any iterable
(10,20, 30) + v1    # fails because tuple doesn't implement our __add__ implementation

Python implements a special dispatching mechanism for the infix operator special methods. Given an expression a + b, the interpreter will perform these steps

If a has __add__, call a.__add__(b) and return result unless it’s NotImplemented.
If a doesn’t have __add__, or calling it returns NotImplemented, check if b has __radd__, then call b.__radd__(a) and return result unless it’s NotImplemented.
If b doesn’t have __radd__, or calling it returns NotImplemented, raise TypeError with an unsupported operand types message.

Do not confuse NotImplemented with NotImplementedError. The first, NotImplemented, is a special singleton value that an infix operator special method should return to tell the interpreter it cannot handle a given operand. In contrast, NotImplementedError is an exception that stub methods in abstract classes may raise to warn that subclasses must implement them.

    # inside the Vector class

    def __add__(self, other):
        pairs = itertools.zip_longest(self, other, fillvalue=0.0)
        return Vector(a + b for a, b in pairs)

    def __radd__(self, other): # more simple implementation
        return self + other

Often, __radd__ can be as simple as that: just invoke the proper operator, therefore delegating to __add__ in this case. This applies to any commutative operator; + is commutative when dealing with numbers or our vectors, but it’s not commutative when concatenating sequences in Python.

    def __add__(self, other):
        pairs = itertools.zip_longest(self, other, fillvalue=0.0)
        return Vector(a + b for a, b in pairs)

    __radd__ = __add__

if an operator special method cannot return a valid result because of type incompatibility, it should return NotImplemented and not raise TypeError. By returning NotImplemented, you leave the door open for the implementer of the other operand type to perform the operation when Python tries the reversed method call.

In the spirit of duck typing, we will refrain from testing the type of the other operand, or the type of its elements. We’ll catch the exceptions and return NotImplemented. If the interpreter has not yet reversed the operands, it will try that. If the reverse method call returns NotImplemented, then Python will raise TypeError with a standard error message like “unsupported operand type(s) for +: Vector and str.”

    def __add__(self, other):
        try:
            pairs = itertools.zip_longest(self, other, fillvalue=0.0)
            return Vector(a + b for a, b in pairs)
        except TypeError:
            return NotImplemented

    def __radd__(self, other):
        return self + other

Overloading * for Scalar Multiplication🔗

What does Vector([1, 2, 3]) * x mean? If x is a number, that would be a scalar product, and the result would be a new Vector with each component multiplied by x—also known as an elementwise multiplication:

Back to our scalar product, again we start with the simplest __mul__ and __rmul__ methods that could possibly work:

    # inside the Vector class

    def __mul__(self, scalar):
        return Vector(n * scalar for n in self)

    def __rmul__(self, scalar):
        return self * scalar

Those methods do work, except when provided with incompatible operands. The scalar argument has to be a number that when multiplied by a float produces another float (because our Vector class uses an array of floats internally). So a complex number will not do, but the scalar can be an int, a bool (because bool is a subclass of int), or even a fractions.Fraction instance.

class Vector:
    typecode = 'd'

    def __init__(self, components):
        self._components = array(self.typecode, components)

    # many methods omitted in book listing, see vector_v7.py
    # in https://github.com/fluentpython/example-code-2e

    def __mul__(self, scalar):
        try:
            factor = float(scalar)
        except TypeError: # scalar can't be converted to float
            return NotImplemented
        return Vector(n * factor for n in self)

    def __rmul__(self, scalar):
        return self * scalar

Using @ as an Infix Operator🔗

The @ sign is well-known as the prefix of function decorators, but since 2015, it can also be used as an infix operator. For years, the dot product was written as numpy.dot(a, b) in NumPy.(3.5+)

Today, you can write a @ b to compute the dot product of two NumPy arrays.

The @ operator is supported by the special methods __matmul__, __rmatmul__, and __imatmul__, named for “matrix multiplication.” These methods are not used anywhere in the standard library at this time, but are recognized by the interpreter since Python 3.5, so the NumPy team—and the rest of us—can support the @ operator in user-defined types. The parser was also changed to handle the new operator (a @ b was a syntax error in Python 3.4).

class Vector:
    # many methods omitted in book listing

    def __matmul__(self, other):
        if (isinstance(other, abc.Sized) and # both should implement __len__ & __iter__
            isinstance(other, abc.Iterable)):
            if len(self) == len(other): # have same length to allow
                return sum(a * b for a, b in zip(self, other)) # using zip and sum operations
            else:
                raise ValueError('@ requires vectors of equal length.')
        else:
            return NotImplemented

    def __rmatmul__(self, other):
        return self @ other

The zip built-in accepts a strict keyword-only optional argument since Python 3.10. When strict=True, the function raises ValueError when the iterables have different lengths. The default is False. This new strict behavior is in line with Python’s fail fast philosophy.

Wrapping-Up Arithmetic Operators🔗

Operator	Forward	Reverse	In-place	Description
+	`__add__`	`__radd__`	`__iadd__`	Addition or concatenation
-	`__sub__`	`__rsub__`	`__isub__`	Subtraction
*	`__mul__`	`__rmul__`	`__imul__`	Multiplication or repetition
/	`__truediv__`	`__rtruediv__`	`__itruediv__`	True division
//	`__floordiv__`	`__rfloordiv__`	`__ifloordiv__`	Floor division
%	`__mod__`	`__rmod__`	`__imod__`	Modulo
divmod()	`__divmod__`	`__rdivmod__`	`__idivmod__`	Returns tuple of floor division quotient and modulo
**, pow()	`__pow__`	`__rpow__`	`__ipow__`	Exponentiation
@	`__matmul__`	`__rmatmul__`	`__imatmul__`	Matrix multiplication
&	`__and__`	`__rand__`	`__iand__`	Bitwise and
\|	`__or__`	`__ror__`	`__ior__`	Bitwise or
^	`__xor__`	`__rxor__`	`__ixor__`	Bitwise xor
<<	`__lshift__`	`__rlshift__`	`__ilshift__`	Bitwise shift left
>>	`__rshift__`	`__rrshift__`	`__irshift__`	Bitwise shift right

Rich Comparison Operators🔗

Group	Infix operator	Forward method call	Reverse method call	Fallback
Equality	`a == b`	`a.__eq__(b)`	`b.__eq__(a)`	`Return id(a) == id(b)`
	`a != b`	`a.__ne__(b)`	`b.__ne__(a)`	`Return not (a == b)`
Ordering	`a > b`	`a.__gt__(b)`	`b.__lt__(a)`	`Raise TypeError`
	`a < b`	`a.__lt__(b)`	`b.__gt__(a)`	`Raise TypeError`
	`a >= b`	`a.__ge__(b)`	`b.__le__(a)`	`Raise TypeError`
	`a <= b`	`a.__le__(b)`	`b.__ge__(a)`	`Raise TypeError`

class Vector:
    # many lines omitted

    def __eq__(self, other):
        return (len(self) == len(other) and
                all(a == b for a, b in zip(self, other)))

In the face of ambiguity, refuse the temptation to guess.

    def __eq__(self, other):
        if isinstance(other, Vector):
            return (len(self) == len(other) and
                    all(a == b for a, b in zip(self, other)))
        else:
            return NotImplemented

Augmented Assignment Operators🔗

Our Vector class already supports the augmented assignment operators += and *=. That’s because augmented assignment works with immutable receivers by creating new instances and rebinding the lefthand variable.

the augmented assignment operators work as syntactic sugar: a += b is evaluated exactly as a = a + b. That’s the expected behavior for immutable types, and if you have __add__, then += will work with no additional code.

However, if you do implement an in-place operator method such as __iadd__, that method is called to compute the result of a += b. As the name says, those operators are expected to change the lefthand operand in place, and not create a new object as the result.

We can summarize the whole idea of in-place operators by contrasting the return statements that produce results in add and iadd in Example 16-19:

__add__ : The result is produced by calling the constructor AddableBingoCage to build a new instance.
__iadd__ : The result is produced by returning self, after it has been modified.