|
| 1 | +.. _annotations-howto: |
| 2 | + |
| 3 | +************************** |
| 4 | +Annotations Best Practices |
| 5 | +************************** |
| 6 | + |
| 7 | +:author: Larry Hastings |
| 8 | + |
| 9 | +.. topic:: Abstract |
| 10 | + |
| 11 | + This document is designed to encapsulate the best practices |
| 12 | + for working with annotations dicts. If you write Python code |
| 13 | + that examines ``__annotations__`` on Python objects, we |
| 14 | + encourage you to follow the guidelines described below. |
| 15 | + |
| 16 | + The document is organized into four sections: |
| 17 | + best practices for accessing the annotations of an object |
| 18 | + in Python versions 3.10 and newer, |
| 19 | + best practices for accessing the annotations of an object |
| 20 | + in Python versions 3.9 and older, |
| 21 | + other best practices |
| 22 | + for ``__annotations__`` that apply to any Python version, |
| 23 | + and |
| 24 | + quirks of ``__annotations__``. |
| 25 | + |
| 26 | + Note that this document is specifically about working with |
| 27 | + ``__annotations__``, not uses *for* annotations. |
| 28 | + If you're looking for information on how to use "type hints" |
| 29 | + in your code, please see the :mod:`typing` module. |
| 30 | + |
| 31 | + |
| 32 | +Accessing The Annotations Dict Of An Object In Python 3.10 And Newer |
| 33 | +==================================================================== |
| 34 | + |
| 35 | + Python 3.10 adds a new function to the standard library: |
| 36 | + :func:`inspect.get_annotations`. In Python versions 3.10 |
| 37 | + and newer, calling this function is the best practice for |
| 38 | + accessing the annotations dict of any object that supports |
| 39 | + annotations. This function can also "un-stringize" |
| 40 | + stringized annotations for you. |
| 41 | + |
| 42 | + If for some reason :func:`inspect.get_annotations` isn't |
| 43 | + viable for your use case, you may access the |
| 44 | + ``__annotations__`` data member manually. Best practice |
| 45 | + for this changed in Python 3.10 as well: as of Python 3.10, |
| 46 | + ``o.__annotations__`` is guaranteed to *always* work |
| 47 | + on Python functions, classes, and modules. If you're |
| 48 | + certain the object you're examining is one of these three |
| 49 | + *specific* objects, you may simply use ``o.__annotations__`` |
| 50 | + to get at the object's annotations dict. |
| 51 | + |
| 52 | + However, other types of callables--for example, |
| 53 | + callables created by :func:`functools.partial`--may |
| 54 | + not have an ``__annotations__`` attribute defined. When |
| 55 | + accessing the ``__annotations__`` of a possibly unknown |
| 56 | + object, best practice in Python versions 3.10 and |
| 57 | + newer is to call :func:`getattr` with three arguments, |
| 58 | + for example ``getattr(o, '__annotations__', None)``. |
| 59 | + |
| 60 | + |
| 61 | +Accessing The Annotations Dict Of An Object In Python 3.9 And Older |
| 62 | +=================================================================== |
| 63 | + |
| 64 | + In Python 3.9 and older, accessing the annotations dict |
| 65 | + of an object is much more complicated than in newer versions. |
| 66 | + The problem is a design flaw in these older versions of Python, |
| 67 | + specifically to do with class annotations. |
| 68 | + |
| 69 | + Best practice for accessing the annotations dict of other |
| 70 | + objects--functions, other callables, and modules--is the same |
| 71 | + as best practice for 3.10, assuming you aren't calling |
| 72 | + :func:`inspect.get_annotations`: you should use three-argument |
| 73 | + :func:`getattr` to access the object's ``__annotations__`` |
| 74 | + attribute. |
| 75 | + |
| 76 | + Unfortunately, this isn't best practice for classes. The problem |
| 77 | + is that, since ``__annotations__`` is optional on classes, and |
| 78 | + because classes can inherit attributes from their base classes, |
| 79 | + accessing the ``__annotations__`` attribute of a class may |
| 80 | + inadvertently return the annotations dict of a *base class.* |
| 81 | + As an example:: |
| 82 | + |
| 83 | + class Base: |
| 84 | + a: int = 3 |
| 85 | + b: str = 'abc' |
| 86 | + |
| 87 | + class Derived(Base): |
| 88 | + pass |
| 89 | + |
| 90 | + print(Derived.__annotations__) |
| 91 | + |
| 92 | + This will print the annotations dict from ``Base``, not |
| 93 | + ``Derived``. |
| 94 | + |
| 95 | + Your code will have to have a separate code path if the object |
| 96 | + you're examining is a class (``isinstance(o, type)``). |
| 97 | + In that case, best practice relies on an implementation detail |
| 98 | + of Python 3.9 and before: if a class has annotations defined, |
| 99 | + they are stored in the class's ``__dict__`` dictionary. Since |
| 100 | + the class may or may not have annotations defined, best practice |
| 101 | + is to call the ``get`` method on the class dict. |
| 102 | + |
| 103 | + To put it all together, here is some sample code that safely |
| 104 | + accesses the ``__annotations__`` attribute on an arbitrary |
| 105 | + object in Python 3.9 and before:: |
| 106 | + |
| 107 | + if isinstance(o, type): |
| 108 | + ann = o.__dict__.get('__annotations__', None) |
| 109 | + else: |
| 110 | + ann = getattr(o, '__annotations__', None) |
| 111 | + |
| 112 | + After running this code, ``ann`` should be either a |
| 113 | + dictionary or ``None``. You're encouraged to double-check |
| 114 | + the type of ``ann`` using :func:`isinstance` before further |
| 115 | + examination. |
| 116 | + |
| 117 | + Note that some exotic or malformed type objects may not have |
| 118 | + a ``__dict__`` attribute, so for extra safety you may also wish |
| 119 | + to use :func:`getattr` to access ``__dict__``. |
| 120 | + |
| 121 | + |
| 122 | +Manually Un-Stringizing Stringized Annotations |
| 123 | +============================================== |
| 124 | + |
| 125 | + In situations where some annotations may be "stringized", |
| 126 | + and you wish to evaluate those strings to produce the |
| 127 | + Python values they represent, it really is best to |
| 128 | + call :func:`inspect.get_annotations` to do this work |
| 129 | + for you. |
| 130 | + |
| 131 | + If you're using Python 3.9 or older, or if for some reason |
| 132 | + you can't use :func:`inspect.get_annotations`, you'll need |
| 133 | + to duplicate its logic. You're encouraged to examine the |
| 134 | + implementation of :func:`inspect.get_annotations` in the |
| 135 | + current Python version and follow a similar approach. |
| 136 | + |
| 137 | + In a nutshell, if you wish to evaluate a stringized annotation |
| 138 | + on an arbitrary object ``o``: |
| 139 | + |
| 140 | + * If ``o`` is a module, use ``o.__dict__`` as the |
| 141 | + ``globals`` when calling :func:`eval`. |
| 142 | + * If ``o`` is a class, use ``sys.modules[o.__module__].__dict__`` |
| 143 | + as the ``globals``, and ``dict(vars(o))`` as the ``locals``, |
| 144 | + when calling :func:`eval`. |
| 145 | + * If ``o`` is a wrapped callable using :func:`functools.update_wrapper`, |
| 146 | + :func:`functools.wraps`, or :func:`functools.partial`, iteratively |
| 147 | + unwrap it by accessing either ``o.__wrapped__`` or ``o.func`` as |
| 148 | + appropriate, until you have found the root unwrapped function. |
| 149 | + * If ``o`` is a callable (but not a class), use |
| 150 | + ``o.__globals__`` as the globals when calling :func:`eval`. |
| 151 | + |
| 152 | + However, not all string values used as annotations can |
| 153 | + be successfully turned into Python values by :func:`eval`. |
| 154 | + String values could theoretically contain any valid string, |
| 155 | + and in practice there are valid use cases for type hints that |
| 156 | + require annotating with string values that specifically |
| 157 | + *can't* be evaluated. For example: |
| 158 | + |
| 159 | + * :pep:`604` union types using `|`, before support for this |
| 160 | + was added to Python 3.10. |
| 161 | + * Definitions that aren't needed at runtime, only imported |
| 162 | + when :const:`typing.TYPE_CHECKING` is true. |
| 163 | + |
| 164 | + If :func:`eval` attempts to evaluate such values, it will |
| 165 | + fail and raise an exception. So, when designing a library |
| 166 | + API that works with annotations, it's recommended to only |
| 167 | + attempt to evaluate string values when explicitly requested |
| 168 | + to by the caller. |
| 169 | + |
| 170 | + |
| 171 | +Best Practices For ``__annotations__`` In Any Python Version |
| 172 | +============================================================ |
| 173 | + |
| 174 | + * You should avoid assigning to the ``__annotations__`` member |
| 175 | + of objects directly. Let Python manage setting ``__annotations__``. |
| 176 | + |
| 177 | + * If you do assign directly to the ``__annotations__`` member |
| 178 | + of an object, you should always set it to a ``dict`` object. |
| 179 | + |
| 180 | + * If you directly access the ``__annotations__`` member |
| 181 | + of an object, you should ensure that it's a |
| 182 | + dictionary before attempting to examine its contents. |
| 183 | + |
| 184 | + * You should avoid modifying ``__annotations__`` dicts. |
| 185 | + |
| 186 | + * You should avoid deleting the ``__annotations__`` attribute |
| 187 | + of an object. |
| 188 | + |
| 189 | + |
| 190 | +``__annotations__`` Quirks |
| 191 | +========================== |
| 192 | + |
| 193 | + In all versions of Python 3, function |
| 194 | + objects lazy-create an annotations dict if no annotations |
| 195 | + are defined on that object. You can delete the ``__annotations__`` |
| 196 | + attribute using ``del fn.__annotations__``, but if you then |
| 197 | + access ``fn.__annotations__`` the object will create a new empty dict |
| 198 | + that it will store and return as its annotations. Deleting the |
| 199 | + annotations on a function before it has lazily created its annotations |
| 200 | + dict will throw an ``AttributeError``; using ``del fn.__annotations__`` |
| 201 | + twice in a row is guaranteed to always throw an ``AttributeError``. |
| 202 | + |
| 203 | + Everything in the above paragraph also applies to class and module |
| 204 | + objects in Python 3.10 and newer. |
| 205 | + |
| 206 | + In all versions of Python 3, you can set ``__annotations__`` |
| 207 | + on a function object to ``None``. However, subsequently |
| 208 | + accessing the annotations on that object using ``fn.__annotations__`` |
| 209 | + will lazy-create an empty dictionary as per the first paragraph of |
| 210 | + this section. This is *not* true of modules and classes, in any Python |
| 211 | + version; those objects permit setting ``__annotations__`` to any |
| 212 | + Python value, and will retain whatever value is set. |
| 213 | + |
| 214 | + If Python stringizes your annotations for you |
| 215 | + (using ``from __future__ import annotations``), and you |
| 216 | + specify a string as an annotation, the string will |
| 217 | + itself be quoted. In effect the annotation is quoted |
| 218 | + *twice.* For example:: |
| 219 | + |
| 220 | + from __future__ import annotations |
| 221 | + def foo(a: "str"): pass |
| 222 | + |
| 223 | + print(foo.__annotations__) |
| 224 | + |
| 225 | + This prints ``{'a': "'str'"}``. This shouldn't really be considered |
| 226 | + a "quirk"; it's mentioned here simply because it might be surprising. |
0 commit comments