11This document describes some caveats about the use of Valgrind with
2- Python. Valgrind is used periodically by Python developers to try
2+ Python. Valgrind is used periodically by Python developers to try
33to ensure there are no memory leaks or invalid memory reads/writes.
44
55If you don't want to read about the details of using Valgrind, there
6- are still two things you must do to suppress the warnings. First,
6+ are still two things you must do to suppress the warnings. First,
77you must use a suppressions file. One is supplied in
88Misc/valgrind-python.supp. Second, you must do one of the following:
99
1010 * Uncomment Py_USING_MEMORY_DEBUGGER in Objects/obmalloc.c,
11- then rebuild Python
12- * Uncomment the lines in Misc/valgrind-python.supp that
11+ then rebuild Python
12+ * Uncomment the lines in Misc/valgrind-python.supp that
1313 suppress the warnings for PyObject_Free and PyObject_Realloc
1414
1515Details:
1616--------
17- Python uses its own allocation scheme on top of malloc called PyMalloc.
18- Valgrind my show some unexpected results when PyMalloc is used.
17+ Python uses its own small-object allocation scheme on top of malloc,
18+ called PyMalloc.
19+
20+ Valgrind may show some unexpected results when PyMalloc is used.
1921Starting with Python 2.3, PyMalloc is used by default. You can disable
2022PyMalloc when configuring python by adding the --without-pymalloc option.
21- If you disable PyMalloc, most of the information in this document and
23+ If you disable PyMalloc, most of the information in this document and
2224the supplied suppressions file will not be useful.
2325
2426If you use valgrind on a default build of Python, you will see
@@ -32,18 +34,19 @@ These are expected and not a problem. Tim Peters explains
3234the situation:
3335
3436 PyMalloc needs to know whether an arbitrary address is one
35- that's managed by it, or is managed by the system malloc.
37+ that's managed by it, or is managed by the system malloc.
3638 The current scheme allows this to be determined in constant
3739 time, regardless of how many memory areas are under pymalloc's
3840 control.
3941
4042 The memory pymalloc manages itself is in one or more "arenas",
41- each a large contiguous memory area obtained from malloc.
42- The base address of each arena is saved by pymalloc
43- in a vector, and a field at the start of each arena contains
44- the index of that arena's base address in that vector.
43+ each a large contiguous memory area obtained from malloc.
44+ The base address of each arena is saved by pymalloc
45+ in a vector. Each arena is carved into "pools", and a field at
46+ the start of each pool contains the index of that pool's arena's
47+ base address in that vector.
4548
46- Given an arbitrary address, pymalloc computes the arena base
49+ Given an arbitrary address, pymalloc computes the pool base
4750 address corresponding to it, then looks at "the index" stored
4851 near there. If the index read up is out of bounds for the
4952 vector of arena base addresses pymalloc maintains, then
@@ -55,14 +58,17 @@ the situation:
5558
5659 to
5760
58- the computed arena address
61+ the arbitrary address pymalloc is investigating
5962
60- pymalloc controls this arena if and only if they're equal.
63+ pymalloc controls this arbitrary address if and only if it lies
64+ in the arena the address's pool's index claims it lies in.
6165
6266 It doesn't matter whether the memory pymalloc reads up ("the
6367 index") is initialized. If it's not initialized, then
6468 whatever trash gets read up will lead pymalloc to conclude
65- (correctly) that the address isn't controlled by it.
69+ (correctly) that the address isn't controlled by it, either
70+ because the index is out of bounds, or the index is in bounds
71+ but the arena it represents doesn't contain the address.
6672
6773 This determination has to be made on every call to one of
6874 pymalloc's free/realloc entry points, so its speed is critical
0 commit comments