-
Notifications
You must be signed in to change notification settings - Fork 60
Expand file tree
/
Copy pathtesting.html
More file actions
846 lines (658 loc) · 47.5 KB
/
testing.html
File metadata and controls
846 lines (658 loc) · 47.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
<!DOCTYPE html>
<html lang="en" data-content_root="../">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Testing — Matplotlib 3.7.5 documentation</title>
<script data-cfasync="false">
document.documentElement.dataset.mode = localStorage.getItem("mode") || "";
document.documentElement.dataset.theme = localStorage.getItem("theme") || "light";
</script>
<!-- Loaded before other Sphinx assets -->
<link href="../_static/styles/theme.css?digest=796348d33e8b1d947c94" rel="stylesheet">
<link href="../_static/styles/bootstrap.css?digest=796348d33e8b1d947c94" rel="stylesheet">
<link href="../_static/styles/pydata-sphinx-theme.css?digest=796348d33e8b1d947c94" rel="stylesheet">
<link href="../_static/vendor/fontawesome/6.1.2/css/all.min.css?digest=796348d33e8b1d947c94" rel="stylesheet">
<link rel="preload" as="font" type="font/woff2" crossorigin href="../_static/vendor/fontawesome/6.1.2/webfonts/fa-solid-900.woff2">
<link rel="preload" as="font" type="font/woff2" crossorigin href="../_static/vendor/fontawesome/6.1.2/webfonts/fa-brands-400.woff2">
<link rel="preload" as="font" type="font/woff2" crossorigin href="../_static/vendor/fontawesome/6.1.2/webfonts/fa-regular-400.woff2">
<link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
<link rel="stylesheet" type="text/css" href="../_static/css/style.css?v=86e00652" />
<link rel="stylesheet" type="text/css" href="../_static/graphviz.css?v=eafc0fe6" />
<link rel="stylesheet" type="text/css" href="../_static/plot_directive.css" />
<link rel="stylesheet" type="text/css" href="../_static/copybutton.css?v=76b2166b" />
<link rel="stylesheet" type="text/css" href="../_static/sg_gallery.css?v=61a4c737" />
<link rel="stylesheet" type="text/css" href="../_static/sg_gallery-binder.css?v=f4aeca0c" />
<link rel="stylesheet" type="text/css" href="../_static/sg_gallery-dataframe.css?v=2082cf3c" />
<link rel="stylesheet" type="text/css" href="../_static/sg_gallery-rendered-html.css?v=1277b6f3" />
<link rel="stylesheet" type="text/css" href="../_static/design-style.1e8bd061cd6da7fc9cf755528e8ffc24.min.css?v=0a3b3ea7" />
<link rel="stylesheet" type="text/css" href="../_static/mpl.css?v=ee6d1e4d" />
<!-- Pre-loaded scripts that we'll load fully later -->
<link rel="preload" as="script" href="../_static/scripts/bootstrap.js?digest=796348d33e8b1d947c94">
<link rel="preload" as="script" href="../_static/scripts/pydata-sphinx-theme.js?digest=796348d33e8b1d947c94">
<script src="../_static/documentation_options.js?v=efff61b2"></script>
<script src="../_static/jquery.js?v3.7.5"></script>
<script src="../_static/doctools.js?v=888ff710"></script>
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../_static/clipboard.min.js?v=a7894cd8"></script>
<script src="../_static/copybutton.js?v=ccdb6887"></script>
<script src="../_static/design-tabs.js?v=36754332"></script>
<script data-domain="matplotlib.org" defer="defer" src="https://views.scientific-python.org/js/script.js"></script>
<script>DOCUMENTATION_OPTIONS.pagename = 'devel/testing';</script>
<script>
DOCUMENTATION_OPTIONS.theme_switcher_json_url = 'https://matplotlib.org/devdocs/_static/switcher.json?v3.7.5-3-gb58d40e6c8';
DOCUMENTATION_OPTIONS.theme_switcher_version_match = '3.7.5';
</script>
<link rel="canonical" href="https://matplotlib.org/stable/devel/testing.html" />
<link rel="search" type="application/opensearchdescription+xml"
title="Search within Matplotlib 3.7.5 documentation"
href="../_static/opensearch.xml"/>
<link rel="icon" href="../_static/favicon.ico"/>
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Writing documentation" href="documenting_mpl.html" />
<link rel="prev" title="Development workflow" href="development_workflow.html" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="docsearch:language" content="en">
</head>
<body data-spy="scroll" data-target="#bd-toc-nav" data-offset="180" data-default-mode="">
<a class="skip-link" href="#main-content">Skip to main content</a>
<input type="checkbox" class="sidebar-toggle" name="__primary" id="__primary">
<label class="overlay overlay-primary" for="__primary"></label>
<input type="checkbox" class="sidebar-toggle" name="__secondary" id="__secondary">
<label class="overlay overlay-secondary" for="__secondary"></label>
<div class="search-button__wrapper">
<div class="search-button__overlay"></div>
<div class="search-button__search-container">
<form class="bd-search d-flex align-items-center" action="../search.html" method="get">
<i class="fa-solid fa-magnifying-glass"></i>
<input type="search" class="form-control" name="q" id="search-input" placeholder="Search the docs ..." aria-label="Search the docs ..." autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false">
<span class="search-button__kbd-shortcut"><kbd class="kbd-shortcut__modifier">Ctrl</kbd>+<kbd>K</kbd></span>
</form>
</div>
</div>
<nav class="bd-header navbar navbar-expand-lg bd-navbar" id="navbar-main"><div class="bd-header__inner bd-page-width">
<label class="sidebar-toggle primary-toggle" for="__primary">
<span class="fa-solid fa-bars"></span>
</label>
<div id="navbar-start">
<a class="navbar-brand logo" href="../index.html">
<img src="../_static/images/logo2.svg" class="logo__image only-light" alt="Logo image">
<img src="../_static/images/logo_dark.svg" class="logo__image only-dark" alt="Logo image">
</a>
</div>
<div class="col-lg-9 navbar-header-items">
<div id="navbar-center" class="mr-auto">
<div class="navbar-center-item">
<ul id="navbar-main-elements" class="navbar-nav">
<li class="nav-item">
<a class="reference internal nav-link" href="../plot_types/index.html">Plot types</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../gallery/index.html">Examples</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../tutorials/index.html">Tutorials</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../api/index.html">Reference</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../users/index.html">User guide</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="index.html">Develop</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../users/release_notes.html">Releases</a>
</li>
</ul>
</div>
</div>
<div id="navbar-end">
<div class="navbar-end-item navbar-persistent--container">
<button class="btn btn-sm navbar-btn search-button search-button__button" title="Search" aria-label="Search" data-toggle="tooltip">
<i class="fa-solid fa-magnifying-glass"></i>
</button>
</div>
<div class="navbar-end-item">
<button class="theme-switch-button btn btn-sm btn-outline-primary navbar-btn rounded-circle" title="light/dark" aria-label="light/dark" data-toggle="tooltip">
<span class="theme-switch" data-mode="light"><i class="fa-solid fa-sun"></i></span>
<span class="theme-switch" data-mode="dark"><i class="fa-solid fa-moon"></i></span>
<span class="theme-switch" data-mode="auto"><i class="fa-solid fa-circle-half-stroke"></i></span>
</button>
</div>
<div class="navbar-end-item">
<div class="version-switcher__container dropdown">
<button type="button" class="version-switcher__button btn btn-sm navbar-btn dropdown-toggle" data-toggle="dropdown">
stable <!-- this text may get changed later by javascript -->
<span class="caret"></span>
</button>
<div class="version-switcher__menu dropdown-menu list-group-flush py-0">
<!-- dropdown will be populated by javascript on page load -->
</div>
</div>
</div>
<div class="navbar-end-item">
<ul id="navbar-icon-links" class="navbar-nav" aria-label="Icon Links">
<li class="nav-item">
<a href="https://gitter.im/matplotlib" title="Gitter" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-gitter"></i></span>
<label class="sr-only">Gitter</label></a>
</li>
<li class="nav-item">
<a href="https://discourse.matplotlib.org" title="Discourse" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-discourse"></i></span>
<label class="sr-only">Discourse</label></a>
</li>
<li class="nav-item">
<a href="https://github.com/matplotlib/matplotlib" title="GitHub" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-github"></i></span>
<label class="sr-only">GitHub</label></a>
</li>
<li class="nav-item">
<a href="https://twitter.com/matplotlib/" title="Twitter" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-twitter"></i></span>
<label class="sr-only">Twitter</label></a>
</li>
</ul>
</div>
</div>
</div>
<div class="navbar-persistent--mobile">
<button class="btn btn-sm navbar-btn search-button search-button__button" title="Search" aria-label="Search" data-toggle="tooltip">
<i class="fa-solid fa-magnifying-glass"></i>
</button>
</div>
<label class="sidebar-toggle secondary-toggle" for="__secondary">
<span class="fa-solid fa-outdent"></span>
</label>
</div>
</nav>
<div class="bd-container">
<div class="bd-container__inner bd-page-width">
<div class="bd-sidebar-primary bd-sidebar">
<div class="sidebar-header-items sidebar-primary__section">
<div class="sidebar-header-items__center">
<div class="navbar-center-item">
<ul id="navbar-main-elements" class="navbar-nav">
<li class="nav-item">
<a class="reference internal nav-link" href="../plot_types/index.html">Plot types</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../gallery/index.html">Examples</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../tutorials/index.html">Tutorials</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../api/index.html">Reference</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../users/index.html">User guide</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="index.html">Develop</a>
</li>
<li class="nav-item">
<a class="reference internal nav-link" href="../users/release_notes.html">Releases</a>
</li>
</ul>
</div>
</div>
<div class="sidebar-header-items__end">
<div class="navbar-end-item">
<button class="theme-switch-button btn btn-sm btn-outline-primary navbar-btn rounded-circle" title="light/dark" aria-label="light/dark" data-toggle="tooltip">
<span class="theme-switch" data-mode="light"><i class="fa-solid fa-sun"></i></span>
<span class="theme-switch" data-mode="dark"><i class="fa-solid fa-moon"></i></span>
<span class="theme-switch" data-mode="auto"><i class="fa-solid fa-circle-half-stroke"></i></span>
</button>
</div>
<div class="navbar-end-item">
<div class="version-switcher__container dropdown">
<button type="button" class="version-switcher__button btn btn-sm navbar-btn dropdown-toggle" data-toggle="dropdown">
stable <!-- this text may get changed later by javascript -->
<span class="caret"></span>
</button>
<div class="version-switcher__menu dropdown-menu list-group-flush py-0">
<!-- dropdown will be populated by javascript on page load -->
</div>
</div>
</div>
<div class="navbar-end-item">
<ul id="navbar-icon-links" class="navbar-nav" aria-label="Icon Links">
<li class="nav-item">
<a href="https://gitter.im/matplotlib" title="Gitter" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-gitter"></i></span>
<label class="sr-only">Gitter</label></a>
</li>
<li class="nav-item">
<a href="https://discourse.matplotlib.org" title="Discourse" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-discourse"></i></span>
<label class="sr-only">Discourse</label></a>
</li>
<li class="nav-item">
<a href="https://github.com/matplotlib/matplotlib" title="GitHub" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-github"></i></span>
<label class="sr-only">GitHub</label></a>
</li>
<li class="nav-item">
<a href="https://twitter.com/matplotlib/" title="Twitter" class="nav-link" rel="noopener" target="_blank" data-toggle="tooltip"><span><i class="fa-brands fa-twitter"></i></span>
<label class="sr-only">Twitter</label></a>
</li>
</ul>
</div>
</div>
</div>
<div class="sidebar-start-items sidebar-primary__section">
<div class="sidebar-start-items__item"><nav class="bd-links" id="bd-docs-nav" aria-label="Section navigation">
<p class="bd-links__title" role="heading" aria-level="1">
Section Navigation
</p>
<div class="bd-toc-item navbar-nav">
<ul class="current nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="contributing.html">Contributing</a></li>
<li class="toctree-l1"><a class="reference internal" href="triage.html">Bug triaging and issue curation</a></li>
<li class="toctree-l1"><a class="reference internal" href="development_setup.html">Setting up Matplotlib for development</a></li>
<li class="toctree-l1"><a class="reference internal" href="development_workflow.html">Development workflow</a></li>
<li class="toctree-l1 current active"><a class="current reference internal" href="#">Testing</a></li>
<li class="toctree-l1"><a class="reference internal" href="documenting_mpl.html">Writing documentation</a></li>
<li class="toctree-l1"><a class="reference internal" href="style_guide.html">Documentation style guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="coding_guide.html">Pull request guidelines</a></li>
<li class="toctree-l1"><a class="reference internal" href="release_guide.html">Release guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="dependencies.html">Dependencies</a></li>
<li class="toctree-l1"><a class="reference internal" href="min_dep_policy.html">Minimum version of dependencies policy</a></li>
<li class="toctree-l1"><a class="reference internal" href="maintainer_workflow.html">Maintainer workflow</a></li>
<li class="toctree-l1 has-children"><a class="reference internal" href="MEP/index.html">Matplotlib Enhancement Proposals</a><input class="toctree-checkbox" id="toctree-checkbox-1" name="toctree-checkbox-1" type="checkbox"/><label class="toctree-toggle" for="toctree-checkbox-1"><i class="fa-solid fa-chevron-down"></i></label><ul>
<li class="toctree-l2"><a class="reference internal" href="MEP/template.html">MEP Template</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP08.html">MEP8: PEP8</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP09.html">MEP9: Global interaction manager</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP10.html">MEP10: Docstring consistency</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP11.html">MEP11: Third-party dependencies</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP12.html">MEP12: Improve Gallery and Examples</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP13.html">MEP13: Use properties for Artists</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP14.html">MEP14: Text handling</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP15.html">MEP15: Fix axis autoscaling when limits are specified for one axis only</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP19.html">MEP19: Continuous Integration</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP21.html">MEP21: color and cm refactor</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP22.html">MEP22: Toolbar rewrite</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP23.html">MEP23: Multiple Figures per GUI window</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP24.html">MEP24: Negative radius in polar plots</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP25.html">MEP25: Serialization</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP26.html">MEP26: Artist styling</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP27.html">MEP27: Decouple pyplot from backends</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP28.html">MEP28: Remove Complexity from Axes.boxplot</a></li>
<li class="toctree-l2"><a class="reference internal" href="MEP/MEP29.html">MEP29: Text light markup</a></li>
</ul>
</li>
</ul>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="license.html">Licenses</a></li>
<li class="toctree-l1"><a class="reference internal" href="color_changes.html">Default color changes</a></li>
</ul>
</div>
</nav>
</div>
</div>
<div class="sidebar-end-items sidebar-primary__section">
<div class="sidebar-end-items__item">
</div>
</div>
<div id="rtd-footer-container"></div>
</div>
<main id="main-content" class="bd-main">
<div class="bd-content">
<div class="bd-article-container">
<div class="bd-header-article">
</div>
<article class="bd-article" role="main">
<section id="testing">
<span id="id1"></span><h1>Testing<a class="headerlink" href="#testing" title="Link to this heading">#</a></h1>
<p>Matplotlib uses the <a class="reference external" href="http://doc.pytest.org/en/latest/">pytest</a> framework.</p>
<p>The tests are in <code class="file docutils literal notranslate"><span class="pre">lib/matplotlib/tests</span></code>, and customizations to the pytest
testing infrastructure are in <a class="reference internal" href="../api/testing_api.html#module-matplotlib.testing" title="matplotlib.testing"><code class="xref py py-mod docutils literal notranslate"><span class="pre">matplotlib.testing</span></code></a>.</p>
<section id="requirements">
<span id="testing-requirements"></span><h2>Requirements<a class="headerlink" href="#requirements" title="Link to this heading">#</a></h2>
<p>To run the tests you will need to
<a class="reference internal" href="development_setup.html#installing-for-devs"><span class="std std-ref">set up Matplotlib for development</span></a>. Note in
particular the <a class="reference internal" href="dependencies.html#test-dependencies"><span class="std std-ref">additional dependencies</span></a> for testing.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>We will assume that you want to run the tests in a development setup.</p>
<p>While you can run the tests against a regular installed version of
Matplotlib, this is a far less common use case. You still need the
<a class="reference internal" href="dependencies.html#test-dependencies"><span class="std std-ref">additional dependencies</span></a> for testing.
You have to additionally get the reference images from the repository,
because they are not distributed with pre-built Matplotlib packages.</p>
</div>
</section>
<section id="running-the-tests">
<h2>Running the tests<a class="headerlink" href="#running-the-tests" title="Link to this heading">#</a></h2>
<p>In the root directory of your development repository run:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">m</span> <span class="n">pytest</span>
</pre></div>
</div>
<p>pytest can be configured via a lot of <a class="reference external" href="http://doc.pytest.org/en/latest/usage.html">command-line parameters</a>. Some
particularly useful ones are:</p>
<table class="table">
<tbody>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">-v</span></code> or <code class="docutils literal notranslate"><span class="pre">--verbose</span></code></p></td>
<td><p>Be more verbose</p></td>
</tr>
<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">-n</span> <span class="pre">NUM</span></code></p></td>
<td><p>Run tests in parallel over NUM
processes (requires <a class="reference external" href="https://pypi.org/project/pytest-xdist/">pytest-xdist</a>)</p></td>
</tr>
<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">--capture=no</span></code> or <code class="docutils literal notranslate"><span class="pre">-s</span></code></p></td>
<td><p>Do not capture stdout</p></td>
</tr>
</tbody>
</table>
<p>To run a single test from the command line, you can provide a file path,
optionally followed by the function separated by two colons, e.g., (tests do
not need to be installed, but Matplotlib should be):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pytest</span> <span class="n">lib</span><span class="o">/</span><span class="n">matplotlib</span><span class="o">/</span><span class="n">tests</span><span class="o">/</span><span class="n">test_simplification</span><span class="o">.</span><span class="n">py</span><span class="p">::</span><span class="n">test_clipping</span>
</pre></div>
</div>
</section>
<section id="writing-a-simple-test">
<h2>Writing a simple test<a class="headerlink" href="#writing-a-simple-test" title="Link to this heading">#</a></h2>
<p>Many elements of Matplotlib can be tested using standard tests. For
example, here is a test from <code class="file docutils literal notranslate"><span class="pre">matplotlib/tests/test_basic.py</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">test_simple</span><span class="p">():</span>
<span class="w"> </span><span class="sd">"""</span>
<span class="sd"> very simple example test</span>
<span class="sd"> """</span>
<span class="k">assert</span> <span class="mi">1</span> <span class="o">+</span> <span class="mi">1</span> <span class="o">==</span> <span class="mi">2</span>
</pre></div>
</div>
<p>Pytest determines which functions are tests by searching for files whose names
begin with <code class="docutils literal notranslate"><span class="pre">"test_"</span></code> and then within those files for functions beginning with
<code class="docutils literal notranslate"><span class="pre">"test"</span></code> or classes beginning with <code class="docutils literal notranslate"><span class="pre">"Test"</span></code>.</p>
<p>Some tests have internal side effects that need to be cleaned up after their
execution (such as created figures or modified <a class="reference internal" href="../api/matplotlib_configuration_api.html#matplotlib.rcParams" title="matplotlib.rcParams"><code class="xref py py-obj docutils literal notranslate"><span class="pre">rcParams</span></code></a>). The pytest fixture
<code class="docutils literal notranslate"><span class="pre">matplotlib.testing.conftest.mpl_test_settings</span></code> will automatically clean
these up; there is no need to do anything further.</p>
</section>
<section id="random-data-in-tests">
<h2>Random data in tests<a class="headerlink" href="#random-data-in-tests" title="Link to this heading">#</a></h2>
<p>Random data is a very convenient way to generate data for examples,
however the randomness is problematic for testing (as the tests
must be deterministic!). To work around this set the seed in each test.
For numpy's default random number generator use:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="n">rng</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">default_rng</span><span class="p">(</span><span class="mi">19680801</span><span class="p">)</span>
</pre></div>
</div>
<p>and then use <code class="docutils literal notranslate"><span class="pre">rng</span></code> when generating the random numbers.</p>
<p>The seed is John Hunter's birthday.</p>
</section>
<section id="writing-an-image-comparison-test">
<h2>Writing an image comparison test<a class="headerlink" href="#writing-an-image-comparison-test" title="Link to this heading">#</a></h2>
<p>Writing an image-based test is only slightly more difficult than a simple
test. The main consideration is that you must specify the "baseline", or
expected, images in the <a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.image_comparison" title="matplotlib.testing.decorators.image_comparison"><code class="xref py py-obj docutils literal notranslate"><span class="pre">image_comparison</span></code></a>
decorator. For example, this test generates a single image and automatically
tests it:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">matplotlib.testing.decorators</span> <span class="kn">import</span> <span class="n">image_comparison</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="nd">@image_comparison</span><span class="p">(</span><span class="n">baseline_images</span><span class="o">=</span><span class="p">[</span><span class="s1">'line_dashes'</span><span class="p">],</span> <span class="n">remove_text</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="n">extensions</span><span class="o">=</span><span class="p">[</span><span class="s1">'png'</span><span class="p">],</span> <span class="n">style</span><span class="o">=</span><span class="s1">'mpl20'</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_line_dashes</span><span class="p">():</span>
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">()</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span> <span class="n">linestyle</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">)),</span> <span class="n">lw</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
</pre></div>
</div>
<p>The first time this test is run, there will be no baseline image to compare
against, so the test will fail. Copy the output images (in this case
<code class="file docutils literal notranslate"><span class="pre">result_images/test_lines/test_line_dashes.png</span></code>) to the correct
subdirectory of <code class="file docutils literal notranslate"><span class="pre">baseline_images</span></code> tree in the source directory (in this
case <code class="file docutils literal notranslate"><span class="pre">lib/matplotlib/tests/baseline_images/test_lines</span></code>). Put this new
file under source code revision control (with <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">add</span></code>). When rerunning
the tests, they should now pass.</p>
<p>It is preferred that new tests use <code class="docutils literal notranslate"><span class="pre">style='mpl20'</span></code> as this leads to smaller
figures and reflects the newer look of default Matplotlib plots. Also, if the
texts (labels, tick labels, etc) are not really part of what is tested, use
<code class="docutils literal notranslate"><span class="pre">remove_text=True</span></code> as this will lead to smaller figures and reduce possible
issues with font mismatch on different platforms.</p>
<p>Baseline images take a lot of space in the Matplotlib repository.
An alternative approach for image comparison tests is to use the
<a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.check_figures_equal" title="matplotlib.testing.decorators.check_figures_equal"><code class="xref py py-obj docutils literal notranslate"><span class="pre">check_figures_equal</span></code></a> decorator, which should be
used to decorate a function taking two <a class="reference internal" href="../api/figure_api.html#matplotlib.figure.Figure" title="matplotlib.figure.Figure"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Figure</span></code></a> parameters and draws the same
images on the figures using two different methods (the tested method and the
baseline method). The decorator will arrange for setting up the figures and
then collect the drawn results and compare them.</p>
<p>For example, this test compares two different methods to draw the same
circle: plotting a circle using a <a class="reference internal" href="../api/_as_gen/matplotlib.patches.Circle.html#matplotlib.patches.Circle" title="matplotlib.patches.Circle"><code class="xref py py-obj docutils literal notranslate"><span class="pre">matplotlib.patches.Circle</span></code></a> patch
vs plotting the circle using the parametric equation of a circle</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">matplotlib.testing.decorators</span> <span class="kn">import</span> <span class="n">check_figures_equal</span>
<span class="kn">import</span> <span class="nn">matplotib.patches</span> <span class="k">as</span> <span class="nn">mpatches</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="nd">@check_figures_equal</span><span class="p">(</span><span class="n">extensions</span><span class="o">=</span><span class="p">[</span><span class="s1">'png'</span><span class="p">],</span> <span class="n">tol</span><span class="o">=</span><span class="mi">100</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">test_parametric_circle_plot</span><span class="p">(</span><span class="n">fig_test</span><span class="p">,</span> <span class="n">fig_ref</span><span class="p">):</span>
<span class="n">red_circle_ref</span> <span class="o">=</span> <span class="n">mpatches</span><span class="o">.</span><span class="n">Circle</span><span class="p">((</span><span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">),</span> <span class="mf">0.2</span><span class="p">,</span> <span class="n">color</span><span class="o">=</span><span class="s1">'r'</span><span class="p">,</span> <span class="n">clip_on</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="n">fig_ref</span><span class="o">.</span><span class="n">add_artist</span><span class="p">(</span><span class="n">red_circle_ref</span><span class="p">)</span>
<span class="n">theta</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="mi">2</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">pi</span><span class="p">,</span> <span class="mi">150</span><span class="p">)</span>
<span class="n">radius</span> <span class="o">=</span> <span class="mf">0.4</span>
<span class="n">fig_test</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">radius</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">cos</span><span class="p">(</span><span class="n">theta</span><span class="p">),</span> <span class="n">radius</span> <span class="o">*</span> <span class="n">np</span><span class="o">.</span><span class="n">sin</span><span class="p">(</span><span class="n">theta</span><span class="p">),</span> <span class="n">color</span><span class="o">=</span><span class="s1">'r'</span><span class="p">)</span>
</pre></div>
</div>
<p>Both comparison decorators have a tolerance argument <code class="docutils literal notranslate"><span class="pre">tol</span></code> that is used to specify the tolerance for difference in color value between the two images, where 255 is the maximal difference. The test fails if the average pixel difference is greater than this value.</p>
<p>See the documentation of <a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.image_comparison" title="matplotlib.testing.decorators.image_comparison"><code class="xref py py-obj docutils literal notranslate"><span class="pre">image_comparison</span></code></a> and
<a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.check_figures_equal" title="matplotlib.testing.decorators.check_figures_equal"><code class="xref py py-obj docutils literal notranslate"><span class="pre">check_figures_equal</span></code></a> for additional information
about their use.</p>
</section>
<section id="creating-a-new-module-in-matplotlib-tests">
<h2>Creating a new module in matplotlib.tests<a class="headerlink" href="#creating-a-new-module-in-matplotlib-tests" title="Link to this heading">#</a></h2>
<p>We try to keep the tests categorized by the primary module they are
testing. For example, the tests related to the <code class="docutils literal notranslate"><span class="pre">mathtext.py</span></code> module
are in <code class="docutils literal notranslate"><span class="pre">test_mathtext.py</span></code>.</p>
</section>
<section id="using-github-actions-for-ci">
<h2>Using GitHub Actions for CI<a class="headerlink" href="#using-github-actions-for-ci" title="Link to this heading">#</a></h2>
<p><a class="reference external" href="https://docs.github.com/en/actions">GitHub Actions</a> is a hosted CI system
"in the cloud".</p>
<p>GitHub Actions is configured to receive notifications of new commits to GitHub
repos and to run builds or tests when it sees these new commits. It looks for a
YAML files in <code class="docutils literal notranslate"><span class="pre">.github/workflows</span></code> to see how to test the project.</p>
<p>GitHub Actions is already enabled for the <a class="reference external" href="https://github.com/matplotlib/matplotlib/">main Matplotlib GitHub repository</a> -- for example, see <a class="reference external" href="https://github.com/matplotlib/matplotlib/actions?query=workflow%3ATests">the Tests
workflows</a>.</p>
<p>GitHub Actions should be automatically enabled for your personal Matplotlib
fork once the YAML workflow files are in it. It generally isn't necessary to
look at these workflows, since any pull request submitted against the main
Matplotlib repository will be tested. The Tests workflow is skipped in forked
repositories but you can trigger a run manually from the <a class="reference external" href="https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow">GitHub web interface</a>.</p>
<p>You can see the GitHub Actions results at
<a class="github reference external" href="https://github.com/your_GitHub_user_name/matplotlib/actions">your_GitHub_user_name/matplotlib</a> -- here's <a class="reference external" href="https://github.com/QuLogic/matplotlib/actions">an
example</a>.</p>
</section>
<section id="using-tox">
<h2>Using tox<a class="headerlink" href="#using-tox" title="Link to this heading">#</a></h2>
<p><a class="reference external" href="https://tox.readthedocs.io/en/latest/">Tox</a> is a tool for running tests
against multiple Python environments, including multiple versions of Python
(e.g., 3.7, 3.8) and even different Python implementations altogether
(e.g., CPython, PyPy, Jython, etc.), as long as all these versions are
available on your system's $PATH (consider using your system package manager,
e.g. apt-get, yum, or Homebrew, to install them).</p>
<p>tox makes it easy to determine if your working copy introduced any
regressions before submitting a pull request. Here's how to use it:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>tox
$<span class="w"> </span>tox
</pre></div>
</div>
<p>You can also run tox on a subset of environments:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>tox<span class="w"> </span>-e<span class="w"> </span>py38,py39
</pre></div>
</div>
<p>Tox processes everything serially so it can take a long time to test
several environments. To speed it up, you might try using a new,
parallelized version of tox called <code class="docutils literal notranslate"><span class="pre">detox</span></code>. Give this a try:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$<span class="w"> </span>pip<span class="w"> </span>install<span class="w"> </span>-U<span class="w"> </span>-i<span class="w"> </span>http://pypi.testrun.org<span class="w"> </span>detox
$<span class="w"> </span>detox
</pre></div>
</div>
<p>Tox is configured using a file called <code class="docutils literal notranslate"><span class="pre">tox.ini</span></code>. You may need to
edit this file if you want to add new environments to test (e.g.,
<code class="docutils literal notranslate"><span class="pre">py33</span></code>) or if you want to tweak the dependencies or the way the
tests are run. For more info on the <code class="docutils literal notranslate"><span class="pre">tox.ini</span></code> file, see the <a class="reference external" href="https://tox.readthedocs.io/en/latest/config.html">Tox
Configuration Specification</a>.</p>
</section>
<section id="building-old-versions-of-matplotlib">
<h2>Building old versions of Matplotlib<a class="headerlink" href="#building-old-versions-of-matplotlib" title="Link to this heading">#</a></h2>
<p>When running a <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">bisect</span></code> to see which commit introduced a certain bug,
you may (rarely) need to build very old versions of Matplotlib. The following
constraints need to be taken into account:</p>
<ul class="simple">
<li><p>Matplotlib 1.3 (or earlier) requires numpy 1.8 (or earlier).</p></li>
</ul>
</section>
<section id="testing-released-versions-of-matplotlib">
<h2>Testing released versions of Matplotlib<a class="headerlink" href="#testing-released-versions-of-matplotlib" title="Link to this heading">#</a></h2>
<p>Running the tests on an installation of a released version (e.g. PyPI package
or conda package) also requires additional setup.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>For an end-user, there is usually no need to run the tests on released
versions of Matplotlib. Official releases are tested before publishing.</p>
</div>
<section id="install-additional-dependencies">
<h3>Install additional dependencies<a class="headerlink" href="#install-additional-dependencies" title="Link to this heading">#</a></h3>
<p>Install the <a class="reference internal" href="dependencies.html#test-dependencies"><span class="std std-ref">additional dependencies for testing</span></a>.</p>
</section>
<section id="obtain-the-reference-images">
<h3>Obtain the reference images<a class="headerlink" href="#obtain-the-reference-images" title="Link to this heading">#</a></h3>
<p>Many tests compare the plot result against reference images. The reference
images are not part of the regular packaged versions (pip wheels or conda
packages). If you want to run tests with reference images, you need to obtain
the reference images matching the version of Matplotlib you want to test.</p>
<p>To do so, either download the matching source distribution
<code class="docutils literal notranslate"><span class="pre">matplotlib-X.Y.Z.tar.gz</span></code> from <a class="reference external" href="https://pypi.org/project/matplotlib/">PyPI</a>
or alternatively, clone the git repository and <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">checkout</span> <span class="pre">vX.Y.Z</span></code>. Copy
the folder <code class="file docutils literal notranslate"><span class="pre">lib/matplotlib/tests/baseline_images</span></code> to the folder
<code class="file docutils literal notranslate"><span class="pre">matplotlib/tests</span></code> of your the matplotlib installation to test.
The correct target folder can be found using:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">c</span> <span class="s2">"import matplotlib.tests; print(matplotlib.tests.__file__.rsplit('/', 1)[0])"</span>
</pre></div>
</div>
<p>An analogous copying of <code class="file docutils literal notranslate"><span class="pre">lib/mpl_toolkits/tests/baseline_images</span></code>
is necessary for testing <code class="docutils literal notranslate"><span class="pre">mpl_toolkits</span></code>.</p>
</section>
<section id="run-the-tests">
<h3>Run the tests<a class="headerlink" href="#run-the-tests" title="Link to this heading">#</a></h3>
<p>To run the all the tests on your installed version of Matplotlib:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">m</span> <span class="n">pytest</span> <span class="o">--</span><span class="n">pyargs</span> <span class="n">matplotlib</span><span class="o">.</span><span class="n">tests</span>
</pre></div>
</div>
<p>The test discovery scope can be narrowed to single test modules or even single
functions:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">m</span> <span class="n">pytest</span> <span class="o">--</span><span class="n">pyargs</span> <span class="n">matplotlib</span><span class="o">.</span><span class="n">tests</span><span class="o">.</span><span class="n">test_simplification</span><span class="o">.</span><span class="n">py</span><span class="p">::</span><span class="n">test_clipping</span>
</pre></div>
</div>
</section>
</section>
</section>
</article>
</div>
<div class="bd-sidebar-secondary bd-toc">
<div class="toc-item">
<div class="tocsection onthispage">
<i class="fa-solid fa-list"></i> On this page
</div>
<nav id="bd-toc-nav" class="page-toc">
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#requirements">
Requirements
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#running-the-tests">
Running the tests
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#writing-a-simple-test">
Writing a simple test
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#random-data-in-tests">
Random data in tests
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#writing-an-image-comparison-test">
Writing an image comparison test
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#creating-a-new-module-in-matplotlib-tests">
Creating a new module in matplotlib.tests
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#using-github-actions-for-ci">
Using GitHub Actions for CI
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#using-tox">
Using tox
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#building-old-versions-of-matplotlib">
Building old versions of Matplotlib
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#testing-released-versions-of-matplotlib">
Testing released versions of Matplotlib
</a>
<ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#install-additional-dependencies">
Install additional dependencies
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#obtain-the-reference-images">
Obtain the reference images
</a>
</li>
<li class="toc-h3 nav-item toc-entry">
<a class="reference internal nav-link" href="#run-the-tests">
Run the tests
</a>
</li>
</ul>
</li>
</ul>
</nav>
</div>
</div>
</div>
<footer class="bd-footer-content">
<div class="bd-footer-content__inner">
</div>
</footer>
</main>
</div>
</div>
<!-- Scripts loaded after <body> so the DOM is not blocked -->
<script src="../_static/scripts/bootstrap.js?digest=796348d33e8b1d947c94"></script>
<script src="../_static/scripts/pydata-sphinx-theme.js?digest=796348d33e8b1d947c94"></script>
<footer class="bd-footer"><div class="bd-footer__inner container">
<div class="footer-item">
<p class="copyright">
© Copyright 2002–2012 John Hunter, Darren Dale, Eric Firing, Michael Droettboom and the Matplotlib development team; 2012–2024 The Matplotlib development team.<br>
</p>
</div>
<div class="footer-item">
<p class="sphinx-version">
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 7.2.6.<br>
</p>
</div>
<div class="footer-item">
<p class="sphinx-version">
Built from v3.7.5-3-gb58d40e6c8.
<br/>
</p>
</div>
</div>
</footer>
</body>
</html>