-
-
Notifications
You must be signed in to change notification settings - Fork 12
Expand file tree
/
Copy pathtesting.html
More file actions
1230 lines (1018 loc) · 86 KB
/
Copy pathtesting.html
File metadata and controls
1230 lines (1018 loc) · 86 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en" data-content_root="../" data-theme="light">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Testing guidelines — NumPy v2.6.dev0 Manual</title>
<script data-cfasync="false">
document.documentElement.dataset.mode = localStorage.getItem("mode") || "light";
document.documentElement.dataset.theme = localStorage.getItem("theme") || "light";
</script>
<!--
this give us a css class that will be invisible only if js is disabled
-->
<noscript>
<style>
.pst-js-only { display: none !important; }
</style>
</noscript>
<!-- Loaded before other Sphinx assets -->
<link href="../_static/styles/theme.css?digest=90905a2f556bf617f1a9" rel="stylesheet" />
<link href="../_static/styles/pydata-sphinx-theme.css?digest=90905a2f556bf617f1a9" rel="stylesheet" />
<link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=8f2a1f02" />
<link rel="stylesheet" type="text/css" href="../_static/graphviz.css?v=4ae1632d" />
<link rel="stylesheet" type="text/css" href="../_static/plot_directive.css" />
<link rel="stylesheet" type="text/css" href="../_static/copybutton.css?v=76b2166b" />
<link rel="stylesheet" type="text/css" href="https://fonts.googleapis.com/css?family=Vibur" />
<link rel="stylesheet" type="text/css" href="../_static/jupyterlite_sphinx.css?v=8ee2c72c" />
<link rel="stylesheet" type="text/css" href="../_static/sphinx-design.min.css?v=95c83b7e" />
<link rel="stylesheet" type="text/css" href="../_static/numpy.css?v=7e165a8b" />
<!-- So that users can add custom icons -->
<script defer src="../_static/scripts/fontawesome.js?digest=90905a2f556bf617f1a9"></script>
<!-- Pre-loaded scripts that we'll load fully later -->
<link rel="preload" as="script" href="../_static/scripts/bootstrap.js?digest=90905a2f556bf617f1a9" />
<link rel="preload" as="script" href="../_static/scripts/pydata-sphinx-theme.js?digest=90905a2f556bf617f1a9" />
<script src="../_static/documentation_options.js?v=538fdff5"></script>
<script src="../_static/doctools.js?v=9bcbadda"></script>
<script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
<script src="../_static/clipboard.min.js?v=a7894cd8"></script>
<script src="../_static/copybutton.js?v=30646c52"></script>
<script src="../_static/jupyterlite_sphinx.js?v=96e329c5"></script>
<script src="../_static/design-tabs.js?v=f930bc37"></script>
<script data-domain="numpy.org/doc/stable/" defer="defer" src="https://views.scientific-python.org/js/script.js"></script>
<script>DOCUMENTATION_OPTIONS.pagename = 'reference/testing';</script>
<script>
DOCUMENTATION_OPTIONS.theme_version = '0.20.0';
DOCUMENTATION_OPTIONS.theme_switcher_json_url = 'https://numpy.org/doc/_static/versions.json';
DOCUMENTATION_OPTIONS.theme_switcher_version_match = 'devdocs';
DOCUMENTATION_OPTIONS.show_version_warning_banner =
true;
</script>
<script>DOCUMENTATION_OPTIONS.search_as_you_type = false;</script>
<link rel="icon" href="../_static/favicon.ico"/>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Typing (numpy.typing)" href="typing.html" />
<link rel="prev" title="numpy.testing.overrides.get_overridable_numpy_array_functions" href="generated/numpy.testing.overrides.get_overridable_numpy_array_functions.html" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="docsearch:language" content="en"/>
<meta name="docsearch:version" content="2.6.dev0" />
<meta name="docbuild:last-update" content="Jul 23, 2026"/>
<script src="../_static/searchtools.js"></script>
<script src="../_static/language_data.js"></script>
<script src="../searchindex.js"></script>
</head>
<body data-default-mode="light">
<div id="pst-skip-link" class="skip-link d-print-none"><a href="#main-content">Skip to main content</a></div>
<div id="pst-scroll-pixel-helper"></div>
<button type="button" class="btn rounded-pill" id="pst-back-to-top">
<i class="fa-solid fa-arrow-up"></i>Back to top</button>
<dialog id="pst-search-dialog">
<form class="bd-search d-flex align-items-center"
action="../search.html"
method="get">
<i class="fa-solid fa-magnifying-glass"></i>
<input type="search"
class="form-control"
name="q"
placeholder="Search the docs ..."
aria-label="Search the docs ..."
autocomplete="off"
autocorrect="off"
autocapitalize="off"
spellcheck="false"/>
<span class="search-button__kbd-shortcut"><kbd class="kbd-shortcut__modifier">Ctrl</kbd>+<kbd>K</kbd></span>
</form>
</dialog>
<div class="pst-async-banner-revealer d-none">
<aside id="bd-header-version-warning" class="d-none d-print-none" aria-label="Version warning"></aside>
</div>
<header id="pst-header" class="bd-header navbar navbar-expand-lg bd-navbar d-print-none">
<div class="bd-header__inner bd-page-width">
<button class="pst-navbar-icon sidebar-toggle primary-toggle" aria-label="Site navigation">
<span class="fa-solid fa-bars"></span>
</button>
<div class="col-lg-3 navbar-header-items__start">
<div class="navbar-item">
<a class="navbar-brand logo" href="../index.html">
<img src="../_static/numpylogo.svg" class="logo__image only-light" alt="NumPy v2.6.dev0 Manual - Home"/>
<img src="../_static/numpylogo_dark.svg" class="logo__image only-dark pst-js-only" alt="NumPy v2.6.dev0 Manual - Home"/>
</a></div>
</div>
<div class="col-lg-9 navbar-header-items">
<div class="me-auto navbar-header-items__center">
<div class="navbar-item">
<nav>
<ul class="bd-navbar-elements navbar-nav">
<li class="nav-item ">
<a class="nav-link nav-internal" href="../user/index.html">
User Guide
</a>
</li>
<li class="nav-item current active">
<a class="nav-link nav-internal" href="index.html">
API reference
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-internal" href="../building/index.html">
Building from source
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-internal" href="../dev/index.html">
Development
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-internal" href="../release.html">
Release notes
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-external" href="https://numpy.org/numpy-tutorials/">
Learn
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-external" href="https://numpy.org/neps">
NEPs
</a>
</li>
</ul>
</nav></div>
</div>
<div class="navbar-header-items__end">
<div class="navbar-item">
<button class="btn btn-sm pst-navbar-icon search-button search-button__button pst-js-only" title="Search" aria-label="Search" data-bs-placement="bottom" data-bs-toggle="tooltip">
<i class="fa-solid fa-magnifying-glass fa-lg"></i>
</button></div>
<div class="navbar-item">
<div class="theme-switch-container dropdown pst-js-only" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Color mode">
<button class="btn btn-sm nav-link pst-navbar-icon theme-switch-button dropdown-toggle" aria-label="Color mode" data-bs-toggle="dropdown">
<i class="theme-switch fa-solid fa-sun fa-lg fa-fw" data-mode="light" title="Light"></i>
<i class="theme-switch fa-solid fa-moon fa-lg fa-fw" data-mode="dark" title="Dark"></i>
<i class="theme-switch fa-solid fa-circle-half-stroke fa-lg fa-fw" data-mode="auto" title="System Settings"></i>
</button>
<ul class="dropdown-menu dropdown-menu-end">
<li><button class="dropdown-item d-flex align-items-center theme-change-button" data-mode="auto"><i class="fa-solid fa-circle-half-stroke fa-lg fa-fw me-1"></i>System Settings</button></li>
<li><button class="dropdown-item d-flex align-items-center theme-change-button" data-mode="light"><i class="fa-solid fa-sun fa-lg fa-fw me-1"></i>Light</button></li>
<li><button class="dropdown-item d-flex align-items-center theme-change-button" data-mode="dark"><i class="fa-solid fa-moon fa-lg fa-fw me-1"></i>Dark</button></li>
</ul>
</div></div>
<div class="navbar-item">
<div class="version-switcher__container dropdown pst-js-only">
<button id="pst-version-switcher-button-2"
type="button"
class="version-switcher__button btn btn-sm dropdown-toggle"
data-bs-toggle="dropdown"
aria-haspopup="listbox"
aria-controls="pst-version-switcher-list-2"
aria-label="Version switcher list"
>
Choose version <!-- this text may get changed later by javascript -->
<span class="caret"></span>
</button>
<div id="pst-version-switcher-list-2"
class="version-switcher__menu dropdown-menu list-group-flush py-0"
role="listbox" aria-labelledby="pst-version-switcher-button-2">
<!-- dropdown will be populated by javascript on page load -->
</div>
</div></div>
<div class="navbar-item"><ul class="navbar-icon-links"
aria-label="Icon Links">
<li class="nav-item">
<a href="https://github.com/numpy/numpy" title="GitHub" class="nav-link pst-navbar-icon" rel="noopener" target="_blank" data-bs-toggle="tooltip" data-bs-placement="bottom"><i class="fa-brands fa-square-github fa-lg" aria-hidden="true"></i><span class="visually-hidden">GitHub</span></a>
</li>
</ul></div>
</div>
</div>
<button class="pst-navbar-icon sidebar-toggle secondary-toggle" aria-label="On this page">
<span class="fa-solid fa-outdent"></span>
</button>
</div>
</header>
<div class="bd-container">
<div class="bd-container__inner bd-page-width">
<dialog id="pst-primary-sidebar-modal"></dialog>
<div id="pst-primary-sidebar" class="bd-sidebar-primary bd-sidebar">
<div class="sidebar-header-items sidebar-primary__section">
<div class="sidebar-header-items__center">
<div class="navbar-item">
<nav>
<ul class="bd-navbar-elements navbar-nav">
<li class="nav-item ">
<a class="nav-link nav-internal" href="../user/index.html">
User Guide
</a>
</li>
<li class="nav-item current active">
<a class="nav-link nav-internal" href="index.html">
API reference
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-internal" href="../building/index.html">
Building from source
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-internal" href="../dev/index.html">
Development
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-internal" href="../release.html">
Release notes
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-external" href="https://numpy.org/numpy-tutorials/">
Learn
</a>
</li>
<li class="nav-item ">
<a class="nav-link nav-external" href="https://numpy.org/neps">
NEPs
</a>
</li>
</ul>
</nav></div>
</div>
<div class="sidebar-header-items__end">
<div class="navbar-item">
<button class="btn btn-sm pst-navbar-icon search-button search-button__button pst-js-only" title="Search" aria-label="Search" data-bs-placement="bottom" data-bs-toggle="tooltip">
<i class="fa-solid fa-magnifying-glass fa-lg"></i>
</button></div>
<div class="navbar-item">
<div class="theme-switch-container dropdown pst-js-only" data-bs-toggle="tooltip" data-bs-placement="bottom" title="Color mode">
<button class="btn btn-sm nav-link pst-navbar-icon theme-switch-button dropdown-toggle" aria-label="Color mode" data-bs-toggle="dropdown">
<i class="theme-switch fa-solid fa-sun fa-lg fa-fw" data-mode="light" title="Light"></i>
<i class="theme-switch fa-solid fa-moon fa-lg fa-fw" data-mode="dark" title="Dark"></i>
<i class="theme-switch fa-solid fa-circle-half-stroke fa-lg fa-fw" data-mode="auto" title="System Settings"></i>
</button>
<ul class="dropdown-menu dropdown-menu-end">
<li><button class="dropdown-item d-flex align-items-center theme-change-button" data-mode="auto"><i class="fa-solid fa-circle-half-stroke fa-lg fa-fw me-1"></i>System Settings</button></li>
<li><button class="dropdown-item d-flex align-items-center theme-change-button" data-mode="light"><i class="fa-solid fa-sun fa-lg fa-fw me-1"></i>Light</button></li>
<li><button class="dropdown-item d-flex align-items-center theme-change-button" data-mode="dark"><i class="fa-solid fa-moon fa-lg fa-fw me-1"></i>Dark</button></li>
</ul>
</div></div>
<div class="navbar-item">
<div class="version-switcher__container dropdown pst-js-only">
<button id="pst-version-switcher-button-3"
type="button"
class="version-switcher__button btn btn-sm dropdown-toggle"
data-bs-toggle="dropdown"
aria-haspopup="listbox"
aria-controls="pst-version-switcher-list-3"
aria-label="Version switcher list"
>
Choose version <!-- this text may get changed later by javascript -->
<span class="caret"></span>
</button>
<div id="pst-version-switcher-list-3"
class="version-switcher__menu dropdown-menu list-group-flush py-0"
role="listbox" aria-labelledby="pst-version-switcher-button-3">
<!-- dropdown will be populated by javascript on page load -->
</div>
</div></div>
<div class="navbar-item"><ul class="navbar-icon-links"
aria-label="Icon Links">
<li class="nav-item">
<a href="https://github.com/numpy/numpy" title="GitHub" class="nav-link pst-navbar-icon" rel="noopener" target="_blank" data-bs-toggle="tooltip" data-bs-placement="bottom"><i class="fa-brands fa-square-github fa-lg" aria-hidden="true"></i><span class="visually-hidden">GitHub</span></a>
</li>
</ul></div>
</div>
</div>
<div class="sidebar-primary-items__start sidebar-primary__section">
<div class="sidebar-primary-item pst-sidebar-collapse"><button id="pst-collapse-sidebar-button" aria-expanded="true" aria-controls="pst-primary-sidebar">
<svg class="pst-icon" role="img" aria-hidden="true" focusable="false" viewBox="0 0 16 16" xmlns="http://www.w3.org/2000/svg">
<path fill="currentColor" d="M3 15.5C2.36232 15.5 1.74874 15.2564 1.28478 14.8189C0.820828 14.3815 0.541576 13.7832 0.504167 13.1467L0.5 13L0.5 3C0.499965 2.36232 0.743605 1.74874 1.18107 1.28478C1.61854 0.820828 2.21676 0.541576 2.85333 0.504167L3 0.5L13 0.5C13.6377 0.499965 14.2513 0.743605 14.7152 1.18107C15.1792 1.61854 15.4584 2.21676 15.4958 2.85333L15.5 3L15.5 13C15.5 13.6377 15.2564 14.2513 14.8189 14.7152C14.3815 15.1792 13.7832 15.4584 13.1467 15.4958L13 15.5L3 15.5ZM3 13.8333L10.5 13.8333L10.5 2.16667L3 2.16667C2.79589 2.16669 2.59889 2.24163 2.44636 2.37726C2.29383 2.5129 2.19638 2.69979 2.1725 2.9025L2.16667 3L2.16667 13C2.16669 13.2041 2.24163 13.4011 2.37726 13.5536C2.5129 13.7062 2.69979 13.8036 2.9025 13.8275L3 13.8333ZM6.65583 10.325L6.5775 10.2558L4.91083 8.58917C4.76735 8.44567 4.68116 8.25476 4.66843 8.05223C4.65569 7.84971 4.71729 7.6495 4.84167 7.48917L4.91083 7.41083L6.5775 5.74417C6.72747 5.59471 6.9287 5.50794 7.14032 5.50148C7.35194 5.49502 7.55809 5.56935 7.7169 5.70937C7.8757 5.8494 7.97525 6.04463 7.99533 6.25539C8.01541 6.46616 7.95451 6.67667 7.825 6.84417L7.75583 6.9225L6.67917 8L7.75583 9.0775C7.89931 9.22099 7.98551 9.41191 7.99824 9.61443C8.01097 9.81695 7.94938 10.0172 7.825 10.1775L7.75583 10.2558C7.61234 10.3993 7.42142 10.4855 7.2189 10.4982C7.01638 10.511 6.81617 10.4494 6.65583 10.325Z"/>
</svg>
<span class="pst-collapse-sidebar-label">Collapse Sidebar</span>
<span class="pst-expand-sidebar-label">Expand Sidebar</span>
</button></div>
<div class="sidebar-primary-item">
<nav class="bd-docs-nav bd-links"
aria-label="Section Navigation">
<p class="bd-links__title" role="heading" aria-level="1">Section Navigation</p>
<div class="bd-toc-item navbar-nav"><ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="module_structure.html">NumPy’s module structure</a></li>
</ul>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="arrays.html">Array objects</a></li>
</ul>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="ufuncs.html">Universal functions (<code class="xref py py-class docutils literal notranslate"><span class="pre">ufunc</span></code>)</a></li>
</ul>
<ul class="current nav bd-sidenav">
<li class="toctree-l1 current active has-children"><a class="reference internal" href="routines.html">Routines and objects by topic</a><details open="open"><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="constants.html">Constants</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.array-creation.html">Array creation routines</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.array-manipulation.html">Array manipulation routines</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.bitwise.html">Bit-wise operations</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.strings.html">String functionality</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.datetime.html">Datetime support functions</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.dtype.html">Data type routines</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.emath.html">Mathematical functions with automatic domain</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.err.html">Floating point error handling</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.exceptions.html">Exceptions and Warnings</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.fft.html">Discrete Fourier Transform</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.functional.html">Functional programming</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.io.html">Input and output</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.indexing.html">Indexing routines</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.linalg.html">Linear algebra</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.logic.html">Logic functions</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.ma.html">Masked array operations</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.math.html">Mathematical functions</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.other.html">Miscellaneous routines</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.polynomials.html">Polynomials</a></li>
<li class="toctree-l2"><a class="reference internal" href="random/index.html">Random sampling</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.set.html">Set routines</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.sort.html">Sorting, searching, and counting</a></li>
<li class="toctree-l2"><a class="reference internal" href="routines.statistics.html">Statistics</a></li>
<li class="toctree-l2 current active has-children"><a class="reference internal" href="routines.testing.html">Test support</a><details open="open"><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_allclose.html">numpy.testing.assert_allclose</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_array_almost_equal_nulp.html">numpy.testing.assert_array_almost_equal_nulp</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_array_max_ulp.html">numpy.testing.assert_array_max_ulp</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_array_equal.html">numpy.testing.assert_array_equal</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_array_less.html">numpy.testing.assert_array_less</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_equal.html">numpy.testing.assert_equal</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_raises.html">numpy.testing.assert_raises</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_raises_regex.html">numpy.testing.assert_raises_regex</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_warns.html">numpy.testing.assert_warns</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_no_warnings.html">numpy.testing.assert_no_warnings</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_no_gc_cycles.html">numpy.testing.assert_no_gc_cycles</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_string_equal.html">numpy.testing.assert_string_equal</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_.html">numpy.testing.assert_</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_almost_equal.html">numpy.testing.assert_almost_equal</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_approx_equal.html">numpy.testing.assert_approx_equal</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.assert_array_almost_equal.html">numpy.testing.assert_array_almost_equal</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.print_assert_equal.html">numpy.testing.print_assert_equal</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.decorate_methods.html">numpy.testing.decorate_methods</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.clear_and_catch_warnings.html">numpy.testing.clear_and_catch_warnings</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.measure.html">numpy.testing.measure</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.rundocs.html">numpy.testing.rundocs</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.suppress_warnings.html">numpy.testing.suppress_warnings</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.overrides.allows_array_function_override.html">numpy.testing.overrides.allows_array_function_override</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.overrides.allows_array_ufunc_override.html">numpy.testing.overrides.allows_array_ufunc_override</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.overrides.get_overridable_numpy_ufuncs.html">numpy.testing.overrides.get_overridable_numpy_ufuncs</a></li>
<li class="toctree-l3"><a class="reference internal" href="generated/numpy.testing.overrides.get_overridable_numpy_array_functions.html">numpy.testing.overrides.get_overridable_numpy_array_functions</a></li>
<li class="toctree-l3 current active"><a class="current reference internal" href="#">Testing guidelines</a></li>
</ul>
</details></li>
<li class="toctree-l2"><a class="reference internal" href="routines.window.html">Window functions</a></li>
</ul>
</details></li>
</ul>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="typing.html">Typing (<code class="xref py py-mod docutils literal notranslate"><span class="pre">numpy.typing</span></code>)</a></li>
</ul>
<ul class="nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="c-api/index.html">NumPy C-API</a></li>
</ul>
<ul class="current nav bd-sidenav">
<li class="toctree-l1"><a class="reference internal" href="array_api.html">Array API standard compatibility</a></li>
<li class="toctree-l1"><a class="reference internal" href="simd/index.html">CPU/SIMD optimizations</a></li>
<li class="toctree-l1"><a class="reference internal" href="thread_safety.html">Thread Safety</a></li>
<li class="toctree-l1"><a class="reference internal" href="global_state.html">Global Configuration Options</a></li>
<li class="toctree-l1"><a class="reference internal" href="security.html">NumPy security</a></li>
<li class="toctree-l1 current active"><a class="current reference internal" href="#">Testing guidelines</a></li>
<li class="toctree-l1"><a class="reference internal" href="distutils_status_migration.html">Status of <code class="docutils literal notranslate"><span class="pre">numpy.distutils</span></code> and migration advice</a></li>
<li class="toctree-l1"><a class="reference internal" href="swig.html">NumPy and SWIG</a></li>
</ul>
</div>
</nav></div>
</div>
<div class="sidebar-primary-items__end sidebar-primary__section">
<div class="sidebar-primary-item">
<div id="ethical-ad-placement"
class="flat"
data-ea-publisher="readthedocs"
data-ea-type="readthedocs-sidebar"
data-ea-manual="true">
</div></div>
</div>
</div>
<main id="main-content" class="bd-main" role="main">
<div class="bd-content">
<div class="bd-article-container">
<div class="bd-header-article d-print-none">
<div class="header-article-items header-article__inner">
<div class="header-article-items__start">
<div class="header-article-item">
<nav aria-label="Breadcrumb" class="d-print-none">
<ul class="bd-breadcrumbs">
<li class="breadcrumb-item breadcrumb-home">
<a href="../index.html" class="nav-link" aria-label="Home">
<i class="fa-solid fa-home"></i>
</a>
</li>
<li class="breadcrumb-item"><a href="index.html" class="nav-link">NumPy reference</a></li>
<li class="breadcrumb-item"><a href="module_structure.html" class="nav-link">NumPy’s module structure</a></li>
<li class="breadcrumb-item"><a href="routines.testing.html" class="nav-link">Test support</a></li>
<li class="breadcrumb-item active" aria-current="page"><span class="ellipsis">Testing guidelines</span></li>
</ul>
</nav>
</div>
</div>
</div>
</div>
<div id="searchbox"></div>
<article class="bd-article">
<section id="testing-guidelines">
<span id="id1"></span><h1>Testing guidelines<a class="headerlink" href="#testing-guidelines" title="Link to this heading">#</a></h1>
<section id="introduction">
<h2>Introduction<a class="headerlink" href="#introduction" title="Link to this heading">#</a></h2>
<p>Until the 1.15 release, NumPy used the <a class="reference external" href="https://nose.readthedocs.io/en/latest/">nose</a> testing framework, it now uses
the <a class="reference external" href="https://pytest.readthedocs.io">pytest</a> framework. The older framework is still maintained in order to
support downstream projects that use the old numpy framework, but all tests
for NumPy should use pytest.</p>
<p>Our goal is that every module and package in NumPy
should have a thorough set of unit
tests. These tests should exercise the full functionality of a given
routine as well as its robustness to erroneous or unexpected input
arguments. Well-designed tests with good coverage make
an enormous difference to the ease of refactoring. Whenever a new bug
is found in a routine, you should write a new test for that specific
case and add it to the test suite to prevent that bug from creeping
back in unnoticed.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>SciPy uses the testing framework from <a class="reference internal" href="routines.testing.html#module-numpy.testing" title="numpy.testing"><code class="xref py py-mod docutils literal notranslate"><span class="pre">numpy.testing</span></code></a>,
so all of the NumPy examples shown below are also applicable to SciPy</p>
</div>
</section>
<section id="testing-numpy">
<h2>Testing NumPy<a class="headerlink" href="#testing-numpy" title="Link to this heading">#</a></h2>
<p>NumPy can be tested in a number of ways, choose any way you feel comfortable.</p>
<section id="running-tests-from-inside-python">
<h3>Running tests from inside Python<a class="headerlink" href="#running-tests-from-inside-python" title="Link to this heading">#</a></h3>
<p>You can test an installed NumPy by <a class="reference internal" href="#numpy.test" title="numpy.test"><code class="xref py py-obj docutils literal notranslate"><span class="pre">numpy.test</span></code></a>, for example,
To run NumPy’s full test suite, use the following:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span>
<span class="gp">>>> </span><span class="n">numpy</span><span class="o">.</span><span class="n">test</span><span class="p">(</span><span class="n">label</span><span class="o">=</span><span class="s1">'slow'</span><span class="p">)</span>
</pre></div>
</div>
<p>The test method may take two or more arguments; the first <code class="docutils literal notranslate"><span class="pre">label</span></code> is a
string specifying what should be tested and the second <code class="docutils literal notranslate"><span class="pre">verbose</span></code> is an
integer giving the level of output verbosity. See the docstring
<a class="reference internal" href="#numpy.test" title="numpy.test"><code class="xref py py-obj docutils literal notranslate"><span class="pre">numpy.test</span></code></a>
for details. The default value for <code class="docutils literal notranslate"><span class="pre">label</span></code> is ‘fast’ - which
will run the standard tests. The string ‘full’ will run the full battery
of tests, including those identified as being slow to run. If <code class="docutils literal notranslate"><span class="pre">verbose</span></code>
is 1 or less, the tests will just show information messages about the tests
that are run; but if it is greater than 1, then the tests will also provide
warnings on missing tests. So if you want to run every test and get
messages about which modules don’t have tests:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">numpy</span><span class="o">.</span><span class="n">test</span><span class="p">(</span><span class="n">label</span><span class="o">=</span><span class="s1">'full'</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span> <span class="c1"># or numpy.test('full', 2)</span>
</pre></div>
</div>
<p>Finally, if you are only interested in testing a subset of NumPy, for
example, the <code class="docutils literal notranslate"><span class="pre">_core</span></code> module, use the following:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">numpy</span><span class="o">.</span><span class="n">_core</span><span class="o">.</span><span class="n">test</span><span class="p">()</span>
</pre></div>
</div>
</section>
<section id="running-tests-from-the-command-line">
<h3>Running tests from the command line<a class="headerlink" href="#running-tests-from-the-command-line" title="Link to this heading">#</a></h3>
<p>If you want to build NumPy in order to work on NumPy itself, use the
<a class="reference internal" href="../dev/spin.html#spin-tool"><span class="std std-ref">spin utility</span></a>.</p>
<p>To run NumPy’s full test suite:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ spin test -m full
</pre></div>
</div>
<p>Testing a subset of NumPy:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ spin test -t numpy/_core/tests
</pre></div>
</div>
<p>For detailed info on testing, see <a class="reference internal" href="../dev/development_environment.html#testing-builds"><span class="std std-ref">Testing builds</span></a></p>
</section>
<section id="running-tests-in-multiple-threads">
<h3>Running tests in multiple threads<a class="headerlink" href="#running-tests-in-multiple-threads" title="Link to this heading">#</a></h3>
<p>To help with stress testing NumPy for thread safety, the test suite can be run under
<a class="reference external" href="https://github.com/Quansight-Labs/pytest-run-parallel">pytest-run-parallel</a>. To install <code class="docutils literal notranslate"><span class="pre">pytest-run-parallel</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ pip install pytest-run-parallel
</pre></div>
</div>
<p>To run the test suite in multiple threads:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ spin test -p auto # have pytest-run-parallel detect the number of available cores
$ spin test -p 4 # run each test under 4 threads
$ spin test -p auto -- --skip-thread-unsafe=true # run ONLY tests that are thread-safe
</pre></div>
</div>
<p>When you write new tests, it is worth testing to make sure they do not fail
under <code class="docutils literal notranslate"><span class="pre">pytest-run-parallel</span></code>, since the CI jobs make use of it. Some tips on how to
write thread-safe tests can be found <a class="reference external" href="#writing-thread-safe-tests">here</a>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Ideally you should run <code class="docutils literal notranslate"><span class="pre">pytest-run-parallel</span></code> using a <a class="reference external" href="https://docs.python.org/3/howto/free-threading-python.html">free-threaded build of Python</a> that is 3.14 or
higher. If you decide to use a version of Python that is not free-threaded, you will
need to set the environment variables <code class="docutils literal notranslate"><span class="pre">PYTHON_CONTEXT_AWARE_WARNINGS</span></code> and
<code class="docutils literal notranslate"><span class="pre">PYTHON_THREAD_INHERIT_CONTEXT</span></code> to 1.</p>
</div>
</section>
<section id="running-doctests">
<h3>Running doctests<a class="headerlink" href="#running-doctests" title="Link to this heading">#</a></h3>
<p>NumPy documentation contains code examples, “doctests”. To check that the examples
are correct, install the <code class="docutils literal notranslate"><span class="pre">scipy-doctest</span></code> package:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ pip install scipy-doctest
</pre></div>
</div>
<p>and run one of:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ spin check-docs -v
$ spin check-docs numpy/linalg
$ spin check-docs -- -k 'det and not slogdet'
</pre></div>
</div>
<p>Note that the doctests are not run when you use <code class="docutils literal notranslate"><span class="pre">spin</span> <span class="pre">test</span></code>.</p>
</section>
<section id="other-methods-of-running-tests">
<h3>Other methods of running tests<a class="headerlink" href="#other-methods-of-running-tests" title="Link to this heading">#</a></h3>
<p>Run tests using your favourite IDE such as <a class="reference external" href="https://code.visualstudio.com/docs/python/testing#_enable-a-test-framework">vscode</a> or <a class="reference external" href="https://www.jetbrains.com/help/pycharm/testing-your-first-python-application.html">pycharm</a></p>
</section>
</section>
<section id="writing-your-own-tests">
<h2>Writing your own tests<a class="headerlink" href="#writing-your-own-tests" title="Link to this heading">#</a></h2>
<p>If you are writing code that you’d like to become part of NumPy,
please write the tests as you develop your code.
Every Python module, extension module, or subpackage in the NumPy
package directory should have a corresponding <code class="docutils literal notranslate"><span class="pre">test_<name>.py</span></code> file.
Pytest examines these files for test methods (named <code class="docutils literal notranslate"><span class="pre">test*</span></code>) and test
classes (named <code class="docutils literal notranslate"><span class="pre">Test*</span></code>).</p>
<p>Suppose you have a NumPy module <code class="docutils literal notranslate"><span class="pre">numpy/xxx/yyy.py</span></code> containing a
function <code class="docutils literal notranslate"><span class="pre">zzz()</span></code>. To test this function you would create a test
module called <code class="docutils literal notranslate"><span class="pre">test_yyy.py</span></code>. If you only need to test one aspect of
<code class="docutils literal notranslate"><span class="pre">zzz</span></code>, you can simply add a test function:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span><span class="w"> </span><span class="nf">test_zzz</span><span class="p">():</span>
<span class="k">assert</span> <span class="n">zzz</span><span class="p">()</span> <span class="o">==</span> <span class="s1">'Hello from zzz'</span>
</pre></div>
</div>
<p>More often, we need to group a number of tests together, so we create
a test class:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># import xxx symbols</span>
<span class="kn">from</span><span class="w"> </span><span class="nn">numpy.xxx.yyy</span><span class="w"> </span><span class="kn">import</span> <span class="n">zzz</span>
<span class="kn">import</span><span class="w"> </span><span class="nn">pytest</span>
<span class="k">class</span><span class="w"> </span><span class="nc">TestZzz</span><span class="p">:</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_simple</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">assert</span> <span class="n">zzz</span><span class="p">()</span> <span class="o">==</span> <span class="s1">'Hello from zzz'</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_invalid_parameter</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">with</span> <span class="n">pytest</span><span class="o">.</span><span class="n">raises</span><span class="p">(</span><span class="ne">ValueError</span><span class="p">,</span> <span class="n">match</span><span class="o">=</span><span class="s1">'.*some matching regex.*'</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>Within these test methods, the <code class="docutils literal notranslate"><span class="pre">assert</span></code> statement or a specialized assertion
function is used to test whether a certain assumption is valid. If the
assertion fails, the test fails. Common assertion functions include:</p>
<ul class="simple">
<li><p><a class="reference internal" href="generated/numpy.testing.assert_equal.html#numpy.testing.assert_equal" title="numpy.testing.assert_equal"><code class="xref py py-func docutils literal notranslate"><span class="pre">numpy.testing.assert_equal</span></code></a> for testing exact elementwise equality
between a result array and a reference,</p></li>
<li><p><a class="reference internal" href="generated/numpy.testing.assert_allclose.html#numpy.testing.assert_allclose" title="numpy.testing.assert_allclose"><code class="xref py py-func docutils literal notranslate"><span class="pre">numpy.testing.assert_allclose</span></code></a> for testing near elementwise equality
between a result array and a reference (i.e. with specified relative and
absolute tolerances), and</p></li>
<li><p><a class="reference internal" href="generated/numpy.testing.assert_array_less.html#numpy.testing.assert_array_less" title="numpy.testing.assert_array_less"><code class="xref py py-func docutils literal notranslate"><span class="pre">numpy.testing.assert_array_less</span></code></a> for testing (strict) elementwise
ordering between a result array and a reference.</p></li>
</ul>
<p>By default, these assertion functions only compare the numerical values in the
arrays. Consider using the <code class="docutils literal notranslate"><span class="pre">strict=True</span></code> option to check the array dtype
and shape, too.</p>
<p>When you need custom assertions, use the Python <code class="docutils literal notranslate"><span class="pre">assert</span></code> statement. Note that
<code class="docutils literal notranslate"><span class="pre">pytest</span></code> internally rewrites <code class="docutils literal notranslate"><span class="pre">assert</span></code> statements to give informative
output when it fails, so it should be preferred over the legacy variant
<code class="docutils literal notranslate"><span class="pre">numpy.testing.assert_</span></code>. Whereas plain <code class="docutils literal notranslate"><span class="pre">assert</span></code> statements are ignored
when running Python in optimized mode with <code class="docutils literal notranslate"><span class="pre">-O</span></code>, this is not an issue when
running tests with pytest.</p>
<p>Similarly, the pytest functions <a class="reference external" href="https://docs.pytest.org/en/stable/reference/reference.html#pytest.raises" title="(in pytest v9.1.1)"><code class="xref py py-func docutils literal notranslate"><span class="pre">pytest.raises</span></code></a> and <a class="reference external" href="https://docs.pytest.org/en/stable/reference/reference.html#pytest.warns" title="(in pytest v9.1.1)"><code class="xref py py-func docutils literal notranslate"><span class="pre">pytest.warns</span></code></a>
should be preferred over their legacy counterparts
<a class="reference internal" href="generated/numpy.testing.assert_raises.html#numpy.testing.assert_raises" title="numpy.testing.assert_raises"><code class="xref py py-func docutils literal notranslate"><span class="pre">numpy.testing.assert_raises</span></code></a> and <a class="reference internal" href="generated/numpy.testing.assert_warns.html#numpy.testing.assert_warns" title="numpy.testing.assert_warns"><code class="xref py py-func docutils literal notranslate"><span class="pre">numpy.testing.assert_warns</span></code></a>,
which are more broadly used. These versions also accept a <code class="docutils literal notranslate"><span class="pre">match</span></code>
parameter, which should always be used to precisely target the intended
warning or error.</p>
<p>Note that <code class="docutils literal notranslate"><span class="pre">test_</span></code> functions or methods should not have a docstring, because
that makes it hard to identify the test from the output of running the test
suite with <code class="docutils literal notranslate"><span class="pre">verbose=2</span></code> (or similar verbosity setting). Use plain comments
(<code class="docutils literal notranslate"><span class="pre">#</span></code>) to describe the intent of the test and help the unfamiliar reader to
interpret the code.</p>
<p>Also, since much of NumPy is legacy code that was
originally written without unit tests, there are still several modules
that don’t have tests yet. Please feel free to choose one of these
modules and develop tests for it.</p>
<section id="using-c-code-in-tests">
<h3>Using C code in tests<a class="headerlink" href="#using-c-code-in-tests" title="Link to this heading">#</a></h3>
<p>NumPy exposes a rich <a class="reference internal" href="c-api/index.html#c-api"><span class="std std-ref">C-API</span></a> . These are tested using c-extension
modules written “as-if” they know nothing about the internals of NumPy, rather
using the official C-API interfaces only. Examples of such modules are tests
for a user-defined <code class="docutils literal notranslate"><span class="pre">rational</span></code> dtype in <code class="docutils literal notranslate"><span class="pre">_rational_tests</span></code> or the ufunc
machinery tests in <code class="docutils literal notranslate"><span class="pre">_umath_tests</span></code> which are part of the binary distribution.
Starting from version 1.21, you can also write snippets of C code in tests that
will be compiled locally into c-extension modules and loaded into python.</p>
<dl class="py function">
<dt class="sig sig-object py" id="numpy.testing.extbuild.build_and_import_extension">
<span class="sig-prename descclassname"><span class="pre">numpy.testing.extbuild.</span></span><span class="sig-name descname"><span class="pre">build_and_import_extension</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">modname</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">functions</span></span></em>, <em class="sig-param"><span class="keyword-only-separator o"><abbr title="Keyword-only parameters separator (PEP 3102)"><span class="pre">*</span></abbr></span></em>, <em class="sig-param"><span class="n"><span class="pre">prologue</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">''</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">build_dir</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">include_dirs</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">more_init</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">''</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#numpy.testing.extbuild.build_and_import_extension" title="Link to this definition">#</a></dt>
<dd><p>Build and imports a c-extension module <em class="xref py py-obj">modname</em> from a list of function
fragments <em class="xref py py-obj">functions</em>.</p>
<dl class="field-list simple">
<dt class="field-odd">Parameters<span class="colon">:</span></dt>
<dd class="field-odd"><dl class="simple">
<dt><strong>functions</strong><span class="classifier">list of fragments</span></dt><dd><p>Each fragment is a sequence of func_name, calling convention, snippet.</p>
</dd>
<dt><strong>prologue</strong><span class="classifier">string</span></dt><dd><p>Code to precede the rest, usually extra <code class="docutils literal notranslate"><span class="pre">#include</span></code> or <code class="docutils literal notranslate"><span class="pre">#define</span></code>
macros.</p>
</dd>
<dt><strong>build_dir</strong><span class="classifier">pathlib.Path</span></dt><dd><p>Where to build the module, usually a temporary directory</p>
</dd>
<dt><strong>include_dirs</strong><span class="classifier">list</span></dt><dd><p>Extra directories to find include files when compiling</p>
</dd>
<dt><strong>more_init</strong><span class="classifier">string</span></dt><dd><p>Code to appear in the module PyMODINIT_FUNC</p>
</dd>
</dl>
</dd>
<dt class="field-even">Returns<span class="colon">:</span></dt>
<dd class="field-even"><dl class="simple">
<dt>out: module</dt><dd><p>The module will have been loaded and is ready for use</p>
</dd>
</dl>
</dd>
</dl>
<p class="rubric">Examples</p>
<div class="try_examples_outer_container docutils container" id="010a2cd2-1550-4300-a778-d9e7dbd4c698">
<div class="try_examples_button_container"><button class="try_examples_button" onclick="window.tryExamplesShowIframe('010a2cd2-1550-4300-a778-d9e7dbd4c698','27524781-4170-4089-b804-e67e9c72cfd3','30484023-7052-4fd5-a450-096d18abca1a','../lite/tree/../notebooks/index.html?path=01d70e9b_ecbc_44d0_827d_a0352f3a0e73.ipynb','None')">Try it in your browser!</button></div><div class="try_examples_content docutils container">
<div class="doctest highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="n">functions</span> <span class="o">=</span> <span class="p">[(</span><span class="s2">"test_bytes"</span><span class="p">,</span> <span class="s2">"METH_O"</span><span class="p">,</span> <span class="s2">"""</span>
<span class="go"> if ( !PyBytesCheck(args)) {</span>
<span class="go"> Py_RETURN_FALSE;</span>
<span class="go"> }</span>
<span class="go"> Py_RETURN_TRUE;</span>
<span class="go">""")]</span>
<span class="gp">>>> </span><span class="s2">mod = build_and_import_extension("testme", functions)</span>
<span class="gp">>>> </span><span class="s2">assert not mod.test_bytes('abc')</span>
<span class="gp">>>> </span><span class="s2">assert mod.test_bytes(b'abc')</span>
</pre></div>
</div>
</div>
</div>
<div id="30484023-7052-4fd5-a450-096d18abca1a" class="try_examples_outer_iframe hidden"><div class="try_examples_button_container"><button class="try_examples_button" onclick="window.tryExamplesHideIframe('010a2cd2-1550-4300-a778-d9e7dbd4c698','30484023-7052-4fd5-a450-096d18abca1a')">Go Back</button><button class="try_examples_button" onclick="window.openInNewTab('010a2cd2-1550-4300-a778-d9e7dbd4c698','30484023-7052-4fd5-a450-096d18abca1a')">Open In Tab</button></div><div id="27524781-4170-4089-b804-e67e9c72cfd3" class="jupyterlite_sphinx_iframe_container"></div></div><script>document.addEventListener("DOMContentLoaded", function() {window.loadTryExamplesConfig("../try_examples.json");});</script></dd></dl>
</section>
<section id="labeling-tests">
<h3>Labeling tests<a class="headerlink" href="#labeling-tests" title="Link to this heading">#</a></h3>
<p>Unlabeled tests like the ones above are run in the default
<code class="docutils literal notranslate"><span class="pre">numpy.test()</span></code> run. If you want to label your test as slow - and
therefore reserved for a full <code class="docutils literal notranslate"><span class="pre">numpy.test(label='full')</span></code> run, you
can label it with <code class="docutils literal notranslate"><span class="pre">pytest.mark.slow</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">pytest</span>
<span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">slow</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_big</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'Big, slow test'</span><span class="p">)</span>
</pre></div>
</div>
<p>Similarly for methods:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">test_zzz</span><span class="p">:</span>
<span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">slow</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_simple</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">assert_</span><span class="p">(</span><span class="n">zzz</span><span class="p">()</span> <span class="o">==</span> <span class="s1">'Hello from zzz'</span><span class="p">)</span>
</pre></div>
</div>
</section>
<section id="setup-and-teardown-methods">
<h3>Setup and teardown methods<a class="headerlink" href="#setup-and-teardown-methods" title="Link to this heading">#</a></h3>
<p>NumPy originally used xunit setup and teardown, a feature of <a class="reference external" href="https://docs.pytest.org/en/stable/index.html#module-pytest" title="(in pytest v9.1.1)"><code class="xref py py-obj docutils literal notranslate"><span class="pre">pytest</span></code></a>. We now encourage
the usage of setup and teardown methods that are called explicitly by the tests that
need them:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">class</span><span class="w"> </span><span class="nc">TestMe</span><span class="p">:</span>
<span class="k">def</span><span class="w"> </span><span class="nf">setup</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'doing setup'</span><span class="p">)</span>
<span class="k">return</span> <span class="mi">1</span>
<span class="k">def</span><span class="w"> </span><span class="nf">teardown</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="nb">print</span><span class="p">(</span><span class="s1">'doing teardown'</span><span class="p">)</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_xyz</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">x</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">setup</span><span class="p">()</span>
<span class="k">assert</span> <span class="n">x</span> <span class="o">==</span> <span class="mi">1</span>
<span class="bp">self</span><span class="o">.</span><span class="n">teardown</span><span class="p">()</span>
</pre></div>
</div>
<p>This approach is thread-safe, ensuring tests can run under <code class="docutils literal notranslate"><span class="pre">pytest-run-parallel</span></code>.
Using pytest setup fixtures (such as xunit setup methods) is generally not thread-safe
and will likely cause thread-safety test failures.</p>
<p><code class="docutils literal notranslate"><span class="pre">pytest</span></code> supports more general fixture at various scopes which may be used
automatically via special arguments. For example, the special argument name
<code class="docutils literal notranslate"><span class="pre">tmp_path</span></code> is used in tests to create temporary directories. However,
fixtures should be used sparingly.</p>
</section>
<section id="parametric-tests">
<h3>Parametric tests<a class="headerlink" href="#parametric-tests" title="Link to this heading">#</a></h3>
<p>One very nice feature of <code class="docutils literal notranslate"><span class="pre">pytest</span></code> is the ease of testing across a range
of parameter values using the <code class="docutils literal notranslate"><span class="pre">pytest.mark.parametrize</span></code> decorator. For example,
suppose you wish to test <code class="docutils literal notranslate"><span class="pre">linalg.solve</span></code> for all combinations of three
array sizes and two data types:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="s1">'dimensionality'</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">25</span><span class="p">])</span>
<span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">parametrize</span><span class="p">(</span><span class="s1">'dtype'</span><span class="p">,</span> <span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">float32</span><span class="p">,</span> <span class="n">np</span><span class="o">.</span><span class="n">float64</span><span class="p">])</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_solve</span><span class="p">(</span><span class="n">dimensionality</span><span class="p">,</span> <span class="n">dtype</span><span class="p">):</span>
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">842523</span><span class="p">)</span>
<span class="n">A</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">(</span><span class="n">size</span><span class="o">=</span><span class="p">(</span><span class="n">dimensionality</span><span class="p">,</span> <span class="n">dimensionality</span><span class="p">))</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">dtype</span><span class="p">)</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">random</span><span class="p">(</span><span class="n">size</span><span class="o">=</span><span class="n">dimensionality</span><span class="p">)</span><span class="o">.</span><span class="n">astype</span><span class="p">(</span><span class="n">dtype</span><span class="p">)</span>
<span class="n">x</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linalg</span><span class="o">.</span><span class="n">solve</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">b</span><span class="p">)</span>
<span class="n">eps</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">finfo</span><span class="p">(</span><span class="n">dtype</span><span class="p">)</span><span class="o">.</span><span class="n">eps</span>
<span class="n">assert_allclose</span><span class="p">(</span><span class="n">A</span> <span class="o">@</span> <span class="n">x</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">rtol</span><span class="o">=</span><span class="n">eps</span><span class="o">*</span><span class="mf">1e2</span><span class="p">,</span> <span class="n">atol</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="k">assert</span> <span class="n">x</span><span class="o">.</span><span class="n">dtype</span> <span class="o">==</span> <span class="n">np</span><span class="o">.</span><span class="n">dtype</span><span class="p">(</span><span class="n">dtype</span><span class="p">)</span>
</pre></div>
</div>
</section>
<section id="doctests">
<h3>Doctests<a class="headerlink" href="#doctests" title="Link to this heading">#</a></h3>
<p>Doctests are a convenient way of documenting the behavior of a function
and allowing that behavior to be tested at the same time. The output
of an interactive Python session can be included in the docstring of a
function, and the test framework can run the example and compare the
actual output to the expected output.</p>
<p>The doctests can be run by adding the <code class="docutils literal notranslate"><span class="pre">doctests</span></code> argument to the
<code class="docutils literal notranslate"><span class="pre">test()</span></code> call; for example, to run all tests (including doctests)
for numpy.lib:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
<span class="gp">>>> </span><span class="n">np</span><span class="o">.</span><span class="n">lib</span><span class="o">.</span><span class="n">test</span><span class="p">(</span><span class="n">doctests</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
</pre></div>
</div>
<p>The doctests are run as if they are in a fresh Python instance which
has executed <code class="docutils literal notranslate"><span class="pre">import</span> <span class="pre">numpy</span> <span class="pre">as</span> <span class="pre">np</span></code>. Tests that are part of a NumPy
subpackage will have that subpackage already imported. E.g. for a test
in <code class="docutils literal notranslate"><span class="pre">numpy/linalg/tests/</span></code>, the namespace will be created such that
<code class="docutils literal notranslate"><span class="pre">from</span> <span class="pre">numpy</span> <span class="pre">import</span> <span class="pre">linalg</span></code> has already executed.</p>
</section>
<section id="tests">
<h3><code class="docutils literal notranslate"><span class="pre">tests/</span></code><a class="headerlink" href="#tests" title="Link to this heading">#</a></h3>
<p>Rather than keeping the code and the tests in the same directory, we
put all the tests for a given subpackage in a <code class="docutils literal notranslate"><span class="pre">tests/</span></code>
subdirectory. For our example, if it doesn’t already exist you will
need to create a <code class="docutils literal notranslate"><span class="pre">tests/</span></code> directory in <code class="docutils literal notranslate"><span class="pre">numpy/xxx/</span></code>. So the path
for <code class="docutils literal notranslate"><span class="pre">test_yyy.py</span></code> is <code class="docutils literal notranslate"><span class="pre">numpy/xxx/tests/test_yyy.py</span></code>.</p>
<p>Once the <code class="docutils literal notranslate"><span class="pre">numpy/xxx/tests/test_yyy.py</span></code> is written, its possible to
run the tests by going to the <code class="docutils literal notranslate"><span class="pre">tests/</span></code> directory and typing:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="n">test_yyy</span><span class="o">.</span><span class="n">py</span>
</pre></div>
</div>
<p>Or if you add <code class="docutils literal notranslate"><span class="pre">numpy/xxx/tests/</span></code> to the Python path, you could run
the tests interactively in the interpreter like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span><span class="w"> </span><span class="nn">test_yyy</span>
<span class="gp">>>> </span><span class="n">test_yyy</span><span class="o">.</span><span class="n">test</span><span class="p">()</span>
</pre></div>
</div>
</section>
<section id="init-py-and-setup-py">
<h3><code class="docutils literal notranslate"><span class="pre">__init__.py</span></code> and <code class="docutils literal notranslate"><span class="pre">setup.py</span></code><a class="headerlink" href="#init-py-and-setup-py" title="Link to this heading">#</a></h3>
<p>Usually, however, adding the <code class="docutils literal notranslate"><span class="pre">tests/</span></code> directory to the python path
isn’t desirable. Instead it would better to invoke the test straight
from the module <code class="docutils literal notranslate"><span class="pre">xxx</span></code>. To this end, simply place the following lines
at the end of your package’s <code class="docutils literal notranslate"><span class="pre">__init__.py</span></code> file:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">...</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test</span><span class="p">(</span><span class="n">level</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">verbosity</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="kn">from</span><span class="w"> </span><span class="nn">numpy.testing</span><span class="w"> </span><span class="kn">import</span> <span class="n">Tester</span>
<span class="k">return</span> <span class="n">Tester</span><span class="p">()</span><span class="o">.</span><span class="n">test</span><span class="p">(</span><span class="n">level</span><span class="p">,</span> <span class="n">verbosity</span><span class="p">)</span>
</pre></div>
</div>
<p>You will also need to add the tests directory in the configuration
section of your setup.py:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="o">...</span>
<span class="k">def</span><span class="w"> </span><span class="nf">configuration</span><span class="p">(</span><span class="n">parent_package</span><span class="o">=</span><span class="s1">''</span><span class="p">,</span> <span class="n">top_path</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="o">...</span>
<span class="n">config</span><span class="o">.</span><span class="n">add_subpackage</span><span class="p">(</span><span class="s1">'tests'</span><span class="p">)</span>
<span class="k">return</span> <span class="n">config</span>
<span class="o">...</span>
</pre></div>
</div>
<p>Now you can do the following to test your module:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span>
<span class="gp">>>> </span><span class="n">numpy</span><span class="o">.</span><span class="n">xxx</span><span class="o">.</span><span class="n">test</span><span class="p">()</span>
</pre></div>
</div>
<p>Also, when invoking the entire NumPy test suite, your tests will be
found and run:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span>
<span class="gp">>>> </span><span class="n">numpy</span><span class="o">.</span><span class="n">test</span><span class="p">()</span>
<span class="go"># your tests are included and run automatically!</span>
</pre></div>
</div>
</section>
</section>
<section id="tips-tricks">
<h2>Tips & Tricks<a class="headerlink" href="#tips-tricks" title="Link to this heading">#</a></h2>
<section id="known-failures-skipping-tests">
<h3>Known failures & skipping tests<a class="headerlink" href="#known-failures-skipping-tests" title="Link to this heading">#</a></h3>
<p>Sometimes you might want to skip a test or mark it as a known failure,
such as when the test suite is being written before the code it’s
meant to test, or if a test only fails on a particular architecture.</p>
<p>To skip a test, simply use <code class="docutils literal notranslate"><span class="pre">skipif</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">pytest</span>
<span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">skipif</span><span class="p">(</span><span class="n">SkipMyTest</span><span class="p">,</span> <span class="n">reason</span><span class="o">=</span><span class="s2">"Skipping this test because..."</span><span class="p">)</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_something</span><span class="p">(</span><span class="n">foo</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>The test is marked as skipped if <code class="docutils literal notranslate"><span class="pre">SkipMyTest</span></code> evaluates to nonzero,
and the message in verbose test output is the second argument given to
<code class="docutils literal notranslate"><span class="pre">skipif</span></code>. Similarly, a test can be marked as a known failure by
using <code class="docutils literal notranslate"><span class="pre">xfail</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">pytest</span>
<span class="nd">@pytest</span><span class="o">.</span><span class="n">mark</span><span class="o">.</span><span class="n">xfail</span><span class="p">(</span><span class="n">MyTestFails</span><span class="p">,</span> <span class="n">reason</span><span class="o">=</span><span class="s2">"This test is known to fail because..."</span><span class="p">)</span>
<span class="k">def</span><span class="w"> </span><span class="nf">test_something_else</span><span class="p">(</span><span class="n">foo</span><span class="p">):</span>
<span class="o">...</span>
</pre></div>
</div>
<p>Of course, a test can be unconditionally skipped or marked as a known
failure by using <code class="docutils literal notranslate"><span class="pre">skip</span></code> or <code class="docutils literal notranslate"><span class="pre">xfail</span></code> without argument, respectively.</p>
<p>A total of the number of skipped and known failing tests is displayed
at the end of the test run. Skipped tests are marked as <code class="docutils literal notranslate"><span class="pre">'S'</span></code> in
the test results (or <code class="docutils literal notranslate"><span class="pre">'SKIPPED'</span></code> for <code class="docutils literal notranslate"><span class="pre">verbose</span> <span class="pre">></span> <span class="pre">1</span></code>), and known
failing tests are marked as <code class="docutils literal notranslate"><span class="pre">'x'</span></code> (or <code class="docutils literal notranslate"><span class="pre">'XFAIL'</span></code> if <code class="docutils literal notranslate"><span class="pre">verbose</span> <span class="pre">></span>
<span class="pre">1</span></code>).</p>
</section>