Pytest使用基础(Pytest usage Foundation)

1 Pytest的运行

1.1 测试搜索

运行pytest时可以指定目录和文件。如果不指定,pytest会搜索当前目录及其子目录中以test_开头或以_test结尾的测试函数。【pytest搜索测试文件和测试用例的过程称为测试搜索】

只要你遵守pytest的命名规则,pytest就能自动搜索所有待执行的测试用例。

  • 测试文件应当命名为 test_.py 或者 _test.py
  • 测试函数、测试类方法应当命名为 test_
  • 测试类应当命名为 Test

 测试文件和测试函数最好以test_开头,但如果先前编写的测试用例遵循的是其他命名规则,也可以修改默认的测试搜索规则。【配置章节介绍】

1.2 测试函数返回结果类型

  • PASSED(.):测试通过
  • FAILED(F):测试失败(也有可能是XPASS状态与strict选项冲突造成的失败)
  • SKIPPD(s):测试未被执行。指定测试跳过执行,可以将测试标记为@pytest.mark.skip(),或者使用 @pytest.mark.skipif() 指定跳过测试的条件
  • xfail(x):预期测试失败,并且确实失败,使用@pytest.mark.xfail()指定你认为会失败的测试用例
  • XPASS(X):预期测试失败,但实际上运行通过,不符合预期
  • ERROR(R):测试用例之外的代码触发了异常,可能由fixture引起,也可能由hook函数引起

 1.3 使用命令行选项

  • –collect-only 选项

可以展示在给定的配置下哪些测试用例会被运行,让你非常方便地在测试运行之前,检查选中的测试用例是否符合预期。

  • -k 选项

允许你使用表达式希望运行的测试用例,如果某测试名是唯一的,或者多个测试名的前缀或后缀相同,那么可以使用表达式来快速定位。

假设希望选中 test_asdict() 和 test_defaults(),那么可以使用 –collect-only 验证筛选情况:

pytest -k "asdict or defaults" --collect-only
  • -m 选项

标记(marker)用于标记测试并分组,以便快速选中并运行。以 test_replace() 和 test_member_access() 为例,它们甚至都不在同一个文件里,如果你希望同时选中它们,那么可以预先做好标记。

import pytest

......
@pytest.mark.run_these_please
def test_member_access():


@pytest.mark.run_these_please
def test_replace():
......

这里我们使用 pytest -m run_these_please 命令就可以同时运行 test_replace() 和 test_member_access()

使用 -m 选项可以用表达式指定多个标记名。

(1)-m “mark1 and mark2” 可以同时选中带有这两个标记的所有测试用例

(2)-m “mark1 and not mark2” 则会选中带有 mark1 的测试用例

(3)-m “mark1 or mark2” 则选中带有 mark1 或者 mark2 的所有测试用例

  • -x, –exitfirst 选项 

正常情况下,pytest会运行每一个搜索到的测试用例,如果某个测试函数被断言失败,或者触发了外部异常,则该测试用例的运行就会到此为止,pytest将其标记为失败后会继续运行下一个测试用例。

通常,这就是我们期望的运行模式。但是在debug时,我们会希望遇到失败即立即停止整个会话,这是 -x 选项就派上用场了。

  • –maxfail=num

-x 选项的特点是,一旦遇到测试失败,就会全局停止。假设你允许pytest失败几次后再停止,则可以使用–maxfail选项,明确指定可以失败几次。

  • -s 与 –capture=method

-s 选项允许终端在测试运行时输出某些结果,包括任何符合标准的输出流信息。-s 等价于 –capture=no。

编写用例时,习惯添加几个print(),以便观察某时刻测试执行到了哪个阶段。

信息捕获方法还有 –capture==fd 和 –capture=sys,使用 –capture=sys 时,sys.stdout / stderr 将被输出至内存;使用 –capture=fd 时,若文件描述符为1或2,则会被输出至临时文件中

  • -l, –showlocals 选项

使用 -l 选项,失败测试用例由于被堆栈追踪,所以局部变量及其值都会显示出来。

使用 -l / –showlocals 选项,在测试失败时会打印出局部变量名和它们的值,这样可以规避一些不必要的print语句

  • –lf, –last-failed 选项

当一个或多个测试失败时,我们常常希望能够定位到最后一个失败的测试用例重新运行。只重新运行上次运行失败的用例(或如果没有失败的话会全部跑)

  • –ff, –failed-first 选项

运行所有测试,但首先运行上次运行失败的测试(这可能会重新测试,从而导致重复的fixture setup/teardown)

  • -v, –verbose 选项

使输出的信息会更详细

  • -q, –quiet 选项

简化输出信息

  • –tb=style 选项

决定捕捉到失败时输出信息的显示方式。某个测试用例失败后,pytest会列举出失败信息,包括失败出现在哪一行、是什么失败、怎么失败的,此过程我们称之为“信息回溯”。

大多数情况下,信息回溯是必要的,他对找到问题很有帮助,但有时也会对多余的信息感到厌烦。

(1)–tb=short:仅输出 assert 的一行以及系统判定内容(不显示上下文)

(2)–tb=line:模式仅使用一行输出显示所有的错误信息

(3)–tb=no:直接屏蔽全部回溯信息

(4)–tb=long:输出最为详尽的回溯信息

(5)–tb=auto:默认值,如果有多个测试用例失败,仅打印第一个和最后一个用例的回溯信息(格式同long)

(6)–tb=native:只输出Python标准库的回溯信息,不显示额外信息

  • –duration=N 选项

加快测试节奏,不关心测试时如何运行的,只统计测试过程中哪几个阶段是最慢的(包括每个测试用例的call、setup、teardown过程)。它会显示最慢的N个阶段,耗时越长越靠前。

————————

1 Pytest的运行

1.1 test search

You can specify directories and files when running pytest. If not specified, pytest searches the current directory and its subdirectories to test_ Start with or_ Test function at the end of test. [the process of pytest searching test files and test cases is called test search]

As long as you follow the naming rules of pytest, pytest can automatically search all test cases to be executed.

  • 测试文件应当命名为 test_.py 或者 _test.py
  • The test function, test class and method should be named test_< something>
  • 测试类应当命名为 Test

Test files and test functions are best represented by test_ At the beginning, but if the previously written test cases follow other naming rules, you can also modify the default test search rules. [introduction to configuration section]

1.2 test function return result type

  • PASSED(.): Test passed
  • Failed (f): test failed (it may also be caused by the conflict between xpass state and strict option)
  • Skippd (s): test not executed. Specify that the test skip execution, and you can mark the test as @ pytest mark. Skip (), or use @ pytest mark. Skipif() specifies the condition for skipping the test
  • Xfail (x): if the expected test fails and does fail, use @ pytest mark. Xfail () specifies the test cases that you think will fail
  • Xpass (x): the expected test failed, but it actually passed and did not meet the expectation
  • Error (R): code outside the test case triggers an exception, which may be caused by fixture or hook function

1.3 using command line options

  • –collect-only 选项

It can show which test cases will be run under a given configuration, so that you can easily check whether the selected test cases meet the expectations before the test runs.

  • -K option

It allows you to use expressions to test cases you want to run. If a test name is unique or multiple test names have the same prefix or suffix, you can use expressions to quickly locate.

Suppose you want to check test_ Asdict() and test_ Defaults(), you can use — collect only to verify the filtering:

pytest -k "asdict or defaults" --collect-only
  • -M option

Marker is used to mark tests and group them so that they can be quickly selected and run. With test_ Replace() and test_ member_ For example, access (), they are not even in the same file. If you want to select them at the same time, you can mark them in advance.

import pytest

......
@pytest.mark.run_these_please
def test_member_access():


@pytest.mark.run_these_please
def test_replace():
......

Here we use pytest – m run_ these_ The please command can run test at the same time_ Replace() and test_ member_ access()

Use the – M option to specify multiple tagnames with an expression.

(1) – M “mark1 and Mark2” all test cases with these two marks can be selected at the same time

(2) – M “mark1 and not Mark2” will select the test case with mark1

(3) – M “mark1 or Mark2” to select all test cases with mark1 or Mark2

  • -x, –exitfirst 选项 

Under normal circumstances, pytest will run every searched test case. If a test function is asserted to fail or an external exception is triggered, the operation of the test case will stop. Pytest will continue to run the next test case after marking it as failed.

Usually, this is the operation mode we expect. However, when debugging, we will want to stop the whole session immediately in case of failure, which is the use of the – x option.

  • –maxfail=num

-The feature of the X option is that once the test fails, it will stop globally. Assuming you allow pytest to fail several times and then stop, you can use the — maxfail option to specify how many times you can fail.

  • -s 与 –capture=method

-The s option allows the terminal to output some results during the test run, including any output stream information that meets the standard- S is equivalent to — capture = No.

When writing use cases, it is customary to add several prints () to observe the stage of test execution at a certain time.

Information capture methods include — capture = = FD and — capture = sys. When — capture = sys is used, sys Stdout / stderr will be output to memory; When — capture = FD is used, if the file descriptor is 1 or 2, it will be output to a temporary file

  • -l, –showlocals 选项

Using the – L option, the failed test case will be traced by the stack, so the local variables and their values will be displayed.

Using the – L / — showlocals option, local variable names and their values will be printed out when the test fails, which can avoid some unnecessary print statements

  • –lf, –last-failed 选项

When one or more tests fail, we often want to locate the last failed test case and rerun it. Rerun only the use cases that failed last time (or all if there is no failure)

  • –ff, –failed-first 选项

Run all tests, but first run the tests that failed the last time (this may retest, resulting in duplicate fixture setup / teardown)

  • -v, –verbose 选项

Make the output information more detailed

  • -q. — quiet option

Simplified output information

  • –tb=style 选项

Decide how to display the output information when capturing failure. After a test case fails, pytest will list the failure information, including the line in which the failure occurs, what the failure is and how it fails. This process is called “information backtracking”.

In most cases, information backtracking is necessary. It is very helpful to find problems, but sometimes he is tired of redundant information.

(1) — TB = short: only one line of assert and the content determined by the system are output (no context is displayed)

(2) — TB = line: the mode uses only one line of output to display all error messages

(3) — TB = No: directly shield all backtracking information

(4) — TB = long: output the most detailed backtracking information

(5) — TB = Auto: default value. If multiple test cases fail, only the backtracking information of the first and last case will be printed (the format is the same as long)

(6) — TB = native: only the backtracking information of Python standard library is output, and no additional information is displayed

  • –duration=N 选项

Speed up the test rhythm, do not care about how the test runs, and only count which stages of the test process are the slowest (including the call, setup and teardown processes of each test case). It will display the slowest n stages, and the longer it takes, the more forward it is.