Slides

# 20. Unit Testing

## 20.1. Introduction: Unit Testing

Testing plays an important role in the development of software. To this point, most of the testing you have done has probably involved
running your program and fixing errors as you notice them. In this chapter, you will learn about a more methodical approach to testing.
Along the way, you will pick up some design techniques. So, let’s get started!

## 20.2. Checking Assumptions With `assert`

Many functions work correctly only for certain parameter values, and produce invalid results (or crash) if given others.
Consider the following function, which computes the sum of the numbers in a range specified by its parameters:

```python
def sumnums(lo, hi):
    """returns the sum of the numbers in the range [lo..hi]"""

sum = 0
    for i in range(lo, hi+1):
        sum += i
    return sum

print(sumnums(1, 3))
print(sumnums(3, 1))
```

Notice that the first call to sumnums produces the correct answer (6), while the second call produces an incorrect answer.
`sumnums` works correctly only if `lo` has a value that is less than, or equal to, `hi`.

This function trusts the calling code to provide parameter values that are valid. If the caller provides a second
parameter that is lower than the first parameter, the function does not produce a correct result. That’s not
the fault of the function; the function isn’t designed to work correctly if `lo` > `hi`.

To make it clear that the function is designed to work correctly only if `lo` <= `hi`, it’s a good idea to state
that as a precondition in the function documentation, like this:

```python
def sumnums(lo, hi):
    """returns the sum of the numbers in the range [lo..hi]

Precondition: lo <= hi
    """
```

Precondition

A **precondition** specifies a condition that must be True if the function is to produce correct results.

A precondition places a constraint on the values of the parameters that the caller can pass and expect to receive a
valid result. Preconditions are boolean expressions – comparisons that you might write in an `if` statement.
We’ll have more to say about preconditions later in the chapter.

Code that calls a function is responsible for passing parameters that satisfy the function’s preconditions. If the
calling code passes values that violate the function’s preconditions, the function isn’t expected to work correctly.
That’s not the function’s fault: it’s the caller’s fault for passing parameters to the function that the function is not
designed to handle correctly. However, it might be a good idea if we designed the function to check for invalid values,
and when it detects them, somehow report that it was called incorrectly.

### 20.2.1. Designing Defensive Functions

A defensive function is a function that checks its parameters to see if they are valid, and responds in an appropriate
way if they are invalid. That raises the question: what should a defensive function do if it receives invalid values?
Should it print an error? Silently ignore the problem and return a default value? Return a special value that indicates
an error? Exit the program? There are several options.

As an example, here is one way we could make `sumnums` defensive:

```python
def sumnums(lo, hi):
    """returns the sum of the numbers in the range [lo..hi]

Precondition: lo <= hi
    """

if lo > hi:
        print('Alert: Invalid parameters to sumnums.')
        return -1

sum = 0
    for i in range(lo, hi+1):
        sum += i
    return sum

print(sumnums(1, 3))
print(sumnums(3, 1))
```

In this version, the function checks to see if the preconditions are violated, and if so, it complains by printing a
message and returns the value -1 to the caller.

Defensive Programming

The strategy of designing functions that check their parameters embodies a principle of software design called
**defensive programming**, in which software checks for invalid inputs and responds in an appropriate way.
Defensive programming is especially important for mission critical systems, but it can be a helpful
strategy in regular software projects, as we’ll soon see.

This is an improvement over the original function, because now, if the function is called with invalid data,
the user will see a message that something is wrong. However, the `if` statement adds three lines of code to the
function. That may not seem like much, but it clutters the code and, in a typical program with several functions,
those if statements will start to feel like undesirable baggage. There’s a better way.

### 20.2.2. The `assert` Statement

Python provides a statement called the `assert` statement that can be used to check function
preconditions. An `assert` statement checks the value of a boolean expression. If the expression is `True`, the
assert statement allows the program to proceed normally. But if the expression is `False`, the assert
statement signals an error and stops the program.

Here’s an example of an assert statement:

```python
x = 1 + 1
assert x == 2
print(x)
```

To see it in action, run the example above. You’ll see the value `2` displayed. The boolean condition `x == 2`
was True, and the assert statement allowed execution to continue.

Try changing the assert statement above as follows:

```python
assert x == 3
```

Run this version of the code, and you’ll see an AssertionError appear. That occurred because the value of the boolean
expression was `False`.

Let’s modify our sumnums function to use an assert statement to check the precondition:

```python
def sumnums(lo, hi):
    """returns the sum of the numbers in the range [lo..hi]

Precondition: lo <= hi
    """

assert lo <= hi

sum = 0
    for i in range(lo, hi+1):
        sum += i
    return sum

print(sumnums(1, 3))
print(sumnums(3, 1))
```

In this version of `sumnums`, we’ve replaced the `if` statement with an `assert` statement. Notice that the boolean
condition of the `assert` statement is the precondition, `lo <= hi`. When the function is called, if the
condition is true, the function completes normally and returns its result. If the condition is false, the program stops
with an AssertionError. So, the first call to `sumnums(1, 3)` succeeds and the result, 6, appears. The second call to
`sumnums(3, 1)` causes the assert to fail and an error appears.

Notice how much more streamlined this version of the function is than the version with the three-line `if` statement.
Here, we’ve added just one line of code to the original version. Using assertions is a relatively low-effort way to
create defensive functions.

Writing assert statements to check preconditions

Writing assert statements to check preconditions is easy. They go at the **beginning** of the function. When you
write an assert statement to check a precondition, if the function comment already contains a precondition, you
often can simply take the precondition and put it directly into the `assert` statement (you might have to tweak it
to make it syntactically legal). If there is no precondition in the function comment, think about how you would
write an if statement to check that the values in the parameters are **correct**, and then put that condition after
the word `assert`.

### 20.2.3. More on `assert` and Preconditions

Let’s discuss for a moment the question of what a defensive function should do when it receives invalid values in its
parameters. By using an `assert` statement to check preconditions, we’ve designed the function to terminate the
program if it is given bad data. Is this the right thing to do? If the program ends abruptly due to an assertion
failure, the user will lose whatever work is in progress. That seems undesirable, to put it mildly.

Although a full discussion of defensive programming and assertions is outside the scope of an introductory programming
textbook, think about this: an assertion error **indicates a bug in the program**. More specifically, the bug is a
logic error that resulted in calling a function with inappropriate parameter values. If a computation is in progress and
a logic error occurs, any results that computation might produce will be faulty. Logic errors often go silently
undetected by users, because they aren’t aware that the output is incorrect. It is better for a user to lose work
than for a logic error to go undetected and produce an invalid result that might be unwittingly used. Therefore,
using assert statements to check function preconditions is entirely appropriate.

Not only will adding assertions to your functions to check preconditions help expose logic errors in your program, it
does so in a way that helps you track them down and fix them quickly. When you don’t use assertions, a function that is
called with incorrect parameters may produce erroneous results that aren’t detected until much later in the program, and
debugging the problem can be difficult to trace back to the source. When you use assertions to check preconditions, a
function that detects a problem will stop immediately, helping you pinpoint the problem much faster. This behavior is
called the **fail fast principle**. You want your program to fail as quickly after a logic error is detected as possible
to help streamline the diagnostic work.

Debugging Assertion Failures

When an `assert` statement that you have written to check a function precondition signals an error at runtime, your first
thought will probably be: “what went wrong? where’s the problem?” It will help if you remember that an assert that
checks a function precondition is there to **catch bugs in code that calls the function**. After all, you put it on the
first line of the function. So, it’s not an indication of a problem in the function: instead, the calling code has a
problem. So, look to see what code called the function. When you’re running your program in a regular Python interpreter,
the full error message will show the exact sequence of calls that triggered the error, and you can tell exactly
which line of code is responsible for providing the incorrect values.

Functions that Cannot Fail

An alternative approach to handling bad input for sumnums would be to design the function so that it works correctly
regardless of whether the low end of the range is specified first or second. For example, we could design it so that
both of the following calls produce correct results:

```python
print(sumnums(1, 3))
print(sumnums(3, 1))
```

It’s not hard to do; I bet you could figure out how to tweak the function to work correctly for both of these
calls without much effort. However, a more important question is: should we do that?

This question doesn’t necessarily have a simple answer, but briefly, there are a couple of considerations that argue
against it. First, refining the function to work correctly for both of these calls will result in a function that is
slightly more complex, and therefore, perhaps more likely to contain bugs. Also, testing will be more involved;
there are more cases to consider.

**Check your understanding**

An `assert` statement displays output if the condition is True.

* True
* Incorrect. Output is displayed only if the condition is False.
* False
* Correct. Output is displayed only if the condition is False.

Consider the following function. Which assert should be added to check its
precondition?

```python
def getfirst(msg):
    """returns first character of msg

Precondition: len(msg) > 0
    """

return msg[0]
```

* assert len(msg) <= 0
* Incorrect. Use the precondition as the condition for the assert.
* assert len(msg) > 0
* Correct. Use the precondition as the condition for the assert.
* assert msg[0]
* Incorrect. Use the precondition as the condition for the assert.
* none of these
* Incorrect. Use the precondition as the condition for the assert.

## 20.3. Testing Functions

Melinda is writing a program that does some mathematical calculations. At the moment, she is working on adding some
functionality to her program that requires rounding numbers to the nearest integer. She would normally use the built-in
Python function `round` to do the job, but her program has a special requirement that numbers should be rounded up if
the fractional portion is .6 or greater, instead of the usual .5 or greater. So, Melinda decides to write a function that
rounds up numbers according to this requirement.

She defines a function `round6` to do the job:

```python
def round6(num):
    """returns num rounded to nearest int if fractional part is >= .6"""

return int(num + .6)
```

This function uses a valid approach to rounding, but is not quite correct (Melinda doesn’t realize it yet — can you
spot the bug?).

Now she needs to test the new code. There are two basic approaches Melinda could take to do her testing:

1. Put the function into the program, modify the program to call the function at the appropriate point, then run the program.
2. Test the function by itself, somehow.

Which do you think will be more efficient?

Melinda’s program does complex mathematical calculations, and asks the user to enter 5 separate pieces of input before
performing the calculations. If she goes with option 1, each time she runs the program to test the function,
she must enter all 5 pieces of input. As you can imagine, that process is cumbersome and will not be very efficient.
Also, if the program output is incorrect, it may be difficult to determine whether the fault is in the new function,
or elsewhere in the program.

Melinda decides to write a separate, short program to help her test her new function. The test program is very simple
— it contains only her new function and a bit of code to get some input, pass it to the function, and display the
result. Here’s what she writes:

```python
def round6(num):
    """returns num rounded to nearest int if fractional part is >= .6"""

return int(num + .6)

# ----- test program -------

x = float(input('Enter a number:'))
result = round6(x)
print('Result: ', result)
```

Before running the program, she jots down some test cases to help her in her testing:

```python
              Input    Expected Output
              -------- ---------------
Test Case 1:       3.5               3
Test Case 2:       3.6               4
Test Case 3:       3.7               4
```

Try running the program with the input values above. Notice that the output isn’t quite right.
Can you figure out how to correct the bug?

After analyzing her logic, Melinda corrects the bug by changing the return statement in the function as follows:

```python
return int(num + .4)
```

She runs the test program again to verify that the function is working correctly. Then, she copies the
`round6` function into her main program, confident that her rounding logic is correct.

The program Melinda wrote to help her test her round6 function is an example of a unit test.

Unit Test

A **unit test** is code that tests a function to determine if it works properly.

A unit test program like this one can dramatically reduce the effort it takes to test a new function, and can reduce the
overall effort involved in adding functionality to a program. The savings tradeoff depends on the amount of effort
required to write the test program, compared to the amount of effort required to test the function in the context of the
main program for which the new function is being developed. Here, the function was relatively simple, and it probably
wouldn’t have taken Melinda too many iterations of testing the function in the context of the main program, with its
five pieces of input. In this scenario, Melinda may not have saved much effort. However, if the function were more
complex, writing a unit test would probably have helped reduce the overall effort. And, using some tricks I’ll show you
in the next sections, you can reduce the amount of effort required to write and run the unit test, making the case for
writing unit tests even more compelling.

### 20.3.1. Automated Unit Tests

The unit test program above is a manual unit test. A **manual unit test** gets input from the user, invokes the code
under test, providing the input supplied by the user, and displays the result. (In our example, `round6` is the code
under test.) Manual unit tests are helpful, but they can be improved in two ways:

1. We can embed the test input directly within the unit test code, so the person running the test doesn’t have to
   come up with the test input or take the time to enter it.
2. We can make the unit test report success or failure, instead of requiring the person running the test to
   look at the output and determine whether the function worked correctly.

We call a unit test that contains its own test input and produces a clear pass/fail indication an **automated
unit test**. Take a look at the following example:

```python
def round6(num):
    return int(num + .4)

# ---- automated unit test ----

result = round6(9.7)
if result == 10:
    print("Test 1: PASS")
else:
    print("Test 1: FAIL")

result = round6(8.5)
if result == 8:
    print("Test 2: PASS")
else:
    print("Test 2: FAIL")
```

This automated unit test invokes the `round6` function on predetermined test input, checks that the function produced the
expected result, and displays a pass / fail message. Run it to see the test PASS messages.

Try editing the round6 function above to introduce Melinda’s original bug, then run it again to see the failure message.
Notice the big advantage of an automated unit test: you can change the function being tested, run the unit test,
and immediately see the test results for a whole series of tests. No hand-entry of test data, and no interpretation of the
results. Clearly, once you have the test written, you can dramatically speed up your edit-test-debug cycle. The downside,
of course, is that the unit test program itself takes more time to develop.

### 20.3.2. Automated Unit Tests with `assert`

To help reduce the amount of effort required to develop an automated unit test, let’s bring the `assert` statement into
play. We can replace each `if` statement in the program above with an assert, as in the program below:

```python
def round6(num: float) -> int:
    return int(num + .4)

# ---- automated unit test ----

result = round6(9.7)
assert result == 10

result = round6(8.5)
assert result == 8

print("All tests passed!")
```

Try running the program above to see the success message. Then, try altering the `round6` function to reintroduce the original
bug, and see how the assertion failure pinpoints that the second test failed.

We can streamline this program even further by eliminating the `result` variable:

```python
assert round6(9.7) == 10
assert round6(8.5) == 8

print("All tests passed!")
```

This is Really Nice. We have a short test program that contains its own test input and displays an automated pass or
fail indication. Writing this program takes very little effort. We have the benefits of an automated test without
having to write much code. Unit test programs are essentially “throw-away” programs that are used only during
development, and it’s important that they can be developed quickly and easily.

### 20.3.3. Unit Tests can have bugs

Unit tests, like the functions they test, can have bugs. So, when you run a unit test and it fails with an assert error,
one of the first questions you need to ask yourself is: “Is the unit test correct?” If the unit test is incorrect, then
you need to correct it, rather than spending time trying to find the bug in the function that the unit test is testing.

For example, consider the following assert:

```python
assert round6(9.2) == 10
```

This unit test is incorrect, because `round6` should produce the value 9, not 10, when given the parameter `9.2`.

**Check your understanding**

Rewrite the following 3 lines of code with a single assert:

```python
result = engage_thruster(22)
if result != 'OK':
    print("Test 2: FAIL")
```

* assert result != 'OK'
* Incorrect. The assert should call the engage\_thruster function. Try again.
* assert engage\_thruster(22) == result
* Incorrect. The assert should not assume the presence of a variable named result.
* assert engage\_thruster(22) != 'OK'
* Incorrect. This assert calls the engage\_thruster function with the correct parameter, but interprets the result differently from the if statement provided.
* assert engage\_thruster(22) == 'OK'
* Correct! This assert checks that the result of invoking engage\_thruster(22) is the value 'OK'.

Consider the following function which is supposed to return the first character of its argument:

```python
def get_first(msg):
    return msg[1]
```

Now, consider this unit test:

```python
assert get_first('Bells') == 'B'
```

This assertion fails. Is the unit test in error, or the function it is testing?

* Unit test
* Incorrect. The assertion correctly checks that get\_first should return the first character in 'Bells', 'B'.
* Tested function
* Correct. The function code returns the second character of the parameter, not the first.
* Both are in error
* Incorrect. The assertion correctly checks that get\_first should return the first character in 'Bells', 'B'.
* Both are correct
* Incorrect. The function code returns the second character of the parameter, not the first.

## 20.4. Designing Testable Functions

Now that you know how to write unit tests using the `assert` statement, it’s important for you
to understand how to write testable functions. Not all functions can be tested.

Consider the following function:

```python
def add(x, y):
    """Adds two numbers and displays the sum"""
    print(x + y)
```

How would you write an assert statement to check that this function works? Think about it a moment.
Would this work?

```python
assert add(2, 3) == 5
```

Answer: no. An assert statement cannot verify that what a function displays on the screen is correct.
It can only check that the contents of variables are correct. This function is not testable.

A **testable function** is a function that produces a result that can be checked by an assert
statement. Generally, it does so in one of three ways:

1. It returns its result
2. It stores its result in a global variable
3. It modifies the state of an object passed as a parameter

Functions that display their output using print are not testable functions.

**Check your understanding**

Is this a testable function?

```python
sum = 0
def add(x, y):
    global sum
    sum = x + y
```

* Yes.
* Correct. In a unit test, an assert statement could check the value of sum after a call to add.
* No.
* Incorrect. The function stores its result in a global variable, which can be checked by an assert statement.

### 20.4.1. Design by Contract

In addition to producing a result that can be checked by an assert statement, a testable function must have a clear
specification. In order to write unit tests for a function, you must have a precise understanding of what the function
should do.

A function specification describes what value the function produces, given its parameter values, and is
generally expressed in the form of a docstring. For example, consider the `sumnums` function given earlier in this
chapter:

```python
def sumnums(lo, hi):
    """returns the sum of the numbers in the range [lo..hi]

Precondition: lo <= hi
    """
    ...
```

The docstring is this function’s specification. Given this specification, you might write a unit test
that contains the following assert:

```python
assert sumnums(1, 3) == 6
```

An alternate way to write the docstring is as follows:

```python
def sumnums(lo, hi):
    """computes the sum of a range of numbers

Precondition: lo <= hi
    Postcondition: returns the sum of the numbers in the range [lo..hi]
    """
    ...
```

This docstring contains three elements: a brief description; a precondition; and a postcondition. We’ve
discussed the concept of a precondition earlier in this chapter. The postcondition is new.

Postcondition

A **postcondition** states the work the function completed by the function if the precondition is satisfied.

Functions that include a precondition and a postcondition in their docstring embody a software engineering idea called
**design by contract**. The idea is that a function specification forms a contract between the function and the code
calling the function. If the code calling the function passes parameters that satisfy the function’s precondition, then
the function should be expected to produce what it says it will produce. If the parameters do not satisfy the function’s
precondition, then the function does not have to produce a valid result. In the design by contract approach, a testable
function is one where the function’s postcondition can be verified by an assert statement.

In this example, you can think of the function’s docstring as promising to calling code: “If you give me two parameters,
lo and hi, such that lo is less than or equal to hi, I promise to return the sum of the numbers in the range lo..hi,
inclusive.”

To write a precondition, think about the parameter values that the function is designed to handle, and write a boolean
expression that expresses what parameter values are valid. For example, consider a function that computes the average weight,
given a total weight and a number of items:

```python
def compute_average(total_weight: float, num_items: float) -> float:
    return total / num_items
```

This function will work if num\_items is greater than zero, but will fail if num\_items is zero. So, an appropriate precondition
would be `num_items > 0`. A complete docstring would look like this:

```python
def compute_average(total_weight: float, num_items: float) -> float:
    """computes the average weight, given `total_weight` of items and `num_items`

Precondition: num_items > 0
    Postcondition: returns average item weight
    """
```

Sometimes, your precondition will be expressed more loosely, using English. Consider this function which extracts the
first word from a string containing text:

```python
def get_first_word(text: str) -> str:
    """extracts the first word from `text`"""

space_loc = text.find(' ')
    return text[0:space_loc]
```

This function will produce nonsense if the string doesn’t contain a space. So, an appropriate precondition might be
“text contains 2 or more words separated by spaces”. The docstring might be:

```python
def get_first_word(text: str) -> str:
    """extracts the first word from `text`

Precondition: `text` contains 2 or more words separated by spaces
    Postcondition: returns the first word in `text`
    """
```

Following the design by contract idea and writing function specifications that include preconditions and postconditions
is an excellent way to design testable functions, because, as we’ll see in the next section, it makes it possible to
reason precisely about what the function should do when given various parameter values. Even if you don’t use
precondition and postcondition terminology in your docstrings, it helps to *think* in those terms.

**Check your understanding**

Consider the following function. What would an appropriate precondition be?

```python
def getfirst(msg):
    """returns first character of msg"""

return msg[0]
```

* len(msg) <= 0
* Incorrect. The function would crash if this condition were true.
* len(msg) > 0
* Correct. The function cannot return the first character of an empty msg.
* msg == ""
* Incorrect. The function would crash if this condition were true.
* none of these
* Incorrect. Try again.

## 20.5. Writing Unit Tests

Once you have designed a testable function, with a clear docstring specification, writing unit tests
is not difficult. In this section, you’ll learn how to do just that.

Let’s start with our `sumnums` function:

```python
def sumnums(lo, hi):
    """computes the sum of a range of numbers

Precondition: lo <= hi
    Postcondition: returns the sum of the numbers in the range [lo..hi]
    """

sum = 0
    for i in range(lo, hi+1):
        sum += i
    return sum
```

As we’ve seen, to write a unit test, you devise test cases for the function, and then write assert statements that call
the function and check that the function produced the expected results. The following assert statements would be
appropriate for a unit test for `sumnums`:

```python
assert sumnums(1, 3) == 6
assert sumnums(1, 1) == 1
```

But what about the following?

```python
assert sumnums(3, 1) == 0
```

Note that `sumnums` produces the value `0` for cases where the `lo` values exceeds the `hi` value, as is the
case in this assert. So, like the first two asserts above, this assert would pass. However, it is not an
appropriate assertion, because the specification says nothing about what the function produces if `lo` is greater than
`hi`.

The unit test should be written such that it passes even if the function implementation is altered in a way that
causes some other value than 0 to be returned if `lo` exceeds `hi`. For example, we might want to redesign the
function to be more efficient — for example, use Gauss’s formula for summing numbers, as in the following:

```python
def sumnums(lo, hi):
    """computes the sum of a range of numbers

Precondition: lo <= hi
    Postcondition: returns the sum of the numbers in the range [lo..hi]
    """

return (hi * (hi + 1) / 2) - (lo * (lo - 1) / 2)
```

This version will produce correct results if the precondition is satisfied. Like the original function, it produces
incorrect results if the precondition is violated — but unlike the original function, the values produced if the
precondition is violated are not necessarily 0.

### 20.5.1. Specification-Based Testing

A key idea to remember when writing a unit test is that your test must always respect the function’s preconditions. The
docstring states what the function should do, with the assumption that parameter values meet the preconditions. It does
not state what the function should do if the parameter values violate the preconditions.

Writing an assert that violates the functions preconditions is not a good idea, because to determine what the function
will produce for that case, you must look into the implementation of the function and analyze its behavior. That is
called **implementation-based testing**, and it leads to brittle tests that are likely to fail if you rework the
function implementation. When you write tests are based only on the function specification, without looking at the
implementation, you are doing specification-based testing.

Specification-Based Tests

Specification-based tests are tests that are designed based only on the information in the function
specification, without considering any of the details in the function implementation.

Specification-based tests are preferred over implementation-based tests, because they are more resilient. They will continue
to pass even if you rework the function implementation.

**Check your understanding**

Consider the following function. Indicate which of the asserts would be
appropriate for a unit test.

```python
def repeat(s: str, num: int) -> str:
    """duplicates a string

Precondition: `num` >= 0
    Postcondition: Returns a string containing `num` copies of `s`
    """
    if num >= 0:
        return s * num
    else:
        return ''
```

* assert repeat('\*', 0) == ''
* Correct. The specification indicates that the function should produce an empty string.
* assert repeat('\*', -1) == ''
* Incorrect. The parameter -1 violates the precondition, and such a test is inappropriate, even though the function would return an empty string in this case.
* assert repeat('-', 5) == '-----'
* Correct. The parameters satisfy the precondition and the value '-----' is appropriate given the postcondition.
* assert repeat('\*', 5) == '\*\*\*'
* Incorrect. The string '\*\*\*' does not conform to the postcondition.

Write assert statements below to test a function
with the following specification. Your asserts should
check that the function produces an appropriate value
for each of the three postcondition cases.

```python
def grade(score):
    """Determines letter grade given a numeric score

Precondition: 0 <= `score` <= 100
    Postcondition: Returns 'A' if 90 <= `score` <= 100,
      'B' if 80 <= `score` < 90, 'F' if 0 <= `score` < 80
    """
```

Note: Line numbers in any assert error messages that appear while
you are developing and testing your answer will not be accurate.

```python
# Write assert statements to test grade()

=====
from unittest.gui import TestCaseGui

class myTests(TestCaseGui):

def testOne(self):
        code = self.getEditorText().replace(' ','').replace('"', '').replace("'", '')
        self.assertTrue(testA and '==A' in code, "Assert tested 90..100")
        self.assertTrue(testB and '==B' in code, "Assert tested 80..90")
        self.assertTrue(testF and '==F' in code, "Assert tested 0..80")
        self.assertFalse(illegal, "No asserts violated preconditions")

myTests().main()
```

The following asserts are just some of several that could have been used.

```python
assert grade(92) == 'A'
assert grade(85) == 'B'
assert grade(69) == 'F'
```

## 20.6. Test-First Development

The idea of unit tests has been around a long time, and most people agree that writing unit tests is a good idea.
However, when deadlines loom and time is at a premium, the unit tests often don’t get written. That’s a problem, because
studies have shown that projects with good unit tests often are more robust, with fewer bugs, than projects that don’t
have good unit tests.

In a traditional development process, when a programmer needs to create a new function, he writes the function, and
then, if he has time, writes a unit test for it. If he doesn’t have time, he doesn’t write the unit test: he tests the
function in the context of the program being developed. One day, someone decided that it might be a good idea
to reverse the order: write the unit test *first*, and then write the function. That led to the idea of
Test-First Development.

Test-First Development

Test-First Development is an approach to writing software that involves writing a unit test for a function before
writing the function.

In this section, we’ll explore the idea of test-first development to see how it can help.

A programmer using Test-First Development writes a new function using the following steps:

1. First, create the function interface and docstring.
2. Next, create a unit test for the function.
3. Run the unit test. It should fail.
4. Write the body of the function.
5. Run the unit test. If it fails, debug the function, and run the test again. Repeat until the test passes.

As an example, suppose that we’re going to write our `sumnums` function using the Test-First methodology. We begin
by creating the interface and docstring:

```python
def sumnums(lo, hi):
    """computes the sum of a range of numbers

Precondition: lo <= hi
    Postcondition: returns the sum of the numbers in the range [lo..hi]
    """
```

Next, we write the unit test for it:

```python
def sumnums(lo, hi):
    """computes the sum of a range of numbers

Precondition: lo <= hi
    Postcondition: returns the sum of the numbers in the range [lo..hi]
    """

assert sumnums(1, 3) == 6
assert sumnums(1, 1) == 1
print("All tests passed!")
```

We run the unit test and it fails.

Next, we implement the body of `sumnums`:

```python
def sumnums(lo, hi):
    """computes the sum of a range of numbers

Precondition: lo <= hi
    Postcondition: returns the sum of the numbers in the range [lo..hi]
    """
    sum = 0
    for i in range(lo, hi):
        sum += i
    return sum

assert sumnums(1, 3) == 6
assert sumnums(1, 1) == 1
print("All tests passed!")
```

Now, run the tests. The tests indicate an assertion error, which points to a bug in the function logic. Fix the bug,
and test again. (If you’re not sure what the bug is, try using **Show in CodeLens** and stepping through the code to help
you figure it out.)

The range function generates numbers in the range `lo` .. `hi - 1`. Our function should include `hi`.
Try adjusting the `hi` parameter for range to `hi + 1`.

Suppose we’re not creating a new function, but modifying an existing one. In Test-First Development,
before making the modification to the function, we write a test for the new functionality. Then,
we modify the function, and use the test to check that the modification worked.

### 20.6.1. Benefits of Test-First Development

There are several benefits to Test-First Development.

1. It ensures that unit tests are written. This tends to lead to higher-quality, robust code, with fewer
   bugs.
2. Writing the tests first helps the programmer to clarify the function specification. It’s not possible to
   write an assert for a function that has a vague function docstring. This process forces the
   programmer to write a clear docstring and to practice specification-based testing, because when the
   tests are written, there is no function implementation to reference.
3. When the programmer writes the function and is ready to test it, the test is all ready to go. There is no
   internal struggle about whether a unit test should be written or not. The programmer runs the test,
   and gets instant feedback about whether the function is working or not.
4. If the function fails to pass the test, the benefits of unit testing in helping the programmer
   to quickly diagnose and fix the problem are instantly available. The test-debug cycle is rapid.
5. When a programmer modifies an existing function for which unit tests already exist, perhaps to add some more functionality,
   the existing unit tests serve as a safety net. They check that the modifications made by the programmer don’t break
   any of the old functionality.
6. The overall development time tends to be reduced. Perhaps counter-intuitively, writing more code (the unit tests)
   actually speeds up the overall development process, because of the benefits imparted by unit testing.
7. Believe it or not, there are psychological benefits. As the programmer works on the project, creating
   little tests and then writing code that passes those tests, there is a sense of accomplishment and satisfaction that
   comes every time a new test passes. Instead of spending hours of frustration debugging a new function in the context
   of a complex program, with few visible results, the test-first progress leads to more visible and regular successes.

I hope you’ll try out Test-First Development on your next assignment and experience some of these benefits
for yourself!

**Check your understanding**

Test-First Development often involves writing more code than traditional development.

* True
* Correct. Test-First Development requires the developer to write unit tests. However, these tests come with significant benefits and can speed up overall development.
* False
* Incorrect. Test-First Development requires the developer to write unit tests. However, these tests come with significant benefits and can speed up overall development.

## 20.7. Testing with pytest

Writing automated unit tests is very helpful in reducing the effort needed to build software. However, the simple
approach described so far is inadequate to help programmers realize the full benefits of unit testing. In this section,
we introduce a unit test framework which addresses some practical issues that crop up when you try to apply
unit testing techniques in software development projects. Here are some of the issues with using plain assertion unit
tests:

* Simple assert-based tests don’t give very good diagnostic information when they fail. It would help to have better
  reporting. For example, when an assert fails, it would help us in diagnosing the error to see what value the function
  actually produced. An AssertionError doesn’t give us that information.
* We need a better way to organize our unit test code. So far, I’ve suggested creating separate programs to hold
  the functions under test together with their unit test code. But that isn’t practical for most projects. For example,
  functions often need to call other functions in the program in order to do their work. It’s not convenient to
  bring those auxiliary functions over into separate test programs.
* We need a way to keep the unit test around and use it even after the function is first developed
  to help us catch bugs that are inadvertently introduced into the function when we make modifications to it.

Unit testing frameworks help to address these issues, by improving error reporting, providing a structure
for programmers to organize their unit tests, and making it possible to leverage existing unit tests when
making enhancements to functions. **pytest** is one unit testing framework that provides these benefits.

For our purposes, the attractive thing about pytest is that writing unit tests with pytest feels a lot like writing unit
tests using plain assert statements. The only difference is that you put your assert statements into test functions.
Here’s an example of how it works:

```python
def round6(num: float) -> int:
    """This function has a bug in it"""
    return int(num + .6)

# ---- automated unit test ----

def test_round6():
    assert round6(9.7) == 10
    assert round6(8.5) == 8

=====

from unittest.gui import TestCaseGui

class pyTests(TestCaseGui):

def testOne(self):
        for item in globals():
            if item.startswith("test_"):
                try:
                    globals()[item]()
                    self.assertEqual(item + " passed", item + " passed", item + " passed")
                except Exception as e:
                    self.assertEqual(item + " failed", item + " passed", item + " failed: " + str(e))

pyTests().main()
```

This code example defines two functions: the function to be tested, `round6`, and a function named `test_round6`
that contains unit test code. When using the pytest approach, you write your unit test as a function whose name must
start with the prefix `test_`. Inside the function, you write normal assert statements to test the desired function.
Notice that you do not write a line to *call* the unit test function. Instead, when you launch pytest to run the unit
tests, pytest scans your script and executes only the functions with the prefix `test_`.

This ActiveCode environment simulates pytest by scanning for and executing functions with a `test_` prefix when you
click **Run**. Go ahead and try it - rename the test\_round6 function to `test_itworks` and try running the test again.

### 20.7.1. Organizing pytest Functions

The example above uses a single pytest function, with both asserts in the same pytest function. The
disadvantage of that approach is that the first failing assert prevents the rest of the asserts from
being tested.

If you want, you can write multiple pytest functions to test a single function. That way, when an
assert fails in one test function, the rest of the pytest functions can still run and report success or
failure.

You can name your pytest functions with names that indicate what they are testing. For
example, try changing the ActiveCode example above so that it defines two test functions: one named
`test_round6_rounds_up`, containing the first assert, and one named `test_round6_rounds_down`,
containing the second assert. Your code should look like this:

```python
def test_round6_rounds_up():
    assert round6(9.7) == 10

def test_round6_rounds_down():
    assert round6(8.5) == 8
```

If you use good pytest function names, when a pytest function has an assertion failure, you can
easily tell what the problem was.

### 20.7.2. Using pytest

To use pytest, you must first install it using the **pip** command. Open your computer’s command line window
(not the Python interpreter) and enter the following command to install:

* Windows:

```python
  pip install pytest
  ```
* Mac/Linux:

```python
  pip3 install pytest
  ```

After you have installed pytest, you run pytest unit tests from the command line window. To run pytest unit
tests, try copying the code from the ActiveCode example above and pasting it into a Python file named (ex.)
**myround.py**. Then, use the **pytest** command to run your tests by opening a command window,
navigating to the folder where you stored myround.py, and executing the following command:

```python
pytest myround.py
```

### 20.7.3. Understanding pytest Failure Reports

When you run the pytest command and an assertion fails, you see a report like this:

```python
=============================== FAILURES ================================
______________________________ test_round6 ______________________________
    def test_round6():
        assert round6(9.7) == 10
>       assert round6(8.5) == 8
E       assert 9 == 8
E        +  where 9 = round6(8.5)

myround.py:8: AssertionError
```

Let’s take a closer look at this report to understand what it’s telling you.

1. First, notice the line with the `>` symbol:

```python
   >       assert round6(8.5) == 8
   ```

The `>` symbol points to the line with the assertion that failed.
2. Next, notice the lines marked `E`:

```python
   E       assert 9 == 8
   E        +  where 9 = round6(8.5)
   ```

This indicates that the call to `round6(8.5)` returned the value 9, instead of the value 8.
   The value `9` is the actual result of the function. Knowing the value actually produced by
   the function can help you to troubleshoot the bug and correct the problem.

### 20.7.4. Integrated Unit Testing with pytest

When you use the pytest framework, you can include pytest test functions in your main program, along with the rest of
your program code. This allows you to keep your tests together with the functions that they test, and you can run either
your program (using the python command) or the unit tests (using the pytest command).

Take a look at this example that shows a function (`round6`, containing a bug), together with a
unit test function (`test_round6`), and a main program that uses `round6`:

```python
 1def round6(num: float) -> int:
 2    return int(num + .6)
 3
 4# ---- automated unit test ----
 5
 6def test_round6():
 7    assert round6(9.7) == 10
 8    assert round6(8.5) == 8
 9
10# ----- main program follows -----
11
12if __name__ == '__main__':
13    num = float(input('Enter a value:'))
14    print('The value rounded is: ' + str(round6(num)))
```

Notice how the main program is inside the `if` statement on line 12. This if condition is true when the program is run
using the **python** command, and allows the main program to execute. When the unit tests are executed using the
**pytest** command, any top-level code outside a function in the python file gets executed when **pytest** scans the
script looking for unit test functions with a `test_` prefix. The `if` condition is false in this scenario, and that
prevents the main program from executing when **pytest** is scanning the script. If that explanation didn’t make total
sense, just remember: in order for pytest to work correctly, any code that is part of the main program must be inside an
`if` statement like the one in this example, so that it doesn’t interfere with pytests’s unit testing process.

**Check your understanding**

Write a pytest unit test function named `test_grade` to test a function
with the following specification. Your asserts should
check that the function produces an appropriate value
for each of the three postcondition cases.

```python
def grade(score):
    """Determines letter grade given a numeric score

Precondition: 0 <= `score` <= 100
    Postcondition: Returns 'A' if 90 <= `score` <= 100,
      'B' if 80 <= `score` < 90, 'F' if 0 <= `score` < 80
    """
```

```python
# Write a pytest unit test function named ``test_grade``

=====
from unittest.gui import TestCaseGui

testA = False
testB = False
testF = False
illegal = False

def grade(score):
    global illegal, testA, testB, testF

if score > 100 or score < 0:
        illegal = True
        return ''
    elif score >= 90:
        testA = True
        return 'A'
    elif score >= 80:
        testB = True
        return 'B'
    else:
        testF = True
        return 'F'

class myTests(TestCaseGui):

def testOne(self):
        code = self.getEditorText().replace(' ','').replace('"', '').replace("'", '')
        self.assertEqual(test_grade(), None, 'test_grade function defined' )
        self.assertTrue(testA and '==A' in code, "Assert tested 90..100")
        self.assertTrue(testB and '==B' in code, "Assert tested 80..90")
        self.assertTrue(testF and '==F' in code, "Assert tested 0..80")

myTests().main()
```

The following is a suggested pytest unit test.

```python
def test_grade():
    assert grade(92) == 'A'
    assert grade(85) == 'B'
    assert grade(69) == 'F'
```

## 20.8. Glossary

assert statement
:   A statement that verifies that a boolean condition is true.

design by contract
:   An approach to designing functions that specifies function behavior using
    preconditions and postconditions.

postcondition
:   Part of a function specification that states the work the function completed by the function if the precondition is
    satisfied.

precondition
:   A boolean condition that the caller of a function must ensure is true
    in order for the function to produce correct results

specification-based tests
:   tests that are designed based only on the information in the function specification, without considering any of
    the details in the function implementation.

unit test
:   Code that tests a function to determine if it works properly

## 20.9. Exercises

> A function named `reverse` takes a string argument, reverses it, and returns the result:
>
> ```python
> def reverse(astring):
>     """Returns the reverse of `astring`"""
> ```
>
> Complete the assert statements in the ActiveCode editor below to create a unit test for `reverse`.
> Your asserts should check that `reverse` works properly for the following test cases (“Input”
> refers to the value passed as a parameter, and “Expected Output” is the result returned from
> `reverse`):
>
> ```python
>              Input    Expected Output
>              -------- ---------------
> Test Case 1: 'abc'    'cba'
> Test Case 2: 'b'      'b'
> Test Case 3: ''       ''
> ```
>
> ```python
> def test_reverse():
>     # Complete the assert statements below
>
>     assert ________________
>     assert ________________
>     assert ________________
> =====
>
> from unittest.gui import TestCaseGui
>
> testABC = False
> testB = False
> testEmpty = False
>
> def reverse(astring):
>     """Returns the reverse of `astring`"""
>     global testABC, testB, testEmpty
>
>     if astring == 'abc':
>         testABC = True
>     elif astring == 'b':
>         testB = True
>     elif astring == '':
>         testEmpty = True
>
>     l = list(astring)
>     l.reverse()
>     return ''.join(l)
>
>
> class myTests(TestCaseGui):
>
>     def testOne(self):
>         test_reverse()
>         code = self.getEditorText().replace(' ','').replace('"', '').replace("'", '')
>         self.assertTrue(testABC and '==cba' in code, "Assert tested 'abc'")
>         self.assertTrue(testB and '==b' in code, "Assert tested 'b'")
>         self.assertTrue(testEmpty and '==\n' in code, "Assert tested ''")
>
> myTests().main()
> ```
>
> If you’re not sure how to get started with this one, review [Writing Unit Tests](WritingUnitTests.html#writing-unit-tests).
>
> ```python
> assert reverse('abc') == 'cba'
> assert reverse('b') == 'b'
> assert reverse('') == ''
> ```
>
> A function named `stripletters` takes a string argument, removes all letters from it, and
> displays the result (see below). However, this function is not testable.
>
> Modify the function so that it can be tested with the assert statements that follow.
>
> ```python
> def stripletters(msg):
>     result = ''
>     for ch in msg:
>         if not ch.isalpha():
>             result += ch
>
>     print(result)
>
> assert stripletters('ab12c') == '12'
> assert stripletters('12') == '12'
>
> =====
>
> from unittest.gui import TestCaseGui
>
> class myTests(TestCaseGui):
>
>     def testOne(self):
>         self.assertEqual(stripletters('ab12c'), '12', "stripletters('ab12c') == '12'")
>         self.assertEqual(stripletters('12'), '12', "stripletters('12') == '12'")
>
> myTests().main()
> ```
>
> Review [Designing Testable Functions](DesigningTestableFunctions.html#designing-testable-functions).
>
> ```python
> def stripletters(msg):
>     result = ''
>     for ch in msg:
>         if not ch.isalpha():
>             result += ch
>
>     return result
> ```