Lab 3: Testing

Topics: Test Plans, Unit Tests

Group work: Group work is allowed for the labs, but each person must do their own coding and each person must turn in an assignment. Copying other peoples' code / code files is not allowed. Copied assignments will receive 0% for everybody involved.

Assignments must build: Assignments that don't build will automatically just be given a flat 50% score. I may still give feedback on what went wrong, and 50% is better than 0%, so turn in something - but ideally, make sure it builds.


 

About

How do you know when you're really done with an assignment or, on the job, a feature or task you're working on?

After you implement a new feature, how can you be sure that you didn't break something you had previously implemented?

If you're working under somebody else, how can you prove to them that your work is complete?

If you're working with somebody else, how can you verify that your teammate's code works properly?

And, how can you save time when testing out your work?

You're probably used to testing out your own programs by running it, manually entering information, and checking that it worked. Often, you have to re-run your program tens or hundreds of times as you add new features and make changes and try to fix bugs. And, as you get bored of doing the same manual tests over and over and over, you resort to just filling in gibberish for your test data and hope that things are working if it runs without crashing - right?

Let the computer do it. In this lab you'll see how you can think in terms of test cases, and how you can write automated unit tests to check all your work.


 

Thinking in Test Cases

Inputs and Outputs

When you boil it down, writing software ends up being about generating outputs with inputs. The core building-blocks are yes/no questions (booleans) that cause a program to branch its behavior or continue looping some commands over and over.

When properly designed, your programs should utilize functions that perform discrete operations: Given some inputs, it is responsible for making some output.

For example, in algebra, we learn about mathematical functions like f(x) = x2.

x is our independent variable - the input.

f(x) (or y) is the dependent variable - the output.

If we plug in "3" as the input, then our output is "9". "2" gives us "4", "5" gives us "25", and so on. This function is responsible for doing one calculation with one variable and outputting one result.

Because we know what should be the output for every input, we can validate its correctness by plugging in values and inspecting if they match the expected output.

We can log some simple test cases by identifying inputs and the expected output for each. The "actual output" is what we would get back from the function when it is actually called in the computer. If the expected output and the actual output don't match, then the test failed and shows us that we need to double-check this function's code.

InputExpected outputActual output
-3 9 f(-3)
-2 4 f(-2)
-1 1 f(-1)
0 0 f(0)
1 1 f(1)
2 4 f(2)
3 9 f(3)
4 16 f(4)

When writing a test case for a function, we should know what inputs and outputs there should be, without seeing the function's implementation. The test isn't about re-implementing the logic to calculate our result - the test is checking if our outputs match.

Example: ReturnOneGreater

Let's say we have this function:

int ReturnOneGreater( int n );

The documentation for the requirement tells us that, for any number n passed in, we should be getting that value plus one back as the output. So, we can write test cases...

InputExpected outputActual output
-3 -2 ReturnOneGreater(-3)
0 1 ReturnOneGreater(0)
99 100 ReturnOneGreater(99)

How would we write out this test case in code? It could be as simple as something like this:

void TestMyFunction()
{
    int input, expectedOutput, actualOutput;
    
    cout << "Test 1" << endl;
    input = -3;
    expectedOutput = -2;
    actualOutput = ReturnOneGreater( -3 );

    if ( actualOutput == expectedOutput )
    {
        cout << "PASS!" << endl;
    }
    else
    {
        cout << "FAIL!" << endl;
    }

    // Test 2...

    // Test 3...
}

Then, any time we want to test our program, we run our test functions!


Test Case Exercises

Come up with at least three test cases for each of the following scenarios. Write out your test cases in a text document to submit as part of this assignment.


Function 1: SumThree

int SumThree( int a, int b, int c )

This function takes in three integers and returns one integer. This function will add the three parameters together and return the output.

#InputsExpected output
1
2
3

Function 2: SumArray

int SumArray( int arr[], int size )

This function takes in an array and the size of that array. It will sum together every item in the array and return the sum as the output.

#InputsExpected output
1
2
3
What would happen if you submit an array of numbers { 1, 3, 5, 2 }, but a size of 2? Make sure to test for as many different scenarios as possible.

Function 3: IsOverdrawn

bool IsOverdrawn( float balance )

This function takes in an account balance, and returns true if it is overdrawn, or false if it is not overdrawn.

#InputExpected output
1
2
3
Are you checking for a balance of 0, too? Is an account with a 0 balance withdrawn? Make sure this functionality is working.

Function 4: GetLength

int GetLength(string word)

This function takes in a string and its output is the length of that string (how many characters are in the string, spaces included.)

#InputExpected output
1
2
3

 

Setting up the project

In your Student Repository, open the CS250-Lab03-Testing project. It will have the following files:

  • tester_program.cpp
  • function1.hpp
  • function2.hpp
  • function3.hpp
  • function4.hpp

All of these source files belong in one project and are not separate programs.

For this lab, there are already several functions included. Each of these functions have logic errors. You will be writing unit tests to test out the expected functionality, and to locate where the error is, then fix it.

For the code part of this lab, you need both correct unit tests and to fix the functions given.

Function stubs are also already provided - you just have to fill them in. The program is built so that it should launch and automatically run the tests without you having to do any coding outside of the tests themselves.

Warning - common error!

It is very common for students who are new to the concept of unit tests to make the following mistake:

REWRITING THE TEST TO PASS, INSTEAD OF WRITING THE TEST TO BE CORRECT.

The tests you write should be logically sound, and if the tests fail, you should be investigating the function being tested to find something to fix. You should NOT, however, rewrite your test so that it passes - this means your function is still incorrect and the test is incorrect, too.

Example: Let’s say you have a function that takes three inputs: a, b, and c. The result should be a + b + c. You write a test that sets a = 2, b = 3, and c = 4. The expected output is 9, but instead the function returns 24. Something is wrong!

WRONG FIX: You change the test so that the expected output is 24 now.

CORRECT FIX: You investigate the function that should be adding these numbers, find that the math is wrong, and fix the algorithm.


 

Running the tests

When you run the program, it will have a menu with options to test each function:

***************************************
**
TESTER
**
***************************************
1. Test AddThree
2. Test IsOverdrawn
3. Test TranslateWord
4. Test GetLength
5. Quit
Test which function ?

When you select one of the functions, it will run the tests and display whether each test passed or failed.

************ Test_AddThree ************
Test_AddThree : Test 1 passed !
Test_AddThree : Test 2 FAILED !
Test_AddThree : Test 3 FAILED !

Based on which test failed, you can look at the inputs and outputs to help you figure out where the logic error is at in these functions.


 

Lab specifications

Each of these .hpp files contain a function to be tested and a test function with one test already written.


 

Function 1: SumThree

The function for the first file is:

int SumThree( int a, int b, int c )
{
    return a + b + b;
}

You can probably already see the logic error here, but don’t fix it yet; you will fix it after you write all the tests.

Inside the Test_SumThree function, there is one test already written:

/* TEST 1 ********************************************/
input1 = 1; input2 = 1; input3 = 1;
expectedOutput = 3;

actualOutput = SumThree( input1, input2, input3 );
if ( actualOutput == expectedOutput )
{
    cout << "Test_AddThree: Test 1 passed!" << endl << endl;
}
else
{
    cout << "Test_AddThree: Test 1 FAILED! \n\t"
    << "Inputs: " << input1 << ", " << input2 << ", " << input3 << "\n\t"
    << "Expected: " << expectedOutput << "\n\t"
    << "Actual: " << actualOutput << endl << endl;
}

This test has the following data:

#InputsExpected output
11, 1, 13

But, a single test isn’t enough to fully cover this function. We can only ensure that the function works if you put the same number in three times - but what about three different numbers? What about a mix of positive and negative numbers?

Using the test cases you wrote up earlier. Make sure your test cases are varied enough; your new tests shouldn’t just be “1 + 1 + 1 = 3”, “2 + 2 + 2 = 6”, “3 + 3 + 3 = 9”, and so on...

For the next two tests, you can set them up by just updating these variables:

/* TEST 2 ********************************************/
// CREATE YOUR OWN TEST
input1 = 0;             // change me
input2 = 0;             // change me
input3 = 0;             // change me
expectedOutput = -1;    // change me

After you’ve updated these, run the tests for the SumThree function again. Your tests should fail because the logic error hasn’t been fixed yet. While the error is pretty obvious for this function, logic errors are not always obvious. This is where unit tests really come in handy.

Fix the logic error in the function, re-run the tests, and make sure everything passes. Once it does, move on to the next file.


 

Function 2: SumArray

int SumArray( int arr[], int size )
{
    int sum = 0;
    for ( int i = 0; i <= size; i++ )
    {
        int sum = 0;
        sum += arr[i];
    }
    return sum;
}

This functions has a logic error. Implement your test cases, run them, locate the logic error, and fix it.


 

Function 3: IsOverdrawn

bool IsOverdrawn( float balance )
{
    if ( balance <= 0 )
    {
        return true;
    }
    else
    {
        return false;
    }
}

This functions has a logic error. Implement your test cases, run them, locate the logic error, and fix it.


 

Function 4: GetLength

int GetLength(string word)
{
    int length = 3;
    word.size();
    return length;
}

This functions has a logic error. Implement your test cases, run them, locate the logic error, and fix it.


 

Turn in

For this function, you will turn in the following:

  • All .hpp files you've changed: function1.hpp, function2.hpp, function3.hpp, function4.hpp
  • The written document where you wrote your test cases out first.