Expert: Zlatko - 7/13/2010

Question
QUESTION: Hi Zlatko,

Here is my attempt below. I did the argv addition program last time but I now want to be able to write a program that will test all possible combinations numbers from 0 to 99 that can be entered into the argv parameter list to be added together and test whether or not the sum was added together correctly. This would be to automatically test whether my code would do the math successfully.  

The test program, though I find, can't enter any numbers through command prompt like any real user would, so I thought I'd just pass the argv and argc arguments into a functadd function that would have parameters of count and param. In that way, I think I could be able to have the program I want to write just loop through all the combinations of two numbers between 0 and 99 and their respective sums and then tell me if the sum is valid or not. (Should all be valid, but this is only a simple addition that's been done) Hope you understand and sorry for the long post.




#include<stdio.h>
#include<string.h>

int funcAdd(int, char**);
int StringToInt(char*);

int main(int argc, char **argv)
{
  int count;
  int sum;
  char** param;
  
  count = argc;
  param = argv;

  sum = funcAdd(count, param);
  printf("Sum = %d\n", sum);
  return 0;
}

int funcAdd(int count, char** param)
{
  int number1 = StringToInt(param[0]);
  int number2 = StringToInt(param[1]);

  int sum = number1 + number2;
  return sum;
}

/*function StringToInt
gets a numeric string from the user and returns the integer
representation of the string
*/
int StringToInt(char* input)
{
  int  number = 0;
  int  inputLen;
  int  i;

  inputLen = strlen(input);
  for (i = 0; i < inputLen; i++)
  {
      int digit = input[i] - '0';
      number *= 10;
      number += digit;
  }
  return number;
}

ANSWER: Hello Mike.

It is certainly possible to write a program to call your addition program with any operands and read the output of the addition program. It is possible but tricky, and I have to admit, I would not write test code in this way. However, the problem is interesting and the idea is useful, so I'll show you how to do it. For this I assume you are using the Windows operating system. I am using Windows specific system functions. If you are using Linux, it is also possible but the function calls are a little different.

The program below calls the adder program with command line arguments and reads the adder's printf output. It is also possible to send data to the adder program as if it was sent from the keyboard, but that just adds complexity. You should try to understand the program below first.

Here is the basic idea. You have a parent process and a child process. The child is the adder program and the parent starts the child. The parent sets up a pipe. A pipe is a thing for communication between a parent and a child. The pipe has 2 ends. The child sends data into one end and the parent reads data from the other end. The parent tells the operating system to connect the child's end of the pipe to the child's stdout, so that anything the child prints to stdout gets sent into the pipe. The parent then reads the messages from the pipe.

Have a look at the code below. I tried to explain what is happening with comments. It is a lot for a beginner to take in.

To run it, you will have to make sure that it is in the same directory as your adder program, and I assumed the adder program is called adder.exe. You can change that if you like. Also, I leave it up to you to parse the adder's response to verify that it worked correctly.

I give credit where credit is due. Parts of this program came from
http://msdn.microsoft.com/en-us/library/ms682499%28VS.85%29.aspx


#include <stdio.h>
#include "Windows.h"

const char* GetLastErrorMessage(void)
{
   static char result[128];
   FormatMessage(FORMAT_MESSAGE_FROM_SYSTEM, NULL, GetLastError(), 0, result, sizeof(result), NULL);
   return result;
}

void ErrorExit(const char* context)
{
   printf("Error: %s: %s\n", context, GetLastErrorMessage());
   fflush(stdout);
   exit(EXIT_FAILURE);
}

/*
This function tries to read a line from the child. It reads one character at a time
until it finds a newline or until the buffer is full. It then returns the number of
characters read.
*/
int ReadLineFromChild(HANDLE pipeEnd, char* result, int resultSpace, int* pipeBrokenFlag)
{
   DWORD dwRead;
   int resultIx = 0;

   while (resultIx < resultSpace)
   {
       char ch;
       if(! ReadFile(pipeEnd, &ch, 1, &dwRead, NULL) )
       {
         /* this happens when child exits */
         if (GetLastError() != ERROR_BROKEN_PIPE)
         {
         printf("Read from child failed: %s (%d)\n", GetLastErrorMessage(), GetLastError());
         }
         else
         {
         *pipeBrokenFlag = 1;
         }

         break;
       }
       result[resultIx++] = ch;
       if (ch == '\n') break;
   }

   /* if there is space, add a null terminator byte */
   if (resultIx < resultSpace) result[resultIx] = 0;
   return resultIx;
}

int runTest(int number1, int number2)
{
   /*
   We need a pipe for getting data from the child.
   The pipe has 2 ends. The parent will use one end for reading and the
   child will use the other end for writing. The operating system will
   connect the child's stdout and stderr to the pipe end.
   */
   HANDLE ChildReadingEnd = NULL; /* Parent reads the child's data from this end */
   HANDLE ChildsWritingEnd = NULL; /* When the child write to stdout or stderr, it will go to this end */
   int childPipeBroken = 0; /* Set to 1 when the parent detects that the child has disconnected from the pipe. */

   SECURITY_ATTRIBUTES saAttr;

   /*
   For creating a process
   */
   char command[128];
   STARTUPINFO startupInfo;
   PROCESS_INFORMATION procInfo;

   /*
   Create pipe handles
   */

   // Set the bInheritHandle flag so pipe handles are inherited.
   saAttr.nLength = sizeof(SECURITY_ATTRIBUTES);
   saAttr.bInheritHandle = TRUE;
   saAttr.lpSecurityDescriptor = NULL;

   /* Create a pipe for the child to parent communication */
   if ( ! CreatePipe(&ChildReadingEnd, &ChildsWritingEnd, &saAttr, 0) )
       ErrorExit("StdoutRd CreatePipe");

   /* This end is for the parent, we don't want the child to inherit this end */
   if ( ! SetHandleInformation(ChildReadingEnd, HANDLE_FLAG_INHERIT, 0) )
       ErrorExit(TEXT("Stdout SetHandleInformation"));

   /*
   This part of the function creates a child process.
   */
   memset(&procInfo, 0, sizeof(procInfo));
   memset(&startupInfo, 0, sizeof(startupInfo));

   startupInfo.cb = sizeof(startupInfo);
   /* this tells the operating system to connect the child's stderr to the writing end of the pipe. */
   startupInfo.hStdError = ChildsWritingEnd;
   /* this tells the operating system to connect the child's stdout to the writing end of the pipe. */
   startupInfo.hStdOutput = ChildsWritingEnd;
   /* this tells the operating system that the startupinfo hStdError and hStdOutput fields have been set */
   startupInfo.dwFlags |= STARTF_USESTDHANDLES;

   printf("Testing %d + %d\n", number1, number2);

   sprintf_s(command, sizeof(command), "adder.exe %d %d", number1, number2);
   if (CreateProcess(
       NULL,
       command,
       NULL,
       NULL,
       TRUE, /* inherit handles. This allows the child to inherit the pipe handles */
       0,
       NULL,
       NULL,
       &startupInfo,
       &procInfo) != 0)
   {
       /*
       Now that we have started the child, we close our version of the ChildsWritingEnd because it
       is for the child to use
       */
       CloseHandle(ChildsWritingEnd);

       /* read from child until the pipe is broken */
       do
       {
         char line[128];
         int readCount = ReadLineFromChild(ChildReadingEnd, line, sizeof(line), &childPipeBroken);
         if (readCount > 0)
         {
         printf("Read %s\n", line);
         }
       } while (!childPipeBroken);

       /* once the pipe is broken, it means the child has exited, so get the exit code */
       while(1)
       {
         DWORD exitCode;
         if (GetExitCodeProcess(procInfo.hProcess, &exitCode) != 0)
         {
         if (exitCode != STILL_ACTIVE)
         {
         printf("Child exited with exit code %d\n", exitCode);
         break;
         }
         }
         else
         {
         printf("GetExitCodeProcess failed: %s\n", GetLastErrorMessage());
         break;
         }
         Sleep(50);
       }

       CloseHandle(procInfo.hThread);
       CloseHandle(procInfo.hProcess);
   }
   else
   {
       ErrorExit("CreateProcess");
   }

   /* Close pipe handles */
   CloseHandle(ChildReadingEnd);
   CloseHandle(ChildsWritingEnd);

   return 0;
}

int main(void)
{
   int op1;
   int op2;
   for(op1 = 0; op1 <= 99; ++op1)
   {
       for(op2 = 0; op2 <= 99; ++op2)
       {
         runTest(op1, op2);
       }
   }

   return 0;
}



You have some errors in your adder program. Here is a corrected version.

#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<Windows.h>

int funcAdd(int, char**);
int StringToInt(char*);

int main(int argc, char **argv)
{
   if (argc != 3) /* ZM added this */
   {
       printf("Wrong argument count\n");
       return EXIT_FAILURE;
   }

   int count;
   int sum;
   char** param;

   count = argc;
   param = argv;

   sum = funcAdd(count, param);
   printf("Sum = %d\n", sum);
   
   fflush(stdout);
   return EXIT_SUCCESS;
}

int funcAdd(int count, char** param)
{
   int number1 = StringToInt(param[1]); /* ZM added this correction. The param[0] is the name of the program */
   int number2 = StringToInt(param[2]);

   int sum = number1 + number2;
   return sum;
}

/*function StringToInt
gets a numeric string from the user and returns the integer
representation of the string
*/
int StringToInt(char* input)
{
   int  number = 0;
   int  inputLen;
   int  i;

   inputLen = strlen(input);
   for (i = 0; i < inputLen; i++)
   {
       int digit = input[i] - '0';
       number *= 10;
       number += digit;
   }
   return number;
}


Finally, I should say that you don't need to test all combinations of input. There are better testing strategies, but that is a discussion for another day.

Best regards
Zlatko

---------- FOLLOW-UP ----------

QUESTION: Hi Zlatko,

here's my attempt, before I read your response above. Thanks for your code, but I think it's little more complicated than I'd like it to be at this point and yes, I saw your errors that you pointed out that I made. In code below, I end up looping and then it would print valid if matches are found. But what I wanted was to be able to have the program run through all combinations of two integers between 0 and 99 added together and yes, I'd like that even though you said it weren't necessary, to verify that all the sums were being added together correctly, which in this case all sums should be valid if I wrote the test program correctly. I know you said better testing strategies are available, but I'd like to be able to figure out how to get this working from where am at at this point. Thank you very much.

#include<stdio.h>
#include<string.h>

int funcAdd(int, char**);
int StringToInt(char*);

int main(int argc, char** argv)
{
  int i;
  int j;
  int test_sum;
  int sum=0;
  int count;
  char** param;

  if(argc!=3)
  {
      puts("ERROR: Usage: add1test <number1> <number2>");
      return -1;
  }

  count = argc;
  param = argv;

  for(i= 0; i<=5; i++)
  {
     for(j=0; j<=5; j++)
     {
        test_sum = i + j;
        printf("test_sum = %d, ", test_sum);

        sum = funcAdd(argc, argv);
        printf("sum = %d ", test_sum);

        if(test_sum == sum)
         printf("valid\n");
        else
         printf("in-valid\n");
     }
  }

  return 0;
}
  
int funcAdd(int count, char** param)
{
  int number1 = StringToInt(param[1]);
  int number2 = StringToInt(param[2]);

  int sum = number1 + number2;
  return sum;
}

/*function StringToInt
gets a numeric string from the user and returns the integer
representation of the string
*/
int StringToInt(char* input)
{
  int  number = 0;
  int  inputLen;
  int  i;

  inputLen = strlen(input);
  for (i = 0; i < inputLen; i++)
  {
      int digit = input[i] - '0';
      number *= 10;
      number += digit;
  }
  return number;
}

ANSWER: Mike, I knew the pipe code would be too complicated. I did it partly for my own education. I had done code like that for Linux but never for Windows. Anyway, now you don't have to take my word for it, you can see for yourself how complicated it is.

Anyway, back to your question. I think you should change funcAdd to be like this:

int funcAdd(char* param1, char* param2)
{
   int number1 = StringToInt(param1);
   int number2 = StringToInt(param2);

   int sum = number1 + number2;
   return sum;
}

The reason is this. You don't really need the count parameter. You see that you don't use it. The function can never use more than two parameters, so there is no point in passing in an array of strings, which is what char** was.

The program to generate all combinations of [0..5] is below.

#include<stdio.h>
#include<string.h>

int funcAdd(char*, char*);
int StringToInt(char*);

int main(int argc, char** argv)
{
   int i;
   int j;
   int test_sum;
   int sum=0;
   char param1[32];
   char param2[32];

   for(i= 0; i<=5; i++)
   {
       for(j=0; j<=5; j++)
       {
         test_sum = i + j;
         printf("test_sum = %d, ", test_sum);

         /*
         The funcAdd takes strings, so in testing the program
         we must first generate the strings
         */
         sprintf(param1, "%d", i);
         sprintf(param2, "%d", j);

         sum = funcAdd(param1, param2);
         printf("sum = %d ", sum); // ZM changed test_sum to sum

         if(test_sum == sum)
         printf("valid\n");
         else
         printf("in-valid\n");
       }
   }

   return 0;
}

int funcAdd(char* param1, char* param2)
{
   int number1 = StringToInt(param1);
   int number2 = StringToInt(param2);

   int sum = number1 + number2;
   return sum;
}

/*function StringToInt
gets a numeric string from the user and returns the integer
representation of the string
*/
int StringToInt(char* input)
{
   int  number = 0;
   int  inputLen;
   int  i;

   inputLen = strlen(input);
   for (i = 0; i < inputLen; i++)
   {
       int digit = input[i] - '0';
       number *= 10;
       number += digit;
   }
   return number;
}



If you really insist on having the old funcAdd, then you will need to add this to main:
   char* params[3];
   params[0] = NULL;
   params[1] = param1;
   params[2] = param2;

and pass params to funcAdd like this:
   sum = funcAdd(3, params);

---------- FOLLOW-UP ----------

QUESTION: so this should be an autotester for which in the original program, the user had to enter in the two numbers via the command line input via argv?

so then it would check that all the valid numbers that a user may enter with argv, would added correctly.

Answer
Yes you could use it as an autotester. You could put it into a separate autotest function, so that it does not clutter up main. There are a number of ways to structure it. You could use a conditional compile in main to call it or you could use a special command line argument that the user would not know about.

The idea of autotest is that you as the developer would want to run through the entire test suite whenever you made changes to your algorithms and wanted to make sure that nothing has broken. That is called regression testing.


Conditional compilation uses the preprocessor #if and #ifdef like this:

#ifdef TEST_MODE
autotest()
{
}
#endif

main()
{
#ifdef TEST_MODE
   autotest();
   return 0;
#endif

   // the rest of the code
}

In the above code, the test code is included in the program only if you define TEST_MODE to the preprocessor. You could do that with
#define TEST_MODE
in your program file, or pass the definition to your compiler, usually with the -D compiler option. How exactly to do that depends on which development tools you are using.

In autotesting you would not want to be told of all the testing operations, you just want to know about the failures. Also you would want to simulate bad user input to ensure that your program handles it. Usually it is the bad user input that is the problem because we as the programmers are in a habit to feeding only good input into our programs. Think of all the security problems you have heard about because of buffer overflows. They are usually the result of bad input taking advantage of bad programming.

As I said, for testing purposes, you don't need to test all combinations of input. If this was a larger program, or even a program for addition of larger integers, you would quickly come to a point where the test would just take too long. Instead, you need to partition the tests into different categories and focus on a typical value from the category and then the border cases of the category. For example,
*two single digit numbers,
*two double digit numbers,
*a combination of single and double digits,
*a three digit number
*a negative number
*zero

If your program is to handle positive numbers, then a border numbers are 1,0,-1. If it is to handle numbers less than 100, then a border numbers are 99,100,101 and such tests should be included.

You can start learning about testing by experimenting and by reading.
http://en.wikipedia.org/wiki/Portal:Software_Testing
is probably a good site to explore.

Best regards
Zlatko