Expert: Ralph McArdell - 1/29/2007

Question
Dear Sir,
I have previously asked a question on how to differentiate integers and alphabets when reading from a .txt file. thank you so much for your kind advise. But too bad my previous question was too vague. Below is the sample .txt file which i would read from.
"
Data Logged on September 16, 2006, at 15:09:27

Datalog_Params

Log_Mode = 2
Sampling_Factor = 8
Defined_Sample_Size = 4000
Captured_Sample_Size = 4000
Delay_Sample = 0
Static_Sample = 10


Smp_No.      AN_EnB_HYG   BY_Cmd_HDA   IJ_MJK_Value
0       -20001       -20000          -69
1       -20001       -20001          121
2       -20001       -20001          -90
.                 .              .                  .
.                 .              .                  .
.                 .              .                  .
.                 .              .                  .
3999        65536      1420544          305   
"
So what I need to do is to actually read the figures from some coloums (for e.g AN_EnB_HYG & IJ_MJK_Value)...from sample no.0 to 3999. And calculate the max, min and average for the figures under AN_EnB_HYG & IJ_MJK_Value.. Every information besides these 2 coloums of figures are redundant to me..Please advise...thanks!  

Answer
In fact you have all the information you require in my answer to your previous question. I assume you have some experience in writing C as you claimed in your original question.

The way to proceed is as follows:

1/ Note that the lines at the start of the file are not required. So read and throw them away until you reach the first line that is not one of the header lines (i.e. the first line of table data). How you determine this is up to you: maybe there are always a fixed number of lines of information and column headings at the start of the file, or maybe you check each line read to see if it contains the table column heading strings (e.g. you see if the first text on the line is "Smp_No."). You can use a modified form of the do while loop using fgets I showed in my original answer. You might find the C string functions such as strncmp of use as well (include string.h).

2/ The table has 4 values per line of which, in the example you give, only 2 are required. Assuming the values are all in the range of values that fit into the int type your compiler uses then you can read the values for each row in the table using fscanf thus:

   int field0 = 0;
   int field1 = 0;
   n = fscanf( inputFile, "%*d%d%*d%d", &field0, &field1 );

That is you read all integer values but ignore the values from the first and third columns (the %*d format specifiers) and keep those from the second and forth columns (the %d format specifiers).

Alternatively, you can treat the unwanted columns as strings and read but ignore them using a %*s format specification:

   n = fscanf( inputFile, "%*s%d%*s%d", &field0, &field1 );

In the code lines above I have specified that the read integer values be placed into scalar integer variables field0 and field1. I would think it more likely that you would read this data into values one line (row) at a time using a loop until the end of the file (or end of the table at least) or you have read enough values and place the values into an array or some other container:

   #define MYPROGRAM_MAX_ROWS_OF_DATA    1000
   #define MYPROGRAM_NUMBER_OF_COLUMNS   2

   /* Array index values for each column */
   #define MYPROGRAM_BY_Cmd_HDA          0
   #define MYPROGRAM_IJ_MJK_Value        1

   ...

   int data[MYPROGRAM_MAX_ROWS_OF_DATA][MYPROGRAM_NUMBER_OF_COLUMNS];

   ...





   int row = 0;
   int  n = 0;
   while ( row < MYPROGRAM_MAX_ROWS_OF_DATA && ! feof(inputFile) )
   {
       n = fscanf( inputFile, "%*d%d%*d%d"
                 , &data[row][MYPROGRAM_BY_Cmd_HDA]
                 , &data[row][MYPROGRAM_IJ_MJK_Value]
                 );
       if ( 2==n )
       {
           ++row; /* OK, increment row count */
       }
       else
       {
           break; /* error or EOF, break out from data reading loop */
       }
   }

In the above code I use a 2 dimension array to store the read values. It can store many rows of data but only two elements can be stored for each row as only two columns are read and stored from each row of data in the file.

The loop continues until either there is no further space to store data or the end of file is reached. Within the loop one row of data is read using fscanf and the format specification string shown above. However the pointers to where to store the read and converted integer values are now the addresses for the elements for the columns of the current row in the array. Note that we require the address of operator ( & ) as we are not specifying (just) the first element of the array (i.e. specifying &data[0][0] would be the same as just specifying data, but this does not work for any of the other elements of the array).

An alternative would be to use a single dimension array of elements of a user defined struct type which contains fields for each of the columns read from the table:

   #define MYPROGRAM_MAX_ROWS_OF_DATA    1000

   ...

   /* Structure to contain the data for each row */
   typedef struct tag_table_data
   {
       int BY_Cmd_HDA;
       int IJ_MJK_Value;
   }
   table_data;

   ...

   table_data data[MYPROGRAM_MAX_ROWS_OF_DATA];

   int row = 0;
   int  n = 0;
   while ( row < MYPROGRAM_MAX_ROWS_OF_DATA && ! feof(inputFile) )
   {
       n = fscanf( inputFile, "%*d%d%*d%d"
                 , &data[row].BY_Cmd_HDA
                 , &data[row].IJ_MJK_Value
                 );
       if ( 2==n )
       {
           ++row; /* OK, increment row count */
       }
       else
       {
           break; /* error or EOF, break out from data reading loop */
       }
   }

Note that you will probably wish to check for various conditions after the loop.  A value of n (the returned value from fscanf) of 2 or EOF can indicate success.

2 because everything was read correctly but the while loop conditions were no longer met as either the limit of the number of rows that could be stored was reached or the end of file was reached, as detected by feof. Both are OK, although you might wish to check the value of row to see if it is maxed out as you may be missing some data if it is. If so increase its value.

EOF because nothing was read and converted for the row before end of file was reached (OK) _or_ there was a read error (BAD). To determine which check the error indicator for the stream using ferror, and if set (ferror returns a non-zero value) then call errno to get the specific error condition code.

If n is another value (e.g. 1) then fscanf failed to read and convert a whole row of values and maybe the data file is corrupted.

The code I show here is for example and demonstration purposes only. It may contain errors and typos etc. If so, then I am sorry and apologise. It is _not_ production quality. For example you may wish to use a more sophisticated memory allocation scheme for storing the read values or use a different container type that supports dynamically allocated elements and resizing etc.

I shall not answer further C questions on this subject. Remember I am a C++ expert not a C expert!