DataCheck

A type mismatch is when the data type of a field in a table (in the structure file) is different than the same field in a particular record in a data file.

Consider a table which has a field called "MyField" of type alpha. When you create a record for this table, the data type for "MyField" on disk will be of type alpha.

However, if you then change the data type of the field in the structure to be Long, the record on disk is not changed; "MyField" is still of type alpha.

So there exists a mismatch between the data types for the field.

A Type mismatch is a feature of 4D.

It was designed in 4D to make life simple for you when designing your database.

For example, when we use another database system (SQL based), everytime you change your data dictionary (your tables/ fields), you have to rebuild the database from scratch. Generally, this can be very painful -- you have to export data, and reimport it.

4D, very conveniently, hides all this from you, by internally converting between data types.

This means that when 4D sees a field on disk of type Alpha, but the structure is of type Long, 4D converts the Alpha to a Long, on the fly.

It is this kind of feature that makes 4D very accessible to the beginning programmer, and it's what has made 4D an excellent, approachable product.

Committed Software has never been able to prove that Type Mismatches cause problems.

We do not like instilling lack of confidence in a feature of 4D without reason, yet in this case, we believe that there is an issue some developers have experienced with their data files. And the issue has a direct correlation to Type Mismatches existing in the data file.

It may still be that Type Mimatches are not the cause. But in many of the cases where we have seen the strange behaviour (see next FAQ entry), Type Mismatches occured. In all cases of this issue, running a simple "TouchAll" algorithm solved the symtoms of the problem.

This does not prove that the Type Mismatch is the cause of the issue. It only shows a correlation between the issue and Type Mismatches.

We are calling this a potential problem because we cannot actually identify the problem, but we have seen direct correlations to a recurring issue. Here, we describe the actual symtoms which lead to the diagnosis of Type Mismatch related issue.

The issue usually occurs with large data files (>200k records, or less records if your records are "very wide"). Generally, we're talking > 200 meg data files.

The issue usually presents itself as a periodic crashing data file. No tool shows any damage. Then, after a period of time, the crashing becomes more frequent. DataCheck begins to show that indexes are damaged. Fixing the index appears to halt the crashing for a while. The crashing then reappears, and worsens over time. The developer finds ways of fixing indexes periodically as it keeps the database "up" longer. Eventually, the developer notices that APPLY TO SELECTION on certain tables is also causing crashes.

The developer tries "Type Mismatch" detection in DataCheck and finds that there are Type Mismatches in the datafile, and in the table that cannot do the APPLY TO SELECTION and had the bad indexes recurring.

In short, the symtoms are: crashing, with increasing frequency, with some "healing" due to index recreation, and the existance of Type Mismatches.

What we have seen is by running a TouchAll algorithm, the database stops crashing. We have seen databases that cannot stay up for more than 2 hours go to a state of being up for weeks straight after running a TouchAll. The TouchAll also removes the Type Mismatches as well.

This is why we see the correlation: files come in crashing, with type mismatches. Nothing else seems wrong with the file. However, removing the Type Mismatches solves the problem. Does that mean the Type Mismatch is the problem or just the same "fix" has the side affect of fixing the Type Mismatch. We cannot be sure.

But the TouchAll algorithm should not have any side affects. So in our opinion, it cannot hurt to run it.

By running this very simple 4D method, you can "touch" all fields in all tables in your datafile. This causes 4D to "re-write" the record from scratch.

Below is nothing "special" -- there is no tricks going on -- no special ordering. You can modify this algorithm all you want and will get the desired results as long as you "$pField->:=$pField->" on every Field in every Table, and save the Record. The algorithm below is included simply for completeness.

This "TouchAll" is something that we suggest you run before you bring any system to production. The reason is that it cannot hurt one bit to do this, and the potential benefits are immeasurable.

  `datacheck_touchall
  `there is no copyright on this code.
  `however, if it breaks something, 
  `  that's your problem.
  `doing ANYTHING to a suspect data
  ` file can cause further damage
  `MAKE BACKUPS before using any tool or 
  `  running any method on suspect data files.
C_LONGINT($table;$field;$rec)
C_POINTER($pFile;$pField)
For ($table;1;Count tables)
  $pFile:=Table($table)
  $aName:="["+Table name($pFile)+"]"
  ALL RECORDS($pFile->)
  FIRST RECORD($pFile->)
  $nRecs:=Records in selection($pFile->)
  For ($rec;1;$nRecs)
    If ($rec%1000=0)
      MESSAGE($aName+" "+String($nRecs-$rec))
    End if 
    For ($field;1;Count fields($pFile))
      $pField:=Field($table;$field)
      $pField->:=$pField->
    End for 
    SAVE RECORD($pFile->)
    NEXT RECORD($pFile->)
  End for 
  UNLOAD RECORD($pFile->)
End for 
FLUSH BUFFERS

return to top Type Mismatches - Is there any other way to resolve them? return to top