| ## Analyze data changes

Examine the content of files to identify non-ready dates, confirm expected date fields, and identify unexpected date fields.

At the same time, identify invalid and reserved values in date fields (such as using a year value of "99" to signify an invalid value). For large files, examine only a sample of records if they use the same record definition.

Specify the modifications needed to convert record formats and file types and to scrub data values.

- Expand date fields
- Remove reserved values
- Remove user codes
- Scrub invalid dates
- Convert date format (for example, 9's complement1 or integer2 to Gregorian)
- Reload file due to key changes
- Convert file type
- Tune to run in batch window and available disk space
- Define a new SQL view (DBA)

1Nines-complement dates are most often used to sort or store records in reverse order. For example, a record with a date of 082297 will usually be seen before one with a date of 082397.

Subtracting each date from 999999 gives 917702 and 917701 respectively, which sort in the opposite order, the 917701 date (08/23/97) coming before the 917702 date (08/22/97).

The theory behind this is that most processing involves the most recent records. Were they sorted normally, each search would read through the entire database before finding the right record. Using nines-complemented dates, the right record should be found in the very beginning of the file. For large files, this could be an enormous performance enhancement.

- COBOL sample code.
- PL/I sample code.

2See Integer date limits

©1997 IBM Corporation