If desired, HEC-DSS will recycle and reuse deleted areas within a file.  This not only reduces the ultimate size of a DSS file that is written to frequently, but also decreases file access times compared to a file that does not use reclamation.  This is especially true when records are deleted and re-written.  The disadvantage is that once space is recycled, it cannot be undeleted and returned to its original state.

File space reclamation is set on by default and can be set to a different level, or off, for a DSS file by calling function zreclaimSet.  for example (ifltab,RECLAIM_NONE).

RECLAIM_NONE               0:            Don't reclaim any space

RECLAIM_DELETED          1:            Reclaim space from deleted records in pathname bin

RECLAIM_BIN_INFO       2:            Reclaim bin space and info/data area in file

RECLAIM_ALL                    3:            Reclaim all possible space


As reclamation is a primary file setting, it generally is done only once, usually before significant writing to the file occurs.   If the reclamation level is changed, it is a good idea to squeeze the file before further use.

We only reclaim significant size segments; that includes deleted records, space from expanded records and deleted catalog sort sequences.  We don't reclaim anything from pathname bins, etc.  We only use reclaimed space for record info/data areas,  not for bins or catalog sort areas (these generally require too much space).

Primary Keys used in reclamation

Current Level:

ifltab[zdssKeys.kreclaimLevel] contains the above reclaim levels.  Usually set in the software and not changed.

File Level:

zdssFileKeys.kreclaimMin (39) is the minimum length of space to use for reclamation.  If you release space smaller than this, it is ignored.  kreclaimMin also indicates if the file allows the use of reclaimed space,  (0 = no, >0 = yes)  (Reclaimed space can be slower).  Generally set at file creation.  Currently 100 words.

zdssFileKeys.kreclaimMaxAvailable (40) is the maximum amount of reclaimed space available in one contiguous segment (we don't do multiple segments).  If the amount of space that you need is larger than this, then you don’t use reclamation; just write at the end of the file.

zdssFileKeys.kreclaimTotal (41) is the total amount of reclaimed space available in the file.

zdssFileKeys.kreclaimTableAddress (42) is the address of the first reclaim table in the file.  it is a 2x array, with the first word being the length for that segment, and the second word being the address of that segment.

zdssFileKeys.kreclaimSegAvailableAdd (43) points to address of reclaim table with available space.  This is the last reclaim table.

zdssFileKeys.kreclaimSegNumber (44) array number (size) that corresponds to kreclaimSegAvailableAdd.  (?????)

zdssFileKeys.kreclaimMaxSegment (45) is the total maximum number of tables that can be used.  After this is reached, releasing space is ignored (time for a squeeze), and number of segments allocated for this file (second int).  Currently 20 tables max in a DSS files

zdssFileKeys.kreclaimSegmentsUsed (46) is the number of tables used.

zdssFileKeys.kreclaimNumber (47) is the number of reclaim pairs for each reclaim table.  Currently 500 pairs

zdssFileKeys.kreclaimSize (48) is the size of the reclaim table. This is kreclaimNumber*2 + 2  = 500 * 2 + 2 = 1002

zdssFileKeys.kreclaimedPaths (49) This is the number of pathnames that have used reclaimed space

zdssFileKeys.kreclaimedSpace (50)   (??????)


How File Space Reclamation Works

When a DSS file is created, a “reclaim table” is created.  The reclaim table will contain reclaim pairs, which is the size of a deleted portion of the file and the address to that portion.  The reclaim table is located following the hash table and is size zdssFileKeys.kreclaimSize.  There can be many reclaim tables in a file, so after one fills up, the end of that table will point to the next one.

Adding to space reclamation:  Deleting a record.  When a record is deleted, the information area, headers and data areas are identified and that size (in long 6) is calculated in function zdeleteInternal.  (This is contiguous space.)   Function zreleaseFileSpace is called, which checks the reclaim level and the minimum amount of space for reclamation against what is being deleted.  zreleaseFileSpace reads the reclaim table (or reclaim array) and goes through it until an empty spot (a zero entry) is found.  Then the deleted size is inserted followed by the address to that space (the reclaim pair.)  The reclaim table is then written back to disk and the amount of reclaim space for the file is updated.

If the reclaim table is full, an extension is created (at EOF) and linked by saving the extension address in the last word of the table (the last 2 words are reserved for extensions.)

Using reclaimed space: Writing a new record.  In zwriteNew, called from zwriteInternal, the amount of space needed for the info area, headers and data are is calculated.  The function zgetFileSpace is called, which will determine if there is sufficient reclamation space available for this record; if not the space will be added to the end of the file.

 zgetFileSpace checks the reclamation level, and sees if the space needed is larger than the minimum (kreclaimMin) reclamation space set and smaller or equal to the maximum available (kreclaimMaxAvailable) reclamation space in the file.  If true, the reclamation table is read and zgetFileSpace walks down the entire table, looking for smallest space that is equal or larger than what is needed.  What is wanted is the smallest space so that larger chunks can be used for larger needs.  When the smallest size that is equal or larger than the needed size is found, the address of that space (from the table) is saved.  The needed size is subtracted from the available size.  That remainder is compared to the minimum length (kreclaimMin).  If it is less, then the reclamation table entry for that space is set to zero.  If it is more, then the entry is set to the remainder size and the address is updated to reflect the use of the space.  kreclaimMaxAvailable is updated, if needed and then the reclaimation table is written to disk and the address of the reclaimed space is returned to zwriteNew.  (If there is insufficient reclamation space, the space needed will be added to the end of the file.)