
Understanding Header and Trailer Processing: In mainframe data processing, it’s essential to handle files that contain Header and Trailer records correctly. Standard sort utilities sort all records — including headers and trailers — unless explicitly handled. To avoid rearranging records post-sort, ICETOOL offers options that allow detail records to be sorted while preserving headers and trailers in their original positions.
What is Header-Detail-Trailer Structure?
A Header-Detail-Trailer file format is a structured layout where:
- Header Record: Contains metadata such as creation date or file ID.
- Detail Records: Core business records that need to be sorted or processed.
- Trailer Record: Summary information like total record count or checksum.
File Structure with Header and Trailer
HEADER20250727001 <- Header record
1111AAAA <- Detail record
3333CCCC <- Detail record
2222BBBB <- Detail record
TRAILER0003 <- Trailer record (3 detail records)
DFSORT Techniques for Header and Trailer Processing
Method 1: Using DATASORT Operator (Recommended)
DATASORT in ICETOOL is built to sort detail records while preserving headers and trailers:
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=INPUT.FILE,DISP=SHR
//OUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//TOOLIN DD *
DATASORT FROM(IN) TO(OUT) HEADER TRAILER USING(CTL1)
/*
//CTL1CNTL DD *
SORT FIELDS=(1,8,CH,A)
/*
How DATASORT Works:
- Identifies the first record as Header.
- Sorts only the Detail records.
- Keeps the last record as Trailer.
- Maintains the original structure without manual rearrangement.
Method 2: Using INREC with Sort Keys
This approach provides more control over header/trailer identification using sort keys:
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
OPTION EQUALS
INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'1')),
IFTHEN=(WHEN=(1,6,CH,EQ,C'HEADER'),OVERLAY=(81:C'0')),
IFTHEN=(WHEN=(1,7,CH,EQ,C'TRAILER'),OVERLAY=(81:C'2'))
SORT FIELDS=(81,1,CH,A,1,8,CH,A)
OUTREC BUILD=(1,80)
/*
Explanation:
- Assigns sort key
81:C'0'for Header – sorted first. - Assigns
81:C'1'for Detail – sorted normally. - Assigns
81:C'2'for Trailer – sorted last. - Removes the temporary key using
OUTREC BUILD=(1,80).
Method 3: Two-Step Process for Complex Scenarios
Use this approach when more control is needed over identifying records:
Step 1: Add Sequence Numbers & Create Symbol Table
//STEP01 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//TEMPIN DD DSN=TEMP.FILE,DISP=(NEW,CATLG,DELETE)
//SYM DD DSN=SYMBOL.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
OPTION COPY
OUTFIL FNAMES=TEMPIN,OVERLAY=(81:SEQNUM,8,ZD)
OUTFIL FNAMES=SYM,REMOVECC,NODETAIL,
TRAILER1=('LASTREC,+',COUNT=(M11,LENGTH=8),80:X)
/*
Step 2: Use Symbols and Final Sort
//STEP02 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SYMNAMES DD DSN=SYMBOL.FILE,DISP=OLD
//SORTIN DD DSN=TEMP.FILE,DISP=OLD
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,8,CH,A,81,8,ZD,A)
INREC IFTHEN=(WHEN=(81,8,ZD,EQ,1),OVERLAY=(90:C'H')),
IFTHEN=(WHEN=(81,8,ZD,EQ,LASTREC),OVERLAY=(90:C'T')),
IFTHEN=(WHEN=(90,1,CH,NE,C'H',AND,90,1,CH,NE,C'T'),
OVERLAY=(90:C'D'))
SORT FIELDS=(90,1,CH,A,1,8,CH,A)
OUTREC BUILD=(1,80)
/* Advanced Techniques
Using SUBSET Operator
For removing header and trailer while preserving specific patterns:
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=INPUT.FILE,DISP=SHR
//OUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//TOOLIN DD *
SUBSET FROM(IN) TO(OUT) INPUT REMOVECC
/*
Handling Multiple Headers and Trailers
When dealing with multiple header/trailer pairs in a single file:
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
OPTION EQUALS
INREC IFTHEN=(WHEN=INIT,OVERLAY=(100:SEQNUM,8,ZD)),
IFTHEN=(WHEN=(1,4,CH,EQ,C'0000'),
OVERLAY=(108:C'HEADER')),
IFTHEN=(WHEN=(1,4,CH,EQ,C'9999'),
OVERLAY=(108:C'TRAILER')),
IFTHEN=(WHEN=(1,4,CH,EQ,C'0100'),
OVERLAY=(108:C'DETAIL'))
SORT FIELDS=(108,7,CH,A,100,8,ZD,A)
/*
SKIPREC and STOPAFT Techniques
Basic Header/Trailer Removal
To skip the first record (header) and stop before the last record (trailer):
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
SKIPREC=1
/*
//* Note: STOPAFT requires knowing exact record count
Dynamic Last Record Calculation
For dynamic trailer handling:
//STEP0100 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=&T1,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//SYSIN DD *
OPTION SKIPREC=1
SORT FIELDS=COPY
OUTREC FIELDS=(SEQNUM,8,ZD,START=0,INCR=1,80:X)
OUTFIL NODETAIL,REMOVECC,
TRAILER1=(C' OPTION COPY,SKIPREC=1,STOPAFT=',1,8,80:X)
/*
//STEP0200 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD SYSOUT=*
//SYSIN DD DSN=&T1,DISP=OLD
/*
OUTFIL Header and Trailer Techniques
Adding Headers and Trailers to Output
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
OUTFIL FNAMES=SORTOUT,
HEADER1=C'HEADER RECORD - SORTED DATA',
TRAILER1=(C'TRAILER - TOTAL RECORDS: ',COUNT=(M11,LENGTH=8))
/*
Multiple Output Files with Different Headers
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//OUT1 DD DSN=OUTPUT1.FILE,DISP=(NEW,CATLG,DELETE)
//OUT2 DD DSN=OUTPUT2.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
OUTFIL FNAMES=OUT1,
INCLUDE=(20,1,CH,EQ,C'A'),
HEADER1=C'TYPE A RECORDS',
TRAILER1=(C'A-TYPE COUNT: ',COUNT)
OUTFIL FNAMES=OUT2,
INCLUDE=(20,1,CH,EQ,C'B'),
HEADER1=C'TYPE B RECORDS',
TRAILER1=(C'B-TYPE COUNT: ',COUNT)
/*
Best Practices and Tips
1. Always Use EQUALS Option
When preserving header/trailer order is critical:
OPTION EQUALS
2. Validate Record Counts
Always verify trailer record counts match detail records:
OUTFIL TRAILER1=(C'DETAIL COUNT: ',COUNT=(M11,LENGTH=8))
3. Handle Variable Length Records
For VB files, account for RDW in position calculations:
INREC IFTHEN=(WHEN=(5,6,CH,EQ,C'HEADER'),OVERLAY=(85:C'H'))
4. Test with Small Datasets
Always test complex header/trailer logic with small, known datasets before production.
5. Document Record Layouts
Clearly document the structure and position of identifiers:
- Position 1-6: Record Type (HEADER/DETAIL/TRAILER)
- Position 7-14: Date (YYYYMMDD)
- Position 15-22: Control Number
- Error Handling and Validation
Validate Header/Trailer Presence
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=INPUT.FILE,DISP=SHR
//TOOLIN DD *
COUNT FROM(IN) FIRST 1 LAST 1 USING(CTL1)
/*
//CTL1CNTL DD *
INCLUDE COND=(1,6,CH,EQ,C'HEADER',OR,1,7,CH,EQ,C'TRAILER')
/*
Check for Missing Records
OUTFIL FNAMES=ERRORS,
INCLUDE=(1,6,CH,NE,C'HEADER',AND,1,6,CH,NE,C'DETAIL',
AND,1,7,CH,NE,C'TRAILER') Examples
Scenario -1: Let’s say we want to SORT an input file by a section that is divided by a constant keyword or symbol or some unique identifier. We can use the WHEN=GROUP function, we can add group numbers after the end of the records. Then we can SORT by the group number and by the key. Finally, we can remove the group numbers.
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//************************************************************
//STEP01 EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=... input file (FB/10)
//SORTOUT DD DSN=... output file (FB/10)
//SYSIN DD *
INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,5,CH,EQ,C'*****'),
PUSH=(11:ID=8))
SORT FIELDS=(11,8,ZD,A,1,5,CH,A)
OUTREC BUILD=(1,10)
/*
Please note the above example is for the FB file… so no padding would occur.
//SYSIN DD *
INREC IFTHEN=(WHEN=INIT,BUILD=(1,4,5:8X,13:5)),
IFTHEN=(WHEN=GROUP,BEGIN=(13,5,CH,EQ,C'*****'),
PUSH=(5:ID=8))
SORT FIELDS=(5,8,ZD,A,13,5,CH,A)
OUTREC BUILD=(1,4,5:13)
/*
Let’s say our input data is having multiple subgroups divided by different unique identifiers…
Input:
HDR 2021/04/10
001 23.05-
002 5213.75+
003 861.51+
004 753.90-
TRL T862143
HDR 2021/04/09
001 282.15+
002 8.00-
003 1496.28+
TRL T201576
HDR 2021/03/11
001 123.86+
002 98.07-
003 61.16+
TRL T031893
HDR 2021/04/09
001 106.27-
002 2308.00+
003 96.72+
004 206.99-
005 208.82-
TRL T201573
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//************************************************************
//STEP01 EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=... input file
//SORTOUT DD DSN=... output file
//SYSIN DD *
IFTHEN=(WHEN=GROUP,BEGIN=(1,3,CH,EQ,C'HDR'),
PUSH=(31:6,10,41:ID=8)),
IFTHEN=(WHEN=(1,3,CH,EQ,C'HDR'),OVERLAY=(49:C'A')),
IFTHEN=(WHEN=(1,3,CH,EQ,C'TRL'),OVERLAY=(49:C'C')),
IFTHEN=(WHEN=NONE,OVERLAY=(49:C'B'))
SORT FIELDS=(31,10,CH,A,
41,8,ZD,A,
49,1,CH,A,
6,9,SFF,D)
BUILD=(1,30)
/*
Output:
HDR 2008/03/11
001 123.86+
003 61.16+
002 98.07-
TRL T031893
HDR 2008/04/09
003 1496.28+
001 282.15+
002 8.00-
TRL T201576
HDR 2008/04/09
002 2308.00+
003 96.72+
001 106.27-
004 206.99-
005 208.82-
TRL T201573
HDR 2008/04/10
002 5213.75+
003 861.51+
001 23.05-
004 753.90-
TRL T862143
Scenario -2: This example illustrates how you can SORT and INCLUDE groups of FB records depending on a value in the first record of each group…
Input File
1RPT.SRIHARI
LINE 1 FOR REPORT 1
LINE 2 FOR REPORT 1
...
1RPT.VICKY
LINE 1 FOR REPORT 2
LINE 2 FOR REPORT 2
...
1RPT.FRANK
LINE 1 FOR REPORT 3
LINE 2 FOR REPORT 3
...
1RPT.DAVID
LINE 1 FOR REPORT 4
LINE 2 FOR REPORT 4
...
INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,4,CH,EQ,C’RPT.’),
PUSH=(31:6,8))
OPTION EQUALS
SORT FIELDS=(31,8,CH,A)
OUTFIL INCLUDE=(31,8,CH,EQ,C’FRANK’,OR,
31,8,CH,EQ,C’SRIHARI’),BUILD=(1,30)
Each report starts with ‘RPT.reptname’ in positions 2-13…
1RPT.SRIHARI
LINE 1 FOR REPORT 1 SRIHARI
LINE 2 FOR REPORT 1 SRIHARI
...
1RPT.VICKY VICKY
LINE 1 FOR REPORT 2 VICKY
LINE 2 FOR REPORT 2 VICKY
...
We use a SORT statement to sort ascending on the reptname…
1RPT.DAVID DAVID
LINE 1 FOR REPORT 4 DAVID
LINE 2 FOR REPORT 4 DAVID
...
1RPT.FRANK FRANK
LINE 1 FOR REPORT 3 FRANK
LINE 2 FOR REPORT 3 FRANK
...
1RPT.SRIHARI SRIHARI
LINE 1 FOR REPORT 1 SRIHARI
LINE 2 FOR REPORT 1 SRIHARI
...
We use an OUTFIL statement to only INCLUDE the records with a reptname of FRANK or SRIHARI…
1RPT.FRANK
LINE 1 FOR REPORT 3
LINE 2 FOR REPORT 3
...
1RPT.SRIHARI
LINE 1 FOR REPORT 1
LINE 2 FOR REPORT 1
