Understanding Header and Trailer Processing: In mainframe data processing, it’s essential to handle files that contain Header and Trailer records correctly. Standard sort utilities sort all records — including headers and trailers — unless explicitly handled. To avoid rearranging records post-sort, ICETOOL offers options that allow detail records to be sorted while preserving headers and trailers in their original positions.
A Header-Detail-Trailer file format is a structured layout where:
HEADER20250727001 <- Header record
1111AAAA <- Detail record
3333CCCC <- Detail record
2222BBBB <- Detail record
TRAILER0003 <- Trailer record (3 detail records) DATASORT in ICETOOL is built to sort detail records while preserving headers and trailers:
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=INPUT.FILE,DISP=SHR
//OUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//TOOLIN DD *
DATASORT FROM(IN) TO(OUT) HEADER TRAILER USING(CTL1)
/*
//CTL1CNTL DD *
SORT FIELDS=(1,8,CH,A)
/* This approach provides more control over header/trailer identification using sort keys:
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
OPTION EQUALS
INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'1')),
IFTHEN=(WHEN=(1,6,CH,EQ,C'HEADER'),OVERLAY=(81:C'0')),
IFTHEN=(WHEN=(1,7,CH,EQ,C'TRAILER'),OVERLAY=(81:C'2'))
SORT FIELDS=(81,1,CH,A,1,8,CH,A)
OUTREC BUILD=(1,80)
/* Explanation:
81:C'0' for Header – sorted first.81:C'1' for Detail – sorted normally.81:C'2' for Trailer – sorted last.OUTREC BUILD=(1,80).Use this approach when more control is needed over identifying records:
//STEP01 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//TEMPIN DD DSN=TEMP.FILE,DISP=(NEW,CATLG,DELETE)
//SYM DD DSN=SYMBOL.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
OPTION COPY
OUTFIL FNAMES=TEMPIN,OVERLAY=(81:SEQNUM,8,ZD)
OUTFIL FNAMES=SYM,REMOVECC,NODETAIL,
TRAILER1=('LASTREC,+',COUNT=(M11,LENGTH=8),80:X)
/* //STEP02 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SYMNAMES DD DSN=SYMBOL.FILE,DISP=OLD
//SORTIN DD DSN=TEMP.FILE,DISP=OLD
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,8,CH,A,81,8,ZD,A)
INREC IFTHEN=(WHEN=(81,8,ZD,EQ,1),OVERLAY=(90:C'H')),
IFTHEN=(WHEN=(81,8,ZD,EQ,LASTREC),OVERLAY=(90:C'T')),
IFTHEN=(WHEN=(90,1,CH,NE,C'H',AND,90,1,CH,NE,C'T'),
OVERLAY=(90:C'D'))
SORT FIELDS=(90,1,CH,A,1,8,CH,A)
OUTREC BUILD=(1,80)
/* For removing header and trailer while preserving specific patterns:
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=INPUT.FILE,DISP=SHR
//OUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//TOOLIN DD *
SUBSET FROM(IN) TO(OUT) INPUT REMOVECC
/* When dealing with multiple header/trailer pairs in a single file:
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
OPTION EQUALS
INREC IFTHEN=(WHEN=INIT,OVERLAY=(100:SEQNUM,8,ZD)),
IFTHEN=(WHEN=(1,4,CH,EQ,C'0000'),
OVERLAY=(108:C'HEADER')),
IFTHEN=(WHEN=(1,4,CH,EQ,C'9999'),
OVERLAY=(108:C'TRAILER')),
IFTHEN=(WHEN=(1,4,CH,EQ,C'0100'),
OVERLAY=(108:C'DETAIL'))
SORT FIELDS=(108,7,CH,A,100,8,ZD,A)
/* To skip the first record (header) and stop before the last record (trailer):
//STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
SKIPREC=1
/*
//* Note: STOPAFT requires knowing exact record count For dynamic trailer handling:
//STEP0100 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=&T1,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//SYSIN DD *
OPTION SKIPREC=1
SORT FIELDS=COPY
OUTREC FIELDS=(SEQNUM,8,ZD,START=0,INCR=1,80:X)
OUTFIL NODETAIL,REMOVECC,
TRAILER1=(C' OPTION COPY,SKIPREC=1,STOPAFT=',1,8,80:X)
/* //STEP0200 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD SYSOUT=*
//SYSIN DD DSN=&T1,DISP=OLD
/* //STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
OUTFIL FNAMES=SORTOUT,
HEADER1=C'HEADER RECORD - SORTED DATA',
TRAILER1=(C'TRAILER - TOTAL RECORDS: ',COUNT=(M11,LENGTH=8))
/* //STEP1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//OUT1 DD DSN=OUTPUT1.FILE,DISP=(NEW,CATLG,DELETE)
//OUT2 DD DSN=OUTPUT2.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
OUTFIL FNAMES=OUT1,
INCLUDE=(20,1,CH,EQ,C'A'),
HEADER1=C'TYPE A RECORDS',
TRAILER1=(C'A-TYPE COUNT: ',COUNT)
OUTFIL FNAMES=OUT2,
INCLUDE=(20,1,CH,EQ,C'B'),
HEADER1=C'TYPE B RECORDS',
TRAILER1=(C'B-TYPE COUNT: ',COUNT)
/* When preserving header/trailer order is critical:
OPTION EQUALS Always verify trailer record counts match detail records:
OUTFIL TRAILER1=(C'DETAIL COUNT: ',COUNT=(M11,LENGTH=8)) For VB files, account for RDW in position calculations:
INREC IFTHEN=(WHEN=(5,6,CH,EQ,C'HEADER'),OVERLAY=(85:C'H')) Always test complex header/trailer logic with small, known datasets before production.
Clearly document the structure and position of identifiers:
//STEP1 EXEC PGM=ICETOOL
//TOOLMSG DD SYSOUT=*
//DFSMSG DD SYSOUT=*
//IN DD DSN=INPUT.FILE,DISP=SHR
//TOOLIN DD *
COUNT FROM(IN) FIRST 1 LAST 1 USING(CTL1)
/*
//CTL1CNTL DD *
INCLUDE COND=(1,6,CH,EQ,C'HEADER',OR,1,7,CH,EQ,C'TRAILER')
/* OUTFIL FNAMES=ERRORS,
INCLUDE=(1,6,CH,NE,C'HEADER',AND,1,6,CH,NE,C'DETAIL',
AND,1,7,CH,NE,C'TRAILER') Scenario -1: Let’s say we want to SORT an input file by a section that is divided by a constant keyword or symbol or some unique identifier. We can use the WHEN=GROUP function, we can add group numbers after the end of the records. Then we can SORT by the group number and by the key. Finally, we can remove the group numbers.
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//************************************************************
//STEP01 EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=... input file (FB/10)
//SORTOUT DD DSN=... output file (FB/10)
//SYSIN DD *
INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,5,CH,EQ,C'*****'),
PUSH=(11:ID=8))
SORT FIELDS=(11,8,ZD,A,1,5,CH,A)
OUTREC BUILD=(1,10)
/*
Please note the above example is for the FB file… so no padding would occur.
//SYSIN DD *
INREC IFTHEN=(WHEN=INIT,BUILD=(1,4,5:8X,13:5)),
IFTHEN=(WHEN=GROUP,BEGIN=(13,5,CH,EQ,C'*****'),
PUSH=(5:ID=8))
SORT FIELDS=(5,8,ZD,A,13,5,CH,A)
OUTREC BUILD=(1,4,5:13)
/*
Let’s say our input data is having multiple subgroups divided by different unique identifiers…
Input:
HDR 2021/04/10
001 23.05-
002 5213.75+
003 861.51+
004 753.90-
TRL T862143
HDR 2021/04/09
001 282.15+
002 8.00-
003 1496.28+
TRL T201576
HDR 2021/03/11
001 123.86+
002 98.07-
003 61.16+
TRL T031893
HDR 2021/04/09
001 106.27-
002 2308.00+
003 96.72+
004 206.99-
005 208.82-
TRL T201573
//N141231A JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//************************************************************
//STEP01 EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=... input file
//SORTOUT DD DSN=... output file
//SYSIN DD *
IFTHEN=(WHEN=GROUP,BEGIN=(1,3,CH,EQ,C'HDR'),
PUSH=(31:6,10,41:ID=8)),
IFTHEN=(WHEN=(1,3,CH,EQ,C'HDR'),OVERLAY=(49:C'A')),
IFTHEN=(WHEN=(1,3,CH,EQ,C'TRL'),OVERLAY=(49:C'C')),
IFTHEN=(WHEN=NONE,OVERLAY=(49:C'B'))
SORT FIELDS=(31,10,CH,A,
41,8,ZD,A,
49,1,CH,A,
6,9,SFF,D)
BUILD=(1,30)
/*
Output:
HDR 2008/03/11
001 123.86+
003 61.16+
002 98.07-
TRL T031893
HDR 2008/04/09
003 1496.28+
001 282.15+
002 8.00-
TRL T201576
HDR 2008/04/09
002 2308.00+
003 96.72+
001 106.27-
004 206.99-
005 208.82-
TRL T201573
HDR 2008/04/10
002 5213.75+
003 861.51+
001 23.05-
004 753.90-
TRL T862143
Scenario -2: This example illustrates how you can SORT and INCLUDE groups of FB records depending on a value in the first record of each group…
Input File
1RPT.SRIHARI
LINE 1 FOR REPORT 1
LINE 2 FOR REPORT 1
...
1RPT.VICKY
LINE 1 FOR REPORT 2
LINE 2 FOR REPORT 2
...
1RPT.FRANK
LINE 1 FOR REPORT 3
LINE 2 FOR REPORT 3
...
1RPT.DAVID
LINE 1 FOR REPORT 4
LINE 2 FOR REPORT 4
...
INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,4,CH,EQ,C’RPT.’),
PUSH=(31:6,8))
OPTION EQUALS
SORT FIELDS=(31,8,CH,A)
OUTFIL INCLUDE=(31,8,CH,EQ,C’FRANK’,OR,
31,8,CH,EQ,C’SRIHARI’),BUILD=(1,30)
Each report starts with ‘RPT.reptname’ in positions 2-13…
1RPT.SRIHARI
LINE 1 FOR REPORT 1 SRIHARI
LINE 2 FOR REPORT 1 SRIHARI
...
1RPT.VICKY VICKY
LINE 1 FOR REPORT 2 VICKY
LINE 2 FOR REPORT 2 VICKY
...
We use a SORT statement to sort ascending on the reptname…
1RPT.DAVID DAVID
LINE 1 FOR REPORT 4 DAVID
LINE 2 FOR REPORT 4 DAVID
...
1RPT.FRANK FRANK
LINE 1 FOR REPORT 3 FRANK
LINE 2 FOR REPORT 3 FRANK
...
1RPT.SRIHARI SRIHARI
LINE 1 FOR REPORT 1 SRIHARI
LINE 2 FOR REPORT 1 SRIHARI
...
We use an OUTFIL statement to only INCLUDE the records with a reptname of FRANK or SRIHARI…
1RPT.FRANK
LINE 1 FOR REPORT 3
LINE 2 FOR REPORT 3
...
1RPT.SRIHARI
LINE 1 FOR REPORT 1
LINE 2 FOR REPORT 1
OUTER JOIN Queries are a valuable tool in SQL, allowing you to retrieve data from…
The SQL JOIN operation enables combining rows from two or more tables based on a…
The Product Owner role has shifted from just being a requirements proxy to a strategic,…
Business Value: In the world of Agile development, the user story has long been the…
The SAFe Scrum Master certification has become one of the most sought-after credentials for Agile…
The Professional Scrum with Kanban (PSK) course enhances your organization's ability to deliver value efficiently…
Effective User interviews play a crucial role in Scrum methodology, helping Product Owners and Scrum…
Product Owners should be well-versed in various user research tools and techniques to effectively understand…