JCL

Header and Trailer – SORT Detail records between these

Understanding Header and Trailer Processing: In mainframe data processing, it’s essential to handle files that contain Header and Trailer records correctly. Standard sort utilities sort all records — including headers and trailers — unless explicitly handled. To avoid rearranging records post-sort, ICETOOL offers options that allow detail records to be sorted while preserving headers and trailers in their original positions.

What is Header-Detail-Trailer Structure?

A Header-Detail-Trailer file format is a structured layout where:

  • Header Record: Contains metadata such as creation date or file ID.
  • Detail Records: Core business records that need to be sorted or processed.
  • Trailer Record: Summary information like total record count or checksum.
File Structure with Header and Trailer
HEADER20250727001   <- Header record
1111AAAA            <- Detail record
3333CCCC            <- Detail record
2222BBBB            <- Detail record
TRAILER0003         <- Trailer record (3 detail records)

DFSORT Techniques for Header and Trailer Processing

Method 1: Using DATASORT Operator (Recommended)

DATASORT in ICETOOL is built to sort detail records while preserving headers and trailers:

//STEP1    EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//IN       DD DSN=INPUT.FILE,DISP=SHR
//OUT      DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//TOOLIN   DD *
  DATASORT FROM(IN) TO(OUT) HEADER TRAILER USING(CTL1)
/*
//CTL1CNTL DD *
  SORT FIELDS=(1,8,CH,A)
/*
How DATASORT Works:
  • Identifies the first record as Header.
  • Sorts only the Detail records.
  • Keeps the last record as Trailer.
  • Maintains the original structure without manual rearrangement.
Method 2: Using INREC with Sort Keys

This approach provides more control over header/trailer identification using sort keys:

//STEP1    EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT  DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN    DD *
  OPTION EQUALS
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(81:C'1')),
        IFTHEN=(WHEN=(1,6,CH,EQ,C'HEADER'),OVERLAY=(81:C'0')),
        IFTHEN=(WHEN=(1,7,CH,EQ,C'TRAILER'),OVERLAY=(81:C'2'))
  SORT FIELDS=(81,1,CH,A,1,8,CH,A)
  OUTREC BUILD=(1,80)
/*

Explanation:

  • Assigns sort key 81:C'0' for Header – sorted first.
  • Assigns 81:C'1' for Detail – sorted normally.
  • Assigns 81:C'2' for Trailer – sorted last.
  • Removes the temporary key using OUTREC BUILD=(1,80).
Method 3: Two-Step Process for Complex Scenarios

Use this approach when more control is needed over identifying records:

Step 1: Add Sequence Numbers & Create Symbol Table
//STEP01   EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.FILE,DISP=SHR
//TEMPIN   DD DSN=TEMP.FILE,DISP=(NEW,CATLG,DELETE)
//SYM      DD DSN=SYMBOL.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN    DD *
  OPTION COPY
  OUTFIL FNAMES=TEMPIN,OVERLAY=(81:SEQNUM,8,ZD)
  OUTFIL FNAMES=SYM,REMOVECC,NODETAIL,
         TRAILER1=('LASTREC,+',COUNT=(M11,LENGTH=8),80:X)
/*
Step 2: Use Symbols and Final Sort
//STEP02   EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SYMNAMES DD DSN=SYMBOL.FILE,DISP=OLD
//SORTIN   DD DSN=TEMP.FILE,DISP=OLD
//SORTOUT  DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN    DD *
  SORT FIELDS=(1,8,CH,A,81,8,ZD,A)
  INREC IFTHEN=(WHEN=(81,8,ZD,EQ,1),OVERLAY=(90:C'H')),
        IFTHEN=(WHEN=(81,8,ZD,EQ,LASTREC),OVERLAY=(90:C'T')),
        IFTHEN=(WHEN=(90,1,CH,NE,C'H',AND,90,1,CH,NE,C'T'),
                OVERLAY=(90:C'D'))
  SORT FIELDS=(90,1,CH,A,1,8,CH,A)
  OUTREC BUILD=(1,80)
/*

Advanced Techniques

Using SUBSET Operator

For removing header and trailer while preserving specific patterns:

//STEP1    EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//IN       DD DSN=INPUT.FILE,DISP=SHR
//OUT      DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//TOOLIN   DD *
  SUBSET FROM(IN) TO(OUT) INPUT REMOVECC
/*

Handling Multiple Headers and Trailers

When dealing with multiple header/trailer pairs in a single file:

//STEP1    EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT  DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN    DD *
  OPTION EQUALS
  INREC IFTHEN=(WHEN=INIT,OVERLAY=(100:SEQNUM,8,ZD)),
        IFTHEN=(WHEN=(1,4,CH,EQ,C'0000'),
                OVERLAY=(108:C'HEADER')),
        IFTHEN=(WHEN=(1,4,CH,EQ,C'9999'),
                OVERLAY=(108:C'TRAILER')),
        IFTHEN=(WHEN=(1,4,CH,EQ,C'0100'),
                OVERLAY=(108:C'DETAIL'))
  SORT FIELDS=(108,7,CH,A,100,8,ZD,A)
/*

SKIPREC and STOPAFT Techniques

Basic Header/Trailer Removal

To skip the first record (header) and stop before the last record (trailer):

//STEP1    EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT  DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN    DD *
  SORT FIELDS=(1,10,CH,A)
  SKIPREC=1
/*
//* Note: STOPAFT requires knowing exact record count
Dynamic Last Record Calculation

For dynamic trailer handling:

//STEP0100 EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT  DD DSN=&T1,DISP=(,PASS),SPACE=(TRK,(1,1),RLSE)
//SYSIN    DD *
  OPTION SKIPREC=1
  SORT FIELDS=COPY
  OUTREC FIELDS=(SEQNUM,8,ZD,START=0,INCR=1,80:X)
  OUTFIL NODETAIL,REMOVECC,
         TRAILER1=(C' OPTION COPY,SKIPREC=1,STOPAFT=',1,8,80:X)
/*
//STEP0200 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT DD SYSOUT=*
//SYSIN DD DSN=&T1,DISP=OLD
/*

OUTFIL Header and Trailer Techniques

Adding Headers and Trailers to Output
//STEP1    EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.FILE,DISP=SHR
//SORTOUT  DD DSN=OUTPUT.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN    DD *
  SORT FIELDS=(1,10,CH,A)
  OUTFIL FNAMES=SORTOUT,
         HEADER1=C'HEADER RECORD - SORTED DATA',
         TRAILER1=(C'TRAILER - TOTAL RECORDS: ',COUNT=(M11,LENGTH=8))
/*
Multiple Output Files with Different Headers
//STEP1    EXEC PGM=SORT
//SYSOUT   DD SYSOUT=*
//SORTIN   DD DSN=INPUT.FILE,DISP=SHR
//OUT1     DD DSN=OUTPUT1.FILE,DISP=(NEW,CATLG,DELETE)
//OUT2     DD DSN=OUTPUT2.FILE,DISP=(NEW,CATLG,DELETE)
//SYSIN    DD *
  SORT FIELDS=(1,10,CH,A)
  OUTFIL FNAMES=OUT1,
         INCLUDE=(20,1,CH,EQ,C'A'),
         HEADER1=C'TYPE A RECORDS',
         TRAILER1=(C'A-TYPE COUNT: ',COUNT)
  OUTFIL FNAMES=OUT2,
         INCLUDE=(20,1,CH,EQ,C'B'),
         HEADER1=C'TYPE B RECORDS',
         TRAILER1=(C'B-TYPE COUNT: ',COUNT)
/*

Best Practices and Tips

1. Always Use EQUALS Option

When preserving header/trailer order is critical:

OPTION EQUALS
2. Validate Record Counts

Always verify trailer record counts match detail records:

OUTFIL TRAILER1=(C'DETAIL COUNT: ',COUNT=(M11,LENGTH=8))
3. Handle Variable Length Records

For VB files, account for RDW in position calculations:

INREC IFTHEN=(WHEN=(5,6,CH,EQ,C'HEADER'),OVERLAY=(85:C'H'))
4. Test with Small Datasets

Always test complex header/trailer logic with small, known datasets before production.

5. Document Record Layouts

Clearly document the structure and position of identifiers:

  • Position 1-6: Record Type (HEADER/DETAIL/TRAILER)
  • Position 7-14: Date (YYYYMMDD)
  • Position 15-22: Control Number
  • Error Handling and Validation
Validate Header/Trailer Presence
//STEP1    EXEC PGM=ICETOOL
//TOOLMSG  DD SYSOUT=*
//DFSMSG   DD SYSOUT=*
//IN       DD DSN=INPUT.FILE,DISP=SHR
//TOOLIN   DD *
  COUNT FROM(IN) FIRST 1 LAST 1 USING(CTL1)
/*
//CTL1CNTL DD *
  INCLUDE COND=(1,6,CH,EQ,C'HEADER',OR,1,7,CH,EQ,C'TRAILER')
/*
Check for Missing Records
OUTFIL FNAMES=ERRORS,
       INCLUDE=(1,6,CH,NE,C'HEADER',AND,1,6,CH,NE,C'DETAIL',
               AND,1,7,CH,NE,C'TRAILER')

Examples

Scenario -1: Let’s say we want to SORT an input file by a section that is divided by a constant keyword or symbol or some unique identifier. We can use the WHEN=GROUP function, we can add group numbers after the end of the records. Then we can SORT by the group number and by the key. Finally, we can remove the group numbers.

//N141231A  JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),
//         NOTIFY=&SYSUID
//************************************************************
//STEP01 EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=... input file (FB/10)
//SORTOUT DD DSN=... output file (FB/10)
//SYSIN DD *
  INREC IFTHEN=(WHEN=GROUP,BEGIN=(1,5,CH,EQ,C'*****'),
         PUSH=(11:ID=8))
  SORT FIELDS=(11,8,ZD,A,1,5,CH,A)
  OUTREC BUILD=(1,10)
/*

Please note the above example is for the FB file… so no padding would occur.

//SYSIN DD *
  INREC IFTHEN=(WHEN=INIT,BUILD=(1,4,5:8X,13:5)),
        IFTHEN=(WHEN=GROUP,BEGIN=(13,5,CH,EQ,C'*****'),
                PUSH=(5:ID=8))
  SORT FIELDS=(5,8,ZD,A,13,5,CH,A)
  OUTREC BUILD=(1,4,5:13)
/*

Let’s say our input data is having multiple subgroups divided by different unique identifiers…

Input:
HDR 2021/04/10
001 23.05-
002 5213.75+
003 861.51+
004 753.90-
TRL T862143
HDR 2021/04/09
001 282.15+
002 8.00-
003 1496.28+
TRL T201576
HDR 2021/03/11
001 123.86+
002 98.07-
003 61.16+
TRL T031893
HDR 2021/04/09
001 106.27-
002 2308.00+
003 96.72+
004 206.99-
005 208.82-
TRL T201573
//N141231A  JOB (123),'XYZ',CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1),
//         NOTIFY=&SYSUID
//************************************************************
//STEP01 EXEC PGM=ICEMAN
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=... input file
//SORTOUT DD DSN=... output file
//SYSIN DD *
  IFTHEN=(WHEN=GROUP,BEGIN=(1,3,CH,EQ,C'HDR'),
         PUSH=(31:6,10,41:ID=8)),
  IFTHEN=(WHEN=(1,3,CH,EQ,C'HDR'),OVERLAY=(49:C'A')),
  IFTHEN=(WHEN=(1,3,CH,EQ,C'TRL'),OVERLAY=(49:C'C')),
  IFTHEN=(WHEN=NONE,OVERLAY=(49:C'B'))
  SORT FIELDS=(31,10,CH,A,
               41,8,ZD,A,
               49,1,CH,A,
               6,9,SFF,D)
  BUILD=(1,30)
/*
Output:
HDR 2008/03/11
001 123.86+
003 61.16+
002 98.07-
TRL T031893
HDR 2008/04/09
003 1496.28+
001 282.15+
002 8.00-
TRL T201576
HDR 2008/04/09
002 2308.00+
003 96.72+
001 106.27-
004 206.99-
005 208.82-
TRL T201573
HDR 2008/04/10
002 5213.75+
003 861.51+
001 23.05-
004 753.90-
TRL T862143

Scenario -2: This example illustrates how you can SORT and INCLUDE groups of FB records depending on a value in the first record of each group…

Input File

1RPT.SRIHARI
 LINE  1 FOR REPORT 1
 LINE  2 FOR REPORT 1
...
1RPT.VICKY
 LINE  1 FOR REPORT 2
 LINE  2 FOR REPORT 2
...
1RPT.FRANK
 LINE  1 FOR REPORT 3
 LINE  2 FOR REPORT 3
...
1RPT.DAVID
 LINE  1 FOR REPORT 4
 LINE  2 FOR REPORT 4
...
INREC IFTHEN=(WHEN=GROUP,BEGIN=(2,4,CH,EQ,C’RPT.’),
              PUSH=(31:6,8))
OPTION EQUALS
SORT FIELDS=(31,8,CH,A)
OUTFIL INCLUDE=(31,8,CH,EQ,C’FRANK’,OR,
                31,8,CH,EQ,C’SRIHARI’),BUILD=(1,30)

Each report starts with ‘RPT.reptname’ in positions 2-13…

1RPT.SRIHARI
 LINE  1 FOR REPORT 1   SRIHARI
 LINE  2 FOR REPORT 1   SRIHARI
...
1RPT.VICKY              VICKY
 LINE  1 FOR REPORT 2   VICKY
 LINE  2 FOR REPORT 2   VICKY
...

We use a SORT statement to sort ascending on the reptname…

1RPT.DAVID               DAVID
 LINE  1 FOR REPORT 4    DAVID
 LINE  2 FOR REPORT 4    DAVID
...
1RPT.FRANK               FRANK
 LINE  1 FOR REPORT 3    FRANK
 LINE  2 FOR REPORT 3    FRANK
...
1RPT.SRIHARI             SRIHARI
 LINE  1 FOR REPORT 1    SRIHARI
 LINE  2 FOR REPORT 1    SRIHARI
...

We use an OUTFIL statement to only INCLUDE the records with a reptname of FRANK or SRIHARI…

1RPT.FRANK
 LINE  1 FOR REPORT 3
 LINE  2 FOR REPORT 3
...
1RPT.SRIHARI
 LINE  1 FOR REPORT 1
 LINE  2 FOR REPORT 1
IFTHEN with SYNCSORT – Click Here SYNSORT Manual: Click Here
Admin

Share
Published by
Admin

Recent Posts

AI Product Owner : Use Cases with Examples and Tools

The Product Owner role has shifted from just being a requirements proxy to a strategic,…

7 months ago

Business Value: Crafting User Stories for Measurable Impact

Business Value: In the world of Agile development, the user story has long been the…

8 months ago

SAFe Scrum Master (SSM) Certification with FAQ and Tips

The SAFe Scrum Master certification has become one of the most sought-after credentials for Agile…

12 months ago

PSK Practice Exam Practice Mode Questions

The Professional Scrum with Kanban (PSK) course enhances your organization's ability to deliver value efficiently…

1 year ago

Effective User Interviews in Scrum Framework

Effective User interviews play a crucial role in Scrum methodology, helping Product Owners and Scrum…

1 year ago

User Research Tools and Techniques for Product Owners

Product Owners should be well-versed in various user research tools and techniques to effectively understand…

1 year ago