Select Records by Example

It’s Plumber Time again… I needed a way to select records based on the first record in the stream (as the selection for a recursive pipeline – more about that later). A popular way to do this is using peekto in a REXX stage to retrieve the key, and then build a selection stage based on that key.

'peekto rec'
key = word(rec,2)
'callpipe (end \) *: | pick w2 == ,'key', | *:'

Although this works, there are some places where the pipe can leak, like when the “key” field contains the character you use as the delimiter of the string (you can fix that by encoding it in hex). But it did not meet my subjective criteria for elegant plumbing…

What I came up with is to use juxtapose to combine the key of the first record with the key of the remaining records, and then have pick select the ones where both keys are identical (implicitly matching the first record obviously). Trying to do this with the live data (prefixing the record with the key) tends to get a bit tricky, so another case where predselect comes handy.

The following is the core of my QBE REXX filter. The argument for the stage is the input range of the record that is used for the matching. The idea is that qbe w3 will select all input records where the third word matches that of the first record.

The spec stage takes care of extracting the specified range and keeps it as a hexadecimal string, so pick can simply compare words to select input records.

parse arg range                                                        
                                                                       
'addpipe (end \ name QBE.REXX:6)',                                     
   '\ *: ',                                                            
   '| o: fanout ',                    /* Original records to pass    */
   '| p: predselect',                                                  
   '| *:',                                                             
   '\ o:',                                                             
   '| spec' range 'c2x 1 , , n ',     /* Extract range as a token    */
   '| z: fanout',                                                      
   '| take',                          /* Keep the first token        */
   '| j: juxtapose',                  /* .. and combine with other   */
   '| x: pick w1 == w2',              /* When identical, pass to     */
   '| p:',                            /* .. predselect for output 0  */
      out1,                                                            
   '\ x:',                            /* Not identical, to           */
   '| p:',                            /* .. predselect to reject     */
   '\ z: ',                                                            
   '| j:'                             /* To juxtapose for pairing    */
                                                                       
return rc * ( rc  12 )                                               

The variable "out1" in the code is set only when the secondary output of the stage is connected. This makes the stage behave like most built-in selection stages. When the secondary output is not connected, the rejected records will be silently dropped. The pre-amble of the stage thus looks like this:

'streamstate all x'                                                    
if words(x) = 1 then out1 = ''        /* Check for secondary output  */
else                                                                   
  do                                                                   
    out1 = '| *.output.1:'                                             
    parse var x . s .                                                  
    parse var s si ':' so .                                            
    call msg si < 12, 1196            /* Reject with secondary input */
  end                                                                  

The error message is issued when the secondary input is connected. The code for the message routine uses the ISSUEMSG Pipeline command.

msg:                        
  parse arg cond, id        
  if cond then              
    do                      
      'ISSUEMSG' id 'ROBQEB'
      exit id               
    end                     
  return