Thursday, July 28, 2011

Oracle Pivot, Can it be Dynamic?

It's usually quite simple to make a PIVOT clause dynamically determine in which columns to place the values. The number of columns (or pivot groups) needs to be known. I haven't yet seen a way to dynamically determine how many columns will be generated by a query.

First of all, here's a quick example of a regular PIVOT. Note that the pivot columns are all hard-coded into the query. "Abe" doesn't show up because his name wasn't in the "IN" list.

/* --- code --- */

   WITH 
     w_data  as
       ( select  2009 yr,  'Bob' nm,  72 score  from dual union all
         select  2010   ,  'Bob'   ,  84        from dual union all
         select  2011   ,  'Bob'   ,  61        from dual union all
         select  2009   ,  'Abe'   ,  27        from dual union all
         select  2009   ,  'Liz'   ,  94        from dual union all
         select  2010   ,  'Liz'   ,  93        from dual union all
         select  2011   ,  'Liz'   ,  81        from dual union all
         select  2010   ,  'Jan'   ,  93        from dual union all
         select  2011   ,  'Jan'   ,  98        from dual 
       )
   select  *
     from  w_data
    PIVOT  ( avg(score)  as av
           FOR  nm  IN
             ( 'Bob' as bob
             , 'Liz' as liz
             , 'Jan' as jan
             )
           )
   ;


/* --- Results --- */

     --  YR     BOB_AV   LIZ_AV   JAN_AV
     --  -----  -------  -------  -------
     --  2009   72       94
     --  2010   84       93       93
     --  2011   61       81       98



The trick is to give each value, on which we want to pivot, it's own unique rank. An analytic function will do this quite nicely. We'll call this 'prepared' data our "Pre-Pivot" data.

/* --- code --- */

   WITH 
     w_data  as
       ( select  2009 yr,  'Bob' nm,  72 score  from dual union all
         select  2010   ,  'Bob'   ,  84        from dual union all
         select  2011   ,  'Bob'   ,  61        from dual union all
         select  2009   ,  'Abe'   ,  27        from dual union all
         select  2009   ,  'Liz'   ,  94        from dual union all
         select  2010   ,  'Liz'   ,  93        from dual union all
         select  2011   ,  'Liz'   ,  81        from dual union all
         select  2010   ,  'Jan'   ,  93        from dual union all
         select  2011   ,  'Jan'   ,  98        from dual 
       )
   , w_pre_pivt  as
       ( select  yr
              ,  nm
              ,  dense_rank() over (order by nm) rnk
              ,  score
           from  w_data
       )
   select  *
     from  w_pre_pivt ;


/* --- Results --- */

     --  YR     NM    RNK   SCORE
     --  -----  ----  ----  ------
     --  2009   Abe   1     27
     --  2010   Bob   2     84
     --  2011   Bob   2     61
     --  2009   Bob   2     72
     --  2010   Jan   3     93
     --  2011   Jan   3     98
     --  2011   Liz   4     81
     --  2010   Liz   4     93
     --  2009   Liz   4     94



Now we'll do a standard PIVOT on the rank column that we just generated in the Pre-Pivot data. In this example we've chosen 5 columns -- if there are more than 5 names, we would only see the top 5 -- if there are less than 5 names in our data then we will see blank columns. We can code our query to create as many or as few columns as we need. Note that we also include the name as one of the measures -- this is so we can identify which pivot-group values belong to which name.

/* --- code --- */

   WITH 
     w_data  as
       ( select  2009 yr,  'Bob' nm,  72 score  from dual union all
         select  2010   ,  'Bob'   ,  84        from dual union all
         select  2011   ,  'Bob'   ,  61        from dual union all
         select  2009   ,  'Abe'   ,  27        from dual union all
         select  2009   ,  'Liz'   ,  94        from dual union all
         select  2010   ,  'Liz'   ,  93        from dual union all
         select  2011   ,  'Liz'   ,  81        from dual union all
         select  2010   ,  'Jan'   ,  93        from dual union all
         select  2011   ,  'Jan'   ,  98        from dual 
       )
   , w_pre_pivt  as
       ( select  yr
              ,  nm
              ,  dense_rank() over (order by nm) rnk
              ,  score
           from  w_data
       )
   select  *
     from  w_pre_pivt
    PIVOT  ( max(nm)     as nm  
           , avg(score)  as av
           FOR  rnk  IN ( 1, 2, 3, 4, 5 )
           )
   ;


/* --- Results --- */

     --  YR     1_NM   1_AV   2_NM   2_AV   3_NM   3_AV   4_NM   4_AV   5_NM   5_AV
     --  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----  -----
     --  2009   Abe    27     Bob    72                   Liz    94
     --  2010                 Bob    84     Jan    93     Liz    93
     --  2011                 Bob    61     Jan    98     Liz    81



And now, if we would prefer to format the output a little we can create a heading row and "union-all" it to the data. Note that the numeric values now need to be converted to character since we are tacking the name values together with the average value measures.

/* --- code --- */

   WITH 
     w_data  as
       ( select  2009 yr,  'Bob' nm,  72 score  from dual union all
         select  2010   ,  'Bob'   ,  84        from dual union all
         select  2011   ,  'Bob'   ,  61        from dual union all
         select  2009   ,  'Abe'   ,  27        from dual union all
         select  2009   ,  'Liz'   ,  94        from dual union all
         select  2010   ,  'Liz'   ,  93        from dual union all
         select  2011   ,  'Liz'   ,  81        from dual union all
         select  2010   ,  'Jan'   ,  93        from dual union all
         select  2011   ,  'Jan'   ,  98        from dual 
       )
   , w_pre_pivt  as
       ( select  yr
              ,  nm
              ,  dense_rank() over (order by nm) rnk
              ,  score
           from  w_data
       )
   , w_pivoted  as
       ( select  *
           from  w_pre_pivt
          PIVOT  ( max(nm)     as nm  
                 , avg(score)  as av
                 FOR  rnk  IN ( 1, 2, 3, 4, 5 )
                 )
       )
   select  NULL            as "Year/Name"
        ,  max( "1_NM" )   as "Prsn-1"
        ,  max( "2_NM" )   as "Prsn-2"
        ,  max( "3_NM" )   as "Prsn-3"
        ,  max( "4_NM" )   as "Prsn-4"
        ,  max( "5_NM" )   as "Prsn-5"
     from  w_pivoted
   UNION ALL
   select  yr
        ,  to_char( "1_AV" ) 
        ,  to_char( "2_AV" ) 
        ,  to_char( "3_AV" ) 
        ,  to_char( "4_AV" ) 
        ,  to_char( "5_AV" ) 
     from  w_pivoted
    order  by 1 nulls first
   ;


/* --- Results --- */

     --  Year/Name   Prsn-1   Prsn-2   Prsn-3   Prsn-4   Prsn-5
     --  ----------  -------  -------  -------  -------  -------
     --              Abe      Bob      Jan      Liz
     --  2009        27       72                94
     --  2010                 84       93       93
     --  2011                 61       98       81

Recursive WITH (simple factorial)

Here's a simple example of a 'Recursive WITH' clause.

When declaring a recursive WITH clause, you need to specify the variables that will be used in the sub-query (in essence, the columns that will be generated by the sub-query). This is done in parenthesis immediately after the name of the sub-query.

The sub-query consists of two queries connected by a "UNION ALL". The first query initializes the column values -- the first row of values. The second query then uses the previous row of values to calculate the next row of values. This continues while the WHERE clause of the second query is true.

The main query simply selects the values and the rows from the results of the recursive WITH as any regular query would select results from a sub-query.

In this example of a recursive WITH sub-query, the "n" column is simply a counter and the process is terminated when it reaches "8". The "val" column is generated by taking the previous "n" value, adding one to it then multiplying by the previous "val" value. The result is a table of numbers with their factorial values (n!).

/* ===== code ===== */  
                                                         
     WITH  
       factorial ( n, val )  as
         ( /*=== The Recursive WITH ===*/
           select  1 as n,  1 as val  
             from  dual
           UNION ALL 
           select  n+1 as n,  (n+1)*val as val
             from  factorial
            where  n < 8
         )
     select  n
          ,  val
       from  factorial
      order  by  n 
     ;
             
                                                                                  
/* --- Results --- */    
                                                         
     --  N   VAL   
     --  --  ------ 
     --  1   1     
     --  2   2     
     --  3   6     
     --  4   24    
     --  5   120   
     --  6   720   
     --  7   5040  
     --  8   40320 
                                                                                  

Calculate & Display a Date-Time Duration as Days-HH:MM:SS

Having two dates, there are various ways to determine the length of the time interval between them. In the following example we're going to display how many days, hours, minutes, and seconds are between the two date values. We'll suppose that if the dates are not in chronological order, then we'll define the interval to be negative.

The sample 'with' data in the examples below uses the TO_DATE function without a format string. To make the examples run, set your session's "NLS_date_format" parameter as follows (execute this command in your connected session):

Execute This:

     ALTER SESSION  set nls_date_format = 'YYYY-MM-DD HH24:MI:SS' ;



In this first example, note that we use the CAST function to change the date values to timestamp values. Subtracting two timestamp values gives us an interval. The EXTRACT function will then pull out the desired elements as shown in the result. So if your source data type is timestamp instead of date, then all you need to do is subtract and extract.

Also note that if you need to deal with dates that are out of chronological order, then your code will have to be a bit more precise. In the following examples this sign is calculated and dealt with in various ways.

/*===== code =====*/

     ALTER SESSION  set nls_date_format = 'YYYY-MM-DD HH24:MI:SS' ;

     with
       w_date_ranges  as 
         ( select  to_date('1964-05-16 23:48:56') dt1, to_date('1964-05-17 10:23:18') dt2  from dual union all
           select  to_date('1985-08-28 22:04:22')    , to_date('1985-08-28 19:15:07')      from dual union all
           select  to_date('1991-03-11 02:05:44')    , to_date('1991-03-11 01:49:38')      from dual union all
           select  to_date('1998-03-07 06:13:18')    , to_date('1998-03-08 06:13:17')      from dual union all
           select  to_date('2001-12-14 01:43:13')    , to_date('2001-12-13 01:43:13')      from dual union all
           select  to_date('2003-11-29 18:53:10')    , to_date('2003-11-30 18:53:11')      from dual union all
           select  to_date('2009-06-10 10:11:48')    , to_date('2009-06-24 09:23:55')      from dual union all
           select  to_date('2009-10-25 00:00:00')    , to_date('2009-10-05 18:47:23')      from dual union all
           select  to_date('2010-01-01 13:27:03')    , to_date('2012-04-13 01:00:27')      from dual
         )
     select  dt1
          ,  dt2
          ,  sign(dt2-dt1)                                                                  as sign
          ,  abs( extract( day     from cast(dt2 as timestamp) - cast(dt1 as timestamp)) )  as ddd
          ,  abs( extract( hour    from cast(dt2 as timestamp) - cast(dt1 as timestamp)) )  as hh
          ,  abs( extract( minute  from cast(dt2 as timestamp) - cast(dt1 as timestamp)) )  as mm
          ,  abs( extract( second  from cast(dt2 as timestamp) - cast(dt1 as timestamp)) )  as ss
       from  w_date_ranges
      order  by 1 ;


/*===== Results =====*/

     DT1                  DT2                  SIGN  DDD  HH  MM  SS 
     -------------------  -------------------  ----  ---  --  --  -- 
     1964-05-16 23:48:56  1964-05-17 10:23:18    1     0  10  34  22 
     1985-08-28 22:04:22  1985-08-28 19:15:07   -1     0   2  49  15 
     1991-03-11 02:05:44  1991-03-11 01:49:38   -1     0   0  16   6 
     1998-03-07 06:13:18  1998-03-08 06:13:17    1     0  23  59  59 
     2001-12-14 01:43:13  2001-12-13 01:43:13   -1     1   0   0   0 
     2003-11-29 18:53:10  2003-11-30 18:53:11    1     1   0   0   1 
     2009-06-10 10:11:48  2009-06-24 09:23:55    1    13  23  12   7 
     2009-10-25 00:00:00  2009-10-05 18:47:23   -1    19   5  12  37 
     2010-01-01 13:27:03  2012-04-13 01:00:27    1   832  11  33  24 



In this example the number of days is found by subtracting the two dates and truncating the decimal value. The TO_CHAR function is used to find the other elements. The difference between dates is added to the system date truncated to midnight. Then looking at the HH24, MI, and SS elements of the date with the TO_CHAR function gives us the results we found in the previous query.

/*===== code =====*/

     ALTER SESSION  set nls_date_format = 'YYYY-MM-DD HH24:MI:SS' ;

     with
       w_date_ranges  as 
         ( select  to_date('1964-05-16 23:48:56') dt1, to_date('1964-05-17 10:23:18') dt2  from dual union all
           select  to_date('1985-08-28 22:04:22')    , to_date('1985-08-28 19:15:07')      from dual union all
           select  to_date('1991-03-11 02:05:44')    , to_date('1991-03-11 01:49:38')      from dual union all
           select  to_date('1998-03-07 06:13:18')    , to_date('1998-03-08 06:13:17')      from dual union all
           select  to_date('2001-12-14 01:43:13')    , to_date('2001-12-13 01:43:13')      from dual union all
           select  to_date('2003-11-29 18:53:10')    , to_date('2003-11-30 18:53:11')      from dual union all
           select  to_date('2009-06-10 10:11:48')    , to_date('2009-06-24 09:23:55')      from dual union all
           select  to_date('2009-10-25 00:00:00')    , to_date('2009-10-05 18:47:23')      from dual union all
           select  to_date('2010-01-01 13:27:03')    , to_date('2012-04-13 01:00:27')      from dual
         )
     select  dt1
          ,  dt2
          ,  decode(sign(dt2-dt1),-1,'-','+')                               as sign
          ,  trunc(abs(dt2-dt1))                                            as ddd
          ,  to_number(to_char( trunc(sysdate)+abs(dt2-dt1), 'HH24' ))      as hh
          ,  to_number(to_char( trunc(sysdate)+abs(dt2-dt1), 'MI'   ))      as mm
          ,  to_number(to_char( trunc(sysdate)+abs(dt2-dt1), 'SS'   ))      as ss
          ,            to_char( trunc(sysdate)+abs(dt2-dt1), 'HH24:MI:SS')  as hhmmss
       from  w_date_ranges
      order  by 1 ;


/*===== Results =====*/

     DT1                  DT2                  SIGN  DDD  HH  MM  SS  HHMMSS  
     -------------------  -------------------  ----  ---  --  --  --  --------
     1964-05-16 23:48:56  1964-05-17 10:23:18    +     0  10  34  22  10:34:22
     1985-08-28 22:04:22  1985-08-28 19:15:07    -     0   2  49  15  02:49:15
     1991-03-11 02:05:44  1991-03-11 01:49:38    -     0   0  16   6  00:16:06
     1998-03-07 06:13:18  1998-03-08 06:13:17    +     0  23  59  59  23:59:59
     2001-12-14 01:43:13  2001-12-13 01:43:13    -     1   0   0   0  00:00:00
     2003-11-29 18:53:10  2003-11-30 18:53:11    +     1   0   0   1  00:00:01
     2009-06-10 10:11:48  2009-06-24 09:23:55    +    13  23  12   7  23:12:07
     2009-10-25 00:00:00  2009-10-05 18:47:23    -    19   5  12  37  05:12:37
     2010-01-01 13:27:03  2012-04-13 01:00:27    +   832  11  33  24  11:33:24



This example uses the MOD function to extract the hours, minutes, and seconds elements. Note that there are a few dates that would give us erroneous results if we didn't add in the "0.00001" -- there is a slight error in Oracle's calculations and this provides the correction.

/*===== code =====*/

     ALTER SESSION  set nls_date_format = 'YYYY-MM-DD HH24:MI:SS' ;

     with
       w_date_ranges  as 
         ( select  to_date('1964-05-16 23:48:56') dt1, to_date('1964-05-17 10:23:18') dt2  from dual union all
           select  to_date('1985-08-28 22:04:22')    , to_date('1985-08-28 19:15:07')      from dual union all
           select  to_date('1991-03-11 02:05:44')    , to_date('1991-03-11 01:49:38')      from dual union all
           select  to_date('1998-03-07 06:13:18')    , to_date('1998-03-08 06:13:17')      from dual union all
           select  to_date('2001-12-14 01:43:13')    , to_date('2001-12-13 01:43:13')      from dual union all
           select  to_date('2003-11-29 18:53:10')    , to_date('2003-11-30 18:53:11')      from dual union all
           select  to_date('2009-06-10 10:11:48')    , to_date('2009-06-24 09:23:55')      from dual union all
           select  to_date('2009-10-25 00:00:00')    , to_date('2009-10-05 18:47:23')      from dual union all
           select  to_date('2010-01-01 13:27:03')    , to_date('2012-04-13 01:00:27')      from dual
         )
     select  dt1
          ,  dt2
          ,  regexp_replace(to_char(dt2-dt1,'9999990.000000'),'^([^\.]*)\..*$','\1')  as ddd
          ,  trunc(mod( abs(dt2-dt1)*24       + 0.00001, 24 ))  as hh
          ,  trunc(mod( abs(dt2-dt1)*24*60    + 0.00001, 60 ))  as mm
          ,  trunc(mod( abs(dt2-dt1)*24*60*60 + 0.00001, 60 ))  as ss
       from  w_date_ranges
      order  by 1 ;


/*===== Results =====*/

     DT1                  DT2                   DDD  HH  MM  SS
     -------------------  -------------------  ----  --  --  --
     1964-05-16 23:48:56  1964-05-17 10:23:18     0  10  34  22
     1985-08-28 22:04:22  1985-08-28 19:15:07    -0   2  49  15
     1991-03-11 02:05:44  1991-03-11 01:49:38    -0   0  16   6
     1998-03-07 06:13:18  1998-03-08 06:13:17     1  23  59  59
     2001-12-14 01:43:13  2001-12-13 01:43:13    -1   0   0   0
     2003-11-29 18:53:10  2003-11-30 18:53:11     1   0   0   1
     2009-06-10 10:11:48  2009-06-24 09:23:55    13  23  12   7
     2009-10-25 00:00:00  2009-10-05 18:47:23   -19   5  12  37
     2010-01-01 13:27:03  2012-04-13 01:00:27   832  11  33  24



This last example just combines everything into one column:
/*===== code =====*/

     ALTER SESSION  set nls_date_format = 'YYYY-MM-DD HH24:MI:SS' ;

     with
       w_date_ranges  as 
         ( select  to_date('1964-05-16 23:48:56') dt1, to_date('1964-05-17 10:23:18') dt2  from dual union all
           select  to_date('1985-08-28 22:04:22')    , to_date('1985-08-28 19:15:07')      from dual union all
           select  to_date('1991-03-11 02:05:44')    , to_date('1991-03-11 01:49:38')      from dual union all
           select  to_date('1998-03-07 06:13:18')    , to_date('1998-03-08 06:13:17')      from dual union all
           select  to_date('2001-12-14 01:43:13')    , to_date('2001-12-13 01:43:13')      from dual union all
           select  to_date('2003-11-29 18:53:10')    , to_date('2003-11-30 18:53:11')      from dual union all
           select  to_date('2009-06-10 10:11:48')    , to_date('2009-06-24 09:23:55')      from dual union all
           select  to_date('2009-10-25 00:00:00')    , to_date('2009-10-05 18:47:23')      from dual union all
           select  to_date('2010-01-01 13:27:03')    , to_date('2012-04-13 01:00:27')      from dual
         )
     select  dt1
          ,  dt2
          ,  lpad( case when (dt2-dt1) < 0 then '- ' end 
                || case when trunc(abs(dt2-dt1)) >= 1 then '('||trunc(abs(dt2-dt1))||') ' end 
                || to_char(trunc(sysdate)+abs(dt2-dt1),'HH24:MI:SS')
                 , 18 
                 ) as "Interval: (days) time"
       from  w_date_ranges
      order  by 1 ;


/*===== Results =====*/

DT1                       DT2                       Interval: (days) time 
------------------------- ------------------------- --------------------- 
1964-05-16 23:48:56       1964-05-17 10:23:18                 10:34:22    
1985-08-28 22:04:22       1985-08-28 19:15:07               - 02:49:15    
1991-03-11 02:05:44       1991-03-11 01:49:38               - 00:16:06    
1998-03-07 06:13:18       1998-03-08 06:13:17                 23:59:59    
2001-12-14 01:43:13       2001-12-13 01:43:13           - (1) 00:00:00    
2003-11-29 18:53:10       2003-11-30 18:53:11             (1) 00:00:01    
2009-06-10 10:11:48       2009-06-24 09:23:55            (13) 23:12:07    
2009-10-25 00:00:00       2009-10-05 18:47:23          - (19) 05:12:37    
2010-01-01 13:27:03       2012-04-13 01:00:27           (832) 11:33:24