Parallel Processing for Huge Number of Records In Internal Table

In some situations, Processing huge number of records in Internal table could take a lot of time. This in turn reduces performance of the entire system.This is where the concept of Parallel Processing comes into the picture. This Blog explains how to use Parallel Processing technique to Process huge volume of data in a much efficient manner.


Basically we are going to split the entire data into small packets and then process them individually by using Function Modules and Work Processes.

1) Create a Custom RFC Function Module

*Import Parameter IM_TAB type CCRCTT_KNA1 (search table type of any table using “where used list” icon/button)

*Export Parameter EX_TAB type CCRCTT_KNA1 to export data through Internal Table

*In source code we will just push data from IM_TAB to EX_TAB (You can write your own logic here based on your requirements)

2. Create a Z/Y Program.

*Top_Include

REPORT zdev_paralell_procs.

DATA : lt_kna1 TYPE TABLE OF kna1, 

       lt_kna1_temp TYPE TABLE OF kna1,

       lt_kna1_final TYPE TABLE OF kna1,

       ls_kna1_final TYPE kna1,

       lv_free_wp TYPE i,

       lv_sucess TYPE i,

       lv_task(2) TYPE c.

*Main Program.

INCLUDE zdev_paralell_procs_top .     " Top Include

INCLUDE zdev_paralell_procs_f01 .     " FORM Include

START-OF-SELECTION.

  "Get_Data(Customers) from KNA1...

  SELECT * FROM kna1 UP TO 19000 ROWS INTO TABLE lt_kna1.    "You Can Fetch As Much You Whant

  "FM to get active work process...  (Go To T-code SM50 and SM51 to View and Process "Work Process"

  CALL FUNCTION 'SPBT_INITIALIZE'

    IMPORTING

      free_pbt_wps                   = lv_free_wp            "It will give the Number of Free Work Process

    EXCEPTIONS

      invalid_group_name             = 1

      internal_error                 = 2

      pbt_env_already_initialized    = 3

      currently_no_resources_avail   = 4

      no_pbt_resources_found         = 5

      cant_init_different_pbt_groups = 6

      OTHERS                         = 7.

  IF sy-subrc <> 0.

    MESSAGE text-001 TYPE 'S' DISPLAY LIKE 'E'.              "Message If No Free Work Process

  ENDIF.

  """Acess Half of the Work Process...                       

  lv_free_wp = lv_free_wp / 2.

  ""Total records in Internal Table...

*******@Dont'n Mind About "Inline Declaration :-)******

  DESCRIBE TABLE lt_kna1 LINES DATA(lv_records).

  "Split the Volume of Data divide by Free Work Process...

  DATA(lv_split) = lv_records / lv_free_wp.

  DATA(lv_split_temp) = lv_split.                            "Temporary Store

  "missed records...

"""11/2 = 5.5 It will take 5. So, will get missed record

  DATA(lv_missed) = lv_split * lv_free_wp.

  IF lv_records GT lv_free_wp.              "Only if the records greater than work process

    DO .

      IF lt_kna1 IS INITIAL.

        EXIT.

      ELSE.

        lv_task = sy-index.                                  "Storing a uniqe number for a task in FM        

        CONCATENATE 'D' lv_task INTO lv_task.                "uniqe number like 'D1'

        CONDENSE lv_task NO-GAPS.

        IF sy-index = 1 AND lv_records LT lv_missed.         "If total records less than lv_missed 

          lv_split = lv_split + ( lv_missed - lv_records ).  "difference add to lv_split

        ELSEIF sy-index = 1 AND lv_records GT lv_missed.     "If lv_missed is less than lv_records

          lv_split = lv_split + ( lv_records - lv_missed ).  "difference add to lv_split

        ELSE.

          lv_split = lv_split_temp.                          "store old value from temp.

        ENDIF.

        APPEND LINES OF lt_kna1 FROM 1 TO lv_split TO lt_kna1_temp.    "push part of the data to IT.

        CALL FUNCTION 'ZDEV_PARALELL_PROCS' STARTING NEW TASK lv_task DESTINATION IN GROUP DEFAULT

          PERFORMING get_data ON END OF TASK                 "With form routin 'Get_Data' using IT

          EXPORTING

            im_tab = lt_kna1_temp.                           "pass internal table

        IF sy-subrc = 0.

          CLEAR lt_kna1_temp.                                "clear temp IT

          DELETE lt_kna1 FROM 1 TO lv_split.                 "Clear part of data from IT KNA1

        ENDIF.

      ENDIF.

    ENDDO.

    ""Wait until processed data get..

    WAIT UNTIL lv_sucess = lv_free_wp.                     "Wait until every Work Process get processed

  ELSE.                                               

    CALL FUNCTION 'ZDEV_PARALELL_PROCS'

      EXPORTING

        im_tab = lt_kna1_temp

      IMPORTING

        ex_tab = lt_kna1_final.

  ENDIF.

*Form_Include

*&  Include           ZDEV_PARALELL_PROCS_F01

*&      Form  GET_DATA

*&---------------------------------------------------------------------*

FORM get_data USING lv_task.

  RECEIVE RESULTS FROM FUNCTION 'ZDEV_PARALELL_PROCS'

  IMPORTING                                       "Data getting back from FM

    ex_tab = lt_kna1_temp.

  lv_sucess = lv_sucess + 1.                      "Store total WP excecuted times value

  APPEND LINES OF lt_kna1_temp TO lt_kna1_final.  "Store data to final IT   

  CLEAR lt_kna1_temp.

ENDFORM.

NOTE : You can use TIMESTAMP to see the duration of execution. 

CLICK HERE to know more about WORK PROCESS SM50.