BATCH DATA COMMUNICATION

BATCH DATA COMMUNICATION


Data Transfer –
Implementing a new software system requires moving data from the present system i.e., legacy system into the R3 system. Data transfer program is an effective and efficient way of transferring large amount of data into the new system, saves time and resources. But more importantly it ensures that accurate data is transferred into R/3.

Two steps are involved in data transfer –

• Conversion- Data is converted from legacy system into required flat file format.
• SAP Data Transfer- Data is automatically entered into the system. A SAP data transfer program reads the prepared data from the flat file and moves it into R/3.

Steps for any data transfer-
• Analysis and cleanup of data in the non-SAP system.
• Extraction of data from the non-SAP system.
• Mapping the data in SAP format.
• Transferring the data to the SAP System.
• Checking the data.


There are three different options to enter data into R/3-
• Automatically, with SAP standard data transfer programs.
• Automatically, by creating own branch input programs
• Manually, by entering the data via the corresponding online transaction



Automatic transfer with a standard data transfer program
This can be done if:
• A standard program exists for the data transfer of a business object in
R/3.
• The data is available in electronic form.
• There is a significant number of records to transfer.
• The cost of converting the legacy data into the required flat file format is acceptable.

Manually transferring business objects
This can be done if:
• No legacy system is available.
• There is only small number of records to enter.
• Translating legacy data into the R/3 structure is more an effort than
manually entering the data.

Using customer specific batch input to transfer business objects
This can be done if:
• No standard program exists to transfer the business object in R/3.
• The data is available in electronic form.
• There is a significant number of records you to transfer.
• Translating legacy data into the structure required by custom
program is easier than manually entering data.

Batch input is a standard procedure for transferring large amount of data into
the R/3 system. It simulates manual data entry. Data consistency is ensured
because batch input uses all the checks conducted on the normal screen.
Using batch input is like entering the data online. Another advantage to
batch input is that there is no need to check the data in advance.

Batch input is a two-step procedure.
It involves a program that creates the batch input session. This session is the data file that includes everything to begin the transaction and the data to be entered on the appropriate screens .The data is not yet in the database tables of R/3 application.
The second steps to process the session, which then actually transfers the data to database table.


Batch Data Communication-

In the process of transferring data, following situations might occur –
• The first is when application data to be replaced is transferred at
once and only once.
• The second situation is to transfer data periodically from external
systems to SAP system and vice versa.
• There is a period of time when information has to be transferred
from existing application, to SAP R/3, and often this process will be
repetitive.

The SAP system offers two primary methods for transferring data into SAP systems from non-SAP systems or legacy system. These methods are collectively called “Batch Input” or “Batch Data Communication”.

The BDC can transfer data from one SAP System to another SAP System or can transfer data from non-SAP System to SAP System too. BDC can be only be used for uploading data into R/3 and so it is not bi-directional. BDC works on the principle of simulating user input for transactional screen, via an ABAP program. Typically the input comes in the form of a flat file. The ABAP program reads this file and formats the input data screen by screen into an internal table (BDCDATA). The transaction is then started using this internal table as the input and executed in the background.
BDC allows you to perform database updates in the background using standard SAP transactions. The resultant entries will be as if the user had manually entered the data via SAP. This means that there is no bypassing of any of the standard SAP consistency checks, authorizations etc.

Advantages offered by Batch Input method:
• Can process large data volumes in batch.
• Can be planned and submitted in the background.
• No manual interaction is required when data is transferred.
• Data integrity is maintained as whatever data is transferred to the table is through transaction. Hence batch input data is submitted to all the checks and validations.

Data Transfer Program –

The batch input program must build all of the input to execute the SAP
transaction.
Two main steps are required:
• To build an internal table containing every screen and every field to be filled in during the execution of an SAP transaction.
• To pass the table to SAP for processing.

Prerequisite for Data Transfer Program
Writing a Data Transfer Program involves following prerequisites:
• Analyzing data from local file
• Analyzing transaction


Analyzing transaction involves following steps:
• The transaction code.
• Which fields require input i.e., mandatory.
• Which fields can be allowed default to standard values.
• The names, types, and lengths of the fields that are used by a transaction.
• Screen number and Name of module pool program behind a particular
transaction.

Once the program name, screen number, field name (screen field
name) are known , the Data Transfer program can be written.
• Declaring internal table
First Internal table similar to structure like local file.
• Declaring internal table like BDCDATA

The data from internal table is not transferred directly to database table, it
has to go through transaction. We need to pass data to particular screen and
to particular screen-field. Data is passed to transaction in particular format,
hence there is a need for batch input structure.
The batch input structure stores the data that is to be entered into SAP
system and the actions that are necessary to process the data. The batch
input structure is used by all of the batch input methods. The same structure can be used for all types of batch input, regardless of whether a session is created in the batch input queue or CALL TRANSACTION is used.

This structure is BDCDATA, which can contain the batch input data for only a single run of a transaction. Hence the processing in a program is done in a loop as follows:
• Create a BDCDATA structure
• Write the structure out to a session or process it with CALL TRANSACTION USING

Within a BDCDATA structure, organize the data of screens in a transaction.
Each screen that is processed in the course of a transaction must be
identified with a BDCDATA record. This record uses the Program, Dynpro, and Dynbegin fields of the structure.
The screen identifier record is followed by a separate BDCDATA record for
each value, to be entered into a field. These records use the FNAM and FVAL fields of the BDCDATA structure. Values to be entered in a field can be any of the following:
• Data that is entered into screen fields.
• Function codes that are entered into the command field. Such function codes execute functions in a transaction, such as Save or Enter.


The BDCDATA structure contains the following fields:
• Program : Name of module pool program associated with the screen. Set this field only for the first record for the screen.
• Dynpro : Screen Number. Set this field only in the first record for the screen.
• Dynbegin : Indicates the first record for the screen. Set this field to X, only for the first record for the screen. (Reset to ‘ ‘ (blank) for all other records.)
• Fnam : Field Name. The FNAM field is not case-sensitive.
• Fval : Value for the field named in FNAM. The FVAL field is case-sensitive. Values assigned to this field are always padded on the right, if they are less than 132 characters. Values must be in character format.

Data is uploaded to internal table by UPLOAD of WS_UPLOAD or GUI_UPLOAD function. Population of BDCDATA for each record of internal table, you need to populate Internal table, which is similar to BDCDATA structure.
All these five initial steps are necessary for any type of BDC interface.
Data Transfer program can call ‘Session Method’ or ‘Call Transaction’.
The initial steps for both the methods are same.
First step for both the methods is to upload the data to internal
table. From Internal Table, the data is transferred to database table
by two ways i.e., Session method and Call transaction.

Session Method –

In this method data is transferred from internal table to database table
through sessions. An ABAP/4 program reads the external data that is to be entered in the SAP System , synchronous processing takes place and stores the data in session , which is to be handled through SM35. Through this method the data can be read by the BDC program from a sequential dataset file. A session stores the actions that are required to enter your data using normal SAP transaction i.e., Data is transferred to session which in turn transfers data to database table.
Session is intermediate step between internal table and database table. Data along with its action is stored in session i.e., data for screen fields, to which screen it is passed, the program name behind it, and how the next screen is processed.
When the program has finished generating the session, the session can be executed to run the SAP transactions in it. The session can be explicitly started and monitored or run in the background processing system. Unless session is processed, the data is not transferred to database table.

BDC_OPEN_GROUP -
Sessions can be created through program by BDC_OPEN_GROUP function.
Parameters to this function are:
• User Name: User name
• Group: Name of the session
• Lock Date: The date on which you want to process the session.
• Keep: This parameter is passed as ‘X’ when session is to be retained
after processing or ‘ ‘ when it is to be deleted.

BDC_INSERT –
This function transfers data to the session.
Parameters to this function are:
• Tcode: Transaction Name
• Dynprotab: BDC Data

BDC_CLOSE_GROUP –
This function closes the BDC Group.
No Parameters.

Some additional information for session processing –

When the session is generated using the KEEP option within the
BDC_OPEN_GROUP, the system always keeps the sessions in the queue,
whether it has been processed successfully or not.
However, if the session is processed, it has to be deleted manually. When session processing is completed successfully while KEEP option was not set, it will be removed automatically from the session queue. Log is not removed for that session.
If the batch-input session is terminated with errors, then it appears in the list of INCORRECT session and it can be processed again. To correct incorrect session , the session can be analyzed. The Analysis function allows to determine which screen and value has produced the error. If there are small errors in data, they can be corrected interactively, otherwise batch input program, which has generated the session or many times even the data file has to be modified.

Call Transaction –

Asynchronous processing takes place and the transactions are triggered at the time of processing itself and so the ABAP program must do the error handling and messages have to be captured at runtime.
In this method, a transaction can be called from the program by
Call transaction using
Mode
Update
Parameter –1 is transaction code.
Parameter –2 is name of BDCTAB table.
Parameter –3 here the execution mode for the transaction is spexcified
A is all screen mode. All the screen of transaction are displayed .
N is no screen mode. No screen is displayed when the transaction is executed.
E is error screen. Only those screens are displayed have error records.
Parameter –4 here update type for the database table updation is specified.
S is for Synchronous update in which data of one table is changed then all the related Tables gets updated. And sy-subrc is returned i.e., sy-subrc is returned for once and all.
A is for Asynchronous update in which data of one table is changed,
the sy-subrc is returned. And then updating of other
affected tables takes place. So if system fails to update other
tables, still sy-subrc returned is 0 (i.e., when first table gets
updated).
Parameter – 5 when updating database table takes place, operation is either
successful or unsuccessful or operation is successful with some
warning. These messages are stored in internal table, which is specified along with MESSAGE statement. This internal table should
be declared like BDCMSGCOLL, a structure available in ABAP/4.

It contains the following fields:
Tcode : Transaction code
Dyname: Batch point module name
Dynumb: Batch input Dyn number
Msgtyp : Batch input message type (A/E/W/I/S)
Msgspra: Batch input Lang, id of message
Msgid : Message id
MsgvN : Message variables (N = 1 - 4)

For each entry, which is updated in database, table message is available in
BDCMSGCOLL. As BDCMSGCOLL is structure, a internal table is declared which can contain multiple records (unlike structure).

Steps for Call Transaction method:
• Internal table for the data (structure similar to the local file)
• BDCTAB like BDCDATA
• UPLOAD or WS_UPLOAD function to upload the data
from local file to itab. (Considering file is local file)
• Loop at itab.
• Populate BDCTAB table.
• Call transaction using
Mode
Update .
• Refresh BDCTAB.
• Endloop.


Difference between Batch Input and Call Transaction in BDC –
Session method -
1) Synchronous processing.
2) Can transfer large amount of data.
3) Processing is slower.
4) Error log is created.
5) Data is not updated until session is processed.
Call transaction.-
1) Asynchronous processing.
2) Can transfer small amount of data.
3) Processing is faster.
4) Errors need to be handled explicitly.
5) Data is updated automatically.
Error Handling in CALL TRANSACTION-
When Session Method updates the records in database table, error records are stored in the log file. In Call transaction there is no such log file available and error record is lost unless handled. Usually there is a need to give report of all the error records i.e., records which are not inserted or updated in the database table. This can be done by the following method:

Steps for the error handling in CALL TRANSACTION:
1. Internal table for the data (structure similar to your local
file)
2. BDCTAB like BDCDATA
3. Internal table BDCMSG like BDCMSGCOLL
4. Internal table similar to 1st internal table
(Third and fourth steps are for error handling)
5. UPLOAD or WS_UPLOAD function to upload the data from
the local file to itab. (Considering file is local file)
6. Loop at itab.
Populate BDCTAB table.
Call transaction using
Mode
Update
Messages .
Perform check.
Refresh BDCTAB.
Endloop.
7. Form check.
IF sy-subrc <> 0. (Call transaction returns the sy-subrc if
updating is not successful).
Call function Format_message.
(This function is called to store the message given by
system and to display it along with record)
Append itab2.
Display the record and message.

Direct Input –

In contrast to batch input, this technique does not create sessions, but stores the data directly. It does not simulate the online transaction. To enter the data into the corresponding database tables directly, the system calls a number of function modules that execute any necessary checks. In case of errors, the direct input technique provides a restart mechanism. However, to be able to activate the restart mechanism, direct input programs must be executed in the background only. Direct input checks the data thoroughly and then updates the database directly.

Common batch input errors -
• The batch input BDCDATA structure tries to assign values to fields which do not exist in the current transaction screen.
• The screen in the BDCDATA structure does not match the right sequence, or an intermediate screen is missing.
• On exceptional occasions, the logic flow of batch input session does not exactly match that of manual online processing. Testing the sessions online can discover by this.
• The BDCDATA structure contains fields, which are longer than the actual definition.
• Authorization problems.

Recording a Batch Input-

A BI recording allows to record a R/3 transaction and generate a program that contains all screens and field information in the required
BDC-DATA format.
SHDB transaction can be used for recording .