Data Staging for SAP Conversion

Purpose

The white paper lists and describes steps needed to perform data conversion for SAP from the staging point of view.

Goals and Objectives

The goal is to enable the staging developers during Data conversion process and help them verify the data converted before it goes into SAP.

Pre-requisites for Developers

The goal is to enable the staging developers during Data conversion process and help them verify the data converted before it goes into SAP.

Developers working on the Staging part should have the following pre requisites.

• Knowledge of Oracle
  • SQL
  • PL/SQL
  • SQL Loader Utility
• Knowledge of MS Excel
• Familiarity with TOAD

 Overview

Data conversion process involves converting/processing the data received from the legacy applications/systems and transforming it into desired output structure of SAP as per the specifications provided.

Functional specification documents provide details about the business rules that are to be applied on the data before it goes into SAP. Staging team does the work of applying those business rules by writing stored procedures in ORACLE.

Pictorial representation of Data conversion process:

 Data conversion process

Steps to be followed

• Understanding and analyzing the requirements of conversion
• Development of procedures for conversion
• Import process
• Processing data
• Versioning
• Exception Reports

Understanding and analyzing the requirements of conversion

• Understanding business rules (from specs) to be applied and analyzing data received from legacy systems
• Getting clarifications from the client on any of the doubts that may arise from step 1
• Preparing a mapping document (using MS excel); listing all the input fields from the data received and output fields as per the structure required for a particular conversion
• Ensuring that the mapping done is correct by getting it verified from a business user

Development of procedures for conversion

Once a mapping document is in place and has been verified by business users, the actual development of code and import process of data is initiated for a particular conversion.

Import process

SQL Loader utility is used to import data into ORACLE tables. Data received from a legacy system could be in various forms. For e.g. it could either be in a tab delimited, CSV or can even be in an excel spreadsheet format. It is important to review the data received is in correct format and ensure that no field / information is missing. Process to import data may differ depending upon the format in which data has been received.

Steps in general that are needed to be performed for the import process:

In case the data received is in tab delimited format, MS Excel can be used to prepare the file before it is used with the SQL Loader tool. Although data can be imported to ORACLE as is, but sometimes the data received is not directly importable into ORACLE tables and has to be converted into a format which is acceptable to the ORACLE.

Select Import Data option available under Data->Import External Data menu option.

• Step 1 will open up an Open file dialog window
• Select a file (Excel, tab delimited or CSV format) containing data using an appropriate path and follow the steps of the Import wizard

Note: An important thing that has to be kept in mind before importing the data is to set the format of all the cells to text. Otherwise, a value larger in size or one with leading zeros might create a problem. Leading zeros get trimmed and a larger value does not get displayed properly in a cell or doesn’t get exported properly.

After the import process completes successfully, a file can be saved as a tab delimited file through MS Excel only. The tab delimited file can then be used by SQL Loader utility to load the data into ORACLE tables.

Processing data

Processing the data imported is one of the major steps in data conversion process. It’s quite possible that in a real life scenario, changes to the code might be required during the course of data conversion for a particular module in the procedures created for processing the data.

Creating separate versions of code

One way to protect and maintain the code written for a particular type of conversion is to write a new stored procedure. That’s the way it’s been mostly done while there have been changes that we were to make in our code during the process of conversion in Wave I. Maintaining separate versions of code like this is a tedious process especially without an integration of a version management tool with the development environment.


Example:

Here is an example of a data conversion process for a module (Sales Order), where there was a requirement to create a separate set of data for all the orders belonging to plant in Canada. In this kind of a scenario, we had created a separate procedure to process the records for Canadian plant. During this conversion process, there have been number of changes, which were done quite frequently to each of these two separate procedures and the nature of changes that were to be made were also different for each of the plants. In a real life scenario, it becomes very difficult to maintain separate versions and also keep track of the changes being made on continuous basis to these different sets of code.

Processed Data

Once the processing is complete, the data has to be delivered to the SAP.

Data Version

Versions of the data are maintained as the data goes into different environments in SAP. It’s required because of the following reasons:

• Delta loads – When a delta load has to be sent for a particular version, it becomes important to know what all data has already been sent so that no duplicate records go into SAP
• Identifying any incorrect data that might have been sent to a particular environment of SAP because of any reason. This helps in keeping track whether incorrect data was sent by Legacy, Staging or it something went wrong at SAP’s end only

Generating Exception records reports

Generating an exception report is one of the most crucial steps in the process of data conversion. An exception report is a log of those records which could not be processed due to the business rules applied as per the functional specs.

An exception report helps the staging team in reporting the users/developers of the legacy system to identify the problems at their end and resolve them and re-send the data to the staging team to process it, so that the same could be uploaded in SAP too. A typical format of an exception report is a collection of raw data fields along with a reason as why those could not get processed. This is sent usually in an MS excel format.

Also, count of records processed and the records that came as part of the raw data is maintained and communicated to the business users/legacy system team to find out how much data got loaded and how much of it failed to load.

Read More

AUTOMATION OF BW QUERIES USING WEB BROWSER

In the fiercely increasing competition amongst corporations it has become mandatory to make quick and sound crucial business decisions based on analysis of business critical data. This is the point where SAP BW comes into play. BW Queries are the core of the reporting and analysis functionality in SAP BW. They provide a flexible and intuitive platform for data analysis that can be developed using the SAP Business Explorer (BEx) Query Designer.


AUTOMATION OF BW QUERIES – OVERVIEW


The Automation of queries in SAP BW is require where the data from BW is extracted by reporting analyst using the BW queries and send to business frequently on a regular basis for their analysis for example monthly or weekly.


Queries using web browser can be configured to be run:
Only Once: We can schedule the BW queries to be run on a specific day or time to be refreshed and sent to the email id of the recipients.
Scheduled Basis: We can schedule the BW queries to be run on a regular basis i.e. every day, specific day of a week or monthly basis.


This document outlines a set of guidelines to automate a query for a given scenario.


PROCEDURE TO AUTOMATE THE QUERIES USING WEB BROWSER:


Step 1: Log in SAP BW using the User Name and Password. Enter the RRMX as transaction code into the transaction window and hit the Enter. New Excel window will open with the additional BEx (Business Explorer) tool bar.

Step 2: Open a query which you want to automate through BEx and refresh the same.

Step 3: Launch the query in web browser.

Tool Button –> Launch in Web Browser –> Query (Default View)

When you click on the Query (Default View) it will pop up the message box asking the user name and password.

While you enter the user name and password it will populate the query in web browser.

Step 4: Modify the output structure of the query according to your requirement using the Variable screen at the left side of the window

Applying filters


If you want to apply any filter on the query then you can do that using filter icon against the characteristics. (Ex. If you have to produce the report for particular business unit only then click on the filter icon in front of Master Cost Centers it will pop up another window. Select the Business unit as per your requirement and click on transfer at the bottom left.)

If you want to set the properties of any characteristic then right click on that it will pop up window and select the properties. It will pop up another properties window that will allow you to change the properties of the characteristics.

Step 5: Information Broadcasting: When you are done with the formatting of the query click on the Information Broadcasting and mention the email addresses of the recipients whom the report needs to be sent then click on Create new Setting with the Wizard.

Step 6: Select on the check box as Zip File, it will create the Zip file for the report then click on continue at the bottom.

Step 7: Enter the subject line for the report and enter the E-mail content in the content window and continue at the bottom.

Step 8: Insert the technical name and description for the automated report.

Step 9: Schedule the report: Select the Create New Scheduling and Periodic All and enter the date which you want to schedule the report for and the schedule as weekly or monthly. Mention the date and time on which you want the report to be sent in front of Next Start at and click on the transfer below.

If you want to execute the report at the time of creation then click on the Execute button at the bottom otherwise close that. Now the report is scheduled for the period you mentioned this will automatically refresh the query in the same format you created on the scheduled date and E-mail this report to the recipients with the report attached in Zip format.
MODIFICATION IN SCHEDULE OF AUTOMATED QUERIES:-
If you want to make changes in the schedule then select the query and click on the Schedule button at the bottom it will pop up the window just create the new schedule and delete the old as well and click on transfer.

The BEx Web query automation is a standalone, comfortable Web application for data analysis that the user can call up using an URL or as an iView in the Enterprise Portal. The Open a query which you want to automate through BEx and offers a wide range of functions for data analysis via various tab pages and the associated view-specific toolbars.
SAP query automation is an information modeling tool which can be used to analyze business data. To help analyze data in a more user friendly manner, SAP BW used to describe any kind of numeric information from a business process level. There are various ways to describe key figures in SAP BW.
Essentially, a query is a database research action with interesting additional functionality like currency scenarios, complex calculation options, and analysis functions. However, analyses born out of queries can be applied flexibly to a multitude of areas in a multidimensional dataset of an SAP BW Info Provider. This is enabled by combining analytical functionality with the provided drilldowns and filter options. Therefore, one query or few queries can often map an entire analytical application.

Read More

ARIS FOR SAP NETWEAVER

Business process management (BPM) allows you to continuously adapt your business processes to new business strategies – modeling processes with different users performing different roles and tasks, and optimizing communication between process owners and IT experts. SAP and IDS Scheer offer a comprehensive BPM solution – SAP NetWeaver and ARIS for SAP NetWeaver. This joint application provides essential elements of a closed-loop BPM solution, from design and configuration, to implementation and execution, to evaluation of the overall process.
There are lots of Business Process Management tools available in market, then why should we choose ARIS for modeling our business processes. Here are some of the features and benefits of ARIS which make this tool different from others.

• ARIS Platform is Highly Scalable and hence complete solution for the Entire Business process Lifecycle – From Web Based Description, analysis, and optimization of business processes and software engineering to SAP Netweaver integration and continuous process controlling.
• ARIS is not just for SAP NetWeaver. ARIS can be used for all SAP initiatives — to design, implement, and monitor SAP, non-SAP and manual processes in an organization.
• The system architecture of the ARIS Platform allows globally active companies to set up distributed scenarios for designing, analyzing and optimizing process, IT and software architectures.
• Business Process Management with SAP NetWeaver and ARIS for SAP NetWeaver provides procedure models, methods, technologies and reference content for modeling, configuring, executing and monitoring these business processes.
• With ARIS Implementation Platform, gap between business and IT is closed. ARIS for SAP Netweaver helps design the process architecture for SAP Solutions that have been optimally adapted to the company’s business process.
• Adds functions to SAP NetWeaver for graphically modeling processes at various levels. At the highest level (Process Architecture Model), the process architecture of a company is built from a purely business perspective, that is, without technical details.

User-Friendly Functionality:

ARIS Business Architect and ARIS Business Designer Web Based Solution enable shared, company wide modeling, analysis, and optimization of business processes, as well as IT Architecture set up via the Internet. With its user-friendly functionality that allows even unskilled occasional users to perform modeling. ARIS Business designer quickly paves the path to professional BPM.

Synchronization between ARIS for SAP Netweaver and SAP Solution Manager:

You can use ARIS for SAP NetWeaver as a modeling environment to import predefined process models as reference content from SAP Solution Manager. You can enhance these models and synchronize them with SAP Solution Manager.
Process models and other configuration elements can be exchanged (synchronized) between ARIS for SAP NetWeaver and SAP Solution Manager. In addition to scope information, structural information is also transferred.
The SAP Solution Manager provides you with the relevant SAP reference models. You can create an implementation project using ARIS for SAP NetWeaver and then synchronize it using the SAP Solution Manager. This enables you, for example, to make SAP reference models available in ARIS for SAP NetWeaver, and adapt and enhance them as required. Finally, you can make the implementation project available again in the SAP Solution Manager, where you can then configure the processes and adapt them to your specific system landscape. The SAP Solution Manager also provides you with an extensive range of functions for monitoring your solutions.

Upload of ARIS Processes to SAP XI:

With the ARIS BPEL export, ARIS BPEL models can be transferred to SAP XI. The elements to be exported must be exported from ARIS Business architect for SAP Netweaver. ARIS creates ZIP files, which can be imported into SAP XI.

Interfaces and development of Add-ons:

ARIS UML Designer is included in the Implementation Platform as a tool for;
Developing add-on functions and interfaces during the implementation. With the addition of classic UML methodology to ARIS, a consistent and uniform visual representation of the requirements of the business world can be obtained. The UML models created in ARIS can be sent via XML to other development tools integrated in Eclipse, which are then available as source code or as J2EE software components from ARIS. This means that software developers can work with (UML) data from ARIS directly in the development environment.

Globally Distributed Process Design with ARIS:

The system architecture of the ARIS platform allows globally active companies to set up distributed scenarios for designing, analyzing and optimizing process, IT and software architectures.
Web-based products such as ARIS Business Architect, ARIS Business Designer and ARIS UML Designer access a centrally set up and managed ARIS Business Server from locations around the world via three-tier architecture. These products are designed for use beyond firewall limits, with a very low bandwidth (e.g. ISDN).

Next Generation of BPM: ARIS embedded in SAP NetWeaver:

In the next SAP NetweaverTM release, the design, modeling and model based configuration will take place in a technically integrated solution, a unified modeling environment as part of the ESR in SAP NetWeaver. There, on the basis of a unified metamodel, users will be able to perform various different role-specific tasks in a unified modeling environment.
SAP AG and IDS Scheer AG leverage the strength of their technologies together to cover the whole business process lifecycle. SAP NetWeaver today offers a comprehensive BPM solution that will be enhanced in future releases providing a unified modeling environment and a powerful combination of Business Process Management and Business Activity Monitoring.

ARIS for SAP NetWeaver contains a consistent description of the process architecture-from enterprise process models to implementation of the processes by SAP Solution Manager, the integration of executable processes in SAP XI, and the applications with SAP Business Workflow.

Some of the key benefits of using ARIS at a glance would be :-

• Get a clear understanding of your process architecture as a starting point for SAP ESA.
• Alignment of business, configuration and service process in one common repository.
• Intuitive and user friendly user interface
• Decentralized Design For centralized optimization
• Available 24 hours a day, 7 days a week and in all locations due to the Web Based Front end.
• Proven , extendable methods for various areas of application
• Highly scalable architecture.

Read More

Living with the Enemy

Just imagine two people who hate each other to death, being forced to live in the same house and continue inflicting pain on each other in every possible way. Do you get the picture?  That is what is happening between Oracle and SAP. To Larry Ellison, there is no other enemy more worse than SAP, and for SAP (even though they downplay it)  the number one in their enemy list is Oracle.

When SAP hates Oracle this much, SAP can not stop being major sales channel for Oracle’s database. Oracle  is still the number one database in the SAP customer base. It’s also no secret that many SAP customers would love to stop paying a premium price for a database that is functionally underused by the SAP product line.

But now there is a hope to SAP,  a new reason that SAP would like to get rid of the traditional relationship between its software and database leader Oracle. If SAP can develop its application to be less dependent on disk database, that  would be the first  step to reduce the dependency.

Speed is Money

That was what was proclaimed  by Mr. Plattner in the Sapphire conference in his key note address. He said the new world of in-memory computing is the next in-thing in the enterprise software.  In-memory database system (IMDS) is designed explicitly for real-time applications and for embedded systems such as set-top boxes, telecom equipment, consumer electronics and other connected gear. The in-memory database minimizes RAM and CPU demands and offers unmatched performance, reliability and development flexibility.

Enterprise software companies could learn from the techniques used by gaming software developing where the in-memory database  usage is already making big impact to get the maximum output from the multi-core CPUs. Mr. Plattner did not promise that SAP is developing in-memory concept into SAP product but he made it very clear that it is the way forward.

Oracle Killer

The desire to kill Oracle is not new found for SAP. As early as 2005, Shai Agassi, the then president of product technology group and member of SAP executive board, elaborated about the company’s programs to improve the in-memory capability of the software. In-memory capability is a new way to manage the data. Largest databases in the world are data warehouses, and these databases get the most complicated queries that they need to process as fast as possible. This requires enormous amount of CPU power. The testing ground for SAP’s new database strategy can be found in a product the company released few years back – the BI Accelerator or BIA. Among its many attributes, BIA is a hardware appliance that runs SAP Analytics incredibly quickly. The potential “Oracle killer” part of BIA comes from its in-memory database functionality, which processes database queries in RAM with no need to access disk-based storage — or even have a relational database at all — which means no costly database license, no massive data-storage farm, no expensive DBAs, and so on.

The idea of in-memory query management at SAP is hardly new. Back in the late 1990s, SAP unveiled LiveCache, an in-memory processor for what was then called the Business Warehouse. LiveCache was a little ahead of its time for lots of reasons, starting with the fact that CPU and memory costs were still relatively high for what SAP had in mind. In the end, LiveCache failed to live up to expectations. But it still survived as the in-memory query engine for SAP’s Advanced Planner and Optimizer (APO) supply-chain product.

LiveCache made history in SAP benchmarking, giving an indication of the response times that are possible using an in-memory database engine. Based on SAP’s own benchmarking standard — the SAP Standard Applications Benchmark — SAP’s hardware partners have had a glorious time leapfrogging each other in recent years to see which could achieve the best response times with LiveCache.

So, It is more of the question of when the killer will arrive. Someday soon, we will have a choice to choose between the status quo and new radical database approach. What will you choose if the newer approach is cheaper and faster and effective?

Read More about  IMDS

Read More

Delivering – Probe SAP

I acted as a midwife assisting to bring ‘Innovation’ mother’s baby – ‘Probe SAP’ to the world. I was the delivery coach and saw every step of the gestation period and how much the mother takes care to bring the baby to the world. It was an uphill task but as with all of our mothers, ‘Innovation’ pulled it through. Kudos to mother Innovation.

Now the midwife’s view. I had the privilege to introduce the baby to the world… What a big expectation it was from different corners for this baby! Fortunately in this case, it was possible to tailor make the baby as per the world’s demand. Even with that flexibility, it was very difficult to meet the expectations. With the privilege also comes the responsibility to take the criticism… ‘I thought the baby would look like Shahrukh Khan, I thought the baby can fly like Superman, I thought the baby would run like Carl Lewis on the first day’ etc. these were the kind of expectations thrown at me during the baby introduction but I am happy the way, we as delivery team worked with expectations and explained to the public what the baby can do.

What the baby can do anyway …

It can analyze any SAP environment and list down all the customizations and come up with the impact of upgrade. The customizations could be anything, any type of objects – Probe recognizes 54 types of objects. The baby has two type of personalities. Inventizer – the uncommitted guy who does not look into the details but interested in getting an overall idea about the person he deals with. Inventizer will give a complete picture of an environment to give anyone an idea on what it takes to maintain or upgrade the environment. The other split personality is Analyzer – the finicky girl who wants to go into every detail and find fault in every little thing. Analyzer goes into the level of identifying the line number where the incompatible element exist to carry out the upgrade. With these, the baby has for sure taken the big leap into the world. Now it requires constant nurturing to turn it into a grown up person to tackle anything in the real world of SAP upgrade and maintenance.

Head mother of Hexaware’s Innovation team, Innovation lead Immanuel has done a great work in directing the team in this delivery. But for his passion and focus in the demanding environment with limited resources, it would be even more difficult child bearing. I do not want to mention any names fearing I will definitely miss out someone. Job well done to the whole innovation team of mothers involved in this.

The baby has taken the first step to run as Carl Lewis!

Read More