Sub : Need to delete data...? for enhancing datasource..?

No need to delete any data in BW or R/3 side for enhancing datasource if loading data into ODS in overwrite mode.

Simply follow the below steps:

1. Add new fields to ODS & cubes and adjust update rules.
2. Clear LBWQ(update queue - Run V3 job only).
3. Clear RSA7(Delta queue - Run infopak to pull data into bw)
4. Move datasource changes to Production, replicate and activate transfer rules.
5. Delete data in LBWG(setup tables).
6. Fill setup tables for historic data.
7. Initialize datasource if required, without data transfer(Zero initialization).
8. Pull data from R/3 to BW ODS in overwrite mode with Repair Full option.
Loading data in overwrite mode, so no problem, Just load again historic data as well. and push delta from ODS to further(ods/cube).
9. Push delta from ODS to CUBE.

More info @ https://www.sdn.sap.com/irj/sdn/thread?threadID=624282&messageID=4393416#4393416

Questions and answers related to T-Code: RSA7(Delta Queue)

This note maintained here for my quick reference and for those dont have SAP Notes access :-)

Question 1:
What does the number in the 'Total' column in Transaction RSA7 mean?
Answer:
The 'Total' column displays the number of LUWs that were written in the delta queue and that have not yet been confirmed. The number includes the LUWs of the last delta request (for repeating a delta request) and the LUWs for the next delta request. An LUW only disappears from the RSA7 display when it has been transferred to the BW System and a new delta request has been received from the BW System.

Question 2:
What is an LUW in the delta queue?
Answer:
An LUW from the point of view of the delta queue can be an individual document, a group of documents from a collective run or a whole data packet from an application extractor.

Question 3:
Why does the number in the 'Total' column, in the overview screen of Transaction RSA7, differ from the number of data records that are displayed when you call up the detail view?
Answer:
The number on the overview screen corresponds to the total number of LUWs (see also question 1) that were written to the qRFC queue and that have not yet been confirmed. The detail screen displays the records contained in the LUWs. Both the records belonging to the previous delta request and the records that do not meet the selection conditions of the preceding delta init requests are filtered out. This means that only the records that are ready for the next delta request are displayed on the detail screen. The detail screen of Transaction RSA7 does not take into account a possibly existing customer exit.

Question 4:
Why does Transaction RSA7 still display LUWs on the overview screen after successful delta loading?
Answer:
Only when a new delta has been requested does the source system learn that the previous delta was successfully loaded into the BW System. The LUWs of the previous delta may then be confirmed (and also deleted). In the meantime, the LUWs must be kept for a possible delta request repetition. In particular, the number on the overview screen does not change if the first delta is loaded into the BW System.

Question 5:
Why are selections not taken into account when the delta queue is filled?
Answer:
Filtering according to selections takes place when the system reads from the delta queue. This is necessary for performance reasons.

Question 6:
Why is there a DataSource with '0' records in RSA7 if delta exists and has been loaded successfully?
Answer:
It is most likely that this is a DataSource that does not send delta data to the BW System via the delta queue but directly via the extractor . You can display the current delta data for these DataSources using TA RSA3 (update mode ='D')

Question 7:
Do the entries in Table ROIDOCPRMS have an impact on the performance of the loading procedure from the delta queue?
Answer:
The impact is limited. If performance problems are related to the loading process from the delta queue, then refer to the application-specific notes (for example in the CO-PA area, in the logistics cockpit area, and so on).
Caution: As of PlugIn 2000.2 patch 3, the entries in Table ROIDOCPRMS are as effective for the delta queue as for a full update. Note, however, that LUWs are not split during data loading for consistency reasons. This means that when very large LUWs are written to the delta queue, the actual package size may differ considerably from the MAXSIZE and MAXLINES parameters.

Question 8:
Why does it take so long to display the data in the delta queue (for example approximately 2 hours)?
Answer:
With PlugIn 2001.1 the display was changed: you are now able to define the amount of data to be displayed, to restrict it, to selectively choose the number of a data record, to make a distinction between the 'actual' delta data and the data intended for repetition, and so on.

Question 9:
What is the purpose of the function 'Delete Data and Meta Data in a Queue' in RSA7? What exactly is deleted?
Answer:
You should act with extreme caution when you use the delete function in the delta queue. It is comparable to deleting an InitDelta in the BW System and should preferably be executed there. Not only do you delete all data of this DataSource for the affected BW System, but you also lose all the information concerning the delta initialization. Then you can only request new deltas after another delta initialization.
When you delete the data, this confirms the LUWs kept in the qRFC queue for the corresponding target system. Physical deletion only takes place in the qRFC outbound queue if there are no more references to the LUWs.
The delete function is intended for example, for cases where the BW System, from which the delta initialization was originally executed, no longer exists or can no longer be accessed.

Question 10:
Why does it take so long to delete from the delta queue (for example half a day)?
Answer:
Import PlugIn 2000.2 patch 3. With this patch the performance during deletion improves considerably.

Question 11:
Why is the delta queue not updated when you start the V3 update in the logistics cockpit area?
Answer:
It is most likely that a delta initialization had not yet run or that the the delta initialization was not successful. A successful delta initialization (the corresponding request must have QM status 'green' in the BW System) is a prerequisite for the application data to be written to the delta queue.

Question 12:
What is the relationship between RSA7 and the qRFC monitor (Transaction SMQ1)?
Answer:
The qRFC monitor basically displays the same data as RSA7. The internal queue name must be used for selection on the initial screen of the qRFC monitor. This is made up of the prefix 'BW, the client and the short name of the DataSource. For DataSources whose name is shorter than 20 characters, the short name corresponds to the name of the DataSource. For DataSources whose name is longer than 19 characters (for delta-capable DataSources only possible as of PlugIn 2001.1) the short name is assigned in Table ROOSSHORTN.
In the qRFC monitor you cannot distinguish between repeatable and new LUWs. Moreover, the data of a LUW is displayed in an unstructured manner there.

Question 13:
Why is there data in the delta queue although the V3 update has not yet been started?
Answer:
You posted data in the background. This means that the records are updated directly in the delta queue (RSA7). This happens in particular during automatic goods receipt posting (MRRS). There is no duplicate transfer of records to the BW system. See Note 417189.

Question 14:
Why does the 'Repeatable' button on the RSA7 data details screen not only show data loaded into BW during the last delta but also newly-added data, in other words, 'pure' delta records?
Answer:
It was programmed so that the request in repeat mode fetches both actually repeatable (old) data and new data from the source system.

Question 15:
I loaded several delta inits with various selections. For which one
is the delta loaded?
Answer:
For delta, all selections made via delta inits are summed up. This
means a delta for the 'total' of all delta initializations is loaded.

Question 16:
How many selections for delta inits are possible in the system?
Answer:
With simple selections (intervals without complicated join conditions or single values), you can make up to about 100 delta inits. It should not be more.
With complicated selection conditions, it should be only up to 10-20 delta inits.
Reason: With many selection conditions that are joined in a complicated way, too many 'where' lines are generated in the generated ABAP source code which may exceed the memory limit.

Question 17:
I intend to copy the source system, i.e. make a client copy. What will happen with may delta? Should I initialize again after that?
Answer:
Before you copy a source client or source system, make sure that your deltas have been fetched from the delta queue into BW and that no delta is pending. After the client copy, an inconsistency might occur between BW delta tables and the OLTP delta tables as described in Note 405943. After the client copy, Table ROOSPRMSC will probably be empty in the OLTP since this table is client-independent. After the system copy, the table will contain the entries with the old logical system name which are no longer useful for further delta loading from the new logical system. The delta must be initialized in any case since delta depends on both the BW system and the source system. Even if no dump 'MESSAGE_TYPE_X' occurs in BW when editing or creating an InfoPackage, you should expect that the delta has to be initialized after the copy.

Question 18.
Am I permitted to use the functions in Transaction SMQ1 to manually control processes?
Answer:
Use SMQ1 as an instrument for diagnosis and control only. Make changes to BW queues only after informing BW Support or only if this is explicitly requested in a note for Component 'BC-BW' or 'BW-WHM-SAPI'.

Question 19.
Despite the delta request only being started after completion of the collective run (V3 update), it does not contain all documents. Only another delta request loads the missing documents into BW. What is the cause for this "splitting"?
Answer:
The collective run submits the open V2 documents to the task handler for processing. The task handler processes them in one or several parallel update processes in an asynchronous way. For this reason, plan a sufficiently large "safety time window" between the end of the collective run in the source system and the start of the delta request in BW. An alternative solution where this problem does not occur is described in Note 505700.

Question 20.
Despite deleting the delta init, LUWs are still written into the DeltaQueue
Answer:
In general, delta initializations and deletions of delta inits should always be carried out at a time when no posting takes place. Otherwise, buffer problems may occur: If you started the internal mode at a time when the delta initialization was still active, you post data into the queue even though the initialization had been deleted in the meantime. This is the case in your system.

Question 21.
In SMQ1 (qRFC Monitor) I have status 'NOSEND'. In the Table TRFCQOUT, some entries have the status 'READY', others 'RECORDED'. ARFCSSTATE is 'READ'. What do these statuses mean? Which values in the field 'Status' mean what and which values are correct and which are alarming? Are the statuses BW-specific or generally valid in qRFC?
Answer:
Table TRFCQOUT and ARFCSSTATE: Status READ means that the record was read once either in a delta request or in a repetition of the delta request. However, this still does not mean that the record has successfully reached the BW. The status READY in the TRFCQOUT and RECORDED in the ARFCSSTATE means that the record has been written into the delta queue and will be loaded into the BW with the next delta request or a repetition of a delta. In any case only the statuses READ, READY and RECORDED in both tables are considered to be valid. The status EXECUTED in TRFCQOUT can occur temporarily. It is set before starting a delta extraction for all records with status READ present at that time. The records with status EXECUTED are usually deleted from the queue in packages within a delta request directly after setting the status before extracting a new delta. If you see such records, it means that either a process which confirms and deletes records loaded into the BW is successfully running at the moment, or, if the records remain in the table for a longer period of time with status EXECUTED, it is likely that there are problems with deleting the records which have already been successfully been loaded into the BW. In this state, no more deltas are loaded into the BW. Every other status indicates an error or an inconsistency. NOSEND in SMQ1 means nothing (see note 378903). However the value 'U' in field 'NOSEND' of table TRFCQOUT is of concern.

Question 22.
The extract structure was changed when the delta queue was empty. Afterwards new delta records were written to the delta queue. When loading the delta into the PSA, it shows that some fields were moved. The same result occurs when the contents of the delta queue are listed via the detail display. Why is the data displayed differently? What can be done?
Answer:
Make sure that the change of the extract structure is also reflected in the database and that all servers are synchronized. We recommend resetting the buffers using Transaction $SYNC. If the extract structure change is not communicated synchronously to the server where delta records are being created, the records are written with the old structure until the new structure has been generated. This may have disastrous consequences for the delta. When the problem occurs, the delta needs to be re-initialized.

Question 23.
How and where can I control whether a repeat delta is requested?
Answer:
Via the status of the last delta in the BW Request Monitor. If the request is RED, the next load will be of type 'Repeat'. If you need to repeat the last load for any reason, manually set the request in the monitor to red. For the contents of the repeat, see Question 14. Delta requests set to red when data is already updated lead to duplicate records in a subsequent repeat, if they have not already been deleted from the data targets concerned.

Question 24.
As of PI 2003.1, the Logistic Cockpit offers various types of update methods. Which update method is recommended in logistics? According to which criteria should the decision be made? How can I choose an update method in logistics?
Answer:
See the recommendation in Note 505700.

Question 25.
Are there particular recommendations regarding the maximum data volume of the delta queue to avoid danger of a read failure due to memory problems?
Answer:
There is no strict limit (except for the restricted number area of the 24-digit QCOUNT counter in the LUW management table - which is of no practical importance, however - or the restrictions regarding the volume and number of records in a database table).
When estimating "soft" limits, both the number of LUWs and the average data volume per LUW are important. As a rule, we recommend bundling data (usually documents) as soon as you write to the delta queue to keep number of LUWs low (this can partly be set in the applications, for example in the Logistics Cockpit). The data volume of a single LUW should not be much larger than 10% of the memory available to the work process for data extraction (in a 32-bit architecture with a memory volume of about 1 GByte per work process, 100 MByte per LUW should not be exceeded). This limit is of rather small practical importance as well since a comparable limit already applies when writing to the delta queue. If the limit is observed, correct reading is guaranteed in most cases.
If the number of LUWs cannot be reduced by bundling application transactions, you should at least make sure that the data is fetched from all connected BWs as quickly as possible. But for other, BW-specific, reasons, the frequency should not exceed one delta request per hour.
To avoid memory problems, a program-internal limit ensures that no more than 1 million LUWs are ever read and fetched from the database per delta request. If this limit is reached within a request, the delta queue must be emptied by several successive delta requests. We recommend, however, to try not to reach that limit but trigger the fetching of data from the connected BWs as soon as the number of LUWs reaches a 5-digit value.

---> Some more related Notes....
873694 - Consulting: Delta repeat and status in monitor/data target
771894 - No data during delta upload: Selection on Z* fields
723935 - Adding the TID display to the DeltaQueue monitor
691721 - Restoring lost data from a delta request
576896 - Checks when PSA contains incorrect data for delta requests
574601 - BW-SAPI: Endless loop when confirming qRFC LUWs
417307 - Extractor package size: Collective note for applications
417189 - BW/SAPLEINS - Online update of delta queue
405943 - Calling an InfoPackage in BW causes short dump
377732 - Collective SAP note SAP BW BCT 2.1C for EBP 2.0 and 3.0

Requirement may come up to add new fields to LO cockpit extractor which is up & running in production environment. This means the extractor is delivering daily deltas from SAP R/3 to BW system .Since this change is to be done in R/3 Production system, there is always a risk that daily deltas of LO cockpit extractor would get disturbed. If the delta mechanism is disturbed (delta queue is broken) then there no another way than doing re-initialization for that extractor. However this re-init is not easy in terms of time & resource. Moreover no organization would be willing to provide that much downtime for live reporting based on that extractor.
As all of us know that initialization of LO Extractor is critical, resource intensive & time consuming task. Pre-requisites to perform fill setup tables are - we need to lock the users from transactional updates in R/3 system, Stop all batch jobs that update the base tables of the extractor. Then we need to schedule the setup jobs with suitable date ranges/document number ranges.
We also came across such scenario where there was a requirement to add 3 new fields to existing LO cockpit extractor 2LIS_12_VCITM. Initialization was done for this extractor 1 year back and the data volume was high.We adopted step by step approach to minimize the risk of delta queue getting broken /disturbed. Hope this step by step procedure will help all of us who have to work out similar scenarios.
Step by Step Procedure:-
1.Carry out changes in LO Cockpit extractor in SAP R/3 Dev system.As per the requirement add new fields to Extractor.These new fields might be present in standard supporting structures that you get when you execute "Maintain Data source" for extractor in LBWE. If all required fields are present in supporting structure mentioned above then just add these fields using arrow buttons provided and there is no need to write user exit code to populate these new fields.However if these fields (or some of the required fields) are not present in supporting structures then you have to go for append structure and user exit code. The coding in user exit is required to populate the newly added fields. You have to write ABAP code in User exit under CMOD & in Include ZXRSAU01.All above changes will ask you for transport request. Assign appropriate development class/Package and assign all these objects into a transport request.
2.Carry out changes in BW Dev system for objects related to this change.Carry out all necessary changes in BW Dev system for objects related to this change (Info source, transfer rules, ODS, Info cubes, Queries & workbooks). Assign appropriate development class/Package and assign all these objects into a transport request
3.Test the changes in QA system.Test the new changes in SAP R/3 and BW QA systems. Make necessary changes (if needed) and include them in follow-up transports.
4.Stop V3 batch jobs for this extractor.V3 batch jobs for this extractor are scheduled to run periodically (hourly/daily etc) Ask R/3 System Administrator to put on hold/cancel this job schedule.
5.Lock out users, batch jobs on R/3 side & stop Process chain schedule on BW.In order to avoid the changes in database tables for this extractor and hence possible risk of loss of data, ask R/3 System Administrator to lock out the users. Also batch job schedule need to be put on hold /cancel.Ask System Administrator to clear pending queues for this extractor (if any) in SMQ1/SMQ2. Also pending /error out v3 updates in SM58 should be processed.On BW production system the process chain related to delta Info package for this extractor should be stopped or put on hold.
6.Drain the delta queue to Zero for this extractor.Execute the delta Info package from BW and load the data into ODS & Info cubes. Keep executing delta Info package till you get 0 records with green light for the request on BW side. Also you should get 0 LUW entries in RSA7 for this extractor on R/3 side.
7.Import R/3 transports into R/3 Production system.In this step we import R/3 transport request related to this extractor. This will include user exit code also. Please ensure that there is no syntax error in include ZXRSAU01 and it is active. Also ensure that objects such as append structure is active after transport.
8.Replicate the data source in BW system.On BW production system, replicate the extractor (data source).
9.Import BW transport into BW Production system.In this step we import BW transport related to this change into BW Production system.
10.Run program to activate transfer rulesExecute program RS_TRANSTRU_ACTIVATE_ALL. Enter the Info source and source system name and execute. This will make sure that transfer rules for this Info source are active
11.Execute V3 job Manually in R/3 sideGo to LBWE and click on Job Control for Application area related to this extractor (for 2LIS_12_VCITM it is application 12). Execute the job immediately and it should finish with no errors.
12.Execute delta Info package from BW systemRun delta Info package from BW system. Since there is no data update, this extraction request should be green with zero records (successful delta extraction)
13.Restore the schedule on R/3 & BW systemsAsk System Administrator to resume V3 update job schedule, batch job schedule and unlock the users. On BW side, restore the process chains schedule.From next day onwards (or as per frequency), you should be able to receive the delta for this extractor with data also populated for new fields.

BI Performance Tuning

-- Contains all information related to BI performance tuning.

Performance Tuning for SAP BW : pdf

Performance Tuning for Queries with Aggregates : pdf

BEx Front-End Performance: Performance Analyses of BEx Analyzer and Web Application in the Local and Wide Area Networks Environment : pdf

SAP BW Data Load Performance Analysis and Tuning : e learning

-- Need SDN User ID and Password to log-on.