Performance problem in production; Please help me out

Hi all,

I'd really appreciate if someone can help me with this.

Every night, the server's SWAP, Sysadmin add more space disk swap every night and for the last 4 days.
I run ADDM report from 22:00 to 04:00 (when the server is running out of memory)
I had the problem of performance of this query:

RECOMMENDATION 4: SQL Tuning, 4.9% benefit (1329 seconds)
      ACTION: Investigate the SQL statement with SQL_ID "b7f61g3831mkx" for 
         possible performance improvements.
         RELEVANT OBJECT: SQL statement with SQL_ID b7f61g3831mkx and 
         PLAN_HASH 881601692

I can't find what the problem is and why it is a source of performance problem, could you help me please

*WORKLOAD REPOSITORY SQL Report*

Snapshot Period Summary

DB Name         DB Id      Instance     Inst Num Release     RAC Host        
------------ ----------- ------------ -------- ----------- --- ------------
****       1490223503 ****             1 10.2.0.1.0  NO  ****

              Snap Id      Snap Time      Sessions Curs/Sess
            --------- ------------------- -------- ---------
Begin Snap:      9972 21-Apr-10 23:00:39       106       3.6
  End Snap:      9978 22-Apr-10 05:01:04       102       3.4
   Elapsed:              360.41 (mins)
   DB Time:              451.44 (mins)

SQL Summary                         DB/Inst: ****/****  Snaps: 9972-9978

                Elapsed 
   SQL Id      Time (ms)
------------- ----------
b7f61g3831mkx  1,329,143
Module: DBMS_SCHEDULER
 GATHER_STATS_JOB
select /*+ no_parallel(t) no_parallel_index(t) dbms_stats cursor_sharing_exact u
se_weak_name_resl dynamic_sampling(0) no_monitoring */ count(*),count("P_PRODUCT
_ID"),count(distinct "P_PRODUCT_ID"),count("NAME"),count(distinct "NAME"),count(
"DESCRIPTION"),count(distinct "DESCRIPTION"),count("UPC"),count(distinct "UPC"),

          -------------------------------------------------------------       

SQL ID: b7f61g3831mkx               DB/Inst: ***/***  Snaps: 9972-9978
-> 1st Capture and Last Capture Snap IDs
   refer to Snapshot IDs witin the snapshot range
-> select /*+ no_parallel(t) no_parallel_index(t) dbms_stats cursor_shari...

    Plan Hash           Total Elapsed                 1st Capture   Last Capture
#   Value                    Time(ms)    Executions       Snap ID        Snap ID
--- ---------------- ---------------- ------------- ------------- --------------
1   881601692               1,329,143             1          9973           9974
          -------------------------------------------------------------       


Plan 1(PHV: 881601692)
---------------------- 

Plan Statistics                     DB/Inst: ***/***  Snaps: 9972-9978
-> % Total DB Time is the Elapsed Time of the SQL statement divided 
   into the Total Database Time multiplied by 100

Stat Name                                Statement   Per Execution % Snap 
---------------------------------------- ---------- -------------- -------
Elapsed Time (ms)                         1,329,143    1,329,142.7     4.9
CPU Time (ms)                                26,521       26,521.3     0.7
Executions                                        1            N/A     N/A
Buffer Gets                                 551,644      551,644.0     1.3
Disk Reads                                  235,239      235,239.0     1.5
Parse Calls                                       1            1.0     0.0
Rows                                              1            1.0     N/A
User I/O Wait Time (ms)                     233,212            N/A     N/A
Cluster Wait Time (ms)                            0            N/A     N/A
Application Wait Time (ms)                        0            N/A     N/A
Concurrency Wait Time (ms)                        0            N/A     N/A
Invalidations                                     0            N/A     N/A
Version Count                                     2            N/A     N/A
Sharable Mem(KB)                                 71            N/A     N/A
          -------------------------------------------------------------       

Execution Plan
---------------------------------------------------------------------------------------------------
| Id  | Operation               | Name    | Rows  | Bytes | Cost (%CPU)| Time     | Pstart| Pstop |
---------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT        |         |       |       | 24350 (100)|          |       |       |
|   1 |  SORT GROUP BY          |         |     1 |   731 |            |          |       |       |
|   2 |   PARTITION RANGE SINGLE|         |  8892 |  6347K| 24350   (1)| 00:04:53 |   KEY |   KEY |
|   3 |    PARTITION LIST ALL   |         |  8892 |  6347K| 24350   (1)| 00:04:53 |     1 |     5 |
|   4 |     TABLE ACCESS SAMPLE | PRODUCT |  8892 |  6347K| 24350   (1)| 00:04:53 |   KEY |   KEY |
---------------------------------------------------------------------------------------------------
 


Full SQL Text

SQL ID       SQL Text                                                         
------------ -----------------------------------------------------------------
b7f61g3831mk select /*+ no_parallel(t) no_parallel_index(t) dbms_stats cursor_
             _sharing_exact use_weak_name_resl dynamic_sampling(0) no_monitori
             ng */ count(*), count("P_PRODUCT_ID"), count(distinct "P_PRODUCT_
             ID"), count("NAME"), count(distinct "NAME"), count("DESCRIPTION")
             , count(distinct "DESCRIPTION"), count("UPC"), count(distinct "UP
             C"), count("ADV_PRODUCT_URL"), count(distinct "ADV_PRODUCT_URL"),
              count("IMAGE_URL"), count(distinct "IMAGE_URL"), count("SHIPPING
             _COST"), count(distinct "SHIPPING_COST"), sum(sys_op_opnsize("SHI
             PPING_COST")), substrb(dump(min("SHIPPING_COST"), 16, 0, 32), 1, 
             120), substrb(dump(max("SHIPPING_COST"), 16, 0, 32), 1, 120), cou
             nt("SHIPPING_INFO"), count(distinct "SHIPPING_INFO"), sum(sys_op_
             opnsize("SHIPPING_INFO")), substrb(dump(min(substrb("SHIPPING_INF
             O", 1, 32)), 16, 0, 32), 1, 120), substrb(dump(max(substrb("SHIPP
             ING_INFO", 1, 32)), 16, 0, 32), 1, 120), count("P_STATUS"), count
             (distinct "P_STATUS"), sum(sys_op_opnsize("P_STATUS")), substrb(d
             ump(min(substrb("P_STATUS", 1, 32)), 16, 0, 32), 1, 120), substrb
             (dump(max(substrb("P_STATUS", 1, 32)), 16, 0, 32), 1, 120), count
             ("EXTRA_INFO1"), count(distinct "EXTRA_INFO1"), sum(sys_op_opnsiz
             e("EXTRA_INFO1")), substrb(dump(min(substrb("EXTRA_INFO1", 1, 32)
             ), 16, 0, 32), 1, 120), substrb(dump(max(substrb("EXTRA_INFO1", 1
             , 32)), 16, 0, 32), 1, 120), count("EXTRA_INFO2"), count(distinct
              "EXTRA_INFO2"), sum(sys_op_opnsize("EXTRA_INFO2")), substrb(dump
             (min(substrb("EXTRA_INFO2", 1, 32)), 16, 0, 32), 1, 120), substrb
             (dump(max(substrb("EXTRA_INFO2", 1, 32)), 16, 0, 32), 1, 120), co
             unt("ANALISIS_DATE"), count(distinct "ANALISIS_DATE"), substrb(du
             mp(min("ANALISIS_DATE"), 16, 0, 32), 1, 120), substrb(dump(max("A
             NALISIS_DATE"), 16, 0, 32), 1, 120), count("OLD_STATUS"), count(d
             istinct "OLD_STATUS"), sum(sys_op_opnsize("OLD_STATUS")), substrb
             (dump(min("OLD_STATUS"), 16, 0, 32), 1, 120), substrb(dump(max("O
             LD_STATUS"), 16, 0, 32), 1, 120) from "PARTNER_PRODUCTS"."PRODUCT
             " sample ( 12.5975349658) t where TBL$OR$IDX$PART$NUM("PARTNER_PR
             ODUCTS"."PRODUCT", 0, 4, 0, "ROWID") = :objn

Dear friend,

Why do you think you have problems with shared pool? In the ASH report you are provided, there was just 2.5 medium active sessions and 170 requests during this period, he is very low with swimming pool shared, problems
you have some queries that use literals, it will be better to replace literals with bind variable if possible, or you can set the init cursor_sharing parameter to force or similar, this is the dynamic parameter.
But it is not so dramatic problem in your case!

From ASHES of your report, we can see that top wait events is "CPU + wait for CPU", "RMAN backup & recovery i/o" and "log file sync" and 65% of your database of waiting time. And even in the background waiting events.
If I understand well report, you have two members in your redo log groups, you have problems with log IO writer speed, check the distribution of files on the disc, newspaper editor is slow causing it to wait for the other sessions. High
processor can be related to rman compression. Best service can we sea GATHER_STATS_JOB consumes 16% of activity 33% consumes rman and only 21% your applications and also there is something running
SQL * more under the sys (?) account. There is from the top of the sql page, this is the sql in your application, if I understand correctly, 'scattered db reading file' event indicates that full scans have a place, is it normal that your application? If Yes, then try using
running in parallel, as we can see in the section "Sessions running PQs Top" your report there is no running in parallel, but as I understand it there are 8 processors, try to use parallel executions or avoid full scans. But consider that
When you do full scans in parallel PGA memory not used CMS, then decrees setting pga_aggregate_target SGA and increase respectively.

Is there another application or a program running on the server except oracle?
Is the performance degradation was strong, I mean yesterday, everything was ok, but today all the evil, or it was good?

Check the reasons for the slow newspaper writer, it can greatly affect performance.
Also of 90% of the performance problems generally because of the poor sql, poor execution plans.
Also if you use automatic memory management, tap Settings, but you must know that in this case the settings will identify the minimum values, this is why define them to lower values in oracle can manage
entirely.
Don't increase your SGA at this stage, get the awr report, use @$ORACLE_HOME/rdbms/admin/awrrpt.sql, check your cover shot.

BUT first, you must change your backup strategy, look at my first post, after that check performance again, before you do that it will be very difficult to help you.

Good luck

Tags: Database

Performance problem in production; Please help me out

Similar Questions

Maybe you are looking for