ORACLE 11g R2 RAC Interconnect latency tuning

We will address some related to the interconnection of network settings at the level of the OS/HW/VMWare and check if there is a difference in latency for different parameters of interconnection.

We cannot test in a real production environment. So, we need to emulate the issue of interconnection somehow in our test environment and compare results (latency) for different HW/VM settings.

Y at - it any way/approach that we can use for testing? For example, to run a script that causes a large number of requests for blocks from one node to another and measure certain parameters.

What related interconnection parameters/settings, we can check in the Oracle and compare tests for different configurations of OS/HW?

Ideally, we need to create a test case that we can reuse and compare the results.

All references are very much appreciated.

We use the following environment:

Oracle SE 11 GR 2 CARS of two nodes.

Windows 2008R2.

VMWare ESXi6.0.

Cisco UCS blades.

Thank you

Yaro

When I wrote Oracle RAC Performance Tuning, I devoted a chapter on optimization of the cluster of interconnection. In order to show the impact of some changes on how interconnection has been configured, I used the freely available iperf utility.

One of the things it's important to know for the Cluster of interconnection is that the device uses UDP. Thus, any tool that you use must be able to send UDP instead of TCP packets. Iperf can do both.

"For example, to run a script that causes a large number of requests for blocks from one node to another and measure certain parameters.

One of the biggest problems with this approach is that you have a lot of variables involved. The buffer Cache, of the shared pool, the system drives, etc, etc, etc. With a tool such as Iperf, you can measure just the performance of the network.

See you soon,.
Brian

Tags: Database

Similar Questions

  • Oracle 11g R2 RAC on IBM with SAN storage

    Hello Experts,

    I design an implementation for RAC Oracle 11 g 2 on AIX 6.1 with IBM's SAN storage. My drawing do not include Oracle ASM, I know, it is recommended to use Oracle ASM, but my purpose is not included

    I read the Guide of Installation of Oracle Grid Infrastructure and I have below questions about storage configurations:

    1. do I need a special configuration to replace the DSO?
    2. is it compulsory to use HACMP (IBM's Clustering solution) because I don't use ASM? Since based on the infrastructure Grid section 3.2 Installation guide (the Storage Configuration shared): [http://download.oracle.com/docs/cd/E11882_01/install.112/e17210/storage.htm] there is a section 3.2.7 "Configuration HACMP multinode disk Heartbeat (MNDHB) for Oracle Clusterware" so my question is it is mandatory to use HACMP if I don't want to use ASM?
    3. in the same above installation guide, mentioned in section 3.2.8 that "to use the raw logical volumes for Oracle Clusterware, HACMP must be installed and configured on all nodes in a cluster." what is raw logical volumes? is it compulsory to use it? Since I'm not on ASM?
    4 installation guide, in sections 3.2.3 to sections 3.2.9 are these prerequisites sections sections 3.2.10 to 3.2.15? or I can simply ask section 3.2.10 to 3.2.15 without applying 3.2.3 to 3.2.9?

    One last Question:

    ASM is integrated with the cluster software, do a separate license or license the license of the ASM cluster cover?

    Kind regards

    NB wrote:
    Hello Levi,

    the information provided is very useful. But what I heard in the installation guide network infrastructure that the raw device needs HCMP while GPFS need not of HACMP, GPFS is a cluster provided by IBM file system.
    Your input please?

    Hello

    Raw disk device (RD) is the LUN (disk) storage mapped on AIX.
    Raw logical volume (RLV) is the LUN (device) created the LVM of AIX.

    If you use devices of raw disk that you don't need HACMP, because AIX (core) has only receive instructions from Oracle to read and write, AIX does not support concurrency control in the raw disks, this control is managed by Oracle.

    If you use raw logical volumes (CLVM) HACMP must be installed and configured on all nodes in the cluster, volume logic is a feature of AIX (IBM) to enable simultaneous mode, you install HACMP, Oracle not manage concurrency control because Oracle does not know that there is an LVM behind.

    You are right, that the new version of GPFS are usable without HACMP.

    Kind regards
    Levi Pereira

    Published by: Levi Pereira on October 8, 2011 17:26 # update the info on gels of features

  • Oracle 11g Release 2 assumption of raw devices to the CCR

    Hi I need one of my db migrate to 10.2.0.3 for 11g release 2
    now, I got to know that there are some problems with raw devices in oracle 11g R2 rac
    so I want the suggestions of experts
    Thanks in advance

    If you are upgrading a database in place (i.e. without reconstruction, without import-export), Oracle has no other choice than to support the application of raw device.
    However, when you install the Clusterware 11.2 it cannot be installed on raw devices - one is not supported!

    Oracle Support Document ID 578455.1 ' announcement of of-Support devices version 12 G GROSS ' announces support to 12 g.

    Hemant K Collette

  • That means 1z0-117 oracle 11g sql tuning now say it includes v12 addition v11

    Hello

    I did the 1z0-117 sql tuning review once.

    Not too far away.

    I studied under and turned off for a while.

    I anticipate taking in 1 month.

    However, I just noticed that the oracle site says 1z0-117 also said that v12 is also included for consideration:

    Oracle 11g sql tuning.

    It makes no sense at all.

    Roger

    However, I just noticed that the oracle site says 1z0-117 also said that v12 is also included for consideration:

    Oracle 11g sql tuning.

    Unless you are looking for something I'm not, what actually is the 1Z0-117 page says: "validated against: review has been validated against Oracle Database 11g Release 2 version 11.2.0.1.0 and database Oracle 12 c 12.1.0.1.0"

    What they actually mean by it is that someone went through all the issues and and asked the question "is still a relevant issue for the release of 12 c to Oracle?'." "  If they find issues that are not valid because of an update/change, then the question will be removed from review (or changed) so that someone who has used 12 c but not 11 g will not at a disadvantage.

    This is * No * means that Oracle has added questions to the review of the capabilities that were introduced in version 12 c.

  • Installing oracle 11g (RAC) on RHEL 6

    Dear all,


    Me and my team will install oracle 11g (RAC) on RHEL 6.
    About mention above I have little doubt it please help me out in this matter.

    In the network section.

    Requirement of intellectual property:

    1 public IP address
    2 private IP
    3. virtual IP address

    Suppose we have two interfaces in each server, eth0 and eth1.

    I put the public IP address in eth0 192.9.25.100 and private 10.79.1.100 eth1 IP address
    Now where put the virtual IP address and what is the procedure.

    Please help me to set the virtual IP address


    Thanks and greetings
    Yvon Das
    S/n, Guwahati

    Published by: Yvon on 23 May 2013 23:44

    Hello

    See the message below:

    http://www.Oracle-base.com/articles/11g/Oracle-DB-11gr2-RAC-installation-on-Oracle-Linux-6-using-VirtualBox.php

    If you need clarification update here.

    Thank you
    A H E E R X

  • Oracle 11g RAC installtion

    Hello

    I would like to know how to install Oracle 11 g RAC by using the virtual machine.

    material:
    RAM: 8 GB
    Hard drive: 320
    CPU: i5

    above hardware is compatible for oracle 11 g RAC?
    Please suggest.

    996253 wrote:
    Hello

    I would like to know how to install Oracle 11 g RAC by using the virtual machine.

    check the link below.

    http://www.Oracle-base.com/articles/11g/Oracle-DB-11gr2-RAC-installation-on-OL5-using-VMware-Server-2.php

    material:
    RAM: 8 GB
    Hard drive: 320
    CPU: i5

    above hardware is compatible for oracle 11 g RAC?
    Please suggest.

    Yes, you can install the help above HW.

  • Deployment of Oracle 11g RAC Release 2 with IBM midsize V7000

    At and deployment 11g rac with IBM midsize V7000 san with aix 6.1
    Is this possible using Oracle CRS without going to the OS like GPFS cluster

    lbn76 wrote:
    At and deployment 11g rac with IBM midsize V7000 san with aix 6.1
    Is this possible using Oracle CRS without going to the OS like GPFS cluster

    Oracle Grid these days use ASM starts for vote and register files storage. So no need for a cooked cluster file system that must be mounted by all nodes. No need either directly with a gross storage logical unit number.

    ASM uses the LUN of the matrix of storage/SAN directly (as devices of rough blocks).

    Generally however, there are several I/O paths to that LUN on the SAN - in which case there is more than 1 device level o/s for this LUN. Multipath driver s/w addresses this and creates a logical (and redundant) device that sits at the top of several devices of i/o path to the level of the o/s.

    You configure ASM to use devices of paths multiples (as devices of rough blocks) - and applying exterrnal, normal or high redundancy necessary when creating starts on these logical multipath devices.

    The grid s/w and RAC database instances can then use ASM starts as cluster storage.

    This makes storage much easier layer management - and also provide a number of useful features too.

    This is also the approach recommended by Oracle Corp. for RAC - use ASM under the layer of management of storage between storage LUNS in the table and one side and CARS on the other.

  • Oracle 11g RAC can support 2 separate databases?

    1. can you Oracle 11 g RAC support 2 separate databases because it requires to SCAN ip cluster with a port number unique listener?
    2. Oracle 11.2 database connections will work correctly if I install a mandatory DNS server in Oracle RAC servers?
    I'm leading a team for a project of 11g and appreciate the answers.
    Thank you
    Satish.

    Oracle 11g RAC will support the 2 separate databases because it requires to SCAN ip cluster with a port number unique listener?

    YES

    Oracle 11.2 database connections will work correctly if I install a mandatory DNS server in Oracle RAC servers?

    YES

  • Will there still be 2 sections to go to oracle RAC exam 1z1-068 12 c as if it was for oracle 11g RAC 1z0-058

    Hello

    Review of Oracle 11 g RAC (1z0-058) required 2 sections to pass.

    As well as reviewing DBA Oracle 12 c (1z0-060).

    in any case there is still 2 sections to review RAC 12 new Oracle 1z1-068 c?

    Thank you Roger

    It is difficult to say - just begun beta review. We will be able to provide more information after the beta and beta analysis post completed.

    Kind regards
    Brandye Barrington

  • use of temporary space in oracle 11g rac

    Hello

    Oracle version: oracle 11g r2

    OS: linux

    storage: asm on exadata

    Please help us understand the use of the temp

    {code}

    SQL > set linesize 1000
    SQL > set pagesize 1000
    SQL > col USERNAME format a20
    SQL > col SPID format a10
    SQL > col MODULE format a30
    SQL > col PROGRAM format a30
    SQL > SELECT s.sid, s.serial #, s.username
    2, p.spid, s.module, p.program
    3, SUM (su.blocks) * tbsp.block_size/1024/1024 mb_used
    4, su.tablespace
    5. FROM v$ sort_usage su
    6, v$ session s
    dba_tablespaces tbsp
    7, $v 8 p process
    9 WHERE su.session_addr = s.saddr
    10 AND su.tablespace = tbsp.tablespace_name
    11 AND s.paddr = p.addr
    GROUP 12 OF
    13 s.sid, s.serial #, s.username, s.osuser, p.spid, s.module,.
    14 p.program, tbsp.block_size, su.tablespace
    15 ORDER BY s.sid;

    USERNAME SID SERIAL # SPID MODULE PROGRAM MB_USED TABLESPACE
    ---------- ---------- -------------------- ---------- ------------------------------ ------------------------------ ---------- -------------------------------
    11 2092 DBSNMP 2861 emagent_SQL_oracle_database Oracle @ [email protected] 8 TMP4

    SQL > select * from v$ temp_space_header;

    FILE_ID BYTES_USED, BYTES_FREE BLOCKS_FREE RELATIVE_FNO BLOCKS_USED NOM_TABLESPACE
    ------------------------------ ---------- ---------- ----------- ---------- ----------- ------------
    TMP1 2 + 12 222274816 33554432 2048 1024 3.6418E
    TMP3 4 + 11 19283968 75497472 4608 1024 3.1595E
    TMP4 5 + 11 19687936 0 0 1024 3.2257E
    TMP5 6 + 13 688638464 0 0 1024 1.1283E

    {code}

    Why isn't bytes_used reclaim as only 8 MB is currently using

    Thank you

    PMP

    I wanted to tell temp is never off allocated. So, if a 'bad' query explodes temp. the only thing you can do is to create a new and drop the old one.

    11 GB temp is clearly ridiculous. Usually, this is caused by the merge join Cartesian paths in your instructions.

    -----------

    Sybrand Bakker

    Senior Oracle DBA

  • RAC interconnect switch?

    Hello

    We are in the process of migrating from a single instance database 10 g to node 2 RAC (Windows Server 2008 OS, EMC storage with 2 SAN,...) and we have some doubts about interconnection.

    We have difficulties in the choice of speed of interconnection correct for the interconnection network, difficulties in the choice of the switch/switches...

    1. because there are 2 knots and 4 Ethernet cables to connect, whether to use one or two switches? With the help of a switch can be a solution, but one switch become a big single point of failure.
    2. the question of whether we can get in performance if we use 2 switches (collage,...)?
    3. as mentioned, there are 4 cables Ethernet, it is recommended to use 1 GB current switches that we use for the public network or buy 1 GB to be used only for the private interconnect switches?
    4. can we use simple port 16 or 8 GigE switches?

    Maybe you can tell me to some switches GigE and SAN (for nodes - storage connection) that you saw that they work without any problem with CARS.

    How can we better deisgn for interconnection networks?

    Thanks in advance!

    user9106065 wrote:

    How can we better deisgn for interconnection networks?

    The best technology for a company of interconnection, especially for Oracle RAC is Infiniband. No Gig-Ethernet.

    Think about the following - one of the objectives of the interconnection is Cache Fusion. Allowing a single node RAC read data blocks to an another RAC cache node via interconnection, instead of having to hit physical disks to read data on disk blocks.

    E/s typical connection is dual fibre channel 2 GB physical disks (old technology - fiber channels today is 4 GB and more). Your interconnection, which is supposed to be faster, it's 1 Gb Ethernet.

    How does make sense, when your Interconnect is slower, with less capacity and latency than the access to the actual physical disk on the pole?

    1 Gb for RAC interconnect is a failureof IMO.

    Looking at what the sellers. UCS provides an Ethernet 10 GB as Interconnect. But then, Cisco has dropped support Inifiband and pushes GigE as the solution. Oracle database machine (and Exadata Storage servers) using 40 GB (two ports) Infiniband RDS running as a protocol for interconnection to the slowest place and more CPU intensive UDP.

    If you look at the architecture of interconnection of 500 bunches faster and the largest in the world, Infiniband for interconnection are used by a total of 2 867 854 cores to a peak of 41,678,845 GFlops.

    GigE is used by a total of 2 510 404 cores to a peak of 26,754,468 GFlops.

    IMO - if you want a good CAR you need good Interconnect and storage layers. And for the first, it is without a doubt Infiniband.

  • Oracle 11g CMO

    Hi all

    am an Oracle 11 g RAC OCÉ, plan to write oracle 11g CMO, can anyone suggest how to proceed?

    How the review will be... is this objective or another criterion?

    How to buy the right? and how much cost in India?

    anytime I write or Oracle Corp. will decide on the dates etc...?

    Please guide me the details.

    Thank you

    Sam.

    The 11g COM is different from all the other reviews.  This is a 2 day practical laboratory exercise.  You are given to an Oracle server and need to perform different tasks, solve problems, etc.

    The review of the CMO is only given to a relatively small number of times throughout the year.  Strictly speaking, there are a lot of times, but they are spread all over the world, so many times at a location any where near you will be weak - maybe one or two a year.

    In addition to the exam, you must also take two practical courses acceptable to Oracle.  You can find more information, including the price in your area by reading and following the links available at the following ADDRESS:

    https://education.Oracle.com/pls/web_prod-PLQ-dad/db_pages.GetPage?page_id=653&get_params=P_ID:52 #tabs - 3

  • upgrade to oracle 9i to Oracle 11 g RAC RAC

    Hi all

    version of DB: oracle 9.2.0.5 RAC
    OS: Linux

    My project team and the customer plans to upgrade oracle 9i RAC to Oracle 11g RAC.
    Please suggest any MOS notes all the links.

    Thank you
    Visu.

    Published by: visu996253 on 8 April 2013 05:30

    version of DB: oracle 9.2.0.5 RAC
    OS: Linux

    My project team and the customer plans to upgrade oracle 9i RAC to Oracle 11g RAC.
    Please suggest any MOS notes all the links.

    Note to master for the upgrades of Oracle database and migration [ID 1152016.1]
    http://www.Oracle.com/technetwork/products/upgrade/index-088044.html
    785351.1 - companion of oracle 11g upgrade
    http://download.Oracle.com/docs/CD/E11882_01/server.112/e17222.PDF - Guide to upgrade the database® Oracle 11 g Release 2 (11.2) E10819-02

    Published by: KR10822864 on 8 April 2013 05:52
    Add more information.

  • Install oracle 11g Client in an application server (HP - UX 64-bit operating system)

    I'm going to install oracle 11g client in an application server (HP - UX 64-bit operating system)
    I have to create the new mount for client installation point?
    Through the application server, I will access (rac) database server.

    I have enough space available in / or root mount point.
    I have space in / opt mounting point, can I make a directory in/opt/OU1/app and install the client at this place in the access database?



    Thank you
    Solaiman

    876149 wrote:
    I'm going to install oracle 11g client in an application server (HP - UX 64-bit operating system)
    I have to create the new mount for client installation point?
    Through the application server, I will access (rac) database server.

    I have enough space available in / or root mount point.
    I have space in / opt mounting point, can I make a directory in/opt/OU1/app and install the client at this place in the access database?

    Thank you
    Solaiman

    You can install in any desired point of installation/directory. Oracle does not give a flying fig this topic. Just put it somewhere that makes sense.

  • Add a node to Oracle 11 g RAC

    Hello.

    If anyone can give a guide short and good to add a node to Oracle 11 g RAC running on Suse Linux 10. ?


    Thank you

    Hello

    Please find below oracle demo,
    http://St-curriculum.Oracle.com/demos/DB/11g/R2/grid_rac/04_clusterware_add_node/clusterware_add_node.swf

    I hope this helps.

    Kind regards
    Anand.

Maybe you are looking for