Aggregator - high memory use of node Cache Server because of deserialization

Hi all

We are currently on version 3.5.3 consistency and have the following configuration:

Cache Key : Long ID

Cache Value :
CacheData {
  Set<Long> refIds1;
  Set<Long> refIds2;
  Set<String> refIds3;
}

After loading the data cache, we're indexing the following:

cache.addIndex( new ReflectionExtractor( "refIds1" ), false, null );
cache.addIndex( new ReflectionExtractor( "refIds2" ), true, null );
cache.addIndex( new ReflectionExtractor( "refIds3" ), false, null );

We had a requirement to collect all the refIds2 of the cache for the following criteria:
#1. Key cache match
#2. All matches refIds1
#3. All matches refIds3

We have implemented the following custom aggregator:

CustomAggregator extends AbstractComparableAggregator implements com.tangosol.io.pof.PortableObject {

  @Override
  public Object aggregate( Set values )
  {
    Set<Long> resultRefIds2 = new HashSet<Long>();
    for(Entry entry : (Set<InvocableMap.Entry>)values)
    {
      CacheData data = (ProductGroup)entry.getValue() ;
      resultRefIds2.addAll( data.getRefIds2() );
    }
    return resultRefIds2 ;
  }
  
  public Object aggregateResults( Collection results )
  {
    Set<Long> localRefIds2 = new HashSet<Long>();
    if( null != results && results.size() > 0 )
    {
     List<ImmutableArrayList> immutableProductGroupArrayList = new ArrayList<ImmutableArrayList>( results );
     for( int immutableArrayCnt = 0; immutableArrayCnt < immutableProductGroupArrayList.size(); immutableArrayCnt++ )
     {
       localRefIds2.addAll( immutableProductGroupArrayList.get( immutableArrayCnt ) );
     }
    } 
    return localRefIds2;
  }
  
}

Following code retrieves the results:

Filter[] filters = new Filter[3];
filters[0] = new InFilter( new KeyExtractor( IdentityExtractor.INSTANCE ), inputKeys );
filters[1] = new ContainsAnyFilter( "refIds1", inputRefIds1 );
filters[2] = new ContainsAnyFilter( "refIds3", inputRefIds3 );
Filter filter = new AllFilter( filters ) ;
CustomAggregator aggregator = new CustomAggregator();
Set<Long> refIds2 = (Set<Long>)cache.aggregate( filter, aggregator );

While we are the results as expected, when entries returns all the criteria of the cache cached to the aggregator, we see many memory due to the deserialization use cached data.

Snippet aggregator:

    for(Entry entry : (Set<InvocableMap.Entry>)values)
    {
      CacheData data = (ProductGroup)entry.getValue() ;
      resultRefIds2.addAll( data.getRefIds2() );
    }

#1. Is there a better way to achieve the same results without having to deserialize data in the cache.
#2. Since we are already indexed data that we are aggregation, is possible to use the indexed data by grouping, rather than extract the cache.

Thanks in advance
Avinash Gagnon

Published by: 972073 on November 19, 2012 12:43

Hi Avinash,

You can avoid the deserialization of objects using indices.
Instead of calling mapEntry.getValue (), use the mapEntry.extract (...) method. If passed Extractor is indexed, attribute is taken between the index.

Kind regards
Alexey

Tags: Fusion Middleware

Similar Questions

ISE 1.3.0.876 high memory use

Hi team,

Kindly help me on our ISE Cisco version 1.3.0.876 that meets the high memory use. It has a HA configuration.

Primary school has 83% and secondary 63%.

Thanks in advance!

Kind regards

Mady

Tehreare some fixes in patches that resolve the problems associated with the use of memory

Last paych 1.3 patch 6 and included the following fix

CSCux53910: patch 1.3 ISE 5 augmented memory of lead for authentic latency

I recommend cosnidering install the latest patches

High memory use of Content Server

We noticed that our server content this morning has been using a lot of memory, up to about 93%. The process back to the top is "helper.exe", it was not transcoding or sending media files from external storage for about an hour now.

What is the typical memory used when the server is idle?

Hey Patrick. What you see is not normal. Either reboot or try running the repair of tcs procedure and see if it corrects the problem.

/ Magnus

Sent by Cisco Support technique iPhone App

On the server with a not too high memory use. DBS
Hi all
I see the use of memory to 100% to this server. I reduced the parameters of the SGA to the minimum values and bounced these DBs, but again I don't see no improvements.

What can we do? I'm the only one currently DBA & I have to solve this problem. Other teams are dependent on my the memory usage reduction.

A few details
++++++++
OS: SunOS
For all instances version: 10.2.0.3

Please come back in case you need more details to help me
++++++++
See you soon,.
Malika
So far you didn't post any detail, such as
-the amount of RAM in the server
-the amount of memory consumed by Oracle
-How many cases, the size of their EAG and their PGAs
- and especially WHY someone at decide between several database on a single server.

---------
Sybrand Bakker
Senior Oracle DBA

Svchost.exe with service Client DHCP [Dhcp] and DNS Client [DNScache] high memory use

I'm using Windows 2003 standard Server SP 2 found problem svchost.exe run service Client DHCP [DHCP] and DNS Client [DNScache]
use a lot around 2 GB of memory. Virus scan try and Hijackthis is not found. I try to restart memory return service.

I have 4 servers and the same symptoms.

Anyone know why it caused?

Hello

Your Windows 7 is better suited for the public of Windows Server on TechNet. Please post your question in the TechNet Windows server forum.

http://social.technet.Microsoft.com/forums/en/category/WindowsServer

vRanger connection 7.1.1 and vCenter 5.5 - open filters causing high memory use vpxd?

Hello

Displays the first time, hoping you can help me. We are big consumers of vRanger connection in our environment, we use multiple instances to save the production, test, dev, VDI, etc. (6 cases total). Recently, we met our vRanger 7.1.1 connection servers and our vCenter to 5.5 Update 2 b. Since then, every evening when our nightly backups occur to our use of memory for vpxd on our vCenter server (process vpxd.exe) increases, and it remains standing and rising again every night until the that vCenter grave service failed due to memory exhaustion. After 2 weeks of troubleshooting with VMware, their senior engineers think that connection vRanger not close correctly filters, leaving a large number of them that causes the memory out is not in the system.

Are you aware of any issues like this being reported elsewhere? I tried to find other peoples identify this problem, but it seems that we are the first. Please help, thanks!

We have not seen this issue in our lab, but we currently 5.5 2d update. I recommend that you open a support casel

You fixed my message high cpu now I get "high memory use. What should I do?

Should I have all the plugins that are currently on my memory? I play pogo games and sometimes large fish.

Create a new profile as a test to see if your profile is the source of the problems.

See "create a profile":
- https://support.Mozilla.org/KB/Profile-Manager-create-and-remove-Firefox-profiles
- http://KB.mozillazine.org/Standard_diagnostic_-_Firefox#Profile_issues
If the new profile works then you can transfer files from a profile in the new profile, but make sure not to copy corrupted files.
- http://KB.mozillazine.org/Transferring_data_to_a_new_profile_-_Firefox

Use of the high memory on Windows 7 without reason...

Hello, I had recently a few problems with my computer after going on vacation for a week and turn it off. When I got home my computer has had some problems of high memory use, I have no idea why. I looked in my processes tab and it shows that Svchost.exe as the highest [150K] usage, but is not a lot to slow down computers. Nothing else under it attracts a lot of use, I tried to disable my anti-virus [Norton] and it still didn't work. All my drivers are up to date.

Data sheet:

Dell Inspiron 560 [2009]

No upgrade graphics card

64 bit

Pentium Dual-Core CPU E5700 @ 3.00 GHz 3 .00GHz

4.00 GB memory installed

Hi Daniel,

Please follow the methods:

Method 1:

You need perform a clean boot to prevent any third party conflicting application from interfering with your computer.

To put the computer in a clean boot State, you must follow the steps in the article mentioned below and check with the question.

How to troubleshoot a problem by performing a clean boot in Windows Vista or Windows 7:
http://support.Microsoft.com/kb/929135

Note: You must follow step 7 of the article mentioned above to recover your computer to a Normal startup after you complete all the steps.

Method 2:

Scan your computer by using the Microsoft Security Scanner by clicking Download now to check if there is no virus:

http://www.microsoftsafetyscanner.com/

Note: The data files that are infected must be cleaned only by removing the file completely, which means that there is a risk of data loss.

Method 3:

Optimize windows 7 for better performance:

TP://Windows.Microsoft.com/is-is/Windows7/optimize-Windows-7-for-better-performance HT

Important: Running chkdsk on the drive if bad sectors are found on the disk hard when chkdsk attempts to repair this area if all available on which data can be lost.

Note: The data files that are infected must be cleaned only by removing the file completely, which means that there is a risk of data loss.

I hope this helps.

High disk use and high memory usage

It seems that my friend helped clean up my laptop I will still have a lot of problems. First my laptop becomes the high memory use. What? The system apparently when I look at the Task Manager. Here's what it looks like:

I don't know how to solve this problem. Keep looking it towards the top online and I haven't found anything that helps. I'm not a tech somehow so when you say something like "have you checked your drivers? I will say that I have no idea how to do that. As you can see the the Service Host: Local System has a lot of memory for some reason use any. And the system takes a lot of space for some strange reason. He is also windows modules Installer. I can't update my laptop Windows 8 for Windows 8.1

Another problem is that I can't get my cell trusts. I go on Internet explore and copy and paste the link that appears on the screen in another tab on him and he always says the same thing 'Open in Internet Explorer' and it still doesn't work!

I should also mention that my friend had to do a system restore to clean all the problems I had before. Who has worked for the most part. Since my laptop was running slower and he's still but a little better.

Any help would be great.

In order to diagnose your problem, we need run Windows performance toolkit, the instructions that are in this wiki

If you have any questions do not hesitate to ask

use of the high memory in the database of node cluster 2

Summary of environmental
Cluster nodes RHEL6.4/11.2.0.3.0/ASM/2
free m
total used free shared buffers cached
MEM: 32096 31723 372 0 238 16277
-/ + buffers/cache: 15206 16889
Swap: 16383 178 16205
Each s 1.0: ps EI o % mem, % cpu, cmd | sort - n - r k 2 | head - 10 Thu Feb 27 16:32:15 2014
4.8 0.2 /u01/app/11.2.0/grid/bin/osysmond.bin
mem % 0.0 4.0 ps EI - o, % cpu, cmd
ora_lms1_orcl2 19.6 1.3
ora_lms0_orcl2 19.6 1.3
/etc/kannel.conf 0.2 0.9/usr/sbin/bearerbox
asm_vktm_ 0.9 0.0 + ASM2
ora_vktm_orcl2 0.8 0.0
/opt/app/Java/JDK1.6.0_12/JRE/bin/Java 7.7 0, 7-server-Xms8192m-Xmx8192m - XX: MaxPermSize = 256 m - XX: NewSize = 2048M - XX: karine
2.5 0.7 /opt/app/java/jdk1.6.0_12/bin/java-Djava.util.logging.config.file=/opt/app/csdtomcat/apache-tomcat-6.0.35/c
/U01/app/11.2.0/grid/bin/ocssd.bin 0.3 0.7
SQL > select value, name from v$ sysstat whose name like '% gc.
THE VALUE NAME
---------- ----------------------------------------------------------------
398431945 gcs sent messages
24465773 gc cr block served
92438 flush time gc cr block
52157 timeout reading gc
100272 expected read gc
failures of reading waiting 0 gc
15 gc waiting read timeouts
77789637 blocks of courses gc served
826771 gc block pine time
130755 block course gc flush times
51119356 blocks of cr gc has received
5027865 gc cr block receive times
61272732 blocks of course gc has received
4268980 block of course gc receive times
181763086 local grants of gc
56125079 grants remote gc
716839431 gc sent kilobytes
101203847 gc saved KB
30105492 compressed blocks of gc
blocks of gc lost 0
blocks of claims gc lost 0
0 gc corrupt blocks
0 gc CPU used by this session
0 player gc bypass grants
time of generation 0 gc cr block
0 gc cr block send time
current block 0 gc send times
Node-1
Each s 1.0: ps EI o % mem, % cpu, cmd | sort - n - r k 2 | head Thu Feb 27 - 10 17:19:47 2014
16.3 51.7 /opt/netvertexp/java/jdk1.6.0_12/jre/bin/java-serveur-Xms8192m-Xmx8192m - XX: MaxPermSize = 256 m - XX: NewSize = 2048M - XX: karine
6.5 0.2 /u01/app/11.2.0/grid/bin/osysmond.bin
0.0% mem 5.0 ps EI - o, % cpu, cmd
2.2 0.0/usr/sbin/tcpdump bond0 w s port 65535 /root/pcapprimary/netvertexp-27Feb_17_15.pcap 3868-i
/U01/app/11.2.0/grid/bin/ocssd.bin 0.4 1.4
19.5 1.2 ora_lms1_orcl1
19.5 1.2 ora_lms0_orcl1
ora_dbw1_orcl1 19.7 0.9
ora_dbw0_orcl1 19.7 0.9
ora_vktm_orcl1 0.9 0.0
Node-2
Each s 1.0: ps EI o % mem, % cpu, cmd | sort - n - r k 2 | head Thu Feb 27 - 10 17:19:07 2014
4.8 0.2 /u01/app/11.2.0/grid/bin/osysmond.bin
mem % 0.0 3.0 ps EI - o, % cpu, cmd
oracleorcl2 0.1 2.5 (LOCAL = NO)
ora_lms1_orcl2 19.6 1.3
ora_lms0_orcl2 19.6 1.3
/etc/kannel.conf 0.2 0.9/usr/sbin/bearerbox
asm_vktm_ 0.9 0.0 + ASM2
ora_vktm_orcl2 0.8 0.0
/opt/app/Java/JDK1.6.0_12/JRE/bin/Java 7.7 0, 7-server-Xms8192m-Xmx8192m - XX: MaxPermSize = 256 m - XX: NewSize = 2048M - XX: karine
2.5 0.7 /opt/app/java/jdk1.6.0_12/bin/java-Djava.util.logging.config.file=/opt/app/csdtomcat/apache-tomcat-6.0.35/c
How to troubleshoot the depth to find the cause of the heavy memory usage. ?
Thank you
hitgon

Hello

vmstat-s

This will give a more detailed overview. Search for active/inactive values and used swap. If active memory is close to the total memory and you have a larger amount of used swap, then you might have the memory pressure.

If the Exchange is not used much, and you have the right percentage of inactive memory, then linux could easily swap memory without a performance problem. In this case, the pressure of memory is very likely.

For a report very detailed on the use of single process

PMAP pid - x

M and the press allows you to sort by memory use. If you are using pmap on bigest processes, then you should have a good starting point was that your memory has disappeared.

Concerning

Thomas

Host memory consumption is higher than the comments its use (incl. Cached) and overhead?

I'm under ESXi 5.1. I don't have any custom reserve configured for memory. I'm running in CVS 'Sandy Bridge' mode.
I noticed earlier than expected host memory consumed on many of my guests. This example comments running Server R2 2012 with the latest version of VMware tools running.
VMware reports
The guest operating system reports
My understanding is that the "Standby" and "Modified" on the client contains the ' cache ', and the 'Cached' memory prompt is the typical response to this question that I see when people seek only to the memory of 'In Use'. In my case, I'm factoring in the memory cache, my guest shows 3805 MB of real memory, yet my host shows 7.68 GB of memory consumed host, with only 68 MB of general consumption. According to my calculations, the virtual machine should be only consume about 4.28 GB of memory.
Post edited by: bradduane Replace the values of text with screenshots.

I think I found my answer in the guide 'Understanding memory resources management' here: http://www.vmware.com/files/pdf/mem_mgmt_perf_vsphere5.pdf

Section "when to retrieve memory host" page 15:

«In the high State, the use of global memory machine virtual comments is smaller than the size of the host memory.» Whether or not the host memory is overcommitted, the hypervisor will not recover memory via hot-air balloon or permutation. (It is true that when the limit memory VM is not set).

If memory free host down to the low threshold, the hypervisor begins to find the memory using the balloon. Balloon happens before free memory has actually reached the soft threshold because it takes time for the balloon pilot to allocate and PIN memory physics reviews. Usually, the balloon pilot is able to retrieve the memory in a timely order to keep free memory in the host above the soft line. »

I checked esxtop, pressing "m" for the display of the memory, and I see that hosts VMKMEM shows 'State', so the only memory of way either currently retrieved is by sharing the page. Only when I start to run out of memory of the host, that this State becomes 'soft' or 'hard', balloon or Exchange will be held in order to free the "free" memory of comments. ESXTOP show my 'minfree"than 940 MB, while it shows that I physically 6960MB of free memory.

Firefox uses a huge amount of memory on my computer. I continue to receive alerts from high memory usage. Can I do to solve this problem?

Firefox uses a huge amount of memory on my computer. I continue to receive alerts from high memory usage. Can I do to solve this problem?

300-400 MB is not huge.

How much RAM do you have?

How AVG reports that Firefox uses?

Windows 7 seems to have unexplained memory use high after a scan of viruses of several hard drives.

My computer:
- 6 GB of installed memory, normal use is about 1.5 GB
- 3 hard disks of the computer (OS [Raid 0]. 2 data readers)
- Windows 7 64 bit
Problem:
- After analysis the entire computer, the use of memory up to 3.5 GB of jumps and stay until the computer is restarted.
- Research in the Task Manager and resource monitor shows the memory used, but running programs don't add to what be close to 3.5 GB
- I tested it with AVG and Avira antivirus, and it happens with both. Then he would lead me to believe it's a matter of Windows.
Troubleshooting:
- Tried it with several antivirus, same problem with ones I've tried.
- Scanned just the OS partition without significant increase in memory usage.
- Scanned two data discs. Memory usage has jumped about 1 GB during the analysis of the first hard drive and another 1 GB when on the second disc. Use of memory are remained high, once the completed scan. Subsequent analyses retains high memory usage.
Question:
- What is causing this behavior? Indexing, or is it some sort of memory leak?
The answer can be found here.

http://social.technet.Microsoft.com/forums/en-us/w7itproperf/thread/50e505cb-00C2-4c86-BF08-e9c926a4982a

Node Manager can be used to restart Weblogic Server Administration?

Hello guys,.
According to my understanding 'Machines' created in Weblogic are the logical representation of define what Node Manager to Point to perform actions on managed servers. But when we place servers on Machine here in my environment I get name "Admin Server" also make a link to a machine.
So far, I thought that assistance from the node Manager we can control the events of life cycle of the managed server. Is it possible to order the start/stop of the Administration also using Node Manager server?
PS: I use Weblogic 12 c and configured NM within my domain.
Kind regards
Puneet

Hello

Yes, short words, it is possible. Check out this doc for more information on how to accomplish this task > http://docs.oracle.com/cd/E24329_01/web.1211/e21050/starting_nodemgr.htm#NODEM219

Kind regards

White

How to determine which process is using high memory

Dear team,
on the single ESXi host we have 192 GB of RAM and on this 5 VMs esxi host who r under tension and a machine virtual is turned off, IF I check ESXi summary and performance tab it show ESXi host uses close to 140 GB of RAM on 192 GB, on the performance tab it show current total current memory usage is 73% and troubleshooting further if I check each VM memory usage then use VM's total memory is not over 50GB then there is show 73% i.e. use of 140 GB, needs your help to check which process is high memory, please find the file attached lot ESXTOP output.
concerning
Mr. VMware

The memory consumed, it's affecting the VM (memory affected by the VM, more general memory pages), i.e. "the memory usage. Active memory is usually much lower and calculated by the hypervisor using a mathematical formula.

André

Aggregator - high memory use of node Cache Server because of deserialization

Similar Questions

Maybe you are looking for