FUSION table on millions of records takes too long
Hi friends,I have two tables with the following data.
CS_DS_DETAIL: 15.8 million lines
SALES_DATA: 28,05 million lines
When I run the following MERGE statement he finished only 1% after 8 hours.
SQL> explain plan for
2 MERGE /*+ INDEX(dtl cs_ds_detail_pk,sd sales_data_pk) */ INTO sales_data sd
3 USING cs_ds_detail dtl
4 ON ( sd.item_id = dtl.item_id
5 AND sd.location_id = dtl.location_id
6 AND sd.sales_date = dtl.week_end_date)
7 WHEN MATCHED THEN
8 UPDATE
9 SET sd.cs_turn_ord = dtl.turn_plug_adj, sd.cs_promo_ord = dtl.promo_plug;
Explained.
SQL> SELECT *
2 FROM TABLE(DBMS_XPLAN.DISPLAY);
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
Plan hash value: 3300673510
----------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
----------------------------------------------------------------------------------------------------
| 0 | MERGE STATEMENT | | 15M| 457M| | 2615K (2)| 08:43:12 |
| 1 | MERGE | SALES_DATA | | | | | |
| 2 | VIEW | | | | | | |
|* 3 | HASH JOIN | | 15M| 4279M| 897M| 2615K (2)| 08:43:12 |
| 4 | TABLE ACCESS BY INDEX ROWID| CS_DS_DETAIL | 15M| 723M| | 1319K (1)| 04:23:52 |
| 5 | INDEX FULL SCAN | CS_DS_DETAIL_PK | 15M| | | 80488 (1)| 00:16:06 |
PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
| 6 | TABLE ACCESS FULL | SALES_DATA | 26M| 6178M| | 929K (4)| 03:05:56 |
----------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("SD"."ITEM_ID"="DTL"."ITEM_ID" AND "SD"."LOCATION_ID"="DTL"."LOCATION_ID" AND
"SD"."SALES_DATE"="DTL"."WEEK_END_DATE")
How can I make it work better. It's a simple Merge statement and we use Oracle 11 g.The tables are not partitioned.
Is there a way I can write this using the simple update rather using Merge?
Anyone think that update gives best performance that merge in this case?
Thank you guys for your time.
~ Reda
Is there a way I can write this using the simple update rather using Merge?
There is nothing wrong with the MERGE, but if you want to use an UPDATE, you can try a modifiable join view:
update
(select sd.cs_turn_ord
,sd.cs_promo_ord
,dtl.turn_plug_adj
,dtl.promo_plug
from sales_data sd
join cs_ds_detail dtl
on sd.item_id = dtl.item_id
and sd.location_id = dtl.location_id
and sd.sales_date = dtl.week_end_date
)
set cs_turn_ord = turn_plug_adj
,cs_promo_ord = promo_plug
;
Whatever it is, I agree with William... get rid of the index indicator!
Tags: Database
Similar Questions
-
Inserting data in the remote table takes too long
Hello
I have a table in the remote database and insert the data into this particular table... takes too long.
Could you please advice me some tips for the DML can quickly...
Concerning
DASThat takes a lot of time, then don't you have an idea how can we send the data fast
Buy more band witdth?
No, honestly. As you can see, the quality of my answers match the quality of your question.
You must locate the bottleneck, where is - this "too much" time has passed.
Is it
collection of data on the local side?
net send?
Actual insertion on remote site?
Maybe you have a condition on the remote site, which causes the data sent to the remote side to local?Impossible to give advice without knowing anything about what is happening.
Concerning
Peter -
Executed scripts takes too long
Hello
I hope for more help with improvement / reducing the amount of time required to get a script to run. I have provided the following script to our CMDB group but they say it take too long to run (full day).
I already use the get-view option to extract the details. Maybe someone can advise how they would improve this script. For the record, we have 2000 + vm in our environment.
Script is attached.
Any help is very appreciated.
Hello, VM_.
Well, nobody likes a slow script, that's for sure. And a script of the day? Double boo to that.
I had a quick glance, and there were some spots of improvement. I have them made here:
Add-PSSnapin VMware.VimAutomation.Core function Get-VMSerial { ## rewrote to take UUID; already being gotten in properties below # param([VMware.VimAutomation.ViCore.Impl.V1.Inventory.VirtualMachineImpl]$VirtualMachine) # $s = ($VirtualMachine.ExtensionData.Config.Uuid).Replace("-", "") param([string]$Uuid) $s = $Uuid.Replace("-", "") $Uuid = "VMware-" for ($i = 0; $i -lt $s.Length; $i += 2) { $Uuid += ("{0:x2}" -f [byte]("0x" + $s.Substring($i, 2))) if ($Uuid.Length -eq 30) { $Uuid += "-" } else { $Uuid += " " } } return $Uuid.TrimEnd()} # Connect to vcenterConnect-VIServer vcenter #Gathering VM settingsWrite-Verbose -Verbose "Gathering VM statistics"$VMReport = @()$Count = 0 ## no need to use Get-VM#Get-VM | % {## moved outside of the Foreach-Object scriptblock, so this Get-View only gets called once, not 2000+ times (not once per VM, just once at all)$filter = @{"Config.Template"="false"}Get-View -ViewType VirtualMachine -Filter $filter -Property Name,Guest.HostName,summary.config.numcpu,summary.config.memorysizemb,summary.config.numEthernetCards,Summary.Config.NumVirtualDisks,Config.Uuid,Parent,Guest.GuestFamily,config.tools.toolsversion,guest.toolsstatus,config.Version,Config.ChangeTrackingEnabled,Datastore,AvailableField,Value | %{ ## $vm is the View of the current VM $vm = $_ ## not used -- removed it # $CustomDetails = Get-VM $_ | Select -ExpandProperty customfields ## rewrote to use Get-View or UpdateViewData() # $LunTierStringArray = Get-VM $_ | Get-Datastore $LunTierStringArray = Get-View -Property Name -Id $vm.Datastore $LunTierString = $LunTierStringArray.Name -split "_" $vms = "" | Select-Object VMName, Cluster, DnsName, TotalCPU, TotalMemory, TotalNics, Disks, DiskTier, SDF, UUID, Folder, OS, ToolsVersion, ToolsStatus, HardwareVersion, CBT, Serial $vms.VMName = $vm.Name ## rewrote to use UpdateViewData() # $vms.Cluster = $(Get-vm $_ | Get-cluster).Name $vms.Cluster = &{$vm.UpdateViewData("Runtime.Host.Parent.Name"); $vm.Runtime.LinkedView.Host.LinkedView.Parent.Name} $vms.DnsName = $vm.Guest.HostName $vms.TotalCPU = $vm.summary.config.numcpu $vms.TotalMemory = $vm.summary.config.memorysizemb $vms.TotalNics = $vm.summary.config.numEthernetCards $vms.Disks = $vm.Summary.Config.NumVirtualDisks $vms.DiskTier = $LunTierString[0] ## getting this in some other way, from the # $vms.SDF = ($_ | Get-Annotation -CustomAttribute 'School/Division/Faculty').Value $vms.SDF = & {$intCustomAttributeKey = ($vm.AvailableField | ?{$_.Name -eq "School/Division/Faculty"}).Key; ($vm.Value | ?{$_.Key -eq $intCustomAttributeKey}).Value} $vms.UUID = $vm.Config.Uuid ## add -Property Name, though, with " | Out-Null", does this even work? # $current = Get-View $vm.Parent | Out-Null $current = Get-View $vm.Parent -Property Name,Parent -ErrorAction:SilentlyContinue $path = $vm.Name do { $parent = $current if($parent.Name -ne "vm"){$path = $parent.Name + "\" + $path} ## add -Property Name, though, with " | Out-Null", does this even work? $current = if ($null -ne $current.Parent) {Get-View $current.Parent -Property Name,Parent -ErrorAction:SilentlyContinue} } while ($current.Parent -ne $null) $vms.Folder = $path $vms.OS = $vm.Guest.GuestFamily $vms.ToolsVersion = $vm.config.tools.toolsversion $vms.ToolsStatus = $vm.guest.toolsstatus $vms.HardwareVersion = $vm.config.Version $vms.CBT = $vm.Config.ChangeTrackingEnabled $vms.Serial = Get-VMSerial -Uuid $vm.Config.Uuid $VMReport += $vms $Count++ ## added Write-Verbose so that the pipeline does not get polluted with strings (for the day that this code is returning objects for further manipulation down the pipeline, instead of going straight to CSV) Write-Verbose -Verbose $Count}#Output$VMReport | Export-Csv vcenter_vm_report.csv_tmp -NoTypeInformation -UseCultureCopy-Item vcenter_vm_report.csv_tmp vcenter_vm_report.csv
I commented on the changes I made (and why) and commented on by the code I replaced. Give a run and see if it is not faster for you / your CMDB crew.
Message has been edited by Matt Boren on February 24, 2015: correction of the piece that gets the path of the inventory of the VM (were able only to return a parent level previously) minor
-
Hello
The following query takes too long (more than 30 minutes), work with 11g.
The table has three columns RID, ida, geometry and index has been created on all of the columns.
The table has about 5,40,000 documents of point geometries.
Please help me with your suggestions. I want to select the geometry in double point where ida = STRING.
SQL > select a.rid, b.rid from totalrecords, totalrecords b where a.ida = 'CORD' and b.idat = 'CORD' and
sdo_equal (a.geometry, b.geometry) = 'TRUE' and a.rid! = b.rid order of 1,2;
concerningHello
Just glad it helped, don't forget to award points ;-).
The SDO_JOIN will use the spatial index for a spatial comparison.
First of all, it will use the index to check which geometries MBR interact, it is the primary filter: http://docs.oracle.com/cd/B28359_01/appdev.111/b28400/sdo_intro.htm#g1000087Secondly, you should apply a space MASK, pair of geometries that comes out of the primary filter is then compared according to the MASK, MASK for example = EQUAL, will check these pair of geoms are equal.
Now, because you have points, the members are the points themselves, so that if their MBR interact, points to interact, which means they are equal.
This means that the result of the SDO_JOIN points, in this case even a self-join, will give you all the points that are equal.But as points are also equal in their car, the join join reflexive sdo, will give you: a = b & one = one but also a b.
To this effect, you set the a.rowid< b.rowid,="" to="" avoid="" a="a" but="" also="" b="">I hope this explains it a little, again read and read great literature (early!), will have a better understanding.
Good luck
Luke
-
Hi friends,
the view that I created on a table takes too long to gimme results
When I try to select * from table_name
Can someone suggest me a solution pls
CREATE OR REPLACE VIEW XXKDD_LATEST_SAL
(RN, MONTH, YEAR, EMPLOYEE_NUMBER, POSITION,)
DEPT, STATUS, TERMINATION_DATE, PAYROLL_NAME, FULL_NAME,
TOP3, BASIC_SALARY)
AS
(
Select "RN", "MONTH", "YEAR", "EMPLOYEE_NUMBER", "POSITION", "PAYROLL_NAME", "DEPT", 'STATUS', 'TERMINATION_DATE', "FULL_NAME", "TOP3", "BASIC_SALARY" of)
ROW_NUMBER() SELECT over (PARTITION BY employee_number ORDER BY employee_number) rn, tp.*
(SOME MONTHS, YEAR, employee_number, position, payroll_name, dept, status, termination_date, full_name,)
ROW_NUMBER () OVER (PARTITION BY employee_number, basic_salary ORDER BY YEAR, MONTH) top3,
DECODE (basic_salary,
100000, 4500,
24000, 1921
basic_salary
) basic_salary
OF kdd_pay_hr_sal_vw
TP command by employee_number, year desc)
WHERE top3 < = 1
))
Select * from XXKDD_LATEST_SALLook in the SQL and PL/SQL FAQ:
-
Data dictionary query takes too long.
Hello
I'm using ORACLE DATABASE 11 g.
The following query takes too long to run and not give the result. As I've tried a few tricks SQL Oracle but it forces developed.
Please guide me what to do? to run this query in a fast paced mode...SELECT distinct B.TABLE_NAME, 'Y' FROM USER_IND_PARTITIONS A, USER_INDEXES B, USER_IND_SUBPARTITIONS C WHERE A.INDEX_NAME = B.INDEX_NAME AND A.PARTITION_NAME = C.PARTITION_NAME AND C.STATUS = 'UNUSABLE' OR A.STATUS = 'UNUSABLE' OR B.STATUS = 'INVALID';
Thanks in advance...Your query is incorrect. It returns all tables if A.STATUS = "UNUSABLE" or B.STATUS = "INVALID". Most likely, you meant:
SELECT distinct B.TABLE_NAME, 'Y' FROM USER_IND_PARTITIONS A, USER_INDEXES B, USER_IND_SUBPARTITIONS C WHERE A.INDEX_NAME = B.INDEX_NAME AND A.PARTITION_NAME = C.PARTITION_NAME AND (C.STATUS = 'UNUSABLE' OR A.STATUS = 'UNUSABLE' OR B.STATUS = 'INVALID');
But the above will return sous-partitionnee tables not valid/no usable index. He ain't no non-sous-partitioned tables partitioned index/index not valid/not usable with same partitions in the form of tables not partitioned with valid/no unusable indexes. If you want to get any table with indexes not valid/not usable, you outer join that's going to hurt even more performance. I suggest you use the UNION:
SELECT DISTINCT TABLE_NAME, 'Y' FROM ( SELECT INDEX_NAME,'Y' FROM USER_INDEXES WHERE STATUS = 'INVALID' UNION ALL SELECT INDEX_NAME,'Y' FROM USER_IND_PARTITIONS WHERE STATUS = 'UNUSABLE' UNION ALL SELECT INDEX_NAME,'Y' FROM USER_IND_SUBPARTITIONS WHERE STATUS = 'UNUSABLE' ) A, USER_INDEXES B WHERE A.INDEX_NAME = B.INDEX_NAME /
SY.
-
Statement Update takes too long to run
Hi all
I am trying to run this update statement. But its takes too long to run.
UPDATE ops_forecast_extract b SET position_id = (SELECT a.row_id FROM s_postn a WHERE UPPER(a.desc_text) = UPPER(TRIM(B.POSITION_NAME))) WHERE position_level = 7 AND b.am_id IS NULL;
SELECT COUNT(*) FROM S_POSTN; 214665
SELECT COUNT(*) FROM ops_forecast_extract; 49366
What could be the reason for the update statement to run so long?SELECT count(*) FROM s_postn a, ops_forecast_extract b WHERE UPPER(a.desc_text) = UPPER(TRIM(B.POSITION_NAME)); 575
Thank youpolasa wrote:
Hi allI am trying to run this update statement. But its takes too long to run.
What could be the reason for the update statement to run so long?
You did not say what means "too long", but a simple and good reason might be that the scalar subquery on "s_postn" using a full table for each run scan. Potentially this subquery is executed for each row of the table "ops_forecast_extract" that satisfies your filter predicates. 'Potentially' due to "filter/subquery optimization" cunning of the Oracle execution engine that tries to cache results of already executed of instances of subquery. Given that the in-memory hash table that contains these cached results is limited in size, the optimization algorithm depends on the sort order of data and could suffer collisions of hash that it is unpredictable, how this optimization works in your particular case.
You can view the execution plan, it should at least tell you how Oracle will run the scalar subquery (that tell you nothing about this "filter/subquery optimization" feature).
Follow the generic guidelines how to generate a useful plan explain output and how to post here:
Could please post a correctly formatted explain you plan output using DBMS_XPLAN. SCREEN, including the 'Predicate information' section below the plan to provide more details about your statement. Please use the noformat} [{noformat} code {noformat}] {noformat} before tag and {noformat} [{noformat} / code {noformat}] {noformat} tag or after the noformat} {{noformat} code {noformat}} {noformat} tag before and after to improve the readability of the outing:
In SQL * more:
SET LINESIZE 130 EXPLAIN PLAN FOR
; SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY); Note that the DBMS_XPLAN package. DISPLAY is only available from 9i on.
In 9i and above, if "Predicate information" section is missing from the DBMS_XPLAN. Output display, but you get the message "Plan table is old version" instead, you must recreate your plan table using the script server '$ORACLE_HOME/rdbms/admin/utlxplan.sql '.
In previous versions, you can run the following in SQL * Plus (on the server) instead:
@?/rdbms/admin/utlxpls
A different approach in SQL * more:
SET AUTOTRACE ON EXPLAIN
; also displays the execution plan.
In order to get a better understanding where your statement passes the time, you might want to turn on SQL tracing as described here:
When your query takes too long...
and after the "tkprof' out here, too.
Kind regards
RandolfOracle related blog stuff:
http://Oracle-Randolf.blogspot.com/SQLTools ++ for Oracle (Open source Oracle GUI for Windows):
http://www.sqltools-plusplus.org:7676 /.
http://sourceforge.NET/projects/SQLT-pp/ -
My daughter Air2 IPad takes too long to save. It is plugged in, connected to wifi and the screen is locked. Any tips?
How long is too long? If it lasts for more than a few hours - and even that is too long in my humble OPINION - then something is wrong. Completely backup, because it will not terminate in any case if its been so long, forced to reboot the device and try again.
To force the reboot your device, press and hold the two buttons of sleep/wake and home for at least ten seconds, until you see the Apple logo.
This may not work either. The backup may be damaged and you will need to remove and start over. However, its easy starting point and it will not hurt anything. If the backup will not end, he has nothing to lose.
-
I am a new user of Iphone, I must he keep plugging my Iphone to the computer during the update download process? especially the download will take too long because it is more than 2 GB and my internet speed is not fast, I can unplug my Iphone until the full download?
Without doubt, but why not just make the day downloading overnight?
-
Why it takes too long to open attachments of the e-mail account?
It takes forever to download attachments from my mail account. I tried to do the following tasks explained on the vista forum: http://windowshelp.microsoft.com/Windows/en-US/help/6b046ae9-1434-4423-9303-400ff6fe686b1033.mspx#ESD, but none of the fixed work possible.
After clicking on open on the box pop up asking if I want to open or save the attachment, it takes too long to download. The transfer window remains open to showing that he is ready to download, but rest in this window of bhox. I press cancel and try to open once again, if the chance that it opens this file, otherwise it takes forever forceing me to cancel. Very small files most of the time, usually between 50 KB should take seconds.
I even tried to save the files, but once again the same process. The transfer case remains open but does not download.
One of the ideas.
Thanks in advance.
Hello
It is not easy to tell exactly what is happening, but it must be something with the e-mail account provider and their page. For me, this case is not a typical problem of Vista, but you can try to find the solution on Vista from Microsoft IT Pro forum.
By the way: you have an alternative email by another provider address? Is it the same situation?
-
HP ENVY 15-j110, takes too long to start
Hey guys I have HP ENVY 15-j110 I bought recently, but now, after a month it takes too long to open and sometimes only a black screen.
He
Product number is this #ACJ F6C58PA
answer as soon as possible.
This could be due to the application running in the back ground and starting applications. Try
Selective startup:
http://Windows.Microsoft.com/en-us/Windows/Run-Selective-startup-system-configuration#1TC=Windows-7
-
Take too long time to come for Windows
Hello
I have Pavilion g6-2031se and of the last two days it takes too long to come to Windows. I faced this problem for the first time and I do not use my notebook for Skype, Facebook and only very light work. Can someone help me with this?
Thank you
Ganga
@Ganga,
No worries, I would like you to try a clean boot. This allows the computer to boot without unnecessary programs or offers to start with it. When you open this document of croll to the section "How to perform a clean boot" once listed there to expand the section for your operating system to get the appropriate instructions.
Once you have made the appropriate changes, restart the computer at least 3 times and see if you notice a difference not how long it takes to start.
Please answer as soon as possible.
Thank you
-
After the upgrade of RAM it take too long to begin and start windows.
Original title: laptop
Hello
I have (HP compaq presario V6615EN)Core 2 duo 1.83 GHz2 GB OF RAMWindows vista Home premium 32-bitIt took 20 seconds to start and windows boot(Yes, only 20 seconds)When I upgraded my ram to 4 GB it takes too long to start and windows bootnearly 160 secondsHello
1 - is the operating frequency of the matches with the 'old' RAM newly installed?
2. how many RAM sticks are there?
Let's try the following steps:Also launch memory diagnostic tool. How will I know if my computer has a memory problem?
http://Windows.Microsoft.com/en-in/Windows-Vista/how-do-i-know-if-my-computer-has-a-memory-problem
If there are 2 RAM sticks, remove one RAM Strip and check the speed. You can also try to swap the RAM sticks and check.
For more help, I suggest that you post your question in the HP support forum.
-
LaserJetPro HP 1102w impression takes too long
Hello
I installed HP 1102w on my laptop using the driver from the CD. Printing is configured using USB and ePrint is disabled.
Printing takes very long (18 s). When I send a print job (even test page), I don't see that it appears in the printer job window liquid. However until I see that some data has been sent to the printer (for example 4 KB / 146kb) time is about 9 seconds. To start sending data up to what the page is printed is another second 8-9 so time total is Reiver 18 seconds in the best of cases.
My laptop is:
Windows Visa Business, 3 GB of RAM, Dual-Core clocked at 1.6 GHZ, SSD drive 120 GB.
I tried to install the XPS (planned it will be faster) driver for this printer however failed to install with the 0x070c error code.
Please have a tip how to print faster (reduce the time of 8 seconds until that data is sent to the printer)
Thank you
Miro.
Hi @MKLI ,
I see by your post that the printer takes too long to print in a USB connection. I would like to help you today.
What kind of documents you print? (PDF, Txt)
Have what applications you tried?
Try a different file or application to see if you have the same results.
Try and print a self-test on the printer hardware to see if it is a hardware or software problem.
Printing a Configuration Page. Select the method two.
Make sure the printer is connected directly to a wall outlet and the USB cable is connected directly to the computer. The USB cable should not be more then 6 feet. Try a different USB port on the laptop. Some ports are higher power then others. You may need to try another USB cable to see if that causes this problem.
Check the power on the computer to see if that is causing the problem.
How to make in the Windows Device Manager?
Go to Device Manager, under Bus for each made USB root hub controllers right click and left click on properties, for the management of the power supply remove the check box for "allow the computer to turn off this device to save energy. It allows the computer to restrict or cut off the power for this unit.
Restart the spooler to print on the laptop.
Click Start (), type services.msc in the Search box and press ENTER.
Find the service named print spooler, right click and select Properties. Click on stop and then click Start. -
LaserJet pro m401dne: fatal error, takes too long to connect with the printer
Downloaded the driver of HP software, started installation and got to the point where it says to connect the printer, did, he recognizes it, start the data transfer and just takes forever. After ten minutes, I get fatal error takes too long to communicate with the printer, if aging persists restart the computer and try again.
I've made this 3 times now and same result.
Since my laptop doesn't have a cd drive, I copied the cd to a flash drive and installed the software from that. Worked perfectly. Something must be missing or corrupted download. Anyway problem solved
Maybe you are looking for
-
When I quit Firefox, I get all stop messages, so I guess he stopped it as it should. But if I go back to my PC (within an hour or two) and click on Firefox mentioned, I get the message 'runningbutnotresponding' and invites him to restart my PC.
-
DV6-6c65el: installation network and another unknown controller
Hello!! I am trying to use my hp pavilion dv6-6c65el Entertainment Notebook PC and use the internet, but I can't use internet. I can't connect to the internet. I need to install the network and the other unknown controller for this computer. This is
-
Compaq Presario V3440TU - drivers for Windows 7/Vista?
Hello I have a Presario V3440TU (p/n: GS115PA #AB5) with Win XP installed on it for a few years now. I would switch to Win 7, as the Win XP from Microsoft support lifecycle policy is now complete. Unfortunately, I can find no drivers Win7 or Vista to
-
Setting up with AT &; T
Hello. I live in England and I am trying to set up my girlfriend with internet access, who lives in the United States. She has AT & T with a DLink modem. We have both found support DLink as useless, and AT & T have not been helpful either, two of the
-
The phone contains updated the software cannot be updated. Any solution?
I have sony xperia arc s lt18i when I connect it to software update that contains his msg to the Softphone of change cannot be updated. any solution for this?