The fastest way to import data for large tables
Hi friends,I recently joined a billing product put up a team for TELECOM giant. Under the procedure of installation/initial tests, we have huge data (size of almost 1.5 billion records / ~ 200 GB of data) in some tables. Currently I use impdp to import. It takes a long time (almost > 9 hours to load these data).
I tried the ways below to import the data (all of these tables is partitioned):
1. normal impdp (single thread that is, parallel = 1): it takes a long time (> 24 hours).
2. normal impdp but partition-wise. It was completed in relatively less amout of time compared to the 1st method.
3 fall drop the index, framed adding another scheme in the same forum that includes data, and then recreate the index (whole process takes about 9 hours)
My questions at all:
1. is there any other way/trick in the book that I can try to put the data in less time
2. what I pointed out that even if I give parallel = 8 or more, sometimes parallel workers are laid and sometimes not. Can someone tell me why?
3. How can I come to know (before running the impdp) that my parallel article will spawn parallel workers or not. As to how I will come to find out, I searched this topic but without success.
I don't know what strategy to follow, because it is an involved and repetitive task for me. Because of the task above, I am not able to focus on other DBA tasks because of what comes to me.
See you soon,.
Malika
assuming that you have a table with 3 clues, then create a script for each index (reblace index_name_n with your index name) and start them at the same time, after the importation of the table is complete
DUMPFILE= or NETWORK_LINK=
DIRECTORY=DATA_PUMP_DIR
LOGFILE=
CONTENT=ALL
PARALLEL=1
JOB_NAME=IMP_
INCLUDE=TABLE_EXPORT/TABLE/INDEX:"IN('')"
TABLES=.
Change the following settings when importing:
increase pga_aggregate_target
increase db_writer_processes
db_block_checking = false
db_block_checksum = false
the value undo and temporary ts autoextend
Set noarchivelogmode
create logs of restoration of 4 GB
HTH
Tags: Database
Similar Questions
-
What is the fastest way to import folders of message in my new identity
I want to spend my addresses end messge files to my new identity thanks Arnie
Because you are using XP and Outlook Express is the only e-mail program that has multiple identities, I guess you mean that? Unless you have a different definition of identity.
For your messages:
In the old identity, firstly, compact all your folders.
In the new identity: file | Import | Messages. Select Microsoft Outlook Express 6 and import from the old identity.
To your address book:
Open the address book in the old identity and file | Export | Address book (wab) to any location on your HARD drive that is easy for you. My Documents is fine. Then, open the address book in the new identity and file | Import | Address book (wab), and then tap on where it was saved.
-
The fastest way to transfer data between two Lacie Thunderbolt external HDs
Hi all. I have a ton of music on a Lacie Rugged 1 TB Thunderbolt/USB 3.0. I use to mix. I have a Macbook Pro 15 "w 2015 / retina with Thunderbolt ports two (2) and I try to use it to transfer all the data from my Lacie Rugged drive to my new Lacie 3 to Desktop HD that was Thunderbolt (2) and USB 3.0 and use also the new Lacie Desktop drive as my primary for backups time machine.
In order to transfer data from the rugged drive (THE most RAPID WAY POSSIBLE) on my new Lacie desktop drive I should? :
(A) connect to each port thunderbolt on the laptop itself?
Or
(B) connect the Lacie Rugged HD to the new Lacie Desktop HD then the Lacie Desktop HD to the thunderbolt on the laptop (daisy chain) port?
Thank you!
MacWorld has done some tests on this and found a slight increase in performance with the devices connected directly to the laptop. So don't try Garland!
-
Option 1:
Insert the statement with:
table view: NOLOGGING
insert mode: APPEND
in ARCHIVELOG mode: noarchive log mode
Option 2:
DEC in NOLOGGING mode
The two options above would generate no redo log. What is the best for performance? I'm loading large or lines (a few million) on a daily basis and it is an intermediate table, so there is no problem when the reprocessing failure.
The answer may depend on the availability of memory and index.
ETG requires you to create indexes subsequently - but each index requires an analysis of the table.
Insert / * + append * / can optimize for indexes by capturing the values of the columns needed as data is loaded, and then creating the index without having to re-read the table.
Concerning
Jonathan Lewis
-
simple, the best way to import data, music computers betwqeen from pc to pc to laptop
I want to transfer music from my pc to my phone, what is the (best) easiest way?
Based computers involved and the amount of data involved. You can transfer data using a Flash DRIVE, burn a CD/DVD-R, on the network, pulling on the hard drive and attaching it to the target machine, etc.. It is not uniform so if you want a more specific answer, provide the missing details on your systems and your data. MS - MVP - Elephant Boy computers - don't panic!
-
The fastest way to transfer files between servers...
I have a question regarding what would be the best option to accelerate the transfer of files between 2 servers to our VMware environment. We plan to spend our GroupWise email from netware to windows and upgrade to Groupwise (v7 to v8) at the same time. We will anticipate moving about ~ 125 gigabytes of data (files ~ 517 000) and expect that it will take several hours. We would like to minimize this time better we can so we wonder what would be the best configuration of things to get there? Two virtual machines on the same host? Two virtual machines on the data store even? On different hosts, or data warehouses? Our configurations are listed below:
-In the course of running vSphere 4.1 ESXi
-Data stored on 2 Dell Equalogic 5500 without
-SAN one in each of the 2 sites, physically about 3 miles away
-3 guests, 2 to a location and 1 in the other (same places the San)
-Transfer of data from Netware 6.5 source, destination will be Windows 2008 R2 DataCenter
-Links gigabit between all the elements of our VMware environment, including the link between the sites (on optical fiber).
-3 guests are able to run all our virtual machines without overloading, so if we go to the virtual computer to another host, it is not a problem.
We would like to know what is the best way to put everything in place for the transfer, so that we can prepare before hand. Any suggestions?
Thanks in advance.
Well, the fastest way would be to offload the clone/copy to the San operation. If your configuration supports VAAI then the clone would happen completely on the SAN by assuming that the source and destination storage device is the same SAN. Would it be different San then the network factors in anything. For what is running on the same host, which would be beneficial for cloning a virtual machine (mainly memory footprint) should not take place on the network.
I hope this helps!
-
The best way to migrate data... Opinion please...
Guys,
I'm working on a solution for a customer who needs to move in a new SAN SAN data.
Existing SAN is attached to the ESX server farm and is very close to the location of a new SAN. In your opinion, what is the best way to move data without interruption to VM ESX farm?
I know we can do SAN replication, but right now, I don't know if the SAN is still the same provider.
I know we can do Storage VMotion. In this case, he should present the new SAN somehow for the existing ESX servers.
I know that we can probably use the converter to migrate virtual machines. This can take some time because we would go over WAN.
What are your thoughts?
Thanks in advance for your comments!
With 4 TB of data, I wouldn't pass on the replication SAN or WAN solution of optimization of the property, and it can fly quickly with 4 TB of data that you use storage vmotion which basically depends on how your WAN links are, if if its quick to go for it. There is not another free solution then this but if you can manage somehow to make backup of system state of all virtual machines, and then you can ship these backup data to remote locations and import, save back to clusters. You have to do the actual calculation how fast you can Transfer data from the siteA-> siteB Mbps? This will give you a clearer picture of how long it will take and picture plan here. I could do that during maintenance or weekend windows that are the least impact to users and systems.
If you found this information useful, please consider awarding points to 'Correct' or 'useful '. Thank you!!!
Kind regards
Stefan Nguyen
VMware vExpert 2009
iGeek Systems Inc.
VMware, Citrix, Microsoft Consultant
-
the fastest way to load an XML file
What is the fastest way to generate an xsd-based oracle table and then load data from a file that conforms to the xml document in the xsd.
I would do something like this:
SQL > insert into tmp_xml
2 values)
3 xmltype (bfilename('TEST_DIR','test.xml'), nls_charset_id ('AL32UTF8'))
4 );
1 row inserted
SQL >
SQL > select x1.col_a, x1.col_b, x1.col_c, x2.*
tmp_xml 2 t
3, xmltable)
4 "/ parent/Child1.
5 passage t.object_value
path of 6 columns col_a varchar2 (10) '@ColumnA '.
7 road of varchar2 (10) col_b '@ColumnB '.
8 road of varchar2 (10) col_c '@ColumnC '.
9, child2 xmltype path "Child2".
(10) x 1
11, xmltable)
12 ' / Child2'
13 passage x1.child2
path of 14 columns col_d varchar2 (10) 'ColonneD '.
15 road of varchar2 (10) col_e "child3/column.
16, road of varchar2 (10) col_f "child3/ColumnF.
17 road of varchar2 (10) col_g "child3/column".
18 road of varchar2 (10) col_h "child3/ColumnH.
19 road to varchar2 (10) col_i 'ColumnI.
(20) x 2
21;
COL_A, COL_B, COL_C COL_D COL_E COL_F COL_G COL_H COL_I
---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ----------
CAF ClientID ClientName 20130121 D 20752,19 AUD
FCA ClientID ClientName M 20130121 235.39 bla Blahdeblah AUD
(you can adjust data types as needed)
-
The fastest way to the files and folder to move
I'm moving a lot of files to another drive and it takes a long time. I do travel in the LR library now and an alternative is to move outside of the LR and use the catalog import option to add (don't know if it's a more quickly).
You can select multiple folders in the LR library to move to the same location or do you need to do one at a time? This would be onen advantage to do outside of LR.
Any thoughts?
You cannot move several files in Lightroom.
The fastest way is to move your "Pictures" folder all OS, then re - link in Lightroom by right-clicking and choosing 'find the missing folder '.
If you need to move several subfolders selectively, then there is no quick way. You move quickly in the BONES and then go rebinding in Lightroom or move them slowly one by one in Lightroom.
-
How to use mobile data for large app download data
How to use mobile data for large app download data
How can I set big data
-
What is the best way to import and work with mxf files in first CC2015?
I am trying to find the best way to import MXF files in first, like many people there seem to be issues.
I have already imported using first CS6 but I got cut short clips and audio dropouts. I decided to start over with CC2015 first, but this time I'd like to do things and once. Should I use prelude first?
I run on a Mac Pro (mid-2010)
OS X El Capitan version 10.11.1
All good, got sorted. I use the prelude.
Thank you!
-
What is the fastest way to convert navigation app 1-level 2-level once a project has already been established?
You cannot change a project to change this setting. You will need to create a new project.
-
Hi Experts,
What is the best way or best practices for access to the session defined in servelt pipeline component?
Please, share your ideas.
Thank you
ankV
As performance is concerned, a lot would depend on as to how your design & logic is implemented and you presented search operations to support the specific performance problems. In fact the operation itself can be not expensive than in-house it is somewhat like a search for key/value of the objects in session/application context name. But having said that, Yes, a potential performance problem could be because it's a synchronized operation. So to avoid searches to be performed during each request, caching the results of a search of a component. And in the majority of cases to solve a 'A' component within your component, you must configure a property reference 'A' in the file properties for your component.
-
the fastest way to deploy on iOS / Alternative to drag the app in itunes?
What is the fastest way to deploy on the iOS device to test?
I myself remember someone has posted a tip on this subject, but I was more focused on android at the time.
I use CS5.5 on Windows
Try here iPhone Configuration utility:
http://www.Apple.com/support/iPhone/Enterprise/
That would get the IPA on a device quickly connect via USB. If you want to send it to a lot of beta testers, then watch https://testflightapp.com/.
-
The best way to pass data when browsing
By taking the case where an action method is called and he chooses navigate to another page.
What is the best way to pass data from bean of old media to the new (probably in the execution of the action method)? I am currently using beans of spring session scope as the support of beans. Take a look at:
It goes between your commandLink or commandButton control:
Maybe you are looking for
-
local folders default reply address is outdated email account
I've recently set up TB (31.4.0, windows 7 pro), import a set of local folders and emails to outlook express. TB has associated the latter with an address e-mail which I use is more, although there are always and sometimes I receive mail on this subj
-
HP Mini 110-4110sc: creator MIE HP Mini 110-4110SC are not not in downloads
Hi people, I have a HP Mini 110-4110sc with broken HARD drive on my hands here and of course without recovery of support has been created by the owner when bought. Also, I found discussions on this forum regarding the download of the image and media
-
Problems connecting to the internet with my wireless router
I have a WRT54GS v7 wireless router that I use with a modem ADSL ZyXEL P-660R-D1. during installation, I was unable to establish a connection to the internet, once my router is properly connected to my desktop computer and my modem. All the correct l
-
I can't start the recovery on my pavilion dv6
I have the recovery DVD 4 and I used them once before and it worked indeed.now I want to restore my laptop, like when I bought it, I put the first DVD rebooted the unit pressed F11 put it says: "Unable to start windows. a recent hardware or software
-
3537/AMD HD8670 stops completely, or wakes up from his sleep
Hi all As the topic says I have improved my system with Windows 10 laptop Dell inspiron 3537 After a new installation for win 10 and install all the drivers of your website with the suggested installation order, the laptop does not completely turn of
