How to tune the loading using SQL LOADER

Hi all

I use SQLLDR to load the file into the target. All the files I am receving is ' | ' delimited. Some of the files size more than 3 GB. It takes a lot of time during loading.

Help me to know if it is possible of tuner SQLLDR.

Here is the example of control file.

DOWNLOAD THE DATA
UTF8 CHARACTER
INFILE ' / var/hulu/ptp/ABC.txt'
BADFILE ' / var/hulu/ptp/ABC.bad'
DISCARDFILE ' / var/hulu/ptp/ABC.dsc'

IN THE TABLE "PTP_ABC".
TRUNCATE

FIELDS TERMINATED BY ' |'
TRAILING NULLCOLS
(
CODE,
DESCRIPTION,
FILLING,
COKE
)

Change you your current sqlldr using external tables, this make the parallel load automatically, or launch sqlldr processes in the background of a single script.

We doing something similar to this with the help of a shell like this script:

nohup sqlldr userid=/ data=data01.dat log=my1.log control=my.ctl bad=my1.bad direct=yes silent=all parallel=true &
nohup sqlldr userid=/ data=data02.dat log=my2.log control=my.ctl bad=my2.bad direct=yes silent=all parallel=true &
nohup sqlldr userid=/ data=data03.dat log=my3.log control=my.ctl bad=my3.bad direct=yes silent=all parallel=true &
nohup sqlldr userid=/ data=data04.dat log=my4.log control=my.ctl bad=my4.bad direct=yes silent=all parallel=true &
nohup sqlldr userid=/ data=data05.dat log=my5.log control=my.ctl bad=my5.bad direct=yes silent=all parallel=true &
nohup sqlldr userid=/ data=data06.dat log=my6.log control=my.ctl bad=my6.bad direct=yes silent=all parallel=true &
nohup sqlldr userid=/ data=data07.dat log=my7.log control=my.ctl bad=my7.bad direct=yes silent=all parallel=true &
...

We are currently opening 20 simultaneous processes and we get an overal 400 000 platelets per second speed. You test what is the maximum number of concurrent processes, you can open.

Course which you need a first step to split the log into smaller chuncks. In my example, the file data.dat got cut into 20 smaller files, and each of them at the same time got loaded in parallel in the database. If you can not split the file, then you should go for external tables.

You can always test direct = true without any parallel option. This should be more than noticiable.

LW

Tags: Database

Similar Questions

  • How to remove the project using sql with the proper sequence

    Hello

    Tried to delete the project failed with query according to the sequence shown below. But get error when running below final sql script (delete from epub_process where of the project = '?')

    SQL error: ORA-02292: integrity constraint (PUB.) PROC_HIST_ID_FK) violated - book of the foundling

    02292 00000 - 'constraint integrity (s.%s) violated - child notebook found. '

    * Cause: attempted to remove a parent key value that was a stranger

    dependence.

    * Action: remove dependencies first then the parent or disable the constraint.

    delete from avm_asset_lock where workspace_id in

    (select avm_devline id, whose name in)

    (select Workspace from epub_project where project = 'prj49024')) ;

    delete from EPUB_PR_HISTORY where project in

    (select project in epub_project where project = "prj49024");

    delete from epub_project where project = "prj49024";

    delete from EPUB_PROC_HISTORY where ID_processus in

    (select ID_processus from epub_process where of the project = "prj49024");

    delete from EPUB_PROC_TASKINFO where id in

    (select ID_processus from epub_process where of the project = "prj49024");

    delete from epub_taskinfo where ID_processus in

    (select ID_processus from epub_process where of the project = "prj49024");

    delete from EPUB_WORKFLOW_STRS where id in

    (select ID from EPUB_IND_WORKFLOW where ID_processus in)

    (select ID_processus from epub_process where of the project = 'prj49024')) ;

    delete from EPUB_IND_WORKFLOW where ID_processus in

    (select ID_processus from epub_process where of the project = "prj49024");

    Get the error when running this script

    remove the epub_process of the project where = "prj49024";

    Please let me know good sequence or how to avoid this mistake

    Thank you

    Sri

    Here are the steps:

    delete from EPUB_PR_HISTORY where project in ('prj213002', 'prj200001');

    delete from EPUB_PROC_HISTORY where ID_processus in

    (select ID_processus from the epub_process of the project where in ('prj213002', 'prj200001'));

    delete from EPUB_PROC_TASKINFO where id in

    (select ID_processus from the epub_process of the project where in ('prj213002', 'prj200001'));

    delete from EPUB_IND_WORKFLOW where ID_processus in

    (select ID_processus from the epub_process of the project where in ('prj213002', 'prj200001'));

    remove the epub_process of the project where in ('prj202001', 'prj213002');

    delete from avm_asset_lock where workspace_id in

    (select avm_devline id, whose name in)

    (sélectionnez project_id dans epub_project où project_id dans ('prj213002', 'prj213002'))) ;

    delete from epub_project where project in ('prj202001', 'prj213002');

    commit;

    Peace

    Shaik

  • How to show the loading (wait screen) with animated image dialog between the screens

    Hello

    Can anyone show how to insert the load between two screens dialog, while the second screen is taken

    loading time?

    I tried the examples given in the forum but no luck

    There is an update about it here:

    http://supportforums.BlackBerry.com/T5/Java-development/sample-quot-Please-wait-quot-screen-part-1/t...

    Note comment added to the article as well - this is a useful Variant.

  • How to display the loading time of the page?

    Hello

    I would like to know how to display the loading time of the page? I know there is a var that must be in the footer, but I don't remember.




    Thanks in advance
    Antonio

    >
    You can use #TIMING # footer region the region to display the time taken to make this region - not sure about a whole page of
    >
    How about this the understands it in the front page template header...

    
    

    And then this in a div in the footer of the page template...

    
    

    See you soon

    Ben

  • How to know the database uses pfile or spfile?

    Hi guru

    How to know the database uses pfile or spfile?



    Thank you

    Thank you for this question of doc!
    Please type in sql * show spfile parameter.
    If it is empty, you do not use a spfile.

    --------------
    Sybrand Bakker
    Senior Oracle DBA

  • How to view clob data using sql

    Hello

    In our database we have a table that is to have a column of type CLOB but now I want to display the data using sql select query but it throws error: "unsupported data Type.

    Could someone let me know how to view the clob data using the select query.

    Oracle DB version: 10.2.0.3

    Thank you

    user562223 wrote:
    Hello

    In our database we have a table that is to have a column of type CLOB but now I want to display the data using sql select query but it throws error: "unsupported data Type.

    Could someone let me know how to view the clob data using the select query.

    Oracle DB version: 10.2.0.3

    Thank you

    Usually a to_char (CLOBDFIELD) circled

    HTH,

    FJFranken

  • How will I know if my Airport Extreme has the latest firmware?  And (not related) how to change the password used to connect to my network?

    I think I bought the latest version of Airport Extreme.  It is the unit which is rectangular, is about 6 to 7 w., etc.  I have some basic questions that I do not understand:

    1. How will I know if I have the latest firmware for this device?  I read a few posts that make it sound as if it was just automatically updates.  Is this true?

    2. can someone tell me how to change the password used to connect to my wireless network?

    Thank you very much!

    Chris

    If a firmware update is available, AirPort Extreme flashes orange.

    The most up-to-date version of the firmware is 7.7.3.

    You can see what version you currently have the following on your Mac...

    Open Finder > Applications > utilities > AirPort Utility

    Click on the image of the AirPort Extreme

    Look for the Version

    If the new firmware was available, you will see a button update here

    To change the wireless network password...

    Click on edit in the window smaller than you watched just to check the firmware version

    Click on the Wireless tab at the top of the next window

    Go back / change the wireless password and enter a new password

    Do the same thing to check

    Click Update at the bottom right of the window and give the airport a minute full for restart

  • How to change the rendering used for video playback in Windows Media Center Version 6 engine

    I usually use other media players, but I would like to first of all using Windows Media Center. However for the moment video playback is not good because of the weird colors (everything is displayed in black and Red instead of normal colors). When you use other media players, I use the Haali rendering engine which gives good results for video playback.

    Can you tell me how to change the rendering used by Windows Media Center?

    Also, I noticed the same weird color effect in Windows Media Player 11.

    Hi Woobee,

    1. What is the brand and model of the computer?

    2. what graphics card use on the computer?

    If you are using an Nvidia graphics card on the computer, in the Nvidia Desktop Manager, you will find an option for saturation to ensure it is set to 100%.

    And if you are any other graphics card on the computer and search for a similar option for saturation.

    If the previous step fails in Windows media player, so be sure that the hue and Saturation are defined by default.

    1. click on the arrow on the tabplaying , point to enhancements, and then clickcolor picker.

    2. do one of the following:

    1. to return to the default color, click thereset link.

    Also, make sure that the video settings are set by default.

    1. start the playback of a file.

    2. click on the arrow on the tabplaying , point to improvements, and then clickvideo settings.

    3. click onreset.

    Check also in Windows Media Center if the

    1. on theStart screen, go to tasks, click settings, clickgeneral, then Visual and sound effects.

    2. undercolors ensure that Windows media center standard is enabled.

    Turn off Visual and sound effects in Windows Media Center

    http://Windows.Microsoft.com/en-us/Windows-Vista/turn-off-Visual-and-sound-effects-in-Windows-Media-Center

    I hope this helps!

    Halima S - Microsoft technical support.

    Visit ourMicrosoft answers feedback Forum and let us know what you think.

  • How to start the database using CRS to start the database instance

    Dear Experts,

    How to start the database using CRS to start the database instance

    As a solution: remove the write permission on the/etc/oratab user database to prevent update to arrive.

    There is no harmful effects known to CRS / RDBMS operation if this workaround solution is used. This has been discussed in Bug 13028802.

  • How to know the tables used in packages of a schema.

    How to know the tables used in packages of a pattern that I have connected.

    SELECT DISTINCT referenced_owner, referenced_name

    Of all_dependencies

    Owner WHERE = "MY_USER_NAME".

    AND type ('PACKAGE', 'BODY of PACKAGE')

    AND referenced_type = 'TABLE '.

  • How to identify the fonts used in text format?

    How to identify the fonts used in text format?

    Hi DSB.

    Open the document in Reader, go to the menu file-> properties, then click on the tag are. You will see a list of the fonts used in the document.

    Another option is to copy the text and paste it into MS Word, where you can simply move your mouse over the text to display the name of the font.

    Kind regards

    Nicos

  • Help, how to query the database using pagination

    Hello

    How to query the database using paging for large data.

    any API for this?

    Thank you in advance!

    Hello

    You can specify XmlQueryContext::Lazy as the type of assessment and only call XmlResults:: next() comes as many times as you need.

    John

  • How to show the loading animation

    There is a 'loading... "animation when the Application is started for the first time."
    I want to know, how can I show the "Loading" a taskflow loading is underway?

    Thank you

    Shahab

    Hello

    unless you are looking for a programming solution, there is no splash screen that display for the loading of the workflow. You can use http://download.oracle.com/docs/cd/E21764_01/apirefs.1111/e12419/tagdoc/af_statusIndicator.html, which when added to the parent page shows a busy for queries (spinning O) status

    Frank

  • How to prevent the loading of the images?

    I'm working on creating a web gallery in Dreamweaver CS3.  The thumbnails are in a horizontal Strip in a layer above the layer of the image in full size, in other words, the Strip lies the part of the image, but only when the image is in a vertical position. (the band has no background, so all that is visible are the actual thumbnails) When you click Show/hide text, in a configuration of rocker, the band fades or shows.   On my computer, it works perfectly. However, once downloaded, it takes forever to load the images.
    The problem: even though I have thumbnail images are defined to preload, all full-size images are loading as well, that takes a lot of time. Is there a way to prevent the images full size of loading, i.e. only when you click on the thumbnail?
    Executives, in which a picture is miniature and the other pictures, is what I normally would use, and have a page for each image. However, for what I have in mind, it will not work (transparency about the parts of an image above, another is as far as I can tell is not possible).

    So how do we stop loading images up to ask?  best regards ~ bill-o

    The problem I had in the past has to do with when you put information in the dialog box it insists on the use of the html version (< for)<) in="" the="" code,="" and="" i="" have="" to="" go="" back="" and="" fix="" them="">

    It is exactly what it OUGHT to do.  You don't need to go back and fix anything.  The code written by this behavior, as I have suggested is appropriate and correct.

  • How to calculate the loading time of BDB startup

    Hi all

    I understand that Berkeley DB at startup it will read a few pages from the disk in the memory cache, but I don't know the exact number of pages that will load. I think that BDB will load default pages as much as possible in the cache until the cache is full. Is this good?

    And how to calculate the cost of time on the loading process? Can it be verified using db_stat?

    Thank you very much

    Hello.

    There is a discussion on how proceed (preload the cache) in this thread:
    Re: a question about read file or cache

    Ben Schmeckpeper

Maybe you are looking for