Performance problem when downloading the CSV files into Oracle tables

We have implemented the Vikas solution ([http://htmldb.oracle.com/pls/otn/f?p=38131:1]) to download CSV files into tables Oracle permanent and it works fine. However, whenever we try to send more than 5,000 records, it takes a lot of time and often wait times. What can we do to improve performance? We are running HTMLDB 2.0.

For example, a CSV file with 12000 documents and 40 columns takes 40 minutes to load (while a recording of CSV 2000 file takes 2 minutes to load).

For large external files, copy us our CSV files to the Oracle server and then make reference to the file in an Oracle external table. Once the external table defined you then load your APEX table via a stored procedure by using:

INSERT INTO your_tbl (columns)
SELECT (columns)
Of external_tbl

Hope that gives you another option to consider.

Jeff

Tags: Database

Similar Questions

  • ADF: How to manage the value zero when downloading the csv file

    Dear all,

    I'm new in the ADF and use 12.2.1. I am downloading of data in the database using af table: inputfile and CSVParser class. Data download correctly in the database where all columns are met, but whenever the csv file has value null (empty) in his column, then CSVParser class does not recognize it and get the next value of columns and assign the previous column. I also tried another one that uses a different class to download date but not get success.

    For Ex:

    ID name Dept

    1 abc Dept1

    2 cde Dept2

    3 Dept3

    In example above two first line are perfectly inserted in the table while the third line Id goes to ID column, due to the value of the column name is null class CSVParser does not recognize it and put Dept3 in the name column.

    Please help how to handle this.

    Thanks in advance.

    Kind regards

    Hello

    Try with this

    UploadedFile file = (UploadedFile)valueChangeEvent.getNewValue();
    InputStream is = null;
    BufferedReader br = null;
    try {
        is = file.getInputStream();
        br = new BufferedReader(new InputStreamReader(is, "UTF-8"));
        String line = "";
        while ((line = br.readLine()) != null) {
             String[] csvCols = line.split(",");
             // do somethinig smart with csvCols
        }
    }finally{
        if (br != null) try{br.close();}catch(Exception ex){br = null;}
        if (is != null) try {is.close();} catch(Exception exxx){is = null;}
    }
    
  • Windows media player has encountered a problem when downloading the file. for additional assistance, click web help.

    Original title: unable to audio stream.

    Unable to audio stream, "windows media player has encountered a problem when downloading the file.  for further assistance, click web help' web help does not work, clicking web help does nothing (no response from program... a lot of negative response from me) stuck with the big pass... vista 64 bit... a lemon from the getgo... problem after problem...

    Hi Joseph,

    What is the file format of the file you are trying to download?

    This problem normally occurs when the Media player could not play the file downloaded or not supported.

    Method 1:

    Step 1: Here is the list of the formats supported by Windows Media player file:

    Information on the types of media files that supports Windows Media Player:

    http://support.Microsoft.com/kb/316992

    Note: See if the file format is in this list, if it is not there, you may need a third-party codec that can read the file.

    Step 2: For more information about codecs, see article given:

    Codecs: Frequently asked questions:

    http://Windows.Microsoft.com/en-us/Windows-Vista/codecs-frequently-asked-questions

    Warning: using third-party software, including hardware drivers can cause serious problems that may prevent your computer from starting properly. Microsoft cannot guarantee that problems resulting from the use of third-party software can be solved. Software using third party is at your own risk.

    Method 2:

    I also suggest you to run the Windows media player Fixit:

    Solve the problems of Windows Media Player video and other media or library

    http://support.Microsoft.com/mats/windows_media_player_diagnostic

    Hope this information helps.

  • Windows Media Player has encountered a problem when downloading the file

    Original title: Windows Media Player does not work

    What this means: Windows Media Player has encountered a problem when downloading the file? and what do I do about it?

    Johnny: What are your exact measurements that lead to the error message?

    Temporary compensation Internet filed and the Cookies Internet Explorer could help.

  • How to download the Csv file with column headers

    Hi all

    This is pavan, using Apex version 4.2.3


    I am trying to download the csv file I followed this link , and I'm able to download excel with headers, when I try to download with headers of this error "'ORA-01858: a non-digit character was found here where was waiting for a digital" I searched in google but could not find the right solution, "


    can anyone help on this please.

    Thanks in advance,


    Kind regards

    Pavan

    This article is 6 years old.

    You should study the solutions that are available for APEX 4.2: data loader or the 'Excel2Collection' plugin (which also manages the CSV files).

    Data Loader

    It is a wizard that generates an Assistant for your application.

    Excel2Collection

    You will use the Excel2Collection (in a single process) to convert the BLOB in a Collection

    Then, in a 2nd address), you just do a "INSERT...". SELECT statement.  Add ' where seq_id > 1 "for files with a header.

    MK

    PS - Use the "EXECUTE IMMEDIATE" article is not necessary.

  • Import a CSV file into a table

    Hello

    I followed this example to download the CSV file in table in the database.
    http://avdeo.com/2008/05/21/uploading-Excel-sheet-using-Oracle-Application-Express-apex/

    It seems that no file is saved in this table - wwv_flow_files

    When I run the procedure laid down in the above example, I get error "No data found".
    Please, help me.

    I use Oracle Apex version 4.0.2

    Thank you!

    See my sample application

    http://Apex.Oracle.com/pls/Apex/f?p=51467:1:3854972221692354:no:

    user: test password: test

  • Load a CSV file into a table like in dataworkshop

    Workshop of data has a function to load a CSV file and create a table based on it, the same, I want to create in my application.



    I went through the forum http://forums.oracle.com/forums/thread.jspa?threadID=334988 & start = 60 & tstart = 0

    but not able to download all the files (application, package HTMLDB_TOOLS and PAGE_SENTRY function) could not find the PAGE_SENTRY function.

    AND when I open this link http://apex.oracle.com/pls/apex/f?p=27746

    I could not run the application. I've provided a CSV file and when I click on SEND, I get the error:

    ORA-06550: line 1, column 7: PLS-00201: identifier ' HTMLDB_TOOLS. PARSE_FILE' must be declared

    tried in apex.oracle.com host as shown in the previous post.

    any help pls..,.

    Another method to load data into the tables..., (as dataworkshop)

    Hello

    I have check app works very well.

    Have you read instructions?
    Load a CSV file in a table
    >
    Create a small csv file as

    col1, col2, col3
    VARCHAR2 (10), Number, "Number (10.2)" "
    Cat, 2, 3.2
    dog, 99, 10.4
    >
    First row must have valid column names. To verify that your first line of titles have no spaces or those who are not words reserved.
    Second line of the CSV file must have column of table data types.

    When you meet these requirements app works perfectly

    Kind regards
    Jari

  • Import from CSV file into oracle using pl/sql - sql loader 3.1

    Hello

    I am trying to learn how to import a CSV file in an oracle table. I tried the import utility in SQL Loader, but it generates specific lines of code to import each line. I would like a loop to go through the CSV file stored on my local disk and transferred it to my table.

    I tried the following, but not luck
    DOWNLOAD THE DATA
    INFILE 'sample_csv.csv '.
    IN THE TEST TABLE
    FIELDS TERMINATED BY ', '.
    ;

    951716 wrote:
    Hello

    I am trying to learn how to import a CSV file in an oracle table. I tried the import utility in SQL Loader, but it generates specific lines of code to import each line. I would like a loop to go through the CSV file stored on my local disk and transferred it to my table.

    line-by-line is slow by slow.
    PL/SQL will be NEVER faster than simple SQL
    INSERT INTO MYTABLE SELECT * FROM EXTERNAL_TBL;

    http://www.orafaq.com/wiki/External_table

  • Load the XML file into Oracle external Table


    I load the data from the XML file into an intermediate table Oracle using external Tables.

    Let's say below, it is my XML file

    < header >
    < A_CNT > 10 < / A_CNT >
    < E_CNT > 10 < / E_CNT >
    < AF_CNT > 10 < / AF_CNT >
    < / header >
    < student >
    <>students-details
    < Student_info >
    < Single_Info >
    < ID > 18 / < ID >
    New York < City > < / City >
    < country > United States < / country >
    < Name_lst >
    < Student_name >
    Samuel < name > < / name >
    Paul < Last_name > < / Last_name >
    < DOB > 19871208 < / DOB >
    Aware of < RecordStatus > < / RecordStatus >
    < / Student_name >
    < Student_name >
    Samuel < name > < / name >
    Paul < Last_name > < / Last_name >
    < DOB > 19871208 < / DOB >

    < TerminationDt > 20050812 < / TerminationDt >
    History of < RecordStatus > < / RecordStatus >
    < / Student_name >
    < / Name_lst >
    < Personal_Info >
    <>men < / Type >
    < 27 > < / Age >
    < / Personal_Info >
    < / Single_Info >
    < / Student_info >

    < student - register >
    class < A >
    < info >
    < detail >
    < ID student > 18 < / student >
    EE < major > < / Major >
    < course-Grades >
    < course > VLSI < / course >
    < degree > 3.0 < / Grade >
    < / course-Grades >
    < course-Grades >
    < course > nanotechnology < / course >
    < degree > 4.0 < / Grade >
    < / course-Grades >
    < / details >
    < detail >
    < ID student > 18 < / student >
    THIS < major > < / Major >
    < / details >
    < / info >
    class < A >
    < Student_Enrol >
    <>students-details
    < student >

    I load this XML data file into a single table using an external Table. Could someone help me please with coding.

    Thank you

    Reva

    Could you please help me how to insert my XML content into that.

    Same as before, try a plain old INSERT:

    insert into xml_pecos

    values)

    XmlType (bfilename ('XML_DIR', "test.xml"), nls_charset_id ('AL32UTF8'))

    );

    But you'll probably hit the same limitation as with the binary XMLType table.

    In this case, you can use FTP to load the file as a resource in the XML DB repository.

    If the XML schema has been registered with the hierarchy enabled then the file will be automatically inserted into the table.

    Could you post the exact statement that you used to save the scheme?

    In the meantime, you can also read this article, I did a few years ago, it covers the XML DB features that may be useful here, including details on how to load the file via FTP:

    https://odieweblog.WordPress.com/2011/11/23/Oracle-XML-DB-a-practical-example/

    And documentation of the course: http://docs.oracle.com/cd/E11882_01/appdev.112/e23094/xdb06stt.htm#ADXDB4672

  • Load the XML file into oracle DB using OWB

    Hi friends,

    I'm new XML data.
    I had a requirememnt to load the xml file into an oracle table.
    Can someone procedure to load the data.

    Thanks in advance

    see this

    http://blogs.Oracle.com/warehousebuilder/2007/09/leveraging_xdb.html

  • Load the XML file into oracle using sql loader

    Hello

    I'm trying to load an xml file into a table using sqlldr.
    I have a table as follows:
    CREATE TABLE xmlloadtable
    ( id number,
     data_xml XMLType
     )
      XmlType data_xml STORE AS CLOB;
    I have a control file that I know is perfectly false:
    LOAD DATA 
    INFILE '/home/oraread/'
    INTO TABLE xmlloadtable 
    (
    id,
    data_xml 
    )
    I googled for the file control and tried in different ways, but of no use.
    I want to load the entire xml file into the table.
    Can someone help me with the correct control file. Any help is appreciated!

    Try this

    LOAD DATA
    INFILE '/home/oraread/'
    INTO TABLE xmlloadtable
    (id,
     file_name          filler,
     data_xml          lobfile(file_name) terminated by eof
    )
    

    You can add the file as a column name field to your table between the id and xml column and remove the word to fill in the .ldr file if you need to keep the name of the file

  • I have a problem in direct charge of path where the default values are not met when using the CSV file to load into the database while his is filled everything in plain insert queries

    -The header of the control file, I added a load direct method(append mode)

    OPTIONS(direct=true,Parallel=true,skip_index_maintenance=true,skip_unusable_indexes=true)

    DATA RELATING TO SUNK COSTS

    ADD

    INSERT INTO TEST_TBL

    (

    col1,

    ...

    Col n-2

    );

    Try to use the classic path instead.  With the direct path, many constraints and triggers, and others are disabled.  Please see the next section of the online documentation.

    https://docs.Oracle.com/database/121/Sutil/GUID-973925DA-8F86-49C1-A707-4564DC3B57AE.htm#SUTIL1330

  • Problem with the date when you load the XML file into Oracle Database 10g

    Hi all

    I have the interface as shown in the screenshot below. In this document, among other things, I am mapping to an element XML file representing a date to an Oracle table column defined as DATE. The source and target columns are highlighted in the screenshot.

    ! http://img223.imageshack.us/img223/1565/odiscr275.jpg!

    When I run the interface, I get the following error message:

    java.lang.IllegalArgumentException to java.sql.Date.valueOf(Date.java:103)

    I guess that this is the conversion of the date!

    I already tried to replace SRC_TRADES. DEAL_DATE with TO_DATE (SRC_TRADES. DEAL_DATE, ' DD/MM/YYYY') on the implementation tab. This feature was not recognized when I executed the interface, so it did not work! The value of date in the XML file is in the format DD/MM/YYYY .

    I guess that Date SQL Oracle functions do not work in the implementation tab. Could someone let me know:

    1. what the Date Conversion function I could use instead?
    2. where can I find a reference to the methods/functions that I use in the implementation tab (if such a reference exists)?

    See you soon.

    James

    Hello.

    Try changing the area of execution at the staging area. Once you change it, write in the map box just SRC_TRADERS. DEAL_DATE. When you use TO_DATE, the source field typu should be varchar2, no date (as it is in store for your data source)

  • Download the .csv file display output white

    Hi all

    We are facing problem when we try to download the output from one page to the .csv format or any other format. After downloading, we not get any data in .csv file.

    Action - Download-> .csv file displays only not all data

    Please help me how to solve this problem.

    Kind regards

    Sushant

    What is the relevant SQL? Do you have any bind variables? They are included in session state?

  • Question to load the XML file into Oracle via ODI: LKM and IKM adjustment

    Hi all

    I am currently building an ODI interface to upload my file system XML local in a 10 g to Oracle database.

    I received the error message shown in the screenshot below:

    ! http://img529.imageshack.us/img529/3491/odiscr175.jpg!

    The main problem I have is, I don't know all what LKM or IKM assign.

    Can someone offer any assistance/advice?

    Thanks in advance.

    James

    James
    You must select the LKM SQL for Oracle (or LKM SQL for SQL) and then within Oracle to integrate Oracle the IKM incremental update or append IKM SQL command.
    Reading XML, you can just use the standard SQL KMs. (when writing, there's a little more necessary artifice, and this can be done by using specific IKMs)
    Hope this helps

Maybe you are looking for

  • Firefox imports only half of my favorites of IE

    When I try to import my favorites from internet explorer, only half of my favorites (a - m) are transferred. Is it possible to increase the capacity of the folder?

  • HP 8600 more

    We have a problem with our printer HP 8600 more. Our operating system is Windows 7. Very often, it's not printing the last line of the page or it is impossible to read. But if I print 5 pages, for example, 4 are perfect, but in the first, the last li

  • is there a limit on the length of maximum dynamic license key?

    Hello Our dynamic keys are 32 characters long. However, we are receiving reports back from users with licensing issues. Download reports are showing the fixed keys 24 characters. Is anyone aware of a maximum key length limit? Thanks for your help

  • Authentication IPSEC Client tunnel computer

    I'm trying to convert our access existing distance from Microsoft Threat Management Gateway Cisco ASA.  Our security people just made me aware that in addition to the Radius authentication against powers AD they want as machine authentication to make

  • my hp printer does not work with my Mac, but because our hp computer. Why? CC = en

    my printer does not work with my Mac, but because our hp computer