Load and read XML file size more than 4 GB

Hi all

My environment is Oracle 10.2.0.4 on Solaris and I process to work with the XML file as below the detail of PL/SQL

1. I read the XML file on the HTTP port in the XMLTYPE column to table.

2. I read value n ° 1 of the table and extract of y to insert another table

On test db, everything is working but I got below error when I use the XML production

ORA-31186: Document contains too many nodes

Current XML size about 100 MB, but the procedure must take over the size of XML file over 4 GB in the future.

Below are some of my code for your info.

1. reading XML by line in variable and inserting into table

LOOP
Utl_http.READ_TEXT (http_resp, v_resptext, 32767);
DBMS_LOB. WriteAppend (v_clob, LENGTH (v_resptext), v_resptext);

END LOOP;

INSERT INTO XMLTAB VALUES (XMLTYPE (v_clob));

2. read the value of the cell in the XML column and extract to insert into another table

DECLARE
CURSOR c_xml IS
(SELECT trim (y.cvalue)
FROM XMLTAB xt.
XMLTable ('/ Table, rows, cells, cell ' PASSAGE xt.) XMLDoc
COLUMNS
cvalueVARCHAR (50)PATH ' / ');

:

:

BEGIN

:

:

OPEN c_xml;
SEEK c_xml INTO v_TempValue;
< generate the insert statement in another table >
OUTPUT WHEN c_xml % NOTFOUND;
NARROW C_xml;

:

END

And another problem is performance problem when the XML file is great, first step to XML content loading to XMLTYPE column slowly.

Could you please suggest any solution to read large XML file and improve performance?

Thank you in advance.

Hiko

Finally, change us source file to XML, CSV format and instead load the XML using SQL Loader.

Tags: Oracle Development

Similar Questions

  • does anyone have example script writing and reading XML file

    someone at - it an example script for writing to variables affecting and reading variables to an xml file.

    Discover the second to the last link on this page:

    http://www.Cisco.com/en/us/partner/products/SW/custcosw/ps1846/products_implementation_design_guides_list.html

    There are some very good examples.

    HTH, please rate all positions!

    Chris

  • Hyberfil.sys file size more than 2 GB! Window 7 Ultimate

    Hello

    I use Windows 7 ultimate 32 bit Os type in its root directory I found hyberfil.sys that show its size 2.36 GB is it normal or I can reduce its size manually change can cause a serious problem.

    Please guide me.

    Thank you

    OPB

    Hiberfil is used for the hibernation.  You can't set the size.  It is a function of the amount of RAM you have.  If you don't use hibernation you can completely eliminate the file by typing powercfg-h off in an elevated command prompt.  (Start > run > cmd > right click > run as administrator)

  • External table - load a log file with more than 4000 bytes per column

    Hello
    I'm trying to import a log file into a database table that has a single column: txt_line
    In this column, I'm trying to fill out a log by record type entry. Each log entry is normally more than 4000 bytes in the outer table, it should be a clob.
    Below is a table of external work that works, but cut all entries after 4000 bytes. How is it is possible to directly load the data into a clob column? All I've found are descriptions where I have a clob-file by file.
    Any help is appreciated
    Thank you



    Source file
    .. more than 4000 bytes...]] .. .more Quen 4000 bytes...]] .. more than 4000 bytes.

    ]] ist the record delimiter

    External table:
    create the table TST_TABLE
    (
    txt_line varchar2 (4000)
    )
    external organization
    (type
    ORACLE_LOADER
    the default directory tmp_ext_tables
    (settings) access
    records delimited by a "]]"
    fields (txt_line char (4000))
    )
    location ("test5.log")
    )
    reject the limit 0
    ;

    user12068228 wrote:

    I'm trying to import a log file into a database table that has a single column: txt_line
    In this column, I'm trying to fill out a log by record type entry. Each log entry is normally more than 4000 bytes in the outer table, it should be a clob.
    Below is a table of external work that works, but cut all entries after 4000 bytes. How is it is possible to directly load the data into a clob column? All I've found are descriptions where I have a clob-file by file.
    Any help is appreciated
    . . . E t c...

    And what did you expect if you define the field source and target column as 4000 characters?

    Try this:

    CREATE TABLE tst_table
     (
       txt_line CLOB
     )
     ORGANIZATION EXTERNAL
     (TYPE oracle_loader
        DEFAULT DIRECTORY tmp_ext_tables
        ACCESS PARAMETERS (
           RECORDS DELIMITED BY ']]'
           FIELDS (txt_line CHAR(32000))
        )
      LOCATION ('test5.log')
     )
    REJECT LIMIT 0
    ;
    

    8 2

  • Multi load: error downloading the text file with more than 5 times

    Hi all

    I am getting below error while trying to load a text file with more than five periods of data through Multi charge.

    " File can not be treated because the number of specified periods exceeds the available periods!" ".

    FLTFILE_TO_PLANNING3
    BUDGET
    30/04/2013
    6
    Q.
    A, C, UD1 UD2 UD3, UD4, UD5, UD6, UD7, UD8, V, V, V, V, V, V
    AC_11102, CC_104204, Local, year 0, target, Final Budget, SA_000000, P_000001, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110
    AC_11102, CC_104207, Local, year 0, target, Final Budget, SA_000000, P_000000, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110
    AC_12542, CC_102101, Local, year 0, target, Final Budget, SA_000000, P_000000, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110
    AC_11102, CC_113242, Local, year 0, target, Final Budget, SA_000000, P_002478, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110
    AC_11102, CC_113243, Local, year 0, target, Final Budget, SA_000000, P_000000, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110
    AC_17802, CC_113244, Local, year 0, target, Final Budget, SA_000000, P_000856, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110
    AC_11102, CC_113511, Local, year 0, target, Final Budget, SA_000000, P_000000, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110
    AC_11102, CC_124111, Local, year 0, target, Final Budget, SA_000000, P_000521, HSP_InputValue, entry, 60, 70, 80, 90, 100, 110

    This file works fine with 5 periods.

    Please notify.

    APR

    Hello

    Yes, table period contol FDM was not properly defined. Now it works fine. Thank you very much.

    APR

  • ___When I have a file I multi page and "Save as", he saves more than just the page that I was clear

    When I have a multi page have e wire (50 pages) and "Save as", he saves more than just the single page, that I was clear (IE. Rank 1-1)... it saves all 50 all over again? How to save just 1 of 50?

    Just answered my own question - if I use 'save a copy' it works... must be a glitch

  • Can we load and unload the files in the run time?

    Can we load and unload the files in the run time?

    For example, there are four files named 'test1.h & test1.c' and another series 'test2.h & test2.c' (I have attached as an attachment to this post).

    test1.h contains the code:

    int variable; variable declared as integer

    test1.c contains the code:

    variable = 1; variable a value assigned

    test1.h contains the code:

    char * variable. variable declared as string

    test1.c contains the code:

    variable = "EXAMPLE"; a string that is assigned to variable

    So here, in this case can I dynamically load / unload first & second group files so that the same variable "variable" name can be used as an integer and string? And if so, how to do?

    Hi pa1majeti,

    You cannot simply consider that some code is loaded into memory or not: all code in a compiled application is permanently in memory unless you divide your application into DLLs that you dynamically load if necessary and remove if is no longer necessary.

    But as I understand from your description, you're better given the amount of memory occupied by the data (i.e. the contents of the variables) rather than the code. If this is the case, your only option is to allocate memory for variables dynamically and free it after it has been used. Now, even if this option is useful if you are using large networks of data, there is little or no effect on the unique variables.

    A final Council, given the huge memory installed on modern computers, are you sure you really need that extra work? I mean, the dynamic allocation of memory involves some additional precautions in programming: check the errors on each function malloc or calloc, check that the pointer is valid before each use, freeing pointers before reassignment in order to avoid memory leaks and so forth, so a balance must always be sought between the memory and the difficulty in programming.

  • Windows 10 Lightroom 6 (not CC) installed and reinstalled but still opens with a HUGE text in the two panels, leaving only a small space to see photos. Preferably and other screens also occupy more than full screen. Appears as tho Lightroom is in failure

    Windows 10 Lightroom 6 (not CC) installed and reinstalled but still opens with a HUGE text in the two panels, leaving only a small space to see photos. Preferably and other screens also occupy more than full screen. Appears as tho Lightroom is in failure Mode (failure mode). Can anyone help?

    Hello

    Click Edit > Preferences

    Go to the interface tab and check the font size: it should be automatic.

    Press enter

    Restart the Lightroom

    Thank you

  • If I put a restriction of password which only allow customers who purchase print it (no copy or modification of the document) have an impact on the quality of the file when people open and read the file on mobile devices like phones and tablets?

    If I put a restriction of password that only allow users who buy to print it (no copies or the modification of the document allowed)-it will have no effect on the quality of the file when people open and read the file on mobile devices like phones and tablets?

    Some mobile devices won't be able to open it at all, but if it is open, the quality is not affected.  Please note and understand the warning you get when you set security: indeed, it is largely ignored by many apps.

  • vCO and reading/writing files

    Hello community,

    I am a newbie with vCO, sorry for the basic question perhaps.

    I created a "workflow" for automatic deployment and customization of VMs. job well enough, the model is cloned, sysprep, vCPUs and RAM set and so one. But the final customization tasks is to assign a unique IP address that is in a list in a plain text file. vCO should open the file, take the first IP address in the first line, remove it and close the file (DHCP is not an option). I can open and read the file with the options of '.readall' and a 'player', but which is not exactly what I want. I tried a few '.readline' which translates as "cannot find function readline" and after I I thought to split the entire string that resulted in "no defined split." I really don't know if this is the right way.

    Can someone please guide me in the right direction, perhaps with a few code snippets?

    Thank you very much!

    S.

    OK, here's another example, which reads the entire file into a string, and then use split() for areas exploded in the elements of an array:

    var fr = new FileReader("C://vcoscripts/demo.txt");
    fr.open();
    
    var complete = fr.readAll();
    System.debug("complete: " + complete);
    var lines = new Array();
    lines= complete.split(/\r\n|\r|\n/);
    System.debug("lines: " + lines);
    for (var linenr in lines) {
        var currentLine = lines[linenr];
        System.debug("currentLine: " + currentLine);
        }
    

    With this, you can use methods of standard table (push(), pop()), which might be easier to use.

    The regular expression to split the lines that I got:

    http://StackOverflow.com/questions/1155678/JavaScript-string-newline-character

    (good reading...)

    Edit: I forgot: with Array.strip (), you can remove an element of a table (in your case the IP address used before writing it back to deposit via FileWriter...)

    Kind regards

    Joerg

  • How can I load and process a file of HTML5 with Javascript in the same way as an XML file?

    I keep a journal in local files of the weeklies that I'm converting to HTML5 of XHTML. I want to extract all 'items' on specific themes of these files and display them in different orders. It is possible to load an XHTML file using the loadXMLDoc function described in http://www.w3schools.com/dom/dom_loadxmldoc.asp , but the same function detects a parse error when it reads a file of HTML5. (For example, the parser expects that the tag of the element one empty end with / >). How to persuade 16.0.1 Firefox to use a proper parser?

    A good place to ask for advice on web development is to the 'Web Standards Development/evangelism' MozillaZine forum.

    Aid to this forum are better informed on issues related to web development.

    You must register on MozillaZine forum site to post in this forum.

  • read xml files in the directory, and then extract the words from their

    I looked in the forums of LabView for a problem similar to mine, but I succumbed to find advice, I need.

    Some time previously, I wrote a LabView program to build a dictionary from an array of strings.  It was really just a list of words that I used to make a crude search engine.

    Now I try to repeat this experience.  However, I'm stuck at the first hurdle.  I have a directory containing a number of folders and files, including .xml files.

    What I have to do is read the individual words in the xml files and then merge the words in a large list.  From there, I should be able to make the dictionary, and I hope that my old search engine still works with the new data.

    The attachment shows my unsuccessful attempt to create a VI that can first index all files in a directory (what it is) and then list only files ending in ".xml" (which it does not).

    Any advice on how to get my word out list would be greatly appreciated!

    Adam

    Is that what you are looking for? (See attached image VI)

  • Disk Cleanup stalling on temporary files for more than 3 hours and eat memory!

    So, whenever I checked my C drive I get different numbers on how much space is left. I deleted 45 GB (sent to remove trash and Perm.) data I didn't need and I always as much as before, if not less. I then tried to run the disk cleanup for that drive and it keeps blocking on "files temporary scanning", which since more than 3 h and no other programs other than windows base and Trend Micro are running and system keeps using more memory. EX: below

    RAM used: 3.1GB/4.0GB

    cache in the processor (i5): 4.2mb/7.7mb

    CPU usage has been the use of 80-90% during this period

    Frequency Max CPU is 109%

    This all started a few days after a windows update.

    It is using 25% CPU usage... as to what it is I don't have any idea. I've used it before and it scanned in less than 45 min.

    Anyone have any suggestions to correct the program that is not responding?

    System information:

    Manufacturer: AUSTeK computer Inc..

    Model: ASUS U52F Notebook series

    Processor: Intel (r) Core i5 CPU M 460 @ 2.53 GHz

    Installed memory (RAM): 4.00 GB

    System type: 64-bit Operating System

    OS: Windows 7 (updated)

    Hello

    This free program allows you to see what is taking place.

    http://www.Jam-software.com/treesize_free/

    Also check how much space system restore uses:

    http://Windows.Microsoft.com/en-us/Windows7/how-much-disk-space-does-System-Restore-require

    _____________________________________

    You can clean your hard drive using these two methods.

    "Delete files using Disk Cleanup" (use CCleaner for now instead of Disk Cleanup)

    http://Windows.Microsoft.com/en-us/Windows/delete-files-using-disk-cleanup#delete-files-using-disk-cleanup=Windows-7

    Using the free CCleaner program:

    http://www.Piriform.com/CCleaner

    In NO case use the registry cleaning Option in CCleaner

    _____________________________________

    Download, install, update and scan your system with the free version of Malwarebytes AntiMalware:

    http://www.Malwarebytes.org/products/malwarebytes_free

    See you soon.

  • Read and write XML files

    I'm working on a Flex application that will be used to save the entries concerning the manufacturing plan. I have two problems to solve:

    1 - can read an XML file to a local drive? (C:/temp/file)

    2. If the system where I call the web service is in offline mode, I have to create a local file that will be read vwhen later, that the system will be in place. How can I create a local file from Flex?

    A web-based application cannot access the local drive without the user's permission.

    Sometimes, people download local files to a server (as demanding as the user

    approval) and then request the file from the server.

    You can use local SharedObject to store data, although there are limits to

    the amount of data.

    The recommended option is to create an AIR application.  He will be able to access

    the local disk after installation.

  • Why is my file no more than 4.7 GB, and what I can do about it?

    Premiere Pro CS3

    PC Windows Vista Ultra

    I've never had this problem before. I edited in 47 minutes of video at 25 minutes. For some reason, I can't put it in yet (msg of error 'failed while defalut preset loading'. I exprorted using the option of film which produced a file of 5.67 GB (too big for DVD). I captured the material under a preset inappropriate (DV NTSC)? Wouldn't not possible to re - capture and republish the material under another preset UGH!

    you have overestimated the depth of my ignorance in the use of this suite!

    OK, so that there is not another misunderstanding on my part, here is what I hear that you are experiencing:

    1.) still won't launch for you. Is this correct? This is a problem, because you will need some (the process of setting up, transcoding and burn your exported to DVD-video file) creation of program to create a video DVD. You can start again alone and not the PrPro called? If this isn't the case, I would like to make a facility still repair to see if you can correct this problem. If not, what other design program you have available?

    2.) you exported using export > movie, but I received no screen the settings dialog. Is this correct? You come to export a file in any format/CODEC, but do not know what were the parameters, because you have never seen them. Is this correct? Use of G-Spot for you tell all the details about this file that you exported. Special attention to the format of the file and its CODEC. This info will be important then.

    3.) you want to burn a DVD-video to play on a drive set-top, or in a computer with a DVD player and DVD player. Is this correct? The file you have held not on a DVD-5, in its form today, but unknown. Is this correct? How have you tried to get this file on the DVD disc? This program has been used?

    I have a feeling that there is a misunderstanding of some of the terms, and possibly what it is you want to accomplish - for me at least.

    I don't want to be condescending, somehow, but I'm going to outline a few steps in obtaining of audio-visual on a DVD-video, just so that we are on the same page:

    1.) in PrPro (subsequently editing software), you change your audio-visual material, as you wish. In CS3, there are basically two ways to go from here.

    (a.) export in yet for creating (which works not not for you)

    (b) export to a file format/CODEC that can still use, then start again independently and import this file (have you tried that?)

    (2.) in yet (the program of creation of the suite), you use the files exported from PrPro and create the necessary navigation (Menus)

    3.) as a reminder, then transcode you (compress the file imported to the format/CODEC MPEG - 2) audio-visual material and then burn to disk, in the proper format.

    Note: the file that you exported is probably not MPEG-2, so is too great to simply copy a DVD disc. More needs to be done, just copy even a DVD compatible MPEG-2 file on disk, to create a DVD-video, it is not a bad thing, because there is still this 'creation' step that needs to be done. Normally, he would be in again (if you run it), but it could be done in other programs, as the author of DVD-video. This may be the only choice if we can not do for you.

    Now, if you knew all this, please forgive me. Just trying to find out where you are now and get information on where you want to go. This is the reason for all the questions. If there is anything you don't understand all the above, please just ask. I want us both on the same 'page '.

    Good luck

    Hunt

Maybe you are looking for