Why read XML taking so long?

The 'answer' is perhaps that "XML is not intended for this purpose... "but here is the situation:

I'm parsing of a directory tree and creating a table of clusters, one for each folder in the tree.  To save it to the 'user-friendly' on the disc format, I chose XML, in part because "it's here", because it is a 'standard', and part because he "embarked" in the file enough data LabVIEW structure it is easy, in principle, to recover the data in the same shape as I initially wrote (i.e. in the form of clusters with a specified TypeDef table).

I did my routine to a directory with about 2700 folders tree.  It took 14 seconds to analyze the tree, about 0.1 second to write to the file (1.7 MB) XML, but over a minute and a half to read it back in and recover the original data.  Incidentally, I made a comparison between the "written data" and "read data", and all the 25 000 elements (the cluster had 9 elements) are the same, so at least to read and write XML "works."

I'm always surprised at the 800-fold difference between reading and writing.  I suspect that this shift is not linear with the "size of the problem", since in the beginning, I tried to do this with a larger painting of 4900 what took 0.25 seconds to write, but I "gave up" after what seemed to be too long to wait (sorry, it's so no scientist, but I think it was 3-4 minutes at least).  However, now that I see that there is a difference of 800-fold for my previous example, I'll wait at least five minutes (800 * 0.25 sec = 200 s, 5 min = 300 sec)... (time goes away...)

Patience rewarded - it took 322 seconds, which is about 1200 times slower (not to criticize my calculations - I'm rounding when I report time, but use the millisecond values when I calculate ratios...).  So the more you do it, it gets the slower.

Hmm - we are going to prove by doing a small folder.  How about one with only 85 files?  Which takes 21 msec to write and 102 msec to read, a factor of 5 only!  Wow, this is certainly not a linear growth.

What happens here?  Is there a problem with the analysis of large XML files?  [I must point out that I use a simple XML File reading (array) for the table (of which will prove be cluster), and then in a for loop do a "Unflatten from XML' to recover the cluster.]  I'm guessing that the loop is to behave linearly, as all clusters are (with the exception of their content) identical, then the part "polynomial time" should be "reading of XML File (array).

Not sure why this should be the case.  It would seem, to me, that since LabVIEW tables are always "identical elements", read a file with 1000 'elements' and turn it into an array of 1000 elements should take approximately 10 times longer than that for an array of 100 elements, unless there is something that is extremely inefficient.  Is this something that can withstand being examined and possibly optimized?

Bob Schor

PS - was curious enough to do more tests.  I started with a 'Master Data Set' of 4906 records, then dealt with 'nested subsets' (which is a subset of the master, a subset of the subset, etc.) to try to compare of"apples with apples" (on a PC, of course).  My samples were 20, 210, 2736 and 4906 records.  The write speed of files has been between 10 K and 24K Records/sec on this beach, which is about nearly linearly with size.  However, a similar measure for the reading of records varies between 3K (for all 20 items) and 15 (for all), reducing the size has increased.  I traced a field of logarithmic data and got a slope of 0.1 for writing (a slope of 0 means speed is linear with the number of cases), but a slope of-0,95 for playback)


Tags: NI Software

Similar Questions

  • I tried to erase and reset my iphone 6 s. But why is it taking so long? For more than 24 hours. Is this normal? The phone displays just the apple logo and wont open. Help, please. Thank you in advance.

    I tried to erase and reset my iphone 6 s. But why is it taking so long? For more than 24 hours. Is this normal? The phone displays just the apple logo and wont open. Help, please. Thank you in advance.

    Hello

    Follow the instructions here:

    If you are unable to update or restore your iPhone, iPad or iPod touch - Apple supports

  • Why Windows Update taking so long?

    Well then I have just updated to Windows XP to Windows Vista, I tried to update my system using windows update. And it was 'Check updates' for a few hours now. Why it takes so long? I mean, the last time I checked, there were only 7 updates for me.

    First, configure the system to boot - How to solve a problem by performing a boot in Windows Vista
    NOTE: If a 3rd party firewall was used, including one installed with a security suite, open Windows Firewall in Control Panel and make sure that the native Vista firewall is now on. No 3rd party firewall will be executed in the State of clean boot.

    Then, go here, please download and Save the Fixit - How to reset the Windows Update components

    With all the programs and browsers closed, run MicrosoftFixit50202.msi and choose aggressive mode. Turn it back on when he finished the race.

    See if the system can update now. If it does not, please copy and paste the last 50 or so lines of WindowsUpdate.log in your response-
    How to read the Windowsupdate.log file

    MowGreen Services update - consumer safety

  • Why is it taking so long to process my order?

    Hey, I ordered CS6 Production Premium on the Education store on September 13 and my order staus is still pending.

    I also downloaded my ID of college on the customer support portal 16 and he says "pending Adobe Response".

    Could someone help me out here regarding what is happening. Why the ordering is not treated? That it becomes quite frustrating waiting for the order to be processed when he said that Adobe would meet my case within a few days of load me my college ID and distribute the software shortly after.

    Can someone help me find out why taking Adobe so long to process my order and how much time yet I have to wait until I can finally use the software...

    Thank you.

    Well, someone who buys the software should be able to start using it right away by downloading and running the free trial of 30 days for the product...

    What about academic validation student, Adobe says that it should normally take about two working days:

    https://www.identit-e.com/default/static/adobecontactus

  • Why Firefox 30 taking so LONG to install?

    I tried to reinstall Firefox from scratch on this machine. However, it takes so long to install I've abandoned now.

    What I was doing wrong and what can I do to put it right? It should not take FOUR HOURS to do this even to halfway through the installation, of course? At one point, I wondered if it was just a loop around between download and install. I like to run two browsers so I can test if the websites showing down in a work in the other. My choices are Firefox and Chrome. Well, they have so far been because Firefox does settle it timely.

    I'm typing this in Chrome and using Windows 7 on a NetBook compatible Windows 8 years. I'm not ready to install Windows 8, so I hope that there is another option. I can not even the thing of widget troubleshooting information in work then I or the machine is really having a bad day.

    Try to use the full installer, offline from here: http://www.mozilla.com/en-US/firefox/all.html

  • Why delete files taking so long in Windows 7?

    I am running windows 7 64 bit on a high-end computer mind nine 8 GB RAm still deleting files takes forever. for example I'm deleting 258 files in a directory as we speak - total size of all files in 458kb (Yes half a megabyte) and windows tells me that it'll take 8 minutes to do! Surely, it should be almost instant? How can I speed up the deletions of files?

    copy of the files also takes an age and makes the system almost unusable.

    Philippe

    It seems slow.  If you are used to use disks formatted FAT32, the stronger NTFS takes significantly longer because it does synchronously things that FAT32 things are started and left under the assumption that they will still complete successfully; and unfortunately that can sometimes cause problems.

    In addition to the suggestion of Ken Blake to find malware, I suggest that you check the installed utilities.  If you have one installed real-time backup programs, some of them can do a real job on the performance.  Try to start safe mode and try some of your slow deletions or copies.  Concerning copies, your antivirus program may cause a big performance hit because they verify each file as it is copied.  Temporarily disable your antivirus can greatly improve speed in many cases.  One thing you might want to try is DEFRAG if you haven't done so recently.  Modern versions of Windows generally do this automatically, but sometimes doing a DEFRAG manual can help.

    Good luck.  I hope this helps.

  • Why vdr stop taking so long?

    When to stop the VM console says linux VDR "Waiting for VMware Data Recovery stop" about 15 times.  It takes about 20 minutes for that one VM stop.  VDR works as I did a backup and restore.  I have something misconfigured to cause the VM to take so long to stop?  Where would I look?

    Stop stop process vmware-tools (and network interfaces) before stopping datarecovery.

    DataRecovery process cannot properly stop without network interface.

    Transition from /etc/rc0.d/K08vmware-tools to /etc/rc0.d/K91vmware-tools and /etc/rc6.d/K08vmware-tools to /etc/rc6.d/K91vmware-tools should solve this problem.

  • Why download Illustrator taking so long?

    I was download the trial for about 5 minutes, and it is not even 1/10. I download it to the desktop, not in the programs/applications folder. It is said will take 2 hours to download. Is this normal? Thank you.

    We do not know. We don't know anything about your network connection or other details. Could be quite normal on a slow connection...

    Mylenium

  • Exchange partition taking so long?

    Hello


    Oracle 11.1.0.7 version.

    Why partition exchange took 10 minutes longer?

    Here is my code

    Table TAB is siloed single list
     CREATE OR REPLACE PROCEDURE load_tab;
   IS
      
      starttime   NUMBER;
      endtime     NUMBER;
      BEGIN
 BEGIN     


EXECUTE IMMEDIATE 'TRUNCATE TABLE TAB_TEMP;'

-- Dropping the index on temp table

 EXECUTE IMMEDIATE 'DROP INDEX TAB_TEMP_IDX01';
 
 BEGIN
 INSERT /*+ append */ INTO TAB_TEMP
 Select col1,col2,...coln from table1@DBLINK1,table2@DBLINK1,table3@DBLINK1 where join conditions;
 
 commit;
 END;
 
 EXECUTE IMMEDIATE 'CREATE INDEX TAB_TEMP_IDX01 ON TAB_TEMP
 (COL1)
 NOLOGGING
 TABLESPACE CSP_INDX
PARALLEL';
 
 
  Gathering stats on temp table;
 
 
 EXECUTE IMMEDIATE 'TRUNCATE TABLE TAB';
 
  starttime := SYS.DBMS_UTILITY.get_time;


 EXECUTE IMMEDIATE 'ALTER TABLE TAB  EXCHANGE PARTITION p1  WITH TABLE TAB_TEMP  WITHOUT VALIDATION UPDATE GLOBAL INDEXES';

  endtime := SYS.DBMS_UTILITY.get_time;
  
  
  logging total time taken for partition exchange on a log table ((endtime-statrtime)/100));


EXECUTE IMMEDIATE 'TRUNCATE TABLE TAB_TEMP';


EXCEPTION Block;

END;

END load_tab;
    Please let me know why partiton Exchange taking so long? here a few times it takes 30 minutes and some time 10-15 minutes.



    I was under the impression that the exchange of partion is nothing but the update of the data dictionary. Please clarify my doubt?




    Thank you
    Mike

    If you don't have a comprehensive index on the partition table, but only local indexes, then more of the optimal approach is:

    (1) create the staging table (similar to the partition table)

    (2) double local indexes on the table to partition in the staging table

    (3) an exchange of partition table using alter table... swap partition.. with table... including the index without validation

    Such an exchange of partition must be second - as sole owner of the materials are exchanged and no treatment/audit/additional validation isn't required.

    Of course, the assumption is that the intermediate table has valid data for the content of the partition it will replace.

  • Why my partitioning process taking so long? (I've waited 1 day it ends)

    Why my partitioning process taking so long? (I've waited 1 day it ends). Please help me, your help is very appreciated.

    1. Please disconnect from external storage. You use a USB Installer?

    2 use wired network connection, if possible.

    3. What is the version of Windows that you are trying to install?

    4. what version of Mac OS X on your Mac?

  • Why does take so long to load from the time wherever I power up until it takes to get to the login screen? help taking too long

    Why does take so long to load from the time wherever I power up until it takes to get to the login screen? help taking too long

    You should take a look at your startup programs and decide what is really necessary and what is nothing more than useless clutter. There is nothing wrong with the startup programs, some may be really useful or even necessary for your personal needs. But keep in mind that many programs build to start when you start the computer and that most of them must not start when the computer starts, all they do is slow down your boot time and clutter of things when you're trying to solve problems, and some are downright harmful!

    I mean, have you really need things like real Planner and Gadwin Print Screen running all the time? These programs will run very well, even if they are not started automatically every time that you start the computer. Go to these programs and change/disable their boot option. Some may be difficult to kill for public assistance services can be useful when you try to control startup programs. Try CodeStuff Starter, it's safe and easy to use, I think you'll like it, download it here:

    http://CodeStuff.tripod.com/products_starter.html>

    Use AutoRuns to check axis start, Richard:

    http://TechNet.Microsoft.com/en-us/sysinternals/bb963902.aspx>

    Use Cclenar-> tools-> start

    http://www.CCleaner.com/download/builds.aspx>

    I can't tell you what to keep, startup items it is a personal preference, but pretty well all the startup items indicated by CodeStuff Starter are safe to disable, and changes are easy to reverse if you change your mind. Searches for items it shows you and decide whether or not these things are really useful or necessary. Some programs will be started at startup without your knowledge and over time, that list can expand to include useless parasites, part of good computer maintenance includes regular control of these startup items and the removal of the unnecessary.

    Suggest that you first spread malware as the cause of the problem by following these steps:

    I. - delete your temporary files

    Clean the system (compensation to all temp/tmp folders and included all the content offline, the tif browser, delete the cookies of compensation.)

    Do a disk cleanup. Click the Start button. in the search box, type disk cleanupand then in the list of results, click Disk Cleanup. Better and easier to use - ccleaner.

    Download the basic version (slim) via

    http://www.Piriform.com/CCleaner/builds>

    The basic version (slim) does not contain the toolbar disgusted

    DO NOT USE ANY cleaning of the advanced options. DO NOT TOUCH THE REGISTRY OR TOOLS. At least not for now.

    Reset

    II. - have your system scanned as a result of security/antivirus installed.

    III. - download, install, update and run both of these tools to a-squared free scans:

    MalwareBytes anti-malware (MBAM)

    http://www.Malwarebytes.org/products/malwarebytes_free>

    SUPERAntiSpyware (SAS)

    http://SUPERAntiSpyware.com/superantispywarefreevspro.html>

    UTC/GMT is 18:16 on Wednesday, January 18, 2012

  • What is the problem with this Forum? Why Adobe taking so long to reply to messages?

    What is the problem with this Forum? Why Adobe taking so long to reply to messages?
    It is in fact 'specific' Forum of Photoshop.
    I usually come back another Adobe forum responses quickly.

    Chime.

    Hi people,

    For what it's worth, there are several of us who try, in addition to our regular work, to keep an eye on two other forums.  Quite simply, we care about the product and try to make sure that we give our customers the best experience as possible.  Plus, what better way to hear about your weak points in order to solve them.  But, as Mylenium and Christmas as well stress this, the forums are officially a user experience.  If you need help fast or official, it isn't the best place to do it.

    Regarding your question of Arab, Rahzah, I answered in two different threads and would be happy to help you with your problem.  Please choose a single thread and stick with it. two tabs is a pain.

    Thank you
    David

    PS - some of us do check the forums on our time, as I did last week at the launch while I was on vacation.

  • How to find the lock and more information why the prg conc taking too long

    Oracle Apps R12-Orader management

    Hi all

    I am new to Oracle Apps, we I run my simultaneous program
    its taking too long to complete. How to find that there is a lock to produce in the table while that simultaneous execution of the program, another way to find why the simultaneous program takes too long to run and how can you speed up the process.

    Yesterday we made a few changes to speed up the process with our DBA to accelerate the simultaneous program

    The first simultaneous program takes 2 h 30 and after these changes taking 1 h 45. Please suggest it's okay

    Connected with the responsibility of the system administrator.

    Manager - set - Manager of the standard.

    Set the value of PROCESS like 15 (instead of 10).

    Set the value of SECONDS of SLEEP as 10 (Changed from 30).

    Manager - set - Manager2 Standard.

    Set the value of PROCESS like 15 (instead of 10).

    Set the value of SECONDS of SLEEP as 10 (Changed from 60).


    Any suggestion on how to find why it takes too much time.

    Any help is highly appricatable

    Thanks and greetings
    Srikkanth.M

    Hello

    I suggest you draw the simultaneous program and analyze the trace file. To do this, mark the simultaneous program as Trace enable (RESP system administrator, Menu simultaneous > program > define) and then run the program and download the trace file after dealing with tkprof. Check which are the staements who take most of the time and try to optimize them.

    It will be useful.

  • V10.11.3 OSX El Capitan taking a long time to open

    Why after update to OSX El Capitan v10.11.3 its taking a long time to open?

    Do you mean that it takes a lot of time for your computer to boot?  There could be several reasons for this.  Start by going to http://www.etresoft.com/etrecheck , download and run EtreCheck, copy paste here the status report.  It goes we will see what is currently on your system, and if there is something on this issue causing the extreme slowness of start-up, we see and advise you on what to do next.

  • computer Dell laptop under Vista Home Basic are taking too long to wake up from its sleep. __

    computer Dell laptop under Vista Home Basic are taking too long to wake up from its sleep.

    http://www.Vistax64.com/tutorials/63567-power-options-sleep-mode-problems.HTML?filter [11] = power % 20Management

    Read the info on sleep disorders in the tutorial above.

    http://www.Vistax64.com/tutorials/198047-power-plan-restore-default-settings.html

    Also read this tutorial; as well as the 'related links' at the bottom of the links above.

    See you soon.

    Mick Murphy - Microsoft partner

Maybe you are looking for

  • Pavilion g6-2212sa > 64-bit Windows 8 > No. Audio devices are installed

    Hi all I followed the steps in the HP support site to use Windows troubleshooting guide to solve the problems with the audio: http://i.imgur.com/p7BDHqX.PNG I updated to the latest driver, audio driver, but there is no device installed http://i.imgur

  • PS/2 keyboard not woking on a T61

    Greetings, I ran int a bit wrong with this keyboard from my friends, it's a "Logitech Internet 350 Keyboard" with a PS/2 connector. The T61 with not a PS/2 port (without a dock) I use a PS/2 to USB converter. In the case of windows, the keyboard pres

  • Hyper-V and Virtual Desktops

    I wish to run five or six virtual desktops, at the start of Windows 7 and maybe later Windows 8 or 10 Windows... What are the specifications of the reasonable system for the server to do this?  I wish I could run Office, the surface web, play streami

  • Error Code 8024402C

    My computer is part of a network, and it has received updates via the network.  I have left the company and cannot receive more updates over the network.  However, my computer is set up to think that he is part of a group then, when I try to download

  • HP compac q not to recognize the power supply cord.

    Tried the portable family factory restore, but failed "plug your laptop AC. The computer is plugged in with the power light on and is in charge of the battery. Help, please! HP Compac Q running Vista. Thank you in advance, Darren