The most effective way to log data and read simultaneously (DAQmx, PDM) high data rates

I want to acquire the data of several Modules cDAQ using several chassis to
high data rates (100 k samples per second if possible). Let's say the measurement time is 10 minutes and we got a large number of channels (40 for example). The measured data is written to a PDM file. I guess, the memory or the HARD disk speed is the limits. For the user, there must be a possibility to view the selection of channels in a graph during the measurement.

My question: what is the best and most effective way to save and read data at the same time?

First of all, I use an architecture of producer-consumer and I don't want to write and display the data in the same loop. I expect two possibilities:

[1] to use the 'DAQmx configure' with the operation 'journal and read' to write the data to a PDM file. To display the data in a second loop, I would create a DVR samples documented and 'sent' the DVR for the second loop, where the data will be displayed in a graph (data value reference). This method has the disadvantage that the data of all channels is copied into memory. Correct me if I'm wrong.

[2] use 'DAQmx configure', but only with the "journal" operation to write the data to a PDM file. To view the selected data, I had read a number of samples of the TDMS file in the second loop (I'm currently writing the TDMS file). In this case, I have only one copy data from the selected channels (not), but there will be more HARD drive accesses necessary.

What is the most effective and efficient solution in this case?

Are there ways to connect and read data with high frequencies of sampling?

Thank you for your help.


You say that the measurement time is 10 minutes. If you have 40 channels and you enjoy all CHs at 100 kHz, it is quite a number of values.

In this case, I always try to approach under the conditions of use. If a measure is only 10 minutes, I just connect all PDM data and create a graphic module that could be in the same loop of consumers where connect you the data. You can always work on the raw data files big offline afterwards, the extraction of all the information you need (have a look at the product called NI DIAdem:

The main issue is that the user needs to see in the graph (or perhaps a chart can be useful too). Lets say that the graph is 1024 pixels wide. It makes no sense to show multiple data to 1024 points, Yes? Every second will produce you 100 data points k per channel. What is the useful information, which should see your username? It depends on the application. In similar cases, I usually use some kind of data reduction method: I use a moving average (Point by point Mean.VI for example) with a size of the interval of 100. This way you get 100 data points of 1000 per channel every second. If you feed your graph every second with these average values, it will be able to data points in 1024 of the store (as a default) by channel (curve), which is a little more than 10 minutes, so that the user will see the entire measurement.

So it depends on the frequency at which you send data to the consumer. For example, collect you values 1024 by iteration of the producer and send it to the consumer. Here you can make a normal means calc or a bearing (according to your needs) and he draw a graphic. This way your chart will display only the values of the last 10 seconds...

Once I programmed some kind of module where I use a chart and not a graph, and the user can specify the interval of the absolute timestamp that is traced. If the data size is larger than the size of the chart in pixels, the module performs an average calculation in order to reduce the number of data points. Of course, if you need to see the raw data, you can specify an interval that is small. It all depends on how you program zoom functions, etc... In my case I hade a rate of 1 Hz, so I just kept all data in RAM limiting the berries to keep 24 hours of data, so that technicians could monitor the system. In your case, given the enormous amount of data, only a file read/write approach can work, if you really need access to all of the RAW data on the fly. But I hope that the values of working capital means will be enough?

Tags: NI Software

Similar Questions

  • The most effective way to import data from Excel in InDesign?

    Hi all

    I'm designing for a prospectus of college which includes 400 + courses list. For the moment, these lists exist as a huge Excel sheet with fields such as course type, course code, description, etc.

    I am familiar with Excel data import in InDesign and the tables/creation of table styles and other formatting, but the problem I have is that the data are in several columns by courses in the Excel worksheet, but will be in a single column per course with several lines in the InDesign document. I can't find a way to easily convert these columns in lines.

    Someone can help me with an effective way to get the data in the page layout without laborious copying and pasting or formatting?

    Thanks in advance!


    In excellent paste / transpose

  • The most effective way to retrieve a number of virtual machines in an org?

    Hi all, what is the most effective way to retrieve a MV of County in an org. I am currently recovering all CDV in an org, then all the vApps, then all of the virtual machines and pushing them into a table, then do a vms.length on the table filled by all virtual machines.

    It takes a horrible time, and there MUST be a better way.

    Open to any suggestion. Thank you

    Good, as administrator of the organization then: (I have not tried as a normal user account)

    var vdcs = new Array();
    var orgVMCount = 0;
    var vcdHost = org.getHost();
    var queryService = vcdHost.getQueryService();
    var expression = new VclExpression(VclQueryOrgVdcField.ORGNAME,, VclExpressionType.EQUALS);
    var filter = new VclFilter(expression);
    var params = new VclQueryParams();
    var resultSet = queryService.queryRecords(VclQueryRecordType.ORGVDC,params);
    while (resultSet != null) {
        var records = resultSet.getRecords(new VclQueryResultOrgVdcRecord);
        for each (var record in records) {
            var vdcVMCount = getVdcVMCount(record.href, vcdHost);
            System.log("VDC "" VM Count: "+vdcVMCount);
        orgVMCount += vdcVMCount;
        resultSet = resultSet.getNextPage();
    System.log("Total Organization VM Count: "+ orgVMCount);
    function getVdcVMCount(vdcHref, host){
        var vmCount = 0;
        var qs = host.getQueryService();
        var exp = new VclExpression(VclQueryVMField.VDC, vdcHref, VclExpressionType.EQUALS);
        var exp2 = new VclExpression(VclQueryVMField.ISVAPPTEMPLATE, false, VclExpressionType.EQUALS);
        var exps = new Array(exp,exp2);
        var expFilter = new VclFilter(exps,VclFilterType.AND);
        var queryParams = new VclQueryParams();
        var vmArray = new Array();
        var rs = qs.queryRecords(VclQueryRecordType.VM,queryParams);
        while (rs != null){
            var vmRecords = rs.getRecords(new VclQueryResultVMRecord);
            vmCount += vmRecords.length;
            rs = rs.getNextPage();
        return vmCount;

    It takes a little more code because the ".ORG" field is not available for the non-Admin queryVMField, but the. VDC IS, so we start by retrieving the TDC for the Org, then for each one, ask the number of vm for VDC.

    Post edited by: Burke - had accidentally initialization of orgVMCount as new Array(); -changed second line to 'var orgVMCount = 0;

  • The most effective way to store lines

    DB version: 10gRel2 and 11G

    If I want to keep a single column as structure, I would like to use associative arrays for performance reasons. Who is the most effective method to store a multicolumn line?

    You guys are using something like this
      type v_emprec_array is table of emp%rowtype INDEX BY BINARY_INTEGER;
      v_emprec v_emprec_array;
      select * bulk collect into v_emprec from emp where empno=7369;
    or this
      TYPE rec_UpdateEmp IS RECORD
              Inpt_emp_id         t_InptId,
              Stat_Code               gt_StatCode,
              Tms_Proc                pa_TmsProc,
              Tms_Po_Flag             pa_TmsPoFlag,
              Sngl_Unit_Flag          pa_SnglUnitFlag,
              Pkt_Profile_Id          gt_ProfileId,
              Pkt_Type                pa_PktType,
              host_inpt_id            pa_HostInptId
          rec                         rec_UpdateEmp;

    Ref Cursor is a better option

    create or replace procedure emp_dtls
     p_sal number,
     p_result out sys_refcursor
     open p_result for
      select ename,sal from emp where sal>p_sal;
    SQL> var myvar refcursor
    SQL> exec emp_dtls(2000,:myvar);
    PL/SQL procedure successfully completed.
    SQL> print myvar
    ENAME             SAL
    ---------- ----------
    JONES            2975
    BLAKE            2850
    CLARK            2450
    SCOTT            3000
    KING             5000
    FORD             3000
    ADAMS            7654
  • The most effective way to the U16-> U32 Array pack for DMA data transfer?

    I have a RT controller who wants to send data to an FPGA using DMA.  In LabVIEW 8.2, I have no choice but to switch to the low blocks of data however U32 my my input data is an array of U16.  This leaves me two choices: 1) wire U16 table directly into the DMA node that will be rude to U32 but I lose 16 bit * #of table 2) Pack the U16 elements in a U32 table before sending it to the bottom.    I have #2 and I have a method that works (and seems to be quite effective) but I have the feeling that I'm doing the hard (or roundabout way).   Is their a simpler (but equally effective) way to do this procedure?  Thank you

    Here's a solution that swaps the bytes and key elements the same. There are several ways to do so.

    (for efficiency, you can place the table 'remodel' inside a case structure then, it happens only if necessary)

  • The most effective way to browse the similar named fields?


    I have a 5-page document on every page that contains appx. 50 similarly named fields.    For example Viol1Num, Viol2Num, Vio3Num...  Viol50Num.

    I'm looking for an effective way to program a loop to watch each field in Javascript, I can do some manipulations in these areas on what the user has entered.

    In FormCalc, I've used the 'foreach' function similar to:

    foreach (Field1, Field2, Field3 Field50)



    However, who gets really long especially when it comes to the following pages, where I have to start adding "topmostSubform.Page2." in front of each domain name so that I can access it from the first page, all fields on the following pages.  Also, I need to do it in Javascript, no FormCalc.

    For example, in JS, I use this loop to mark all fields read-only:

    for (var nPageCount = 0; nPageCount <; nPageCount++) {
     var oFields = xfa.layout.pageContent(nPageCount, "field");
     var nNodesLength = oFields.length;
     for (var nNodeCount = 0; nNodeCount < nNodesLength; nNodeCount++) {
     oFields.item(nNodeCount).access = "readOnly";

    How could I do something similar to that for I can browse each field and perform actions on it without having to list every single domain name?

    I tried to change to watch the fields instead of field properties, but I couldn't run.

    Thank you.

    I solved my problem.   It took a few fighting in javascript using xfa.resolveNode.

    I have 5 pages, each consisting of a series of 60 fields named Viol1Num, Viol2Num, Viol3Num... Viol60Num.

    If when the javascript is executed, it detects an empty field, then insert a '3' in it.

    Here it is the javascript code that runs to the second page of this document.

       While (LoopCounter< 61)="" {="">
       If ((LoopCounter! = 21) & (LoopCounter! = 22)) {}
       If ((xfa.resolveNode("topmostSubform.Page2.Viol"_+_LoopCounter_+_"Num").rawValue == null) |) ((xfa.resolveNode("topmostSubform.Page2.Viol"_+_LoopCounter_+_"Num").rawValue == "")) {
       xfa.resolveNode("topmostSubform.Page2.Viol"_+_LoopCounter_+_"Num").rawValue = 3;
       LoopCounter = LoopCounter + 1
  • The most effective way to create a rendering iPad3


    I published three issues in my app now, in which case the last two had made for the iPad3. Both times I followed the same steps, but I find them to be a unnescessarily bit of time, so I wonder if there are faster ways to achieve this.

    Here's my way of doing it;

    1. I have a folio entirely converted in 768 x 1024. The entire development process is done on iPad2 with images of 144dpi.

    2. it's all over, and I'm ready to duplicate the entire folio in 1536 x 2048.

    3. in inDesign, I create a new folio, with the same name and the details (but with 1536 x 2048)

    4. I have an appointment online for the producer of Folio, click on my way to the new empty folio and click 'Add '.

    -This is my main question. Is it possible to add many/all articles at once? As I do, is to add one and a single article in time, and it becomes very painful and long.

    5. After all the items are added, apparently fine, but I can't publish the folio. It gives me an error, indicating that the folio contains a content in the wrong resolution.

    6. I go to inDesign and update of each article. (even once, would take a long time)

    7 folio rendering is finished and ready to publish.

    Is there a better way to do this? Advice would be greatly appreciated.


    Do not use Add. Use the import multiple items ordered... and that's where a sidecar.xml file is very important.


  • The most effective way to insert form data into multiple tables and w/autonum

    Hi all, this is my first post here. I am new to APEX and PL/SQL. I have ColdFusion and SQL, and I'm used to perform CRUD operations in my CF Tags application files. From what I've read so far, so if this is accurate, it seems that it is more efficient to perform most of the CRUD operations on the database side, I hope that for advice on the best way to accomplish what I ask - even if I'm trying to do is better off in my new APEX application I am open to this information as well.

    I have a form whose data must reside in two tables. Table A is a one-to-one relationship table and stores General information for each request. Table B is a table of one-to-many observations and should store unique records for each type of comment entered on the form, of which there are 3 comment fields on the form, I'd end up with 3 disks in table B for 1 in table A. I am generating an AutoNumber for table A with a sequence and relaxation. I want to do is take the AutoNumber I just created and use it for insertion in the Table B each insertion I have to do. This is where I am stuck.

    I would like to comment on 1) where I should put all these operations and 2) how best to go about what I want to do coding. In ColdFusion, it is as simple as inserting the record in Table A, immediately asking to table A more recent recording and storing (auto) ID in a variable and then using this var for any other table insertions, I had to do. I know it must be at least a little easier in the Oracle world, I just need a nudge in the right direction. Thank you!


    You're pretty close to not having worked with oracle a lot.

    This section is backwards:

    insert into sch.tbl_main(cust_lname,cust_fname,acct_num)
    select "SCH"."TBL_MAIN_SEQ".currval
          from dual;

    Even if it will work if you only have a single user, it's a potential bug with two or more users. First select the key (if it's first use nextval rather than currval) can use it in the insert statement.

    See you soon,.
    Janet Tyson

    Published by: Tyson Janet on June 23, 2011 10:48

  • What is the most effective way to change a demo, training and evaluation of the recording mode?

    Often I want to record a software simulation in demonstration, training and evaluation, or at least in two modes.

    After registration, I have to make some changes, for example, I want to change "click field Contact must be completed" to "in the Contact field, click the fill button. So if I want to use videos as a demo, training and evaluation, I have to review even three times. Is there an easier way to do it?

    N ° you must edit each output separately.

  • What is the most effective way to bring up the 1 of 4 images at any time?

    Thanks for reading this.  I have the same exact image need to attend different places in my program.  Right now, I created a movieclip with the image and I pasted in my step 4 different times and given to each instance a different instance name.  I would make a more effective program if I changed the location in the script of the video and has said the Stop program on a different image for each image, I need?

    What I'm trying to do, is to have a user to select from 4 different options and if they choose option 1, for example, the image of LED for option 1 light up and the other 3 lights will be off.  So, right now, if they choose option 1 I put ownership .visible to false for the other 3 images.  Is there a better way?

    The visible property control is very good.  If there is no need to show three of four, and they are identical in all cases, it may be better to simply have an and control its position.

  • The most effective way to understand what quarter an object is in relation to another

    Compares an object to another, and I wonder if there is a better way to determine which quarter an object is in relation to another.


    Currently I am doing

    if (object1.x > object2.x){
     if(object1.y > object2.y){
      //quarter 1
     } else {
      //quarter 0
    } else {
     if(object1.y > object2.y){
      //quarter 2
     } else {
      //quarter 3

    Thanks in advance.

    Google search using the terms "AS3 ATAN2 track object" you're likely to find many things that talks following the mouse... just replace the mouse with the object you want to point to.

  • The most effective way to put an internal radius to a border using edge animate.

    Hi all..

    I'm looking for a little help in the resolution of this one...

    If we were to create a rectangle and make this transparent rectangle, then assign a color to the border, then do tell... 3px we'd see a nice rectangle with rounded corners.

    But... How can I add an inner radius to my border...

    If I were coding CSS I would like to do the following, but I would like to know the edge way to do

    Thanks much as always!

    If I were in the css:

    #container {
        float: left;
        border-radius: 3px;
        background-color: #383838;
        width: 174px;
        height: 174px;
    #container:before {
        border-radius: 3px;
        border: 2px solid #383838;;
        content: '';
        width: 170px;
        height: 170px;
        position: absolute;

    You can set as shown in the screenshot attached:



  • The most effective way to create this color wheel?

    I know there are several ways to create a color circle, but in this case, there is a slight difference (ie. feature white) between the hidden images.

    I was wondering how you guys approached this creation. Thank you. Much appreciated!


    The white line can be created by using the rotation tool or the transform command. In both cases, the degrees of rotation is determined by dividing the number of strokes by 360 degrees. Rotation tool, you can make a duplicate order to repeat the rotation, make sure that your rotation a copy not the original.

    Then it's you if you want the line color to match the background, or use the path search tools to cut the railways making individual piece of the wheel. This can be handy if you want to move a piece of the wheel outwards and resize it perhaps until drew attention to it.

  • What is the most effective way to run this sql

    I have a sql like this:
    ((select q_avg24 from view1 where years=2009 and qtr=1) - (select q_avg24 from view1 where years=2010 and qtr=1)) as sub_s1_avg24,
    ((select q_avg24 from view2 where years=2009 and qtr=1) - (select q_avg24 from view2 where years=2010 and qtr=1)) as sub_s2_avg24,
    ((select q_avg24 from view3 where years=2009 and qtr=1) - (select q_avg24 from view3 where years=2010 and qtr=1)) as sub_s3_avg24,
    ((select q_avg24 from view4 where years=2009 and qtr=1) - (select q_avg24 from view4 where years=2010 and qtr=1)) as sub_s4_avg24
    from dual
    As you can see, the sql calculate a field value to another view (View1 view2 View3, view4).
    It returns the difference between the 2 years.

    In this sql, you can see she will solicit the views of the 2 x 4 = 8 times to get result table.
    Each query sql (select statement) return a result within 5 seconds, then... for total = 8 X 5 = 40secs...

    40 dry is not so big problem... but in fact, I have at least 5 games queries (select statements) with other parameters are not an entry for the calculation of this query...
    There are a lot of your time in SQL that results from: (8X5secs) x 5 = 200secs...

    I would like to know is there a solution better if we face such problems?
    Thank you!!!

    You can use as... not tested...
    But if you provide additional information on the data, I think we can think best solution

    select sum(decode(id,1,decode(yers,2009,av,0),0))-sum(decode(id,1,decode(yers,2010,av,0),0)) av1,
           sum(decode(id,2,decode(yers,2009,av,0),0))-sum(decode(id,2,decode(yers,2010,av,0),0)) av2,
           sum(decode(id,3,decode(yers,2009,av,0),0))-sum(decode(id,3,decode(yers,2010,av,0),0)) av3,
           sum(decode(id,4,decode(yers,2009,av,0),0))-sum(decode(id,4,decode(yers,2010,av,0),0)) av4
      select 1 id,q_avg24 av,years
      from view1
      where years in (2009,2010)
      and qtr = 1
      union all
      select 2 id,q_avg24,years
      from view2
      where years in (2009,2010)
      and qtr = 1
      union all
      select 3 id,q_avg24,years
      from view3
      where years in (2009,2010)
      and qtr = 1
      union all
      select 4 id,q_avg24,years
      from view4
      where years in (2009,2010)
      and qtr = 1)

    Published by: JAC on March 30, 2012 13:31

  • the most effective way to browse the MTS files

    I have been using mts for about 2 years now.

    When I get out of my car my camera sd I usually dump and go first.

    However, it becomes quite old and it takes a hell of a lot of time to go through 5-10 this way sd cards.

    Y at - it an application that I can use to view the mts files, as if it was a qt movie?

    with .mov can I just plug the card into my sd drive and browse in the finder. do not have to import or open a nle.

    I know that .mov is native in osx but y at - it a way to get a card that would allow my mac to treat the multilateral trading system the same?

    I think you need to CS6 for her to work.

Maybe you are looking for