How to calculate daily, hourly average for the very large data set?

Some_Timestamp

"Parameter1" Parameter2
1 JANUARY 2015 02:00. 00:000000 AM - 07:002341.4534676341.4534
1 JANUARY 2015 02:00. 01:000000 AM - 07:002341.4533676341.3
1 JANUARY 2015 03:04. 01:000000 PM - 07:005332.3533676341.53
1 JANUARY 2015 03:00. 01:000046 PM - 07:0023.3436434.4345
JANUARY 2, 2015 05:06.01:000236 AM - 07:00352.3343543.4353
JANUARY 2, 2015 09:00. 01:000026 AM - 07:00234.453453.54
3 FEBRUARY 2015 10:00. 01:000026 PM - 07:003423.3534634.45
FEBRUARY 4, 2015 11:08. 01:000026 AM - 07:00324.3532534534.53

We have data as above a table and we want to calculate all the days, the hourly average for large data set (almost 100 million).  Is it possible to use the sql functions of analysis for better performance instead of the ordinary average and group of?  Or any other way better performance?

Don't know if that works better, but instead of using to_char, you could use trunc. Try something like this:

select
    trunc(some_timestamp,'DD'),
    avg(parameter1)
from
   bigtable
where
   some_timestamp between systimestamp-180 and systimestamp
group by
   trunc(some_timestamp,'DD');

Hope that helps,

dhalek

Tags: Database

Similar Questions

  • [8i] how to calculate a moving average?

    I am trying to calculate a moving average over a specific period of time, but I can't quite figure out how to do this. For this particular problem, I am also stuck on a 8i database.

    Some examples of data for the problem:
    CREATE TABLE     my_data
    (     order_no     CHAR(10)
    ,     seq_nbr          CHAR(4)
    ,     area_id          CHAR(4)
    ,     start_date     DATE
    ,     unit_time     NUMBER(7,2)
    );
    
    INSERT INTO my_data VALUES ('0000567890','0010','A001',TO_DATE('05/01/2000','mm/dd/yyyy'),0.34);
    INSERT INTO my_data VALUES ('0000567890','0020','A001',TO_DATE('05/02/2000','mm/dd/yyyy'),0.78);
    INSERT INTO my_data VALUES ('0000567890','0030','A002',TO_DATE('05/03/2000','mm/dd/yyyy'),0.91);
    INSERT INTO my_data VALUES ('0000567890','0040','A003',TO_DATE('05/03/2000','mm/dd/yyyy'),0.27);
    INSERT INTO my_data VALUES ('0000123456','0010','A001',TO_DATE('04/01/2001','mm/dd/yyyy'),0.39);
    INSERT INTO my_data VALUES ('0000123456','0020','A001',TO_DATE('04/02/2001','mm/dd/yyyy'),0.98);
    INSERT INTO my_data VALUES ('0000123456','0030','A002',TO_DATE('04/03/2001','mm/dd/yyyy'),0.77);
    INSERT INTO my_data VALUES ('0000123456','0040','A003',TO_DATE('04/03/2001','mm/dd/yyyy'),0.28);
    INSERT INTO my_data VALUES ('0000123123','0010','A001',TO_DATE('12/01/2001','mm/dd/yyyy'),0.31);
    INSERT INTO my_data VALUES ('0000123123','0020','A001',TO_DATE('12/02/2001','mm/dd/yyyy'),0.86);
    INSERT INTO my_data VALUES ('0000123123','0030','A002',TO_DATE('12/03/2001','mm/dd/yyyy'),0.82);
    INSERT INTO my_data VALUES ('0000123123','0040','A003',TO_DATE('12/03/2001','mm/dd/yyyy'),0.23);
    INSERT INTO my_data VALUES ('0000111111','0010','A001',TO_DATE('06/01/2002','mm/dd/yyyy'),0.29);
    INSERT INTO my_data VALUES ('0000111111','0020','A001',TO_DATE('06/02/2002','mm/dd/yyyy'),0.84);
    INSERT INTO my_data VALUES ('0000111111','0030','A002',TO_DATE('06/03/2002','mm/dd/yyyy'),0.78);
    INSERT INTO my_data VALUES ('0000111111','0040','A003',TO_DATE('06/03/2002','mm/dd/yyyy'),0.26);
    INSERT INTO my_data VALUES ('0000654321','0010','A001',TO_DATE('05/01/2003','mm/dd/yyyy'),0.28);
    INSERT INTO my_data VALUES ('0000654321','0020','A001',TO_DATE('05/02/2003','mm/dd/yyyy'),0.88);
    INSERT INTO my_data VALUES ('0000654321','0030','A002',TO_DATE('05/03/2003','mm/dd/yyyy'),0.75);
    INSERT INTO my_data VALUES ('0000654321','0040','A003',TO_DATE('05/03/2003','mm/dd/yyyy'),0.25);
    It is a sample of the actual data, which includes the lines around 50K, and is the result of a query, not a table. In addition, the actual data also have a reference number attached to each order, and the end result will be be partitioned by party id number and the region. This sample represents basically something, you might see a unique part number. In addition, these records at the time spacing is not predictable. You might have a single year that has 20 ordered, and another year (or years) that does not.

    Here is where I am now in the problem. For example, suppose I want to calculate a moving average over 2 years (average for the 2 years preceding the date of the current row). For my actual data, I probably want to be able to change the period of time (at different times in several queries) on which the moving average is calculated.

    I thought that using the analytical function AVERAGE with a window clause might be the way to go, but I don't know exactly how to use the window clause. I can even be completely wrong interpret what is its use, but if all goes well, it should still show what I'm after.
    SELECT     area_id
    ,     AVG(tot_area_unit_hrs)     
         OVER     (
              PARTITION BY     area_id
              ORDER BY     min_start
              ROWS          BETWEEN     --2 years prior to min_start of current row
                        AND     CURRENT ROW
              )
    ,     --something to indicate the time period the average is over     AS time_period
    FROM     (
         SELECT     order_no
         ,     area_id
         ,     MIN(start_date)     AS min_start
         ,     SUM(unit_time)     AS tot_area_unit_hrs
         FROM     my_data
         GROUP BY     order_no
         ,          area_id
         )
    ORDER BY     area_id
    ,          time_period
    As you can see from the above query, I want to calculate the sum of the unit_time for each order/area_id combo, then roll the way to this total time. (I'm the average total time that a product passes through each area_id per order).

    I want to see results like something along those lines... I really don't like how the time period is identified, as long as I can pay by him in chronological order, and he tells me what period of time it covers. The way I show with period_start and period_end in the results of the sample below is just a way that I thought to do this.
    area_id   period_start period_end   avg_unit_time   
    ----------------------------------------------------
    A001      4/30/1998    5/1/2000     1.120           
    A001      3/31/1999    4/1/2001     1.245           
    A001      11/30/1999   12/1/2001    1.220           
    A001      5/31/2000    6/1/2002     1.223           
    A001      4/30/2001    5/1/2003     1.153           
    A002      5/2/1998     5/3/2000     0.910            
    A002      4/2/1999     4/3/2001     0.840            
    A002      12/2/1999    12/3/2001    0.833            
    A002      6/2/2000     6/3/2002     0.790            
    A002      5/2/2001     5/3/2003     0.783            
    A003      5/2/1998     5/3/2000     0.270            
    A003      4/2/1999     4/3/2001     0.275            
    A003      12/2/1999    12/3/2001    0.260            
    A003      6/2/2000     6/3/2002     0.257            
    A003      5/2/2001     5/3/2003     0.247            
    Any suggestions?

    Hello

    You're close enough. You want a window based on the BEACH, no LINES.
    LINES BETWEEN... means that you know exactly how many lines will be in the window, and you don't care what range of values that represents.
    RANGE BETWEEN... means that you know the exact scope of the ORDER BY value to include in the window, which could mean any number of lines.

    When you say "RANGE BETWEEN x BEFORE...". "where the words ORDER BY id of a DATE, x is considered as the number of days. (Which is consistent with the arithmetic DATE Oracle).

    Try this:

    SELECT     area_id
    ,     min_start - 730          AS period_start
    ,     min_start             AS period_end
    ,     AVG(tot_area_unit_hrs)
         OVER     (
              PARTITION BY     area_id
              ORDER BY     min_start
              RANGE BETWEEN     365 * 2     PRECEDING
                    AND     CURRENT ROW
              )               AS avg_unit_time
    FROM     (
         SELECT     order_no
         ,     area_id
         ,     MIN(start_date)     AS min_start
         ,     SUM(unit_time)     AS tot_area_unit_hrs
         FROM     my_data
         GROUP BY     order_no
         ,          area_id
         )
    ORDER BY     area_id
    ,          period_end
    ;
    

    Output of your sample data:

    AREA PERIOD_STA PERIOD_END AVG_UNIT_TIME
    ---- ---------- ---------- -------------
    A001 5/2/1998   5/1/2000           1.120
    A001 4/2/1999   4/1/2001           1.245
    A001 12/2/1999  12/1/2001          1.220
    A001 6/1/2000   6/1/2002           1.223
    A001 5/1/2001   5/1/2003           1.153
    A002 5/4/1998   5/3/2000            .910
    A002 4/4/1999   4/3/2001            .840
    A002 12/4/1999  12/3/2001           .833
    A002 6/3/2000   6/3/2002            .790
    A002 5/3/2001   5/3/2003            .783
    A003 5/4/1998   5/3/2000            .270
    A003 4/4/1999   4/3/2001            .275
    A003 12/4/1999  12/3/2001           .260
    A003 6/3/2000   6/3/2002            .257
    A003 5/3/2001   5/3/2003            .247
    

    The period_starts are not quite what you posted, but, if I understand you correctly, you don't mind that.
    There should be no problem of partitioning an additional column.

  • my phone asking me always Apple ID password of the account that I did not create. I have this account in my settings, but I don't know how to remove it. Thanks for the reply.

    my phone asking me always Apple ID password of the account that I did not create. I have this account in my settings, but I don't know how to remove it. Thanks for the reply.

    There are several places an identifier Apple is used on an iPhone.

    • If it's the Apple of a former owner of the phone ID, and it is in settings/iCloud grace to find my iPhone activated, ONLY the person who set up the Apple ID can remove (see: find my iPhone Activation Lock: a mechanism of extraction of the previous owner)
    • If it's the Apple ID in Settings/iTunes & App Store just log out, then log back with the correct ID of Apple.
    • Do the same for iMessage and Facetime, if that is where the Apple ID is used.
    • If he is asked during the updating of the apps means that applications have been purchased or downloaded using this code from Apple. To resolve this issue, you must remove the application, and then buy with your Apple ID.
  • How will I know if I am being charged for the use of data on the mini2 ipad

    How will I know if I am to be charged or billed for usage on my ipad mini data 2?

    Hello

    You can be charged for the use of data on an ipad, if it has a SIM card

    Cellular data.

    If your home wifi is unlimited you will not be charged more.

    Unless your WiFi has a hat of data ie download limit.

    See you soon

    Brian

  • How to make a good UI for the selection of the channels with DAQmx

    A lot of my Labview programming accompanied verisons them previous LabVIEW (pre Labview 7), and there are a ton of new features I am digesting (currently using Labview 8.2).

    My question is (in general), how people schedule their user interface for the selection of channels and others when using hardware DAQmx?  What I have in mind is a system compacDAQ with a number of modules.  The user would be able to choose which channels to watch.  With the help of MAX as part of this process seems counter intuative and somewhat static (yes I know you can change out there, but it feels more like something you would use for something that doesn't really change much).  I wish that all the UI within the VI.

    Programmers use the task of creating screw for this or something else?

    Thanks in advance.

    ChuckNRC,

    I tend to avoid using MAX and create all my channels/tasks using the DAQmx functions.  There are also having some DAQmx features that allow you to save/remove channels, tasks and scales to MAX.

  • How can I recover my pc for the first time that I bought?

    Hello
    How can I recover my pc for the first time that I bought? because my pc is so slow... I have a compaq laptop cq61... and I want to set up the first time... How can I?  I don't have a recovery disk because when I bought it dosnt have a recovery in it disc. Please help me. Thank you

    Hello

    I'm happy to help you

    Glancing at the screen shot you posted, your looks of partition fine recovery (recovery (D).

    Save your files on an external hard drive or flash drive (depending on how many you have).

    Once that is done, disconnect all connected to your PC, but the AC adapter.

    Start the PC and the HP welcome screen, press the F11 key to start the Recovery dialog box and follow the prompts.

    It should take 15 to 30 minutes and your PC needs to be restored to its factory settings.

    Then just reinstall all programs and files that does not come with the PC.

    You will need the files or the installation of program diskettes. You cannot back up installed programs and reinstall them using a backup.

    Paul

  • How can I change an icon for the device in Windows 7?

    How can I change an icon for the device in Windows 7?

    In 'Control Panel\Devices and printers' I want to change the icon of a device that there are listed.

    Hi Dave Tavres

    I'm afraid to say that you would not be able to change this icon.
    This icon is provided by the hardware manufacturer, and is included in the drivers and architecture has no Windows allows to change this information.
    However, you can use the third party applications to change. Use your favorite search engine to find on the Internet.

    Important note: This response contains a reference to third party World Wide Web site. Microsoft provides this information as a convenience to you. Microsoft does not control these sites and no has not tested any software or information found on these sites; Therefore, Microsoft cannot make any approach to quality, security or the ability of a software or information that are there. There are the dangers inherent in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

    It will be useful.

    Thank you, and in what concerns:
    Shekhar S - Microsoft technical support.

    Visit our Microsoft answers feedback Forum and let us know what you think.

  • I tried to delete some unused programs of my had drive and the PC deleted wlidcl.dll. How can I get this program for the PC works as before?

    I tried to delete some unused programs of my had drive and cleared PC wlidcl.dll How can I get this program for the PC works as before? The PC works now as slow as a snail climbs.  What do you suggest me? Thank you

    Norman.--

    E-mail address is removed from the privacy *.

    original title: wlidcl.dll

    Hello

    I suggest to follow the steps below and check if the problem persists:

    Not one:

    Uninstall Windows Live essentials followed the procedure described in the link:

    http://Windows.Microsoft.com/en-us/Windows7/uninstall-or-change-a-program

    Step b:

    Download and install Windows Live essential from the link:

    http://explore.live.com/Windows-Live-Essentials?OS=other

  • How can I create a shortcut for the recycling bin in my office?

    Original title: Recycle bin shortcut.

    Anyone know how I can create a shortcut for the recycling bin in my office context menu (context menu)?

    Sorry Manu B G, but I think I understand what he wants exactly. This isn't a desktop shortcut but a shortcut context Menu. He wants a right-click on any empty space in the office and click Recycler Bin shortcut in the menu to go to the basket. Any way to do it?

  • How can I reset Adobe Acrobat for the default settings?

    How can I reset Adobe Acrobat for the default settings?

    Hi sharonc7 ,

    You can try to reset the preferences, here is a link on how to reset preferences: How to reset preference settings in Acrobat format.

    Also. Visit this link, which deals with the same subject: How to reset Adobe to its original settings? (Acrobat Reader) .

    Thank you

    Shivam

  • could you please tell me how to start to make flowers for the party and is the best soft

    could you please tell me how to start to make flowers for the party and is the best soft

    I've never done this, then go to desktop applications Adobe Creative Cloud | Adobe Creative Cloud and click on programs to read a description of what each program does

  • How to group by field derived for the field value below?

    Hi all

    I class field with the name of CLASS_FLD data item, I want to group by on left(CLASS_FLD,2).

    How to write him group by for the left(class_FLD,2) of expression above?

    I used earlier messages based on the syntax below but I am unable to make the Group

    <? for-each - group: row; xdoxslt:left(./CLASS_FLD,2)? > <? type: xdoxslt:left (current-group () / CLASS_FLD, 2); ' ascending '; data-type = "text"? >

    Thank you and best regards,

    1157496 wrote:

    Give me the syntax for the first group of lines BY expression counts.

    and also how the syntax would be if he is Businessunit group then group by expression (left(account,2)

    Mean you nested groups, first group BUSINESSUNIT and then other group ACCOUNT

    If yes then the internal group based on the ACCOUNT, we could watch as below

    for-each - Group: Current - Group (); xdoxslt:Left(./Account,2)? >

    For example

    . . . .

  • How to migrate first cc Pro for the complete package?

    How to migrate first cc Pro for the complete package?

    I just checked that you have added creative cloud apps on your account membership. If all goes well it should work for you now.

    Please let us know if you need assistance.

  • How can I save my picture for the web and devices?

    How can I save my picture for the web and devices?

    I can't find the option when I want to save the image

    Hi jacquelineb,

    What version of Photoshop are you using?

    If you are using the latest version of CC, please consult export images Adobe Photoshop for mobile and web. Tutorials Photoshop Adobe CC

    Let us know if that helps.

    Kind regards

    Assani

  • trying to cancel my plan, how do hard, would facilitate respect for the customer, it is a parody

    trying to cancel my plan, how do hard, would facilitate respect for the customer, it is a parody

    Hello

    Please contact support by calling/chat for cancellation requests and billing queries:

    Contact the customer service

    * Be sure to stay connected with your Adobe ID before accessing the link above *.

    You can also check the help below document:

    https://helpx.Adobe.com/x-productkb/policy-pricing/cancel-membership-subscription.html

    Please go through the Adobe - General conditions of subscription as well.

    Kind regards

    Sheena

Maybe you are looking for