How to calculate daily, hourly average for the very large data set?
Some_Timestamp | "Parameter1" | Parameter2 |
---|---|---|
1 JANUARY 2015 02:00. 00:000000 AM - 07:00 | 2341.4534 | 676341.4534 |
1 JANUARY 2015 02:00. 01:000000 AM - 07:00 | 2341.4533 | 676341.3 |
1 JANUARY 2015 03:04. 01:000000 PM - 07:00 | 5332.3533 | 676341.53 |
1 JANUARY 2015 03:00. 01:000046 PM - 07:00 | 23.34 | 36434.4345 |
JANUARY 2, 2015 05:06.01:000236 AM - 07:00 | 352.33 | 43543.4353 |
JANUARY 2, 2015 09:00. 01:000026 AM - 07:00 | 234.45 | 3453.54 |
3 FEBRUARY 2015 10:00. 01:000026 PM - 07:00 | 3423.353 | 4634.45 |
FEBRUARY 4, 2015 11:08. 01:000026 AM - 07:00 | 324.35325 | 34534.53 |
We have data as above a table and we want to calculate all the days, the hourly average for large data set (almost 100 million). Is it possible to use the sql functions of analysis for better performance instead of the ordinary average and group of? Or any other way better performance?
Don't know if that works better, but instead of using to_char, you could use trunc. Try something like this:
select trunc(some_timestamp,'DD'), avg(parameter1) from bigtable where some_timestamp between systimestamp-180 and systimestamp group by trunc(some_timestamp,'DD');
Hope that helps,
dhalek
Tags: Database
Similar Questions
-
[8i] how to calculate a moving average?
I am trying to calculate a moving average over a specific period of time, but I can't quite figure out how to do this. For this particular problem, I am also stuck on a 8i database.
Some examples of data for the problem:
It is a sample of the actual data, which includes the lines around 50K, and is the result of a query, not a table. In addition, the actual data also have a reference number attached to each order, and the end result will be be partitioned by party id number and the region. This sample represents basically something, you might see a unique part number. In addition, these records at the time spacing is not predictable. You might have a single year that has 20 ordered, and another year (or years) that does not.CREATE TABLE my_data ( order_no CHAR(10) , seq_nbr CHAR(4) , area_id CHAR(4) , start_date DATE , unit_time NUMBER(7,2) ); INSERT INTO my_data VALUES ('0000567890','0010','A001',TO_DATE('05/01/2000','mm/dd/yyyy'),0.34); INSERT INTO my_data VALUES ('0000567890','0020','A001',TO_DATE('05/02/2000','mm/dd/yyyy'),0.78); INSERT INTO my_data VALUES ('0000567890','0030','A002',TO_DATE('05/03/2000','mm/dd/yyyy'),0.91); INSERT INTO my_data VALUES ('0000567890','0040','A003',TO_DATE('05/03/2000','mm/dd/yyyy'),0.27); INSERT INTO my_data VALUES ('0000123456','0010','A001',TO_DATE('04/01/2001','mm/dd/yyyy'),0.39); INSERT INTO my_data VALUES ('0000123456','0020','A001',TO_DATE('04/02/2001','mm/dd/yyyy'),0.98); INSERT INTO my_data VALUES ('0000123456','0030','A002',TO_DATE('04/03/2001','mm/dd/yyyy'),0.77); INSERT INTO my_data VALUES ('0000123456','0040','A003',TO_DATE('04/03/2001','mm/dd/yyyy'),0.28); INSERT INTO my_data VALUES ('0000123123','0010','A001',TO_DATE('12/01/2001','mm/dd/yyyy'),0.31); INSERT INTO my_data VALUES ('0000123123','0020','A001',TO_DATE('12/02/2001','mm/dd/yyyy'),0.86); INSERT INTO my_data VALUES ('0000123123','0030','A002',TO_DATE('12/03/2001','mm/dd/yyyy'),0.82); INSERT INTO my_data VALUES ('0000123123','0040','A003',TO_DATE('12/03/2001','mm/dd/yyyy'),0.23); INSERT INTO my_data VALUES ('0000111111','0010','A001',TO_DATE('06/01/2002','mm/dd/yyyy'),0.29); INSERT INTO my_data VALUES ('0000111111','0020','A001',TO_DATE('06/02/2002','mm/dd/yyyy'),0.84); INSERT INTO my_data VALUES ('0000111111','0030','A002',TO_DATE('06/03/2002','mm/dd/yyyy'),0.78); INSERT INTO my_data VALUES ('0000111111','0040','A003',TO_DATE('06/03/2002','mm/dd/yyyy'),0.26); INSERT INTO my_data VALUES ('0000654321','0010','A001',TO_DATE('05/01/2003','mm/dd/yyyy'),0.28); INSERT INTO my_data VALUES ('0000654321','0020','A001',TO_DATE('05/02/2003','mm/dd/yyyy'),0.88); INSERT INTO my_data VALUES ('0000654321','0030','A002',TO_DATE('05/03/2003','mm/dd/yyyy'),0.75); INSERT INTO my_data VALUES ('0000654321','0040','A003',TO_DATE('05/03/2003','mm/dd/yyyy'),0.25);
Here is where I am now in the problem. For example, suppose I want to calculate a moving average over 2 years (average for the 2 years preceding the date of the current row). For my actual data, I probably want to be able to change the period of time (at different times in several queries) on which the moving average is calculated.
I thought that using the analytical function AVERAGE with a window clause might be the way to go, but I don't know exactly how to use the window clause. I can even be completely wrong interpret what is its use, but if all goes well, it should still show what I'm after.
As you can see from the above query, I want to calculate the sum of the unit_time for each order/area_id combo, then roll the way to this total time. (I'm the average total time that a product passes through each area_id per order).SELECT area_id , AVG(tot_area_unit_hrs) OVER ( PARTITION BY area_id ORDER BY min_start ROWS BETWEEN --2 years prior to min_start of current row AND CURRENT ROW ) , --something to indicate the time period the average is over AS time_period FROM ( SELECT order_no , area_id , MIN(start_date) AS min_start , SUM(unit_time) AS tot_area_unit_hrs FROM my_data GROUP BY order_no , area_id ) ORDER BY area_id , time_period
I want to see results like something along those lines... I really don't like how the time period is identified, as long as I can pay by him in chronological order, and he tells me what period of time it covers. The way I show with period_start and period_end in the results of the sample below is just a way that I thought to do this.
Any suggestions?area_id period_start period_end avg_unit_time ---------------------------------------------------- A001 4/30/1998 5/1/2000 1.120 A001 3/31/1999 4/1/2001 1.245 A001 11/30/1999 12/1/2001 1.220 A001 5/31/2000 6/1/2002 1.223 A001 4/30/2001 5/1/2003 1.153 A002 5/2/1998 5/3/2000 0.910 A002 4/2/1999 4/3/2001 0.840 A002 12/2/1999 12/3/2001 0.833 A002 6/2/2000 6/3/2002 0.790 A002 5/2/2001 5/3/2003 0.783 A003 5/2/1998 5/3/2000 0.270 A003 4/2/1999 4/3/2001 0.275 A003 12/2/1999 12/3/2001 0.260 A003 6/2/2000 6/3/2002 0.257 A003 5/2/2001 5/3/2003 0.247
Hello
You're close enough. You want a window based on the BEACH, no LINES.
LINES BETWEEN... means that you know exactly how many lines will be in the window, and you don't care what range of values that represents.
RANGE BETWEEN... means that you know the exact scope of the ORDER BY value to include in the window, which could mean any number of lines.When you say "RANGE BETWEEN x BEFORE...". "where the words ORDER BY id of a DATE, x is considered as the number of days. (Which is consistent with the arithmetic DATE Oracle).
Try this:
SELECT area_id , min_start - 730 AS period_start , min_start AS period_end , AVG(tot_area_unit_hrs) OVER ( PARTITION BY area_id ORDER BY min_start RANGE BETWEEN 365 * 2 PRECEDING AND CURRENT ROW ) AS avg_unit_time FROM ( SELECT order_no , area_id , MIN(start_date) AS min_start , SUM(unit_time) AS tot_area_unit_hrs FROM my_data GROUP BY order_no , area_id ) ORDER BY area_id , period_end ;
Output of your sample data:
AREA PERIOD_STA PERIOD_END AVG_UNIT_TIME ---- ---------- ---------- ------------- A001 5/2/1998 5/1/2000 1.120 A001 4/2/1999 4/1/2001 1.245 A001 12/2/1999 12/1/2001 1.220 A001 6/1/2000 6/1/2002 1.223 A001 5/1/2001 5/1/2003 1.153 A002 5/4/1998 5/3/2000 .910 A002 4/4/1999 4/3/2001 .840 A002 12/4/1999 12/3/2001 .833 A002 6/3/2000 6/3/2002 .790 A002 5/3/2001 5/3/2003 .783 A003 5/4/1998 5/3/2000 .270 A003 4/4/1999 4/3/2001 .275 A003 12/4/1999 12/3/2001 .260 A003 6/3/2000 6/3/2002 .257 A003 5/3/2001 5/3/2003 .247
The period_starts are not quite what you posted, but, if I understand you correctly, you don't mind that.
There should be no problem of partitioning an additional column. -
my phone asking me always Apple ID password of the account that I did not create. I have this account in my settings, but I don't know how to remove it. Thanks for the reply.
There are several places an identifier Apple is used on an iPhone.
- If it's the Apple of a former owner of the phone ID, and it is in settings/iCloud grace to find my iPhone activated, ONLY the person who set up the Apple ID can remove (see: find my iPhone Activation Lock: a mechanism of extraction of the previous owner)
- If it's the Apple ID in Settings/iTunes & App Store just log out, then log back with the correct ID of Apple.
- Do the same for iMessage and Facetime, if that is where the Apple ID is used.
- If he is asked during the updating of the apps means that applications have been purchased or downloaded using this code from Apple. To resolve this issue, you must remove the application, and then buy with your Apple ID.
-
How will I know if I am being charged for the use of data on the mini2 ipad
How will I know if I am to be charged or billed for usage on my ipad mini data 2?
Hello
You can be charged for the use of data on an ipad, if it has a SIM card
Cellular data.
If your home wifi is unlimited you will not be charged more.
Unless your WiFi has a hat of data ie download limit.
See you soon
Brian
-
How to make a good UI for the selection of the channels with DAQmx
A lot of my Labview programming accompanied verisons them previous LabVIEW (pre Labview 7), and there are a ton of new features I am digesting (currently using Labview 8.2).
My question is (in general), how people schedule their user interface for the selection of channels and others when using hardware DAQmx? What I have in mind is a system compacDAQ with a number of modules. The user would be able to choose which channels to watch. With the help of MAX as part of this process seems counter intuative and somewhat static (yes I know you can change out there, but it feels more like something you would use for something that doesn't really change much). I wish that all the UI within the VI.
Programmers use the task of creating screw for this or something else?
Thanks in advance.
ChuckNRC,
I tend to avoid using MAX and create all my channels/tasks using the DAQmx functions. There are also having some DAQmx features that allow you to save/remove channels, tasks and scales to MAX.
-
How can I recover my pc for the first time that I bought?
Hello
How can I recover my pc for the first time that I bought? because my pc is so slow... I have a compaq laptop cq61... and I want to set up the first time... How can I? I don't have a recovery disk because when I bought it dosnt have a recovery in it disc. Please help me. Thank youHello
I'm happy to help you
Glancing at the screen shot you posted, your looks of partition fine recovery (recovery (D).
Save your files on an external hard drive or flash drive (depending on how many you have).
Once that is done, disconnect all connected to your PC, but the AC adapter.
Start the PC and the HP welcome screen, press the F11 key to start the Recovery dialog box and follow the prompts.
It should take 15 to 30 minutes and your PC needs to be restored to its factory settings.
Then just reinstall all programs and files that does not come with the PC.
You will need the files or the installation of program diskettes. You cannot back up installed programs and reinstall them using a backup.
Paul
-
How can I change an icon for the device in Windows 7?
How can I change an icon for the device in Windows 7?
In 'Control Panel\Devices and printers' I want to change the icon of a device that there are listed.
Hi Dave Tavres
I'm afraid to say that you would not be able to change this icon.
This icon is provided by the hardware manufacturer, and is included in the drivers and architecture has no Windows allows to change this information.
However, you can use the third party applications to change. Use your favorite search engine to find on the Internet.Important note: This response contains a reference to third party World Wide Web site. Microsoft provides this information as a convenience to you. Microsoft does not control these sites and no has not tested any software or information found on these sites; Therefore, Microsoft cannot make any approach to quality, security or the ability of a software or information that are there. There are the dangers inherent in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.
It will be useful.
Thank you, and in what concerns:
Shekhar S - Microsoft technical support.Visit our Microsoft answers feedback Forum and let us know what you think.
-
I tried to delete some unused programs of my had drive and cleared PC wlidcl.dll How can I get this program for the PC works as before? The PC works now as slow as a snail climbs. What do you suggest me? Thank you
Norman.--
E-mail address is removed from the privacy *.
original title: wlidcl.dll
Hello
I suggest to follow the steps below and check if the problem persists:
Not one:
Uninstall Windows Live essentials followed the procedure described in the link:
http://Windows.Microsoft.com/en-us/Windows7/uninstall-or-change-a-program
Step b:
Download and install Windows Live essential from the link:
-
How can I create a shortcut for the recycling bin in my office?
Original title: Recycle bin shortcut.
Anyone know how I can create a shortcut for the recycling bin in my office context menu (context menu)?
Sorry Manu B G, but I think I understand what he wants exactly. This isn't a desktop shortcut but a shortcut context Menu. He wants a right-click on any empty space in the office and click Recycler Bin shortcut in the menu to go to the basket. Any way to do it?
-
How can I reset Adobe Acrobat for the default settings?
How can I reset Adobe Acrobat for the default settings?
Hi sharonc7 ,
You can try to reset the preferences, here is a link on how to reset preferences: How to reset preference settings in Acrobat format.
Also. Visit this link, which deals with the same subject: How to reset Adobe to its original settings? (Acrobat Reader) .
Thank you
Shivam
-
could you please tell me how to start to make flowers for the party and is the best soft
could you please tell me how to start to make flowers for the party and is the best soft
I've never done this, then go to desktop applications Adobe Creative Cloud | Adobe Creative Cloud and click on programs to read a description of what each program does
-
How to group by field derived for the field value below?
Hi all
I class field with the name of CLASS_FLD data item, I want to group by on left(CLASS_FLD,2).
How to write him group by for the left(class_FLD,2) of expression above?
I used earlier messages based on the syntax below but I am unable to make the Group
<? for-each - group: row; xdoxslt:left(./CLASS_FLD,2)? > <? type: xdoxslt:left (current-group () / CLASS_FLD, 2); ' ascending '; data-type = "text"? >
Thank you and best regards,
1157496 wrote:
Give me the syntax for the first group of lines BY expression counts.
and also how the syntax would be if he is Businessunit group then group by expression (left(account,2)
Mean you nested groups, first group BUSINESSUNIT and then other group ACCOUNT
If yes then the internal group based on the ACCOUNT, we could watch as below
for-each - Group: Current - Group (); xdoxslt:Left(./Account,2)? >
For example
. . . .
-
How to migrate first cc Pro for the complete package?
How to migrate first cc Pro for the complete package?
I just checked that you have added creative cloud apps on your account membership. If all goes well it should work for you now.
Please let us know if you need assistance.
-
How can I save my picture for the web and devices?
How can I save my picture for the web and devices?
I can't find the option when I want to save the image
Hi jacquelineb,
What version of Photoshop are you using?
If you are using the latest version of CC, please consult export images Adobe Photoshop for mobile and web. Tutorials Photoshop Adobe CC
Let us know if that helps.
Kind regards
Assani
-
trying to cancel my plan, how do hard, would facilitate respect for the customer, it is a parody
Hello
Please contact support by calling/chat for cancellation requests and billing queries:
* Be sure to stay connected with your Adobe ID before accessing the link above *.
You can also check the help below document:
https://helpx.Adobe.com/x-productkb/policy-pricing/cancel-membership-subscription.html
Please go through the Adobe - General conditions of subscription as well.
Kind regards
Sheena
Maybe you are looking for
-
Hi chaps and chapessesHope you can help I have a problem with my A60, there is strong during playback through Windows Media with all that it combined stuff like WOW, but when you use AVRack, JetAudio or RealPlayer, the volume is about half of the vol
-
What is the size of my Apple iCloud account storage?
What is the size of my Apple iCloud account storage?
-
Sidebar calendar shows only orange... not the data either
My calendar loads in the bar but shows only an orange page. There is nothing else... no data at all. I tried the tools without result; It only allows me to remove the calendar.
-
How can I get a new e-mail address as my primary address? &#62;?
I want to remove my current e-mail address and use another as my primary address. How can I do this?
-
(Redirected) No internet: error 651
Hello world! I formatted my secondary 1 to drive HARD and installed ISO image file windows 7 pro. Windows work fine. Except Internet. Error 651. Is it possible to solve this problem without any third party software? I have already searched all the i