Getting Sum, Count, and Distinct Count of a file

Hi all, this is a question of UNIX.

I have a large flat file with millions of records.
col1 | col2. col3
1. a | b
2. c | d
3. e | f
3. g | h
footer *.

I'm supposed to do to calculate the sum of the [= 9] col1 col1 County [= 4] and a separate account of col1 [= c3]

I would like if you avoid external commands like AWK. Also, can we do the same thing by creating a function?

Please keep in mind that the file is huge

Thanks in advance

Yes, I suppose that it is very much inappropriate for answering this question here, I just like awk too] :))

where x is the file:

sum(col1) : awk -F\| '{col1+=$1}END{print col1}' x
count(col1): wc -l x | awk '{print $1-2}'
count(distinct col1): cut -d\| -f1 x|sort -u|wc -l|awk '{print $1-2}'

Perl would be a much more effective tool to use here... try a perl forum :)

Tags: Database

Similar Questions

  • Get the width and height of the image

    Hi scripters

    I try to get the width and height of my image file. Here is my code

    myPicFile = File("c:\\ar.eps");

    h = myPicFile.height;

    w = myPicFile.width;

    Alert (h)

    Copy the following code generates error

    ---------------------------
    Script alert
    ---------------------------
    undefined
    ---------------------------
    Ok
    ---------------------------

    How can I get the height and the width of my image? is there another method?

    Thanks in advance

    concerning

    a you are the

    Dear Arul,

    Here, my code is in InDesign. I don't know that we get the size of the Image directly. ?

    Throught InDesign [you using the InDesign Script then] Script here I pasted the code below...

    = Image height / width =.

    myDoc var = app.activeDocument;

    myPicFile = file ("E:\\HariharaSudhan\\Blue hills.jpg");

    App.place (file (myPicFile);

    = Then u please select the Image file in InDesign and then you run the script = / /.

    mySel var = app.selection [0];

    var limit = mySel.geometricBounds;

    var height = related [2] - bound [0];
    var width = Bound [3] - bound [1];

    Alert ("Image Height:" + height + ""+"Image width:" + width);

    = End: getting the Image height / width =.

    Please let me know if you have any questions.

    Thank you & best regards

    Thierry T.R.Harihara

  • Get incremental counter/sound to work along side with action with usb-6008 with labview tia sal22

    Get incremental counter/sound to work along the coast with usb-6008 with labview tia sal22

    Hi all

    I can get this vi to work if they are distinct from the vi but I can't join them together

    Example of my error:

    If buffers are set to 0 the freq counter increment works, but no sound
    If the buffers are set to 1 the audio works fine but is not increment the Freq counter
    If the buffers are attached to more 1 clicks and pops are coming

    That's what I'm doing:
    (1) have the frequency of increment of my internal sound card to a certain level as .01hz a second until he gets to 20 000 hz

    (2) use my device usb-6008 daq, which is connected to the same machine to measure the voltage at the same time. (I am in a position very low voltages between 1-5volts)

    (3) output to a worksheet text file which will show you:

    time in seconds, frequency, voltage
    0,400.01,2

    1,400.02,2.5

    2,400.03,1

    I'm a bit confused about how connect the increment and the audio during the measurements with the usb-6008 housing on the same machine
    at the same time and in the same VI.

    Anyone have any ideas?  I'm using labview 8.5
    TIA sal22

    Ha ha you have been deceived by a dynamic thread. Insert a convert from Dynamic Data Express VI (Palette to own: Signal handling screw Express) between the daq read and build the array function. Then it won't work. Now the value in the dynamic data is only converted to a numeric value

  • Impossible to get Min, Max and median of the values in the date range values

    Hello

    I had a requirement as to show the data of each charge group of wise men as '< 100' ' 100-199 "" 200-299 "" 300-399 "400-499, 500-599 600-699 700-799 800-899 900-999 > = 1000 '"»

    With the query be able to get the count between the beach and the total below. But impossible to get the Min and Max values for this range. For example if the County < 100 is 3 then in these 3, the lowest value is need to display in the min. Idem for Max column also.

    In the light of the median value on these values.

    Thanks in advance.



    Requirement is as below:
    State < 100 100-199, 200-299 300-399 400-499, 500-599 600-699 700-799 800-899 900-999 > = 1000 Min Total median Max
    AK 1 2 0 4 1 4 4 35 35 4 1 $25 $85 850 $1,200
    AL 0 0 2 27 10 17 35 2 2 35 0 $103 100-$1 500 750


    * "QUERY ' * '"

    WITH t AS
    (SELECT 'AL' State, 12 DUAL FROM VALUE
    UNION ALL
    SELECT 'AL' State, 67 FROM DUAL VALUE
    UNION ALL
    SELECT 'AL' State, 23 FROM DUAL VALUE
    UNION ALL
    SELECT 'AL' State, 12 DUAL FROM VALUE
    UNION ALL
    SELECT 'AL' State, 12 DUAL FROM VALUE
    UNION ALL
    SELECT 'AL' State, 78 FROM DUAL VALUE
    UNION ALL
    SELECT 'AL' State, 34 FROM DUAL VALUE
    UNION ALL
    SELECT 'AL' State, 4 DUAL FROM VALUE
    UNION ALL
    SELECT 'AL' State, 12 DUAL FROM VALUE
    UNION ALL
    SELECT 'AL' State, 15 VALUE FROM DUAL
    UNION ALL
    SELECT "AZ" State, FROM DUAL VALUE 6
    UNION ALL
    SELECT "AZ" State, 123 FROM DUAL VALUE
    UNION ALL
    SELECT "AZ" State, 123 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 23 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 120 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 456 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 11 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 24 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 34 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 87 DUAL FROM VALUE
    UNION ALL
    SELECT 'MY' State, 23 FROM DUAL VALUE
    UNION ALL
    SELECT 'MY' State, 234 DUAL FROM VALUE
    UNION ALL
    SELECT 'MY' State, 789 FROM DUAL VALUE
    UNION ALL
    SELECT "HD" State, VALUE FROM DUAL 54321).
    -End of test data
    AS T1
    (SELECT State,
    NVL (COUNT (DECODE (VALUE, 0, 0)), 0) '< 100 ',.
    NVL (COUNT (DECODE (VALUE, 1, 1)), 0) '100-199.
    NVL (COUNT (DECODE (VALUE, 2, 2)), 0) '200-299.
    NVL (COUNT (DECODE (VALUE, 3, 3)), 0) '300-399.
    NVL (COUNT (DECODE (VALUE, 4, 4)), 0) '400-499.
    NVL (COUNT (DECODE (VALUE, 5, 5)), 0) '500-599,'
    NVL (COUNT (DECODE (VALUE, 6, 6)), 0) '600-699.
    NVL (COUNT (DECODE (VALUE, 7, 7)), 0) '700-799.
    NVL (COUNT (DECODE (VALUE, 8, 8)), 0) '800-899.
    NVL (COUNT (DECODE (VALUE, 9, 9)), 0) '900-999. "
    NVL (COUNT (DECODE (VALUE, 10, 10)), 0) ' > = 1000.
    (SELECT STATE,
    CASE
    WHAT VALUE < 100 THEN 0
    WHAT A VALUE BETWEEN 100 AND 199 THEN 1
    WHAT VALUE BETWEEN 200 AND 299, THEN 2
    WHAT VALUE BETWEEN 300 AND 399 THEN 3
    WHAT VALUE BETWEEN 400 AND 499 THEN 4
    WHAT VALUE BETWEEN 500 AND 599 5 THEN
    WHAT VALUE BETWEEN 600 AND 699 6 THEN
    WHAT VALUE BETWEEN 700 AND 799 THEN 7
    WHAT VALUE BETWEEN 800 AND 899 8 THEN
    WHAT VALUE FROM 900 TO 999 9 THEN
    WHAT VALUE > = 10 THEN 1000
    END
    VALUE
    T)
    GROUP BY State)
    SELECTION STATE,
    "< 100."
    "100-199.
    "200 299",
    "300-399.
    "400-499.
    '500-599,'
    "600-699.
    "700-799.
    "800-899.
    "900-999."
    "> = 1000."
    '< 100 '.
    + "100-199.
    + "200-299.
    + '300-399.
    + '400-499.
    + "500-599.
    + '600-699.
    + "700-799.
    + "800-899.
    + '900-999 ".
    + ' > = 1000.
    in total,.
    less ("< 100",)
    "100-199.
    "200 299",
    "300-399.
    "400-499.
    '500-599,'
    "600-699.
    "700-799.
    "800-899.
    "900-999."
    ("> = 1000 ') min_val,.
    largest ("< 100",)
    "100-199.
    "200 299",
    "300-399.
    "400-499.
    '500-599,'
    "600-699.
    "700-799.
    "800-899.
    "900-999."
    ("> = 1000 ') max_val
    FROM t1
    /

    Why not keep it simple?

    WITH t AS
(SELECT 'AL' state, 12 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 67 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 23 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 12 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 12 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 78 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 34 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 4 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 12 VALUE FROM DUAL
UNION ALL
SELECT 'AL' state, 15 VALUE FROM DUAL
UNION ALL
SELECT 'AZ' state, 6 VALUE FROM DUAL
UNION ALL
SELECT 'AZ' state, 123 VALUE FROM DUAL
UNION ALL
SELECT 'AZ' state, 123 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 23 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 120 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 456 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 11 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 24 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 34 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 87 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 23 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 234 VALUE FROM DUAL
UNION ALL
SELECT 'MA' state, 789 VALUE FROM DUAL
UNION ALL
SELECT 'MH' state, 54321 VALUE FROM DUAL)
SELECT state
     , NVL( COUNT( case when VALUE < 100 then 0 end ), 0 ) "<100"
     , NVL( COUNT( case when VALUE between 100 and 199 then 0 end ), 0 ) "100-199"
     , NVL( COUNT( case when VALUE between 200 and 299 then 0 end ), 0 ) "200-299"
     , NVL( COUNT( case when VALUE between 300 and 399 then 0 end ), 0 ) "300-399"
     , NVL( COUNT( case when VALUE between 400 and 499 then 0 end ), 0 ) "400-499"
     , NVL( COUNT( case when VALUE between 500 and 599 then 0 end ), 0 ) "500-599"
     , NVL( COUNT( case when VALUE between 600 and 699 then 0 end ), 0 ) "600-699"
     , NVL( COUNT( case when VALUE between 700 and 799 then 0 end ), 0 ) "700-799"
     , NVL( COUNT( case when VALUE between 800 and 899 then 0 end ), 0 ) "800-899"
     , NVL( COUNT( case when VALUE between 900 and 999 then 0 end ), 0 ) "900-999"
     , NVL( COUNT( case when VALUE >= 1000 then 0 end ), 0 ) ">=100"
     , count( value ) "total"
     , min( VALUE ) "min"
     , max( VALUE ) "max"
     , avg( VALUE ) "avg"
     , median( value ) "median"
from t
group by state

  • Get the number of DISTINCT from several columns

    How can I get the number of DISTINCT from several columns?
    SQL> select count(distinct ename) from emp;

COUNT(DISTINCTENAME)
--------------------
                  14

SQL> select count(distinct ename, job) from emp;
select count(distinct ename, job) from emp
       *
ERROR at line 1:
ORA-00909: invalid number of arguments

    Hello

    select count(distinct ename || job) from emp;

    Kind regards

  • Still not get the filtering and the SUMIF with visible cells

    Hi all

    I know it is a pain for you people, but I'm really stuck with this problem.

    I tried to use the answers I found in the community and the User Guide, but I have to be closed to this kind of explanation.

    Once again:

    In the column A is a series of dates

    In column B the value corresponding

    In the foot of B line, I want the SumOf only the visible cells that I filtered before.

    So far I have always the sum of all the lines, filtered or not.

    I added another column with a True/False value and tried to use it but get the same result.

    I'm pretty slow, so I guess that I'll need your help once more.

    It is the sum before filtering:

    And that's the sum after filtering:

    I thank you for your time and patience.

    Lopez

    Lopez Hello,

    Table filtering only affects the rows that are displayed. Hidden lines are always included in the calculations of formulas referring to them.

    Your formula, = SUM (B2:B16) summarizes the content of all the B2 to B16 cells, including those that are hidden.

    There are several other functions in the family of the SUM. SUM, SUMIF and SUMIF. ENS are of interest here.

    SUMIF (English: SUMIF) accepts a condition and only adds the cells on the lines which is assembled as a condition.

    SUMIF. ENS (English: SUMIFS) accepts several conditions and only adds the cells on the rows that meet all the conditions.

    For the example stated in your post, must be the form you need:

    = SUMIF. ENS(B; A; "> = 02/01/2016." A; "" < 03/01/2016 ")"

    (NOT tested. The dates should be placed in two cells and these cells referenced in the formula. post a reply with any error message if the formula does not work as written).

    The lines not included in the amount need not be hidden.

    If the formula is inserted into column B, it must be in a line defined as a footer line using the menu item in the same position as the highlight in this screenshot:

    Footer lines can be placed on foot (bottom) of a table.

    Kind regards

    Barry

  • How do I get the toolbar and the sidebar to display in the Finder?

    How do I get the toolbar and the sidebar to display in the finder window? The choices are grayed out in the view.

    Hello!

    You clicked on dock finder, take the cursor to the top and clicked on display and then click on hide menu bar .it toolbar is hidden.

    See the side bar, tool bar again click on view - once again, click Show toolbar.

    Thank you!

  • How to get the phone and messages back in my dock apps? And for some reason, I discover now all my app pages from the middle of my phone and not at the top?

    How can I get the phone and messages apps in my dock? And for some reason, I discover now all my app pages from the middle of my phone and not at the top?

    Try

    Settings > general > reset > reset home screen presentation.

    Note: All other applications will be organised by alphabetical order.

  • I have trouble getting my videos and pictures OF my shared albums. Once I shared them, I can't copy or save on my phone.

    I have trouble getting my videos and pictures OF my shared albums. Once I shared them, I can't copy or save on my phone.

    After you have shared photos and videos, they are always on your phone, unless you delete them.

  • Forgot, my apple Id and password, so I, just, decided to reset, my, phone, thru, bis, new, Tunes, account, I thought, after reset, phone, I would get right through, and I didn't, I, fonts, remember, my, e mail, I, defined, phone, up, with, someone, please

    Forgot, my apple Id and password, so I, just, decided to reset, my, phone, thru, bis, new, Tunes, account, I thought, after reset, phone, I would get right through, and I didn't, I, fonts, remember, my, e mail, I, defined, phone, up, with, someone, please, help

    These can help:

    If you have forgotten your Apple - Apple Support ID

    If you forgot your Apple ID - Apple Support password

    If you forgot the answers to your questions of security of Apple ID - Apple Support

  • How can I move the tab bar to the bottom, and how can I get my reload and stop buttons back?

    The new version of Firefox are repositioned my bar tabs at the top and I can't move it to the bottom. I tried everything: config, but browser.tabs.ontop was already set to false - very curious, because this should have resolved my problem in the first place.

    In the new version of Firefox, I can't get my reload and stop buttons back once again.

    Hi narab.

    You can use the classic theme restaurateur to bring all back them again. You can find more details here: https://support.mozilla.org/en-US/kb/how-to-make-new-firefox-look-like-old-firefox

  • I want to merge 2 Thunderbird profiles into one, if I get all emails and address book.

    I got a new PC and installed Thunderbird on it on 1 April and sent e-mails again. Then, I went back to the old PC and did a full back up all of my book e-mail and address. When I installed the back upward on the new PC I don't see new emails as well as the 'old' or the address book. How to merge back to the top with the current data so I see all of my old and new emails and two address books?

    Using a Pop or IMAP e-mail account?

    Re "made a full to the top of all my email and address book."
    How did you do that? Have you used "Mozbackup" tool to copy the entire profile or have you the emails to export as .eml files using ImportExporttool addon or manually save as .eml files? Or manually you the profile folder, then the Mail folder, then mail the account name and then copied all the files? Please explain.

    Did you export address books or access the profile folder and copy the .mab files? Are - what you are talking about "Personal address book" and "Collected addresses" or others that you have created?

    Currently, what do you see? Can you only see the old imported stuff or only new emails?

    How many profiles you have in the folder Profiles? You have more than one?
    A default profile name will be letters and numbers, for example: abc1d234.default and usually located here:
    C:\Users\UserName\AppData\Roaming\Thunderbird\Profiles\

    How to move to a new PC: http://kb.mozillazine.org/Move_to_a_new_PC

    If you have a profile and emails only you see are new emails, then ImportExport addon for backup/backup tool use these emails as .eml files and save them in the new folder on the desktop.

    ImportExporttool: https://freeshell.de//~kaosmos/mboximport-en.html

    Then on the old computer, install the Mozbackup tool and use it to backup the entire profile.

    Copy the file *.pcv on the desktop on the new computer.
    Install Mozbackup on new computer and use it to restore the old profile on new computer. Then use ImportExporttool to get stored in Thunderbird .eml files.

    Re: address books:
    http://KB.mozillazine.org/Moving_address_books_between_profiles

    MozBackup tool should move the default address books, but the other address books can must be exported in form of files LDIF or .csv files and then imported into the new profile on the new computer.

  • HP Elitebook 820 G2: Get drivers, software and firmware error page.

    Hi, I had this problem while HP go for HP Elitebook 820 G2

    A click on > download options > Get drivers, software and firmware... I get this error page

    http://h20565.www2.HP.com/hpsc/SWD/public/readIndex?sp4ts.SN=5CG5435L98 & sp4ts. OID = 7343194 & lang = to & SP...

    Downloads for this product are not available through this Web site.

    For the company's products, you can find English American filing of HPE software downloads.
    For the HP products and home offices, visit the HP customer service.
    If you need additional assistance, you can contact your local HP representative for more information on the specific update methods available for this product.

    @JimmyC

    Try here.

    http://h20564.www2.HP.com/hpsc/SWD/public/readIndex?sp4ts.Oid=7343194 & swLangOid = 8 & swEnvOid = 4192

    REO

  • Hi there, I been using MacBook Air and IPad2 for awhile. Recently, I lost my MacBook Air and am interested open ICloud to check my folder/old records. Can I get directions to the opportunity to see my files in I have the cloud.

    Hi there, I been using MacBook Air and IPad2 for awhile. Recently, I lost my MacBook Air and am interested open ICloud to check my folder/old records. Can I get directions to the opportunity to see my files in I have the cloud.

    I hope that you are not in a surprise but a MacBook Air not backup to iCloud. It can record data in iCloud drive (if you saved it specifically to this place) and can synchronize Photos, Contacts, etc to iCloud if you had set it to do. You can use any computer and connect to https://www.icloud.com/ to see what is stored. I hope this is good news and that it contains what you hope.

  • where can I get 10.9 and 10.10 for my mac so my last is 10.8.

    I jumped OS X 10.9 and 10.10 on my system at home that I need a stable system during this release cycle and had no time to fix potential problems. If it's not broken, don't fix it! 10.11 wreaks havoc on my system to work and I had to go back to 10.7.5. 10.7.5 to 10.11 problems included a complete failure to load which means that I couldn't not installed other applications apple app store. I also had problems with basic system services and could not solve them, so cancelled.  I want to move forward on my home machine to a later, stable version but am not prepared to try from 10.11 of previous problems. I love the feature that allows the restore via the time machine and would like to get 10.9 and 10.10 then running, but can not find a download on the app store. Any help would be appreciated.

    You should NOT be dried at least download (but stop installation) who releases when then came out.  In this way they would be sitting in your App Store-> purchases tab.

    But if you have NEVER downloaded them they aren't available for you any longer.  You can move to El Capitan or stay where you are.

    Suggest at least download El Capitan to enter your App Store-> tab purchases in case you have this reversal on El Capitan in the future.

    If you must absolutely have the Mavericks or Yosemite, take a go-Genius Bar and ask if they will install the or the older OS X releases.  If you have a good reason, they can perform this service.

Maybe you are looking for

  • HP pavilion g6-227sa: my laptop does not detect my wifi!

    Recently, I changed my victory windows 8 for win 7. "I downloaded" network drivers "via the net and installed on my laptop, but to my surprise it detect other wifis, but it cannot detect my wifi. "I should say when I go on ' network and sharing Cente

  • Satellite C660 won't boot from USB

    Hello I have a problem.After updating the BIOS to the latest version of the portable computer cannot boot from the usb drive.In the start menu, this option is present, but cannot start. I'm sorry. Google translate.

  • Satellite L650 - 10 d - sound crackling and shaped point on screen

    I have a L650 - 10 d, bought about 1 month ago. 1. I have a problem, whenever I listen to music, play games, watch movies my his gal (emits a crackling sound) once every 20 to 40 seconds. I reinstalled windows, I installed the latest drivers, I did a

  • Why it starts at the beginning?

    In older versions of Windows Media Player - the system picked up where the last/story/cd/whatever song arrested and the next time I opened the player is where he would have started playing.  In more recent versions of Windows Media Player (that I cur

  • How to set up my old e-mail account on my new computer?

    My old E-mail need Hello, recently my computer succumbed to a virus and it does not start, my son built me a new computer and install windows 7 that I was too frustrated with windows vista. I use email a lot and it is necessary for a lot of all my ta