How can I improve record speed of 100 millions of queries?

How can I improve record speed of 100 millions of queries?

When I ask several condition they have queries slow about 200 to 500 seconds.

3070066 wrote:

How can I improve record speed of 100 millions of queries?

When I ask several condition they have queries slow about 200 to 500 seconds.

There are 4 ways.

Faster hardware, faster disks, more big pipes, I/O etc. In other words, reduce costs (depending on the time) to the IO has 100 million lines.

Parallel processing. Instead of a single process serialized bed a 100 million lines, using a 100 in parallel process, each reading of 1 million rows.

Planning of roads I/O better and faster the query whether does not read a 100 million lines, or to read a 100 million lines (of say only 1 KB each), reading a 100 million entries index (for example, 50 bytes).

Rethink the requirement of the business that requires reading/treatment one lines 100 million. Or rethink the model of data requiring a 100 million lines to deal with this requirement of the company.

There is no option 5, which uses a magic wand that can be rough to make this process quick. Analyze the performance of the query (based on the links already provided in responses above) is the first step in the process of identifying the real problem and how to solve it.

Tags: Database

Similar Questions

  • How can I improve the performance of my compaq presario V2000, its very slow

    How can I improve the performance of my compaq presario V2000, its very slow!

    According to many things (specifications, you have now installed vs the programs he came originally with programs, etc.)-it may or may not work better that ever he does now.  However, in terms of nothing other than the software and others on the subject - there are some things you can do to optimize performance.

    Search for malware:

    Download, install, execute, update and perform analyses complete system with the two following applications:

    Remove anything they find.  Reboot when necessary.  (You can uninstall one or both when finished.)

    Search online with eSet Online Scanner.

    The less you have to run all the time, most things you want to run will perform:

    Use Autoruns to understand this all starts when your computer's / when you log in.  Look for whatever it is you do not know using Google (or ask here.)  You can hopefully figure out if there are things from when your computer does (or connect) you don't not need and then configure them (through their own built-in mechanisms is the preferred method) so they do not - start using your resources without reason.

    You can download and use Process Explorer to see exactly what is taking your time processor/CPU and memory.  This can help you to identify applications that you might want to consider alternatives for and get rid of all together.

    Do some cleaning and dusting off this hard drive:

    You can free up disk space (will also help get rid of the things that you do not use) through the following steps:

    Windows XP should take between 4.5 and 9 GB * with * an Office suite, editing Photo software, alternative Internet browser (s), various Internet plugins and a host of other things installed.

    If you are comfortable with the stability of your system, you can delete the uninstall of patches which has installed Windows XP...
    http://www3.TELUS.NET/dandemar/spack.htm
    (Especially of interest here - #4)
    (Variant: http://www.dougknox.com/xp/utils/xp_hotfix_backup.htm )

    You can run disk - integrated into Windows XP - cleanup to erase everything except your last restore point and yet more 'free '... files cleaning

    How to use disk cleanup
    http://support.Microsoft.com/kb/310312

    You can disable hibernation if it is enabled and you do not...

    When you Hibernate your computer, Windows saves the contents of the system memory in the hiberfil.sys file. As a result, the size of the hiberfil.sys file will always be equal to the amount of physical memory in your system. If you don't use the Hibernate feature and want to reclaim the space used by Windows for the hiberfil.sys file, perform the following steps:

    -Start the Control Panel Power Options applet (go to start, settings, Control Panel, and then click Power Options).
    -Select the Hibernate tab, uncheck "Activate the hibernation", and then click OK. Although you might think otherwise, selecting never under "Hibernate" option on the power management tab does not delete the hiberfil.sys file.
    -Windows remove the "Hibernate" option on the power management tab and delete the hiberfil.sys file.

    You can control the amount of space your system restore can use...

    1. Click Start, right click my computer and then click Properties.
    2. click on the System Restore tab.
    3. highlight one of your readers (or C: If you only) and click on the button "settings".
    4 change the percentage of disk space you want to allow... I suggest moving the slider until you have about 1 GB (1024 MB or close to that...)
    5. click on OK. Then click OK again.

    You can control the amount of space used may or may not temporary Internet files...

    Empty the temporary Internet files and reduce the size, that it stores a size between 64 MB and 128 MB...

    -Open a copy of Microsoft Internet Explorer.
    -Select TOOLS - Internet Options.
    -On the general tab in the section 'Temporary Internet files', follow these steps:
    -Click on 'Delete the Cookies' (click OK)
    -Click on "Settings" and change the "amount of disk space to use: ' something between 64 MB and 128 MB. (There may be many more now.)
    -Click OK.
    -Click on 'Delete files', then select "Delete all offline content" (the box), and then click OK. (If you had a LOT, it can take 2 to 10 minutes or more).
    -Once it's done, click OK, close Internet Explorer, open Internet Explorer.

    You can use an application that scans your system for the log files and temporary files and use it to get rid of those who:

    CCleaner (free!)
    http://www.CCleaner.com/
    (just disk cleanup - do not play with the part of the registry for the moment)

    Other ways to free up space...

    SequoiaView
    http://www.win.Tue.nl/SequoiaView/

    JDiskReport
    http://www.jgoodies.com/freeware/JDiskReport/index.html

    Those who can help you discover visually where all space is used.  Then, you can determine what to do.

    After that - you want to check any physical errors and fix everything for efficient access"

    CHKDSK
    How to scan your disks for errors* will take time and a reboot.

    Defragment
    How to defragment your hard drives* will take time

    Cleaning the components of update on your WIndows XP computer

    While probably not 100% necessary-, it is probably a good idea at this time to ensure that you continue to get the updates you need.  This will help you ensure that your system update is ready to do it for you.

    Download and run the MSRT tool manually:
    http://www.Microsoft.com/security/malwareremove/default.mspx
    (Ignore the details and download the tool to download and save to your desktop, run it.)

    Reset.

    Download/install the latest program Windows installation (for your operating system):
    (Windows XP 32-bit: WindowsXP-KB942288-v3 - x 86 .exe )
    (Download and save it to your desktop, run it.)

    Reset.

    and...

    Download the latest version of Windows Update (x 86) agent here:
    http://go.Microsoft.com/fwlink/?LinkId=91237
    ... and save it to the root of your C:\ drive. After you register on the root of the C:\ drive, follow these steps:

    Close all Internet Explorer Windows and other applications.

    AutoScan--> RUN and type:
    %SystemDrive%\windowsupdateagent30-x86.exe /WUFORCE
    --> Click OK.

    (If asked, select 'Run'). --> Click on NEXT--> select 'I agree' and click NEXT--> where he completed the installation, click "Finish"...

    Reset.

    Now reset your Windows with this FixIt components update (you * NOT * use the aggressive version):
    How to reset the Windows Update components?

    Reset.

    Now that your system is generally free of malicious software (assuming you have an AntiVirus application), you've cleaned the "additional applications" that could be running and picking up your precious memory and the processor, you have authorized out of valuable and makes disk space as there are no problems with the drive itself and your Windows Update components are updates and should work fine - it is only only one other thing you pouvez wish to make:

    Get and install the hardware device last drivers for your system hardware/system manufacturers support and/or download web site.

    If you want, come back and let us know a bit more information on your system - particularly the brand / model of the system, you have - and maybe someone here can guide you to the place s x of law to this end.  This isn't 100% necessary - but I'd be willing to bet that you would gain some performance and features in making this part.

  • How can I get (record) web pages to work when im offline

    How can I get (record) web pages to work when im offline. I can't find any setting for this.can help you

    Try the Read It Later addon.
    https://addons.Mozilla.org/en-us/Firefox/addon/7661/

  • Sony Cam AS15 Action does not focus under water. How can I improve underwater video quality?

    The AS15 provides a very good quality video over water, but as soon as it is lowered underwater, it is not focused. The video quality is very bad, the images are totally blurred. It seems that it does not focus properly.

    How can we improve quality?


  • How can I use the speed dial on my Motorola W490?

    How can I use the speed dial on my motorola W490? I have followed the instructions on the manual book, but did not work.

    The result, I received the screen is "unauthorized."

    Pls help. THX.

    Desiree-

    Can you tell us what steps you took to set your speed dial and use it?

    W490 User Guide

    On page 61, you can find information about adding dial numbers to speed up and 62 mentions how you dial. For example if you have saved your Uncle Paul as Dáil speed 3, you simply hold the button 3 to call this contact.

    Mark

    Support Forums Manager

  • I've upgraded to IE9, how can I improve the quality of my picture of the webcam for video calls?

    I've recently updated to IE9.  The quality of my web cam image is dark and resolution of RL is unrecognizableand similar to the shadow.  How can I improve the quality of the image during video calls, for example over Skype or yahoo?   I have a Toshiba door 'ge'.

    Hello

    1. What is the number of brand and model of the computer and the webcam?

    2. What is the operating system installed on the computer?

    Try to uninstall then install web camera drivers and check if the problem is solved or not.

    Uninstall or change a program

    Note: this article applies to Windows Vista and Windows 7

    Also check out the following link:

     
    Let us know if the problem is solved or not.
  • How can I improve the performance of my pc hp pavilion

    When I try to use the pc, it is very very slow. programs like e-mail search on the net, watch video (cannot watch it) are all slow. How can I improve performance?

    Hello

    You could have a look at this link on the improvement of performance. May be useful for you. Talking about the general house keeping deleting the temporary files, defrag and such.

  • How can I improve the graphics subscore for windows 7 64 bit?

    How can I improve graphics subscore for windows 7 64 bit

    the graphics is 5.0 and I used to improve, but how?
     
    Original title: computer Preformance

    There is nothing you can do. The score you get is comparable to what this laptop model should get.

    http://www.notebookcheck.NET/review-Acer-Aspire-5741G-334G50Mn-notebook.28688.0.html

    http://www.notebookcheck.NET/ATI-Mobility-Radeon-HD-5470.23698.0.html

  • How can I create a slide show 100% width, but have the caption in different positions for each slide.

    How can I create a slide show 100% width, but have the caption in different positions for each slide.

    There is placement of the legend only one for all the images in a slide show.

    The slideshow widget is specialized to create slideshows of the image. The widget of composition is a much more flexible widget which allows any combination of images and text in the trigger containers or containers of target of the widget. A widget of composition could be used to create a slide show with several legends by image and/or legends of different places for each image. The disadvantage of the widget of composition is that, due to the nature flexible widget, very little is automated for you and so it is much more intensive work to create a slideshow of images using the widget of composition.

    Thank you

    Sanjit

  • I want to get multiple clips horizontally through the frame, how can I control the speed at which the move?

    I want to get multiple clips horizontally through the frame, how can I control the speed at which the move?

    And the second

  • How can I adjust my speed of clip at 0%?

    How can I adjust my speed of clip at 0%?

    I can't do it. Is a minimum value: 1.00

    http://i.imgur.com/6ziCmo3.png

    This method works, but it is rather abrupt. He suddenly stops and takes up all of a sudden.

    Many people prefer the easing in the gel just a little and then acceleration out of it just a little. To do this requires the time remapping effect.

    You might want to invest some time with this seminar. I watched it live and it's pretty good.

    emiere-Pro-CC-and-after-effects-CC/ http://blogs.adobe.com/creativelayer/Ask-a-Pro-Webinar-Series-Controlling-Time-in-Adobe-PR

  • Everytime I open a PDF the reader opens up to 800%. How can I fix to open at 100%?

    Everytime I open a PDF the reader opens up to 800%. How can I fix to open at 100%?

    Preferences | Display of the page. Zoom: 100%.

  • How can I improve Import / Export performance?

    Hello

    How can I improve Import / Export performance?
    Also of the good links to learn more about Datapump please?

    Thank you

    Felipe.

    Dear Felipe,

    Please take a look at this excellent notes by Chris Foot datapump:

    * http://www.dbazine.com/blogs/blog-cf/chrisfoot/blogentry.2006-06-08.1893574515
    * http://www.dbazine.com/blogs/blog-cf/chrisfoot/blogentry.2006-06-02.2786242018
    * http://www.dbazine.com/blogs/blog-cf/chrisfoot/blogentry.2006-05-26.3042156388
    * http://www.dbazine.com/blogs/blog-cf/chrisfoot/blogentry.2006-05-19.6918984141

    Kind regards

    Francisco Munoz Alvarez
    http://www.oraclenz.com

  • How can you remove recorded on all google search results: home at the start of firefox?

    I like to keep google search, as it is in terms of use as the search engine. I do not wish to change this. I would like to reset or clear the search results of the search bar on the start about page however: home. Is this possible and how can I do? East - somethings that I have to reset in the about.config file? I have already reset the mstone, adjustment and the results are still there. Thanks a lot for helping me with this problem.

    Follow these steps to delete the recorded data (form) in a drop-down list:

    1. Click on the (empty) input field on the web page to open the drop-down list
    2. Select an entry in the drop-down list
    3. Press the DELETE key (on a Mac: shift + delete) to remove it.
  • How can I get my page to 100% my facebook page and other web pages are so small, I can't see them...

    In google chrome, you have a key at the top you can click on and scroll down to the percentage, you want your page to be 75% or 100%... My pages are popping up to 25% for children to see or read, how can I change that to where I used my full page... which means that the police on my larger page?

    CTRL + 0 (zero) resets default zoom

    CTRL ++ zoom in

    CTRL + - zoom out

Maybe you are looking for