Design of a queue for the Amazon price search

I am working on a system to retrieve pricing infomration for a list of items via the Amazon API (and ultimately, hopefully, purchase some of them).
I am thinking of implementing the entire process in Oracle. I have come up with a design below. Does this make sense?

The items are listed in a TABLE, the prices will be listed in a child TABLE.
The actual XML response would be stored outside the DB due to space constraints. 

Restrictions:
     1) One request batch per second (request batch may have two API requests)
     2) 2000 request batches per hour
     3) Request batch has timestamp
          i) 15 minute timeout
          ii) timestamp is encrypted (with full request)

Steps:
     1) Create FIFO queue (TABLE) for price lookup
     2) Create DBMS_SCHEDULER job
          i) hourly interval
          ii) pushes items into queue
          iii) first request
               - uses current timestamp
               - activates second job
          iv) succeeding requests increments original timestamp by one second
          v) run until 2000 request batches in queue, or no more items
     3) Create DBMS_SCHEDULER job
          i) secondly interval
          ii) pops one request batch off queue
               - if queue is empty, deactivate job
          iii) retrieves response (UTL_HTTP.REQUEST_PIECES)
          iv) processes/stores response
          v) exit
Published by: Brian Tkatch on 15 July 2012 13:31

The QA process can be executed as a DBMS_JOB/DBMS_SCHEDULER process. The master process (this place every second) can be started as a job. It can start in turn also the slave (waiting) processes that do the real work, that jobs.

It works the same way as MTS/multi-threaded server Oracle Shared Server aka. A process of Dispatcher that places work on a virtual circuit. Sharing process that takes a request for circuit work and does the actual work.

The problem with DBMS_JOB/DBMS_SCHEDULER in my opinion, is that you do not control the exact time of the execution of the work process. He could start the second specified. He could start a few seconds later. And without real second precision, your code might violate the agreement of its use with Amazon. However, with QA you can control it with your code to the second.

A simpler method would be DBMS_PIPE - but IMO, he is not sufficiently robust enough to run tight control.

Tags: Database

Similar Questions

  • What is a good way to use the queues for the model of consumers/producers?

    Hi all

    I am following the model of consumers/producers to use the queue to synchronize the following process: the producer is a loop to produce a number N, I will put each number generated in a table and after each 5 numbers generated, I put the table in the queue and pass it on to the consumer. I have to wait the use by consumers of the data and it will then remove the item from queue so that producers will have the chance to produce another 5 numbers. As I put the maximum size of the queue one, I expect that the producer and the consumer turns to produce / consume all five numbers and the opportunity to another. Here is my code

    When the checkbox is false, the code will be

    For the first 5 numbers, product will generate every thing right and put it in the table, and it's going to pass the array to the quere so that the consumer will have the chance to loop through the table. I except the procude loop will continue only when the queue is available (i.e. all items are deleted), but it seems that once the consumer starts the loop loop of the product will continue (if the indicator x + 1 and x 2 will be changed to numbers). But this isn't what I want, I know there must be something wrong, but I can't say it is.

    dragondriver wrote:

    As you say in 1, sequency structure to enforce the order of execution, that's why I put it here, in this example, the simple question, I replace the complete code with increase in the number, in the real case, the first markers + 1 and + 2 must be performed in this order.

    Mikeporter says:
    1. get rid of all the structures of the sequence. None of them are nothing but apply a work order which would have been the same without them.

    So even if you delete the sequence structure, there will be a fixed & defined order and it is because LabVIEW follows the MODEL of FLOW OF DATA.

    Data flow model (more precisely in the context of LabVIEW): a block diagram node runs when it receives the required inputs. When a node is running, it produces output data and transmits data to the next node in the path of the data stream. The flow of data on the nodes determines the order of execution of the VIs and functions on the block diagram (click here for reference).

    Now in your code, just remove the sequence structure will not make you order will be going to stay the same, but you need to do some very minor changes (as thread of the error in loop, before that he go to the node "Élément Dequeue").

    Come to the main point: it's a good way to use the queue for the consumer/pmodel that?
    The model you are using (and qualifying as consumer/pmodel) is much too deviated from the original consumer/pmodel which model.

    dragondriver wrote:

    For the second, Yes, it's my fault for delete, though. I'm actually the example of model of producer/consumer design pattern, but I do not pay attention to the while loop in the part of the consumer.

    While loops (two producers & consumers) are the essential part of this architecture and cannot be deleted. You can start your code using standard model.

  • So, I have creative cloud... It came with Photoshop and Lightroom... could I uninstall Lightroom and install Illustrator for the same price?

    It came with Photoshop and Lightroom... could I uninstall Lightroom and install Illustrator for the same price?

    No, you can not install Illustrator instead of Lightroom. You can buy Adobe Illustrator creative Cloud app unique: pricing and membership creative cloud plans | Adobe Creative Cloud

  • can I load the creative cloud on more than one computer for the same price.  I would like to put on our other computer.

    I have the creative cloud on my laptop, but find I would like to have on my desk where there is more room.  I can load it on a second computer for the same price and if so, how do I change the email address to download?

    Hello

    Creative desktop cloud applications can be downloaded and installed on multiple computers, regardless of the operating system. However, the activation is limited to two machines per person associated with membership. See terms of use for more information. Learn how to turn off a Creative License on a cloud machine.

    You can download creative cloud under the following link: -.

    https://helpx.Adobe.com/creative-cloud/help/download-install-app.html

  • Why am I not eligible for the discounted price of $29.99 per month subscription CS full?

    So if I am already a customer subscribed to the photography program Photoshop, why am I not eligible for the price reduced from $29.99 per month subscription (offer that ends may 29) complete CS?

    Terms and Conditions are the following:

    VOID WHERE PROHIBITED OR RESTRICTED BY LAW. Qualifying customers may buy a full subscription Adobe® Creative Cloud™, with an annual commitment for a discounted price. Offer valid for the purchase of an annual plan, which requires a 12 month contract. This offer is only available to clients who have a Creative Suite edition or individual product in one of the following versions (CS3.x, CS4, or CS5.x or CS6), and who buy directly from the Adobe Store or by calling a regional call center of Adobe. This offer is not available for education, OEM or volume licensing customers. Residents of the affected countries to embargo are not eligible. This offer is limited to 1 one 1 purchase a creative cloud full annual membership by customer. Offers subject to the laws and control laws in U.S. exports where the recipient resides. Offer cannot be assigned, traded, sold, transferred, or combined with any other discount or offer or exchanged for cash or other goods and services. The offer is valid until 29 may 2015 PT and may be changed without notice.

    In my view, it is of the affected line: "this offer is only available to customers who have a Creative Suite edition or individual product in one of the following versions (CS3.x, CS4, or CS5.x or CS6)...". »

    Maybe it's because my edition of Photoshop is CC 2014? Rather than a CS3, CS4 etc?

    No idea.

    Hello

    As you said, the offer is for customers who own a previous product of perpetual Creative Suite (CS3 - CS6) rather than a subscription.

    You can switch the plan of photography to full creative cloud for the normal price.

    Thank you

    Bev

  • How to add the default value for the "Full - Text Search" box

    Hi all

    I am looking for the ability to add a default value for the "Full text search" box in the search form.

    Example:

    When the user accesses the search page it get default values for some fields (defined in the rule as default values) such as

    xField1 = txt1 and xFiel2 = txt2.

    and Furthermore we need full-text = Hello world.

    Is this possible?

    Another way that we use is to open the search by URL with coreContentOnly = 1 as a form:

    http:// < Server >/cs/idcplg? IdcService = GET_DOC_PAGE & to Action = GetTemplatePage & to Page = STANDARD_QUERY_PAGE & coreContentOnly = 1 & to xField1 = txt1 & to xFiel2 = txt2

    It is possible to add full-text criteria here? Somethig like: & text = Hello World

    Thank you

    Leon

    It is not possible in a very obvious way, not to mention that it is a rather strange request.  I can't really imagine a realistic use for this case.

    Anyway, without a messy customization, you should provide a default query with an FTX inside element text setting (i.e. "IdcService = GET_DOC_PAGE & Action = GetTemplatePage & Page = STANDARD_QUERY_PAGE & QueryText =Hello World" "")

    To do this in a profile in the secondary effects of the profile section, add the following:

    <$dpPromote("QueryText",>Hello everyone") $>.

  • Size of queue for the server reports

    Hello

    How to increase the size of the queue for our reports, good that now it shows only 1,000 records.

    Thank you
    Sanjay

    Hello Sanjay

    As an option. You can have a maximum of a single element of the queue in the configuration of your server. If you have nothing in the queue, the default value, 1000, will remain in force.

    The queue item to specify the maximum number of jobs that can be held in each of the queues of reports. Reporting Services OracleAS components three of the queue:

    a queue of scheduled tasks

    a work in progress queue

    a queue of tasks completed

    The queue item provides the value allowed for each of these components.

    This element applies only to the work queue. Thus, if the number of jobs is greater than the specified maximum value, this work queue automatically purge his older works. Planned work queue, and the queue of work in progress are not affected. By default the reports server queue size is 1000 jobs.

    If you increase the size of the queue at more than 3000 and use reports (rwrqm.exe) queue manager to monitor the queue, the queue manager may fail. When a queue of 3000 or higher is necessary, use Oracle Enterprise Manager 10 g or reports Servlet (rwservlet) to manage and monitor the queue of jobs to report server.

    Configuration file is located in $ORACLE_HOME/reports/Server

    Kind regards
    Myth

    Published by: myth on July 7, 2010 11:09

  • For the field price GP business portal

    By using the following:

    GP 2010

    BP 5.0

    When you create a requisition in Business Portal the item price field gathers at the current cost of the item of GP.  The user wants the field to always be $0, so they can enter the amount in dollars.  They don't want to shoot the current cost of the GP.  Is there a way to get there?

    Thank you

    Jennifer

    Hello

    Check out these resources:

    Microsoft Dynamics - help and how-to (technical support options)
    http://www.Microsoft.com/en-us/Dynamics/default.aspx

    Microsoft Dynamics - community, Forums & Forum search
    http://community.dynamics.com/

    I hope this helps.

    Rob Brown - Microsoft MVP<- profile="" -="" windows="" expert="" -="" consumer="" :="" bicycle=""><- mark="" twain="" said="" it="">

  • Problem in designing a custom field for the display of paragraph

    Hello

    I need to develop a custom field that is supposed to act as a RichTextField with the additional of the hyperlink feature as seen in the web pages. I have a few paragraphs of text with an id associated with each of them. A paragraph may have link to another paragraph. I think that the hyperlink is actually like a button.

    My screen will consist of a single paragraph. If any hyperlink (id) in this paragraph is preesed then the screen reloads with another paragraph (id in a hurry).

    My problem is to develop a custom field for the display of this paragraph.

    Point: several lines of text without interruption. Some of the text may be bold, italic. a text can be the hyperlink.

    Now I'm stuck. I have no ideas how I can start with. Can someone give me please suggestions as to how I can start with?

    Thanks in advance

    Bika

    I have a different Suggestion.

    If you are using an ActiveRichTextField, you get two things:

    (1) the ability to shape a field using different fonts:

    Comment - formatting text in a RichTextField
    Article number: DB-00124
    http://www.BlackBerry.com/knowledgecenterpublic/livelink.exe/fetch/2000/348583/800332/800505/800345/...

    (2) the ability to create "links" automatically using string models

    How to-to comparison models in the BlackBerry smartphone to provide a user experience integrated applications
    Article number: DB-00525
    http://www.BlackBerry.com/knowledgecenterpublic/livelink.exe/fetch/2000/348583/800332/800505/800608/...

    Using models of string, I think you'd be able to create ' hot-link in your text, which the run Menu item was able to process you want.

    Just an idea.  Hope this works for you.

  • Data for the Eloqua Web search is secure

    I recently put up a newspaper in the system using a form and functionality of Web of Eloqua database search.

    From the point of view of users, it worked perfectly by only to redirect users to a page/hypersite if they correspond to their user name (email address) with a password (sent to them by DM).

    My question is, how secure is the Web database search for similar implementation and are there underlying technical problems I should be aware of?

    While the platform Eloqua itself is secure (including data search), its course in the fact that someone cannot access your data without authorization. However, because all data is stored in plain text in Eloqua, we absolutely do not recommend using it for something as password storage. Depending on where your company is located you can actually break the law by storing the password in plain text data. Therefore, while your data is safe, your password storage space is not, and anyone who has access to your instance Eloqua also has access to these thousands of usernames and passwords, many quite possibly shared with other sites outside your own.

    Bottom line, you should never store any data that must be encrypted in Eloqua, because we don't offer any type of data outside of text unencrypted storage.

  • Where is the text for the index and search tabs saved?

    It is specifically in RoboHelp 9 - although the answer may be the same for other versions.

    I wonder, when WebHelp is created, where to get the text to use in the indexing and search tabs?

    Specifically, I'm looking for (and change), the text in the Index tab that reads, "type in the keyword to find:" and the text in the Search tab that reads, "Type in the words to search for:

    I suspect these two phrases are in a file .lng somewhere, and just this file needs to be changed...

    Any ideas?

    The mentioned Colum .lng file are accessible more easily by clicking on file > project settings > Advanced button > LNG tab

    Under LNG, you will see everything most of the default labels for different items. It is better to work here on the source of LNG, rather than trying to change the output WebHelp that will be replaced later the next time that you build.

    Thank you

    John Daigle

    Adobe Certified RoboHelp and Captivate instructor

    Evergreen, Colorado

    www.showmethedemo.com

  • How to transfer my license Adobe Creative Suite Design and Web Premium for the new computer laptop?

    Hello

    I recently bought a new laptop and want to transfer my license Adobe Creative Suite Design and Web Premium on my new computer laptop. Please tell us the steps how to disable my old laptop and transfer/install to my new laptop.  Thank you very much.

    Concerning

    Angie

    You should not disable since you are allowed to have two active facilities.  But if you don't want to have a backup available facility, and then to turn it off, just to open one of the applications and choose disable in the Help menu.  That will disable the suite together.

    To install on the new machine, simply install it yourself you did initially.  If you downloaded it initially then you'd find a link for it to yoru online Adobe account.  Otherwise, you will find a link to the next page:

    Download and Installation Help-

    https://helpx.Adobe.com/download-install.html

    If you still can not find a link then indicate what version of the suite you need and can probably provide a more direct link.

  • Adobe media encoder cs5.5 fails to open the queue for the first export

    Adobe media encoder cs5.5 fails to open the queue when first CS5.5 exporting. I use an Mpeg2 normative action and choosing export immediate option works very well. After having chosen the option of queue, Adobe media encoder will show its splash screen... Then nothing. Does anyone else have this problem?

    Premeiere CS5.5

    ASUS Board

    CORE I5 2400 K

    16 GB of Ram

    NVIDIA GTX 560Ti (software hack)

    2 TB internal dedicated video player

    Try trashing the preferences of the user for media encode.  It is usually the cause of this type of problem.

  • The use of queues for the buffering

    Hello

    In my application, I'm scanning my computer com ports, seeking information on the buffer. What I want to do is run a Comsumer s/producer achetiecture so that I can constantly scan the com ports, and as soon as I get on the buffer data, it is sent to a queue. Queue, and then runs another loop that makes some manipulations of database, and it is this manipulation, I want to go back and keep sweeping him agrees more data.  My problem is I want to be able to start and stop the program at any time, but I don't want to lose any information on the queue buffer. The manipulation of the database takes about 800 ms and com ports scan takes about 50 ms.

    Basically, I want program blocking when I hit the "stop" button control, but if it contains data on the buffer, I want that he do the manipulation of data to all data on the buffer, then the output so I don't lose any information.

    Thank you.

    Gerardo

    Since you have a queue, why not just add a Boolean value to your queue (ie: cluster) that you change value when the button is pressed.  It will inform the other part of the program in order to process the remaining data, then stop.  Very simple with little extra treatment.

    R

  • How to open the queue for the printer HP Deskjet F2120 all-in-one printing?

    My OS is Windows 7 Home Premium. I had a problem with being stuck in the print queue, print jobs what has caused the printer become inoperative.

    The print queue is now disabled, but I had to download a diagnostic tool from HP which has scanned the printer and cleared the queue. That fixed the problem, but in the past when I had this same problem, I just access the leading printing and deleted stuck jobs. I couldn't find a way to get to the queue print this time and I hope someone out there can tell me what I did right.

    Hello

    • Click Start
    • Click on devices and printers
    • Right-click your printer and choose "see what's printing.

    Tricky

Maybe you are looking for