Design of a queue for the Amazon price search
I am working on a system to retrieve pricing infomration for a list of items via the Amazon API (and ultimately, hopefully, purchase some of them).
I am thinking of implementing the entire process in Oracle. I have come up with a design below. Does this make sense?
The items are listed in a TABLE, the prices will be listed in a child TABLE.
The actual XML response would be stored outside the DB due to space constraints.
Restrictions:
1) One request batch per second (request batch may have two API requests)
2) 2000 request batches per hour
3) Request batch has timestamp
i) 15 minute timeout
ii) timestamp is encrypted (with full request)
Steps:
1) Create FIFO queue (TABLE) for price lookup
2) Create DBMS_SCHEDULER job
i) hourly interval
ii) pushes items into queue
iii) first request
- uses current timestamp
- activates second job
iv) succeeding requests increments original timestamp by one second
v) run until 2000 request batches in queue, or no more items
3) Create DBMS_SCHEDULER job
i) secondly interval
ii) pops one request batch off queue
- if queue is empty, deactivate job
iii) retrieves response (UTL_HTTP.REQUEST_PIECES)
iv) processes/stores response
v) exit
Published by: Brian Tkatch on 15 July 2012 13:31
The QA process can be executed as a DBMS_JOB/DBMS_SCHEDULER process. The master process (this place every second) can be started as a job. It can start in turn also the slave (waiting) processes that do the real work, that jobs.
It works the same way as MTS/multi-threaded server Oracle Shared Server aka. A process of Dispatcher that places work on a virtual circuit. Sharing process that takes a request for circuit work and does the actual work.
The problem with DBMS_JOB/DBMS_SCHEDULER in my opinion, is that you do not control the exact time of the execution of the work process. He could start the second specified. He could start a few seconds later. And without real second precision, your code might violate the agreement of its use with Amazon. However, with QA you can control it with your code to the second.
A simpler method would be DBMS_PIPE - but IMO, he is not sufficiently robust enough to run tight control.
Tags: Database
Similar Questions
-
What is a good way to use the queues for the model of consumers/producers?
Hi all
I am following the model of consumers/producers to use the queue to synchronize the following process: the producer is a loop to produce a number N, I will put each number generated in a table and after each 5 numbers generated, I put the table in the queue and pass it on to the consumer. I have to wait the use by consumers of the data and it will then remove the item from queue so that producers will have the chance to produce another 5 numbers. As I put the maximum size of the queue one, I expect that the producer and the consumer turns to produce / consume all five numbers and the opportunity to another. Here is my code
When the checkbox is false, the code will be
For the first 5 numbers, product will generate every thing right and put it in the table, and it's going to pass the array to the quere so that the consumer will have the chance to loop through the table. I except the procude loop will continue only when the queue is available (i.e. all items are deleted), but it seems that once the consumer starts the loop loop of the product will continue (if the indicator x + 1 and x 2 will be changed to numbers). But this isn't what I want, I know there must be something wrong, but I can't say it is.
dragondriver wrote:
As you say in 1, sequency structure to enforce the order of execution, that's why I put it here, in this example, the simple question, I replace the complete code with increase in the number, in the real case, the first markers + 1 and + 2 must be performed in this order.
Mikeporter says:
1. get rid of all the structures of the sequence. None of them are nothing but apply a work order which would have been the same without them.So even if you delete the sequence structure, there will be a fixed & defined order and it is because LabVIEW follows the MODEL of FLOW OF DATA.
Data flow model (more precisely in the context of LabVIEW): a block diagram node runs when it receives the required inputs. When a node is running, it produces output data and transmits data to the next node in the path of the data stream. The flow of data on the nodes determines the order of execution of the VIs and functions on the block diagram (click here for reference).
Now in your code, just remove the sequence structure will not make you order will be going to stay the same, but you need to do some very minor changes (as thread of the error in loop, before that he go to the node "Élément Dequeue").
Come to the main point: it's a good way to use the queue for the consumer/pmodel that?
The model you are using (and qualifying as consumer/pmodel) is much too deviated from the original consumer/pmodel which model.dragondriver wrote:
For the second, Yes, it's my fault for delete, though. I'm actually the example of model of producer/consumer design pattern, but I do not pay attention to the while loop in the part of the consumer.
While loops (two producers & consumers) are the essential part of this architecture and cannot be deleted. You can start your code using standard model.
-
It came with Photoshop and Lightroom... could I uninstall Lightroom and install Illustrator for the same price?
No, you can not install Illustrator instead of Lightroom. You can buy Adobe Illustrator creative Cloud app unique: pricing and membership creative cloud plans | Adobe Creative Cloud
-
I have the creative cloud on my laptop, but find I would like to have on my desk where there is more room. I can load it on a second computer for the same price and if so, how do I change the email address to download?
Hello
Creative desktop cloud applications can be downloaded and installed on multiple computers, regardless of the operating system. However, the activation is limited to two machines per person associated with membership. See terms of use for more information. Learn how to turn off a Creative License on a cloud machine.
You can download creative cloud under the following link: -.
https://helpx.Adobe.com/creative-cloud/help/download-install-app.html
-
So if I am already a customer subscribed to the photography program Photoshop, why am I not eligible for the price reduced from $29.99 per month subscription (offer that ends may 29) complete CS?
Terms and Conditions are the following:
VOID WHERE PROHIBITED OR RESTRICTED BY LAW. Qualifying customers may buy a full subscription Adobe® Creative Cloud™, with an annual commitment for a discounted price. Offer valid for the purchase of an annual plan, which requires a 12 month contract. This offer is only available to clients who have a Creative Suite edition or individual product in one of the following versions (CS3.x, CS4, or CS5.x or CS6), and who buy directly from the Adobe Store or by calling a regional call center of Adobe. This offer is not available for education, OEM or volume licensing customers. Residents of the affected countries to embargo are not eligible. This offer is limited to 1 one 1 purchase a creative cloud full annual membership by customer. Offers subject to the laws and control laws in U.S. exports where the recipient resides. Offer cannot be assigned, traded, sold, transferred, or combined with any other discount or offer or exchanged for cash or other goods and services. The offer is valid until 29 may 2015 PT and may be changed without notice.
In my view, it is of the affected line: "this offer is only available to customers who have a Creative Suite edition or individual product in one of the following versions (CS3.x, CS4, or CS5.x or CS6)...". »
Maybe it's because my edition of Photoshop is CC 2014? Rather than a CS3, CS4 etc?
No idea.
Hello
As you said, the offer is for customers who own a previous product of perpetual Creative Suite (CS3 - CS6) rather than a subscription.
You can switch the plan of photography to full creative cloud for the normal price.
Thank you
Bev
-
How to add the default value for the "Full - Text Search" box
Hi all
I am looking for the ability to add a default value for the "Full text search" box in the search form.
Example:
When the user accesses the search page it get default values for some fields (defined in the rule as default values) such as
xField1 = txt1 and xFiel2 = txt2.
and Furthermore we need full-text = Hello world.
Is this possible?
Another way that we use is to open the search by URL with coreContentOnly = 1 as a form:
http:// < Server >/cs/idcplg? IdcService = GET_DOC_PAGE & to Action = GetTemplatePage & to Page = STANDARD_QUERY_PAGE & coreContentOnly = 1 & to xField1 = txt1 & to xFiel2 = txt2
It is possible to add full-text criteria here? Somethig like: & text = Hello World
Thank you
Leon
It is not possible in a very obvious way, not to mention that it is a rather strange request. I can't really imagine a realistic use for this case.
Anyway, without a messy customization, you should provide a default query with an FTX inside element text setting (i.e. "IdcService = GET_DOC_PAGE & Action = GetTemplatePage & Page = STANDARD_QUERY_PAGE & QueryText =
Hello World " "")To do this in a profile in the secondary effects of the profile section, add the following:
<$dpPromote("QueryText",>$dpPromote("QueryText",>
Hello everyone ") $>. -
Size of queue for the server reports
Hello
How to increase the size of the queue for our reports, good that now it shows only 1,000 records.
Thank you
SanjayHello Sanjay
As an option. You can have a maximum of a single element of the queue in the configuration of your server. If you have nothing in the queue, the default value, 1000, will remain in force.
The queue item to specify the maximum number of jobs that can be held in each of the queues of reports. Reporting Services OracleAS components three of the queue:
a queue of scheduled tasks
a work in progress queue
a queue of tasks completed
The queue item provides the value allowed for each of these components.
This element applies only to the work queue. Thus, if the number of jobs is greater than the specified maximum value, this work queue automatically purge his older works. Planned work queue, and the queue of work in progress are not affected. By default the reports server queue size is 1000 jobs.
If you increase the size of the queue at more than 3000 and use reports (rwrqm.exe) queue manager to monitor the queue, the queue manager may fail. When a queue of 3000 or higher is necessary, use Oracle Enterprise Manager 10 g or reports Servlet (rwservlet) to manage and monitor the queue of jobs to report server.
Configuration file is located in $ORACLE_HOME/reports/Server
Kind regards
MythPublished by: myth on July 7, 2010 11:09
-
For the field price GP business portal
By using the following:
GP 2010
BP 5.0
When you create a requisition in Business Portal the item price field gathers at the current cost of the item of GP. The user wants the field to always be $0, so they can enter the amount in dollars. They don't want to shoot the current cost of the GP. Is there a way to get there?
Thank you
Jennifer
Hello
Check out these resources:
Microsoft Dynamics - help and how-to (technical support options)
http://www.Microsoft.com/en-us/Dynamics/default.aspxMicrosoft Dynamics - community, Forums & Forum search
http://community.dynamics.com/I hope this helps.
Rob Brown - Microsoft MVP<- profile="" -="" windows="" expert="" -="" consumer="" :="" bicycle="">-><- mark="" twain="" said="" it="">->
-
Problem in designing a custom field for the display of paragraph
Hello
I need to develop a custom field that is supposed to act as a RichTextField with the additional of the hyperlink feature as seen in the web pages. I have a few paragraphs of text with an id associated with each of them. A paragraph may have link to another paragraph. I think that the hyperlink is actually like a button.
My screen will consist of a single paragraph. If any hyperlink (id) in this paragraph is preesed then the screen reloads with another paragraph (id in a hurry).
My problem is to develop a custom field for the display of this paragraph.
Point: several lines of text without interruption. Some of the text may be bold, italic. a text can be the hyperlink.
Now I'm stuck. I have no ideas how I can start with. Can someone give me please suggestions as to how I can start with?
Thanks in advance
Bika
I have a different Suggestion.
If you are using an ActiveRichTextField, you get two things:
(1) the ability to shape a field using different fonts:
Comment - formatting text in a RichTextField
Article number: DB-00124
http://www.BlackBerry.com/knowledgecenterpublic/livelink.exe/fetch/2000/348583/800332/800505/800345/...(2) the ability to create "links" automatically using string models
How to-to comparison models in the BlackBerry smartphone to provide a user experience integrated applications
Article number: DB-00525
http://www.BlackBerry.com/knowledgecenterpublic/livelink.exe/fetch/2000/348583/800332/800505/800608/...Using models of string, I think you'd be able to create ' hot-link in your text, which the run Menu item was able to process you want.
Just an idea. Hope this works for you.
-
Data for the Eloqua Web search is secure
I recently put up a newspaper in the system using a form and functionality of Web of Eloqua database search.
From the point of view of users, it worked perfectly by only to redirect users to a page/hypersite if they correspond to their user name (email address) with a password (sent to them by DM).
My question is, how secure is the Web database search for similar implementation and are there underlying technical problems I should be aware of?
While the platform Eloqua itself is secure (including data search), its course in the fact that someone cannot access your data without authorization. However, because all data is stored in plain text in Eloqua, we absolutely do not recommend using it for something as password storage. Depending on where your company is located you can actually break the law by storing the password in plain text data. Therefore, while your data is safe, your password storage space is not, and anyone who has access to your instance Eloqua also has access to these thousands of usernames and passwords, many quite possibly shared with other sites outside your own.
Bottom line, you should never store any data that must be encrypted in Eloqua, because we don't offer any type of data outside of text unencrypted storage.
-
Where is the text for the index and search tabs saved?
It is specifically in RoboHelp 9 - although the answer may be the same for other versions.
I wonder, when WebHelp is created, where to get the text to use in the indexing and search tabs?
Specifically, I'm looking for (and change), the text in the Index tab that reads, "type in the keyword to find:" and the text in the Search tab that reads, "Type in the words to search for:
I suspect these two phrases are in a file .lng somewhere, and just this file needs to be changed...
Any ideas?
The mentioned Colum .lng file are accessible more easily by clicking on file > project settings > Advanced button > LNG tab
Under LNG, you will see everything most of the default labels for different items. It is better to work here on the source of LNG, rather than trying to change the output WebHelp that will be replaced later the next time that you build.
Thank you
John Daigle
Adobe Certified RoboHelp and Captivate instructor
Evergreen, Colorado
-
Hello
I recently bought a new laptop and want to transfer my license Adobe Creative Suite Design and Web Premium on my new computer laptop. Please tell us the steps how to disable my old laptop and transfer/install to my new laptop. Thank you very much.
Concerning
Angie
You should not disable since you are allowed to have two active facilities. But if you don't want to have a backup available facility, and then to turn it off, just to open one of the applications and choose disable in the Help menu. That will disable the suite together.
To install on the new machine, simply install it yourself you did initially. If you downloaded it initially then you'd find a link for it to yoru online Adobe account. Otherwise, you will find a link to the next page:
Download and Installation Help-
https://helpx.Adobe.com/download-install.html
If you still can not find a link then indicate what version of the suite you need and can probably provide a more direct link.
-
Adobe media encoder cs5.5 fails to open the queue for the first export
Adobe media encoder cs5.5 fails to open the queue when first CS5.5 exporting. I use an Mpeg2 normative action and choosing export immediate option works very well. After having chosen the option of queue, Adobe media encoder will show its splash screen... Then nothing. Does anyone else have this problem?
Premeiere CS5.5
ASUS Board
CORE I5 2400 K
16 GB of Ram
NVIDIA GTX 560Ti (software hack)
2 TB internal dedicated video player
Try trashing the preferences of the user for media encode. It is usually the cause of this type of problem.
-
The use of queues for the buffering
Hello
In my application, I'm scanning my computer com ports, seeking information on the buffer. What I want to do is run a Comsumer s/producer achetiecture so that I can constantly scan the com ports, and as soon as I get on the buffer data, it is sent to a queue. Queue, and then runs another loop that makes some manipulations of database, and it is this manipulation, I want to go back and keep sweeping him agrees more data. My problem is I want to be able to start and stop the program at any time, but I don't want to lose any information on the queue buffer. The manipulation of the database takes about 800 ms and com ports scan takes about 50 ms.
Basically, I want program blocking when I hit the "stop" button control, but if it contains data on the buffer, I want that he do the manipulation of data to all data on the buffer, then the output so I don't lose any information.
Thank you.
Gerardo
Since you have a queue, why not just add a Boolean value to your queue (ie: cluster) that you change value when the button is pressed. It will inform the other part of the program in order to process the remaining data, then stop. Very simple with little extra treatment.
R
-
How to open the queue for the printer HP Deskjet F2120 all-in-one printing?
My OS is Windows 7 Home Premium. I had a problem with being stuck in the print queue, print jobs what has caused the printer become inoperative.
The print queue is now disabled, but I had to download a diagnostic tool from HP which has scanned the printer and cleared the queue. That fixed the problem, but in the past when I had this same problem, I just access the leading printing and deleted stuck jobs. I couldn't find a way to get to the queue print this time and I hope someone out there can tell me what I did right.
Hello
- Click Start
- Click on devices and printers
- Right-click your printer and choose "see what's printing.
Tricky
Maybe you are looking for
-
My macPro syncs with iOS devices
When I create new contacts or take photos it does not download to my MacPro.
-
Don't know if there is an answer to this question. I see others asking also, what is the Maximum memory card format that will probably work in the 6 d? I checked the manual, I checked store descriptions, no one seems to know the answer to this questi
-
Hi, I'm interesting to add more ram, but is it ok for HP Pavilion dv6 - 6179er 16 GB of ram? Will it work?
-
How can I stop my data execution prevention be disabled
My data execution prevention keeps going to the people with reduced mobility. even after the fix Microsoft program
-
Is it possible to keep my laser ready for instant printing printer?
I have a bespoke system that involves a fairly instant impression once that the software produces a result, the problem is that the laser printer goes back through his vivid procedure at the beginning (which is once ok) is there a way to maintain as