Problem of incremental update in the treatment of updates

Hi all

I use ODI 11.1.1 and IKM Oracle incremental update. I have the following problem:

My target table contains the following keys: NAME EVENT_ID, location_id and following columns, VALUE

And assume that my source table that contains the same columns and a TIME column.

If the source table is as follows:

EVENT_ID LOCATION_ID NAME TIME VALUE

1                1                   T         1          1

1                1                   T         2          2

If these two lines are loaded into the staging table I$ at the same time and the target table does not contain already the (1,1) combination for event_id, location_id 2 new lines will be inserted into the fact table. I would not have only one line with the most recent value. The problem is that step 'make update' will never report the second row as 'U' because there is no such thing as the combined update key in the fact table.

How can I get the desired behavior?

Hello

There are several ways to do so. It depends mainly on your cardinality.

APPROACH 1 (VERY BAD): filter

  • Place a filter with this text

    • EVENT_ID | LOCATION_ID | TIME IN (SELECT EVENT_ID |) LOCATION_ID | MAX (TIME) FROM TABLE GROUP BY EVENT_ID, LOCATION_ID)

APPROACH 2: make a subselect interface

  1. create a temporary with interface

    • EVENT_ID = EVENT_ID
    • LOCATION_ID = LOCATION_ID
    • TIME = MAX (TIME)
  2. then put this interface (yellow) in your source and mark as subselect
  3. make an inner join

METHOD 3: VIEW

  1. give an opinion on your db

    • create view MYVIEW AS SELECT EVENT_ID, location_id, MAX (TIME) FROM TABLE GROUP BY EVENT_ID location_id
  2. reversing the trend
  3. inner join

APPROACH 4: FUNCTION ANALYTICS (good)

Use an analytic function. See for example Re: 2nd Max Sal. But what will little customization.

However, a variable is not a solution.

Let us know

Tags: Business Intelligence

Similar Questions

  • backup incremental update feature

    Hi all

    can you explain this feafure when you take the copy of the image and then backup incremental and when
    data file is corrupted I have to restore the copy of the data file, and then apply an incremental backup.
    I do this planning, but I do not write the COPY OF THE file of DATA WITH TAG RECOVER command

    I take a copy of the image and then make this table spaces and make sure is the default value for the data base and ten make change
    DML, stop immediately and drop datafile can restore and recover
    restore the data file 5
    recover datafile 5


    without any incremental update command, feature and now my database is up-to-date? can you explain to me an example please?


    Thank you very much

    We must use the control switch that will make the controlfile to point to the backup of the data on the disk files.
    For example: you lost the data user 6 file

    RMAN > switch datafile 6 to copy;
    RMAN > recover datafile 6;
    RMAN > alter database datafile 6 online;

    The problem with incremental updates, is that, after any failure, as above, if restore us it, it will change the name and location of the data file. After restoration manual you rename them to original location.

    In case of failure of the database

    RMAN > switch database to copy.
    RMAN > restore database;
    RMAN > alter database open

    Published by: renee_mathieu on August 5, 2012 07:54

  • What should I use the incremental update for IKM Oracle?

    IM missing some conditions before using the incremental update

    I have a simple integration of an Oracle source data source table directly to another target Oracle data source table

    I'd like to run a simple merger on a periodic basis

    Source table that contains data, the target table is created and set in the model

    Project created, put two tables in mapping

    LKM specified LKM SQL for Oracle (I won't use DBLink)

    Key of the target table are defined

    I go to set the IKM but cannot choose the incremental update option

    Probably a new problem user but can't seem to find a solution

    So it's a stale issue that I have long found the answer and thought I would post it here just in case someone else is beginner and difficulties as well

    For the incremental update be visible the following must be in place:

    (1) the IKM that supports the update must be visible in the project, as shown in the diagram above - that's what our ODI comes with

    (2) in the logical scheme of the model to the target in the context table to see its properties

    (3) set a goal on the properties of incremental update

    (4) now put target in context in the physical tab

    (5) IKM Oracle Fusion is now visible in the knowledge of integration Module

  • Invisible signatures by the incremental update does not have any signs?

    Hi all!

    Sorry if here is not the right place for my question. If this is the case, is - can someone inform in the right place?

    Well, I am trying to build a tool to sign a PDF of incremental update using Delphi [].

    I am beginner in PDF structure and the brevity of the time made me perhaps a bit inattentive and desperate.

    After 2 months of hard fighting, I managed to get Acrobat Reader know my PDF (initially created by a java tool called SignServer) has a signature and recognize the certificate but it says that the certification is not valid because 'the byte range is invalid.

    I analyzed the PDF file in a hex editor and the byte range signature so any other object compensates seems correct.

    My PDF test is here: https://drive.Google.com/open?ID=0B0KKmaB-a0Z4SDZfNXdZRWZEem8

    Can you tell me what the problem with this PDF?

    Many thanks for any help!

    Specifications and PDF language

  • The problem with Windows update - automatic updates service won't start not (error 126)

    original title: the problem with Windows update

    Please help me to get Windows Update operational return.

    XP Media Center Edition, Version 2002, SP3

    Had a problem with "Generic host process for Win32 has encountered a problem and needs to close."  Follow-up of the instructions on this link:
    http://support.Microsoft.com/kb/931852

    Seems to have cleared the error message.  So far so good.

    However, now a Windows Update does not work.  Security Center Windows were told to go to control panel > system and use the Automatic Updates tab.  But, if I go to control panel > system, the Automatic Updates tab does not appear.

    I also went to the control panel > Services.  Tried to start the automatic updates service.  Received this error message: could not start the service automatic updates on Local computer.  Error 126: The specified module could not be found

    I already followed several instructions on this subject:

    http://answers.Microsoft.com/en-us/Windows/Forum/windows_xp-windows_update/cannot-access-Windows-automatic-update-after/aeaec175-01f4-47ed-8F97-55b854af4220

    In the thread above, I was following the post on 30 August.  According to Option A, I completed step 5 for the uninstallation of McAfee Security.  At this time, my network card has stopped working.  On the recommendation of the officer, I posted to the Malwarebytes forum.  The officer determined Malwarebytes that I don't have a malicious software.  I ended up posting on forum McAfee to get my network card works again.  Then I reinstalled McAfee and start to run Windows Update.

    I started with Option A, step 8 to reset Internet Explorer using http://support.microsoft.com/kb/923737.  The difficulty it has worked very well.

    Then Option A, step 9 says reset Windows Update using http://support.microsoft.com/kb/971058.  When I ran the difficulty in default or aggressive mode, I got an error message saying: he cannot start wuauserv.  The message asks me to verify that the user has privileges to take this step.  Well, I checked through user accounts who am the only one user set up on this PC and I have administrator privileges.  Help me please run this difficulty or recommend another way to get Windows Update works again.

    [1]... looks like you are saying that my system is so wet that a clean install is the (only?) way to go.

    [2] if I do a clean install, not only what I need to reinstall Windows and all my applications, I also have to reinstall the drivers... Etc. ?

    [3] I think I have the disks supplied with the PC system. Is it better to use than to use the recovery partitiion?

    [4A]... is there a possibility that some of [my data are] infected?

    [4] preventing me from simply infect the whole PC again?

    A1. Yes, and it was my recommendation in August, too.

    A2. Yes.

    A3. Only not really well using the restore Partition hidden to make the "destructive recovery" will include all drivers, etc., that have been installed at the factory.

    A4a. Yes, a real possibility.

    A4b. You run at least three 3 tests on the backup data before restoring one (1) with your installed & updated up-to-date anti-virus application (for example, Microsoft Security Essentials) and two 2 with good reputation, free online scans (e.g., ESET;) Bit Defender; Kaspersky; HouseCall; (F - Secure).

    If you have saved your data on CD/DVD, the disc scanner. If you have backed up your data to a flash drive, SD card, or another external drive AND KB971029 is installed, analyze the drive.

    If you have any questions or need additional help, please start a new thread in this forum: http://answers.microsoft.com/en-us/windows/forum/windows_xp-system

    Once again, good luck!

  • Problem with Windows Update after the removal of Virus

    My operating system is Win Vista 32 bit, had some virus closed my windows updates, have a view of the white page. Checked that my services and win update is in automatic mode has started, so I don't know why my updater gives me a red x saying windows updater is turned off.  Any suggestions?

    [Original title: Windows Update]

    https://answers.Microsoft.com/en-us/protect/Forum/protect_other-protect_scanning/problem-with-Windows-Update-after-removing-virus/3a8dd279-2732-4693-88bf-f5cea4afca96>

    It makes more sense. How to reset the Windows Update components?

    Yet, as indicated in The ball , the search results links do not work. I also used several browsers and DuckDuckGo. Maybe it's because the Windows Update Agent was changed some time ago and the article is no longer valid.

    EDIT: If the OP is also struggling to access the link I suggest using repair Windows updates available from tweaking.com. I suggest also using the Registry backup of the same author before hand.

    Repair the Windows updates

    http://www.tweaking.com/content/page/repair_windows_updates.html

    Registry backup
    http://www.tweaking.com/content/page/registry_backup.html

    I hope this helps.

  • Problem area: windows update; Error: TRUST_E_NOSIGNATURE (0x800b0100) __Message: installation error prevented the service pack 2 windows systems (KB948465) _

    Problem area: windows update; Error: TRUST_E_NOSIGNATURE (0x800b0100)
    Message: Installation error prevented the service pack 2 windows systems (KB948465)

    http://support.Microsoft.com/kb/956702

    «Error message when you try to use Microsoft Update or Windows Update Web sites to install updates: «0x800B0100»»

    Method for Windows Vista or Windows Server 2008 to resolve this issue, run the tool (Checksur.exe) system update. To do this, follow these steps:

    1. Download and run the system update tool. This tool runs an ad hoc analysis of inconsistencies that might prevent future maintenance operations. For more information about how to download and run the Checksur.exe tool, click on the number below to view the article in the Microsoft Knowledge Base:

      947821 (http://support.microsoft.com/kb/947821/) Description of the update readiness tool for Windows Vista system
    2. Try to install updates again.

    http://support.Microsoft.com/kb/947821/

    "Description of the preparation tool system update for Windows Vista for Windows Server 2008 and for Windows 7"

    If the above solves the problem:

    It comes with Vista, upgrade install and activate Forum.

    You will get the best help for any problem of Update/Service Pack update Vista Forum; the link below:

    http://social.answers.Microsoft.com/forums/en-us/vistawu/threads

    When you repost here, kindly include this error Code, and exactly what is happening when you try to update.

    In this way, you will receive the best help.

    See you soon.

    Mick Murphy - Microsoft partner

  • How I download 80070103 windows update is update safe what is the problem with this update

    Just got the update installation today to download update 80070103 time ever I go to download the update fail what is the problem with this update & how do I get updated.

    Hello

    Thanks for posting in the Microsoft Community.

    If I understand correctly that you have problems with Windows Update.

    Please answer these questions:

    1. What is the update's KB number?

    2 have you made any changes to the computer before the show?

    Let us try the following methods:

     

    Method 1: Run the following fix - it:

     

    The problem with Microsoft Windows Update is not working:

    http://support.Microsoft.com/mats/windows_update/

    Method 2: See the following article:

    Windows Update 80070103 error code:

    http://Windows.Microsoft.com/en-us/Windows-Vista/Windows-Update-error-80070103

    Hope the helps of information. We know if you need help. We will be happy to help you.

  • problems trying to open pdf files. said the treatment works cannot open file

    original title: rayexceed

    Hi, I have problems trying to open pdf files. said the treatment works cannot open file. everything worked fine before and when it started I'm not too sure.my security is up to date when I try to open a pdf file, it says file may be damaged and then I press ok and it goes on a blank page in aid of processor.please of work

    Work does not have PDF files. You need Acrobat Reader or Foxit Reader.

  • Problem with Windows Update after the new installation of Windows 7 Pro the Lenovo ThinkPad X 200

    I'm trying to do a fresh install of Windows 7 Pro on a Lenovo ThinkPad X 200. The product key is easy to find on the sticker of the ThinkPad. However, the version of Windows is hidden (for example, I can't check that my ThinkPad had originally installed Windows 7 Pro). I was able to install by using a Pro Win 7 SP1 CD ROM I bought for another installation. I used the ThinkPad I was trying to make the new facility on the product key. I was able to activate the Windows 7 Pro using the phone call and my cell phone. I checked the activation! However, I have problems with Windows update. The product key may be a different version of Windows 7. If so, why did it install and activate OK? Y at - it a fix for this problem or I have to buy a new version of Windows 7 Pro? Please note that I have not yet installed all drivers Lenovo ThinkPad as I can't seem to install correctly. Thank you!

    Since the issue is updated to Windows, it is not a product number key.

    You can reset the Windows Update components by running the Fixit on this page. But if there is malware present, she will continue to reset the connection to the update servers:

    How to reset the Windows Update components

    http://support.Microsoft.com/kb/971058

    Suggest you download and save the Fixit. Then configure the system before the clean boot by running:

    How to troubleshoot a problem by performing a clean boot in Windows Vista/Windows 7

    http://support.Microsoft.com/kb/929135

    Once the Fixit has been downloaded and the system is started in the pure State, check that the native Windows Firewall is now on if a 3rd party firewall has been used previously. Now run the Fixit and choose the default mode. Restart once it's done and see if the system can be connected to the update servers. If he can't, then rerun the Fixit and choose aggressive mode. Turn it back on when he finished the race and updates.

  • Problem installation Vista updated and now the Vista operating system unbootable - help

    Here it is the description of what has past.  Ive been through everything I can do and am now stuck with a code of 0 X 4001100200001005.

    I worked with Microsoft Vista forum group of answers, but I'm not back upward and running if Im going back to HP for assistance.  I hope that we can achieve together.  This all started when started Vista updates failing.  I got the 1 3 install stuck on the 2nd about three weeks ago.

    Here is a list of what I've done so far
    I have an old HP DV9410us for a year which will not restart after installing updates.  It comes to configuring updates step 1 from 3 to about 98% and then stops.

    It then says unable to start windows and start the repair program windows for about 5-10 minutes.  He said that you want to use the system restore.  I tell Yah Why Not.  Then, it will stop and start the cycle.

    The error in the Startup Repair States the following

    After a bunch of tests successful

    Root cause found:

    c:\ntoskrnl.exe is corrupted.

    Repari Action: Repair of files
    Result: failed.  Error code = 0xa
    Duration 6521ms

    Action repair system restore
    Result: Completed successfully

    He then returns to the cycle all over again.

    I turned the clock to October with F10 at startup but that did nothing.

    I checked the hard drive for the short and complete and checked everything.

    I dug my recovery disks I did ago about 10 months after I bought and used to return to the top of my hardrive.  I was all ready for a MAC at this point.

    I went back to the HP Recovery program and tried to restore an earlier version of a well-known operating system.  The Manager of recovery disk wont work so I used the Recovery Manager disk.  There is no old configs to this true its in step 2.

    I'm now trying to perform a factory image recovery.
    Reload the operating system.  It started and got to 13% after 20 minutes and gave me the following error code.

    During this process, I have an error that had the code 0 x 4001100200001005.

    It is the reformatting of repeat of the Windows Partition.

    Is this a Vista update problem?  Have you seen this before?

    I'm doing a factory restore and will let you know if this operation fails.  I just wanted to get this thread started.

    I really appreciate this forum and any help you could give me.

    Message edited by ftrnow on 12/30/2008 08:17

    If you had an error in "configuration update 1 3-3 3 ' which would be a problem with windows update.

    to fix this, you'll need a factory image recovery if you can not access other modes for windows. But youy mentioned that you had a 0xx000x... (similar to code) error... try checking the hard drive and memory modules health... threr could be a problem with these devices, you can access it through the bios settings (turning on laptop, the moment you see hp invent screen, press f10) go to the diagnostics section, run the memory test and that hard drive test

    very probably with the error you have, you might have a hard drive problem.

    most of my clients (I am a specialist of the laptop of beak) with that problem and stuck on a boot loop or may not switch from windows, 95% of them had a hard drive failure.

  • Load with the incremental update of the IKM Oracle

    Hi Experts,

    According to my understanding, incremental load is that the new data (with insert append or load incremental (update/insert or merged with or without behavior SCD)).

    While peek into the code of the IKM KM here it is my understanding:

    Incremental update: given the PK defined in the target data store, that KM will check line by line and put insert/update (change data capture).

    Append: Blindly block for the table insertion target and any changed data will be captured. It will not truncate and insert so that to avoid duplicates in the data, have a PK defined in the target table, so whenever the duplicate data comes it can be prevented or go for CKMs.


    Now my doubt is,


    When you use the incremental km update: the scenario is I have an incremental load today, inserted for example 200000 files today and tomorrow other 200000 records added (may include updates of certain lines in the previous loaded 2,00,000 documents. Now it will scan 4,00,000 (yesterday + today) and seek changes, I mean to update or insert

    Because according to my understanding will treat this KM row by row all records (my understanding is correct?). If it reads each record and are looking for a change or not change it seems to me that his time and performance issues?  CDC is right approach in this scenario or the implementation of SDC on all columns in the table?


    Regarding the large number of records coming daily, updated incremental if IKM checking all records for update or insert or no change, in my opinion, this isn't a performance wise and time to compare source and target value. This KM eliminate comparing itself the Chronogram of the source to target those who does not charge any change in any of the previous column value?



    Sorry if this is a silly question. Just trying to figure which can be better charge strategy, especially when I have millions of records entering source daily.


    SSX I remember ealier JeromeFr our expert member in the community, said Partioned Exchange to treat only to process the given month data when you manage tables partitioned at the database level.


    Best regards

    ASP.








    Hi ASP_007,

    Charge, as opposed to full reload, does indeed only new (and possibly changed) data. There are 3 main ways to do this:

    • Set up a filter in your mapping interface/load only the data including the upper date to a variable (which holds the last loading date).
    • Use the framework of the CDC in ODI. There are several JKMs. The solution optimal is probably the Golden Gate, one, but you must purchase this additional license. mRainey wrote about this several times: http://www.rittmanmead.com/2014/05/goldengate-odi-perfect-match-12c-1/
    • Retrieve all the data in the source and allow an incremental update of the IKM define what already exists.

    Of course, the first two still will take a little more time to develop, but it will be much faster in terms of performance because you treat data less.

    It is for the part "Extract", get data from the source.

    Now, you must know how to "integrate" into your target. There are different strategies as insert Append, incremental update, Type 2 SCD...

    • Indeed insert Append won't update. It will only insert lines. It is a good approach for a full charge, or for an additional charge when you want to insert data. There is an option in most of the IKMs Append (control) to truncate the table before inserting (or delete all the lines if you do not have the privileges to truncate).
    • Incremental update: there are different IKMs for this and some may have better performance than others depending on your environment. I recommend you to try a few and see which is more fast for you. For example ' IKM Oracle incremental update (MERGE) "could be faster than 'IKM Oracle incremental update. I personally often use a slightly modified version of ' IKM Oracle incremental update (MERGE) for Exadata ' to avoid using a work table (I$ _) and perform the merge directly into the target table. The last approach works well with the CDC when you know that all data are new or changed and needs to be treated.
    • SCD2: To maintain your dimensions needing SCD2 behavior.

    So in answer to your questions:

    Because according to my understanding will treat this KM row by row all records (my understanding is correct?). If it reads each record and are looking for a change or not change it seems to me that his time and performance issues?

    Some of the IKMs will do it line by line, others will do it based on a game. This is why it is important to check what he does and he spots.

    CDC is right approach in this scenario or the implementation of SDC on all columns in the table?

    Yes certainly, you will have less data to be processed.

    Regarding the large number of records coming daily, updated incremental if IKM checking all records for update or insert or no change, in my opinion, this isn't a performance wise and time to compare source and target value. This KM eliminate comparing itself the Chronogram of the source to target those who does not charge any change in any of the previous column value?

    Yes, by using ' IKM Oracle incremental update (MERGE) for Exadata ' with the strategy of 'NONE '. This means that he will not try to see the rows from the source is already in the target.

    PS; I am remember ealier JeromeFr our expert member in the community, said Partioned Exchange to treat only to process the given month data when you manage tables partitioned at the database level.

    It is a good approach when you want to reload an entire partition (if you have a monthly charge in a monthly partition or a daily load in a daily score for example). It is easier to set up to load the new lines only. But if you need to update things in the source, you can use incremental update strategy on an intermediate table that contains the same data that your partition and then create the swap partition in a further step.

    Thanks for the mention.

    Be sure to close your other discussions.

    It will be useful.

    Kind regards

    JeromeFr

  • Hello I have a problem since my installation of the update of Yosemite on my Mac, I can't open Indesign CS6

    Hello

    Hello I have a problem since my installation of the update of Yosemite on my Mac, I can't open Indesign CS6.

    I have a problem since my installation of the update of Yosemite on my Mac, I can open more Indesign and yet I installed JavaForOSX2014-001, but it changed nothing someone could help me?

    Please do not post the same question multiple times. If someone can help, they will do it, but be patient - we are all volunteers righteous, users like you. I'll delete your other messages.

  • difference between the incremental update of the IKM oracle and incremental update IKM oracle (PL - SQL)

    Hello

    What is the difference between the incremental update of the IKM oracle and updated incremental IKM oracle (PL - SQL) and incremental update IKM oracle (line by line).

    Thank you

    Papai

    The only difference is that the second using plsql for incremental update. He also to manage clob issues well enough.

    If you need to know more you can read the description of each KM section.

    Incremental update IKM Oracle (PL-SQL)

    -------

    Description:

    -Knowledge integration module

    -Integrates data into an Oracle table from target in incremental update mode using PL/SQL.

    -Non-existent rows are inserted. already existing lines are updated.

    -Data can be controlled. Data invalid are isolated in the error Table and can be recycled.

    -This KM uses PL/SQL to perform the inserts and updates until and blob columns are supported. Please see the restrictions.

    -When you use this module with a source table logged, it is possible to synchronize the deletions.

    Restrictions:

    -When working with the logged data, if the "synchronize destruction of the newspaper" are executed, the lines deleted on the target are engaged

    -The data are updated even if not changed (upgrade from any (e)

    -The number of lines (number of inserts/changes) is not available because the transactions are performed using PL/SQL

    -Comparison of the data is performed using the key to update defined in the interface. It must be set.

    -L'option TRUNCATE does not work if the target table is referenced by another table (foreign key)

    -Options FLOW_CONTROL, and STATIC_CONTROL call the Module knowledge check to isolate invalid data (if no CKM is defined, an error occurs). These two options should be set to NO in the case where an integration Interface meets a TEMPORARY target data store.

    -L' FLOW_TABLE_OPTION option is set by default to NOLOGGING. Set it to a space if the interface is running on an Oracle 7 database

    -Deletions are committed regardless of the VALIDATION option

    Incremental update of the IKM Oracle

    -------------

    DESCRIPTION:

    -Integrates data into an Oracle table from target in incremental update mode.

    -Non-existent rows are inserted. already existing lines are updated.

    -Data can be controlled. Data invalid are isolated in the error Table and can be recycled.

    -When you use this module with a source table logged, it is possible to synchronize the deletions.

    REQUIREMENTS:

    -The update key defined in the interface is required.

    RESTRICTIONS:

    -When working with the logged data, if the "synchronize destruction of the newspaper" are executed, the lines deleted on the target are engaged

    -L'option TRUNCATE does not work if the target table is referenced by another table (foreign key)

    -Options FLOW_CONTROL, and STATIC_CONTROL call the Module knowledge check to isolate invalid data (if no CKM is defined, an error occurs).

    These two options should be set to NO in the case where an integration Interface meets a TEMPORARY target data store.

    -L' FLOW_TABLE_OPTION option is set by default to NOLOGGING. Set it to a space if the interface is running on an Oracle 7 database

    -Deletions are committed regardless of the VALIDATION option

    -L' ANALYZE_TARGET option will allow to assess correct statistics only if the VALIDATION is set to Yes. Otherwise, the IKM gather statistics based on old data.

    -Default UPDATE option is TRUE, which means by default it is assumed that there is at least one column nonkey specified in a target data store.

  • I am new to Muse, so I am following the tutorial for coffee of Katie.  I downloaded the sample files.  The problem is not that all the files in the folder as in the tutorial.  The color swatch for example is not there.  The updated CC on the road Katies.m

    I am new to Muse, so I am following the tutorial for coffee of Katie.  I downloaded the sample files.  The problem is not that all the files in the folder as in the tutorial.  The color swatch for example is not there.  The CC file to get started Katies.mulib is also not if you can't automatically download all the necessary files.  Can anyone help?

    found in another area

    Where can I find updated cc enroute Katies.mulib file?

Maybe you are looking for