Optimize the BSO - SQL Loader data loading

Hello

From left to right on the fields, I put the sparse dims sorted and in some order first then the Dense dims, then data?  Or is this Dense first and then Sparse and then data?   Seems there are conflicting ideas here...

optimal order is most certainly rare members fors, followed by dense members.  To confirm the best order, take you cube and if there is no data in it, then charge a small amount using a lock of excel and send it, if there is, fine, then make a column for EAS export (level 0 is fine) and look at the exported file. The order of the dimensions is the optimal order of reloading.

Tags: Business Intelligence

Similar Questions

  • Import data... Wizard creates the file SQL Loader ctl with columns out of order

    4.1.1.19 SQL Developer version. Connected to Oracle XE to test this.

    I'm trying to understand what the problem with the data in my import file when finally, I realize that the Import Wizard of data did not care how I traced the columns at all. The ctl SQL Loader file generated by the Wizard expects that the columns of data in my file to match the order that they appear in the definition of the table and not how they have been mapped in the wizard. Manually edit the ctl file is a workaround. Has anyone else seen elsewhere?

    I see that this is a bug.

  • Trouble with the adapter SQL Loader

    Hi all

    I'm trying to sue the adapter SQL Loader to collect my SQL server instance performance statistics.  I followed the instructions in Appendix A of this document: https://www.vmware.com/files/pdf/solutions/Monitoring-Business-Critical-Applications-VMware-vCenter-Operations-Manager-white-paper.pdf

    Our environment is one vCOps 5.8 vCloud Suite Enterprise license licensed.  SQL server SQL 2008 R2 and is reasonably up-to-date on patches, etc.  (CUA8, I think)

    I've gotten to the point where the database is logged in and my query is running, but it seems that the values returned by the query correspond to default values (0).  I don't know why this is happening, but because the newspapers seem to show the actual values from the database:

    2014-02-10 17:29:50, 438 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - #SELECT "SQL LOADER" ADAPTERKIND, query

    "SQL Server PERFMON STATS" RESOURCEKIND,.

    GETUTCDATE() TIMESTAMP,

    'Agent' RESOURCEKIND,

    "SVC-mgmt-sql SQL Server" RESOURCENAME,.

    (LTRIM (RTRIM (replace ([parameter], ':', '-'))) + "|" + counter_name) METRICNAME1,

    Sum (cntr_value) VALUE1

    FROM sys.dm_os_performance_counters

    WHERE

    (GETUTCDATE() > = (CONVERT (datetime, ' 02/10/2014 17:9:0 ', 101))) AND

    (' 01 / 01/2000 ' < (CONVERT (datetime, ' 02/10/2014-17:29:50 ', 101))) AND

    counter_name in)

    "User connections"

    "Compilations SQL/s."

    "SQL Recompilations/sec."

    "Server memory target (KB)"

    "Total server memory (KB)"

    "Scriptures delayed/s."

    "Checkpoint pages/sec.

    "Page life expectancy."

    'Memory waiting requests',

    "IO Page lock expects."

    "Wait worker."

    "Writing waiting to connect."

    "IO expects network")

    GROUP BY (LTRIM (RTRIM (replace ([parameter], ':', '-'))) + "|" + counter_name)

    ORDER OF TIMESTAMP

    2014-02-10 17:29:50, 467 com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB DEBUG [Collector worker thread 8] (20583) - try to connect...

    2014-02-10 17:29:50, 478 com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB DEBUG [Collector worker thread 8] (20583) - connected...

    2014-02-10 17:29:50, 490 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - QueryExecutionTime = 12

    2014-02-10 17:29:50, 509 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - strMetricCount = 0

    2014-02-10 17:29:50, 533 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 538 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 571 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 12921

    2014-02-10 17:29:50, 612 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 624 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 624 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 10

    2014-02-10 17:29:50, 626 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 626 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 626 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 72869

    2014-02-10 17:29:50, 626 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 626 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 626 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 17

    2014-02-10 17:29:50, 627 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 627 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 627 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 72869

    2014-02-10 17:29:50, 627 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 628 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 628 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 10644

    2014-02-10 17:29:50, 628 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 628 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 628 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 77

    2014-02-10 17:29:50, 629 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 629 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 629 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 0

    2014-02-10 17:29:50, 629 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 629 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 630 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 425692

    2014-02-10 17:29:50, 630 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 630 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 630 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 11747062

    2014-02-10 17:29:50, 630 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 631 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 631 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 29360128

    2014-02-10 17:29:50, 631 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 631 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 631 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 29360128

    2014-02-10 17:29:50, 632 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getResourceKeyFromResultSet (20583) - work on resource: SVC - mgmt - sql SQL Server

    2014-02-10 17:29:50, 632 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.isResourceRenameAllowed - rename audit allowed for resource key {resourceName = SVC - mgmt - sql SQL Server & adapterKindKey = SQL LOADER & resourceKindKey = SQL Server PERFORMANCE Monitor STATS}

    2014-02-10 17:29:50, 632 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - resource of treatment: SVC-mgmt-sql SQL Server extract the metric values are 0

    2014-02-10 17:29:50, 632 DEBUG [Collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB (20583) - number of records processed 13 - used memory (MB): free Mem in 1757 (Mo): 1104

    2014-02-10 17:29:50, 632 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.getDataFromDB - for request #= "SQL LOADER' SELECT ADAPTERKIND,.

    "SQL Server PERFMON STATS" RESOURCEKIND,.

    GETUTCDATE() TIMESTAMP,

    'Agent' RESOURCEKIND,

    "SVC-mgmt-sql SQL Server" RESOURCENAME,.

    (LTRIM (RTRIM (replace ([parameter], ':', '-'))) + "|" + counter_name) METRICNAME1,

    Sum (cntr_value) VALUE1

    FROM sys.dm_os_performance_counters

    WHERE

    (GETUTCDATE() > = %f)) AND

    (' 01 / 01/2000 ' < %t)) AND

    counter_name in)

    "User connections"

    "Compilations SQL/s."

    "SQL Recompilations/sec."

    "Server memory target (KB)"

    "Total server memory (KB)"

    "Scriptures delayed/s."

    "Checkpoint pages/sec.

    "Page life expectancy."

    'Memory waiting requests',

    "IO Page lock expects."

    "Wait worker."

    "Writing waiting to connect."

    "IO expects network")

    GROUP BY (LTRIM (RTRIM (replace ([parameter], ':', '-'))) + "|" + counter_name)

    ORDER OF TIMESTAMP

    RecordCount = 13 FilteredRecordCount = 0

    2014-02-10 17:29:50, 632 DEBUG [Collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.addDefaultData - Default Metrics size after a data loop = 13

    2014-02-10 17:29:50, 633 INFO [collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.addDefaultData - number of default values for this piece of data = 13

    2014-02-10 17:29:50, 633 INFO [collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.DataReader.read - 26 returned query parameters

    2014-02-10 17:29:50, 634 INFO [collector worker thread 8] com.integrien.adapter3.generalsqldataloader.DataReader.updateResourceTimeMapAndRemoveDuplicates (20583) - 26 data 0 resource comments who have been sent the last time.

    2014-02-10 17:29:50, 634 INFO [collector worker thread 8] (20583) com.integrien.adapter3.generalsqldataloader.GeneralSQLDataLoaderAdapter.groupByResource - grouping of resources

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 634 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 635 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2014-02-10 17:29:50, 636 DEBUG [Collector worker thread 8] (20583) com.integrien.alive.common.adapter3.AdapterBase.addMetricData - 1 added metrics to collect the result for the "SVC - mgmt - sql SQL Server" resource, resId = 20584, adapter "GeneralSQLDataLoaderAdapter".

    2

    Has anyone out there dealt with this or found a way around this kind of problem?  There only seems to be a lot of information regarding the use of this (or of the) adapters.

    Thank you

    Jason

    IS to use the option you gave.

    Looks like that time is not the same timezone between solutions if he just started working after 5 to 6 hours. Make sure you that you don't need to use a different zone such as GMT.

  • What is the cause of the error SQL * Loader-925: the ulprep error: extraction?

    Hello

    I use SQL Loader to load a file into a partitioned table. However, the performance charger throws the following error message:

    SQL * Loader-925: the ulprep error: extraction
    ORA-01031: insufficient privileges

    Please can someone explain what the possible cause of this error?

    Thanks in advance
    Annie

    try to view the logfile too. the log file can be useful.

    ORA-01031: insufficient privileges-> indicates the user account does not have access to the table.

  • Optimize the conversion of SGL data chain

    As the title indicates, I'm looking to pronounce on a more efficient way (as appropriate) to convert a string into an array of SGL.

    The detachment is just to get feedback on the best ways of doing things from a personal educational point of view. I don't actually have to improve performance... (Just keep improving my skills of codeing.)

    'More effective' here must take to be less cycles of CPU and low memory footprint, in that order. (That is, most small load on CPU is preferred, if smaller memory footprint is at the expense of the other cycles CPU, the hit on memory footprint is preferred).

    The length of the input string comes from a TCP read and contains only the raw bytes of the expected (in this case, SGL) format without any header/footer of meta-data.

    Currently, my implementation is very simple and given that the expected table size is known, I am relying on the loop being smart in his automatic indexation allowance.

    For the conversion, I do a right type-Gallery of 4 characters at once.

    The code below, the for loop is simply intended to give a 'real' output boolean if the string length is the length of expected as determined by the input string 'number of track points. No additional error handling is required or necessary.

    I suspect the "subset of string" combined with the iterations (for loop) are the two worst performance in this VI hits, but I can't think of a good way to do the conversion as a table, and if I could, then I should at least do a conversion in 'string to array U8' first.

    Thanks to all who took the time to read and watch. Your time and your contribution is appreciated as always!

    Q

    Hi Q,

    Why don't convert you directly in a table of SGL? No need of a loop FOR, nor for StringSubset...

  • SQL Loader with the line terminator

    Dear friends,

    I use Oracle 10 g R2.

    Need to load some records to a text file to Oracle.

    It's my control file:
    LOAD DATA 
INFILE 'D:\load\dat\FinNote.dat' 
BADFILE 'D:\load\bad\FinNote.bad' 
DISCARDFILE 'D:\load\dat\discard\FinNote.dsc' 
INSERT INTO TABLE FinNote FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"' TRAILING 
NULLCOLS
( IDNo, RegDate date 'YYYY-MM-DD HH24:MI:SS', Col1, Remark, UserID, Status)
{code}

My text file has semicolon as a row delimiter and | as column delimiter. Now, how can I add the semicolon as row delimiter. Please guide me.

I followed the method given in this site: 
http://www.exforsys.com/tutorials/oracle-10g/oracle-10g-sql-loader-input-data-and-datafiles.html


*INFILE datafile_name ["str terminator_string"]*

{code}
LOAD DATA 
INFILE 'D:\load\dat\CRS_FinNote.dat' ";"
BADFILE 'D:\load\bad\CRS_FinNote.bad' 
DISCARDFILE 'D:\load\dat\discard\CRS_FinNote.dsc' 
INSERT INTO TABLE CRS_FinNote FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"' TRAILING 

NULLCOLS
( StudentNo, IDate date 'YYYY-MM-DD HH24:MI:SS', Topic, Remark, UserID, Status)
    and received the error:

    SQL * Loader-500: could not open the file (D:\load\dat\CRS_FinNote.dat)
    SQL * Loader-555: unrecognized property treatment option
    SQL * Loader-509: System error: the operation completed successfully.
    SQL * Loader-2026: the charge was dropped because SQL Loader cannot continue.
    SQL * Loader-513: cannot close the file (D:\load\dat\CRS_FinNote.dat)
    SQL * Loader-559: Error closing file
    SQL * Loader-509: System error: the storage control block address is invalid.

    Thanks in advance.
    Nith

    Published by: user645399 on June 16, 2010 10:35

    And, if your CRS_FinNote.dat look like below (line/record delimited; then)

    1245|2000-05-16|Oracle|The Best|AAA|Online;
294|2000-04-03|Dotnet|Best|BBB|Pending;
300|2010-03-28|C++|The Best|CCC|Offline;

    Scott@ORCL > select utl_raw.cast_to_raw (';') under "hexadecimal;" double;

    in hexadecimal
    --------------------------------------------------------------------------------
    3B

    Then I change user645399.ctl with below:

    LOAD DATA
INFILE 'd:\csv\CRS_FinNote.dat' "str X'3B'"
BADFILE 'd:\csv\FinNote.bad'
DISCARDFILE 'd:\csv\FinNote.dsc'
INSERT INTO TABLE FinNote FIELDS TERMINATED BY '|' OPTIONALLY ENCLOSED BY '"' TRAILING
NULLCOLS
( IDNo, RegDate date 'YYYY-MM-DD HH24:MI:SS', Col1, Remark, UserID, Status)

    The idea received a link from http://www.orafaq.com/forum/t/140062/0/.

    C:\Documents and Settings\Girish Sharma>sqlldr scott/tiger@orcl control=d:\csv\user645399.ctl log=user645399.log

Enter user-name: scott/tiger

Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options

SCOTT@orcl> select * from finnote;

      IDNO REGDATE              COL1       REMARK     USERID     STATUS
---------- -------------------- ---------- ---------- ---------- ----------
      1245 16-may-2000 00:00:00 Oracle     The Best   AAA        Online
       294 03-apr-2000 00:00:00 Dotnet     Best       BBB        Pending
       300 28-mar-2010 00:00:00 C++        The Best   CCC        Offline

SCOTT@orcl>

    HTH
    Girish Sharma

  • Deploy the mapping of sql loader

    Hello
    I have a problem with OWB when I try to start a mapping (sql loader) after deployment. The problem is that when I start the execution of the mapping, sql loader control file is created not under Unix (where I set the location of the file entry and control file), but in the pc with my client OWB. For this reason execution fails because it can't find the input file (that I put in the unix machine).
    Is there a configuration for the client or the server?

    Thank you

    start from unix machine

  • SQL Loader with decimal comma

    Dear all,

    I am loading data for a German client and the files shipped contain numeric data with decimal comma. For example 123.5 instead of 123.5

    I've been Googling like crazy on this one and there is a lot of chatter on the definition of NLS_LANG and/or NLS_NUMERIC_CHARACTERS, but none of this seems to work in the environment SQL Loader.

    I guess I'll end up tinkering together something complicated with TO_NUMBER or another function, but I really can't believe SQL Loader isn't a simple switch to return semicolon.

    Or I forgot something?

    Advice would be a great help.

    Kind regards
    Alan Searle

    You can use something like:

    Load data
    INFILE...
    BadFile...
    ....
    fields completed by...
    (
    ...
    col1 "translate (: col1, ','.')", ".
    ...
    )

    Published by: SKU on January 28, 2009 04:41

  • Need help with SQL Loader

    I have a third-party site that transforms the information into a csv summary. I can't change the format, and they are not ready to give directly from db and db, so I try and then import it. I'm trying to use SQL Loader, but I can't seem to make it work. I looked in the documentation (VERY rare examples) as well as several websites and I can't find anything that can help out me.

    Control file:
    LOAD DATA
INFILE 'Daily_Diagnostics_test.csv'
BADFILE 'Diag_failed.csv'
APPEND
INTO TABLE ROAD_DIAGS
FIELDS TERMINATED BY ","
--WHEN VEHICLE != 'Vehicle' AND VEHICLE != 'Total'
(VEHICLE, DRIVER, START_DATE, START_TIME, START_TZ, END_DATE, END_TIME, END_TZ, 
 TM_DRV_MINS, TM_IDLE_MINS, TM_PTO_MINS, TRIP_TM_MINS,
 FUEL_DRV_GALS, FUEL_IDLE_GALS, FUEL_PTO_GALS, FUEL_TRIP_GALS, FUEL_START_GALS,
 MPG, DRV_MPG, START_ODOM, END_ODOM, DIFF_ODOM, AVG_MPH, MAX_MPH, MAX_RPM,
 DATE1, TM1, TZ1, ADDRESS_NUM1, ADDRESS_STREET1, ADDRESS_ST1, ADDRESS_ZIP1, ADDRESS_COUNTY1,
 DATE2, TM2, TZ2, ADDRESS_NUM2, ADDRESS_STREET2, ADDRESS_ST2, ADDRESS_ZIP2, ADDRESS_COUNTY2,
 STOP_CT, FAULT_CODE, HARD_BRAKE)
    Note that the line WHEN is commented out, because I can't make it work.

    and here is a sample of the data:
    Vehicle,Driver,Start Date,Start Time,Timezone,End Date,End Time,Timezone,Driving (hh:mm),Idle (hh:mm),PTO (hh:mm),Trip (hh:mm),Driving (gals),Idle (gals),PTO (gals),Trip (gals),Before Trip (gals), MPG,Driving MPG,Start,End,Trip Distance (miles),Average(MPH),Maximum(MPH),Maximum RPM(rpm),Date,Time,Timezone,Address,Street,City,State,Zip,County,Date,Time,Timezone,Address,Street,City,State,Zip,County,Stop Count,Fault Code,Hard Brake,
26033, ,01/23/08,12:40 AM,MST,01/23/08,12:42 AM,MST,0,3,0,3,0,0.12,0,0.12, ,0,0,295829.22,295829.22,0,0,6.84, ,01/23/08,12:23 AM,MST,0,COUNTY ROAD 108,SAVAGE,MT,59262,RICHLAND,01/23/08,12:23 AM,MST,0,COUNTY ROAD 108,SAVAGE,MT,59262,RICHLAND,0,4, ,
26033, ,01/23/08,12:43 AM,MST,01/23/08,04:03 AM,MST,126,74,0,200,0,0,0,0, ,0,0,295829.22,295914.97,85.75,25.72,67.11, ,01/23/08,12:23 AM,MST,0,COUNTY ROAD 108,SAVAGE,MT,59262,RICHLAND,01/23/08,03:56 AM,MST,0,COUNTY ROAD 125,SIDNEY,MT,59270,RICHLAND,8,3, ,
26033, ,01/23/08,04:03 AM,MST,01/23/08,12:29 PM,MST,317,190,0,507,0,0,0,0, ,0,0,295914.97,296139.9,224.94,26.64,65.87, ,01/23/08,03:56 AM,MST,0,COUNTY ROAD 125,SIDNEY,MT,59270,RICHLAND,01/23/08,12:29 PM,MST,756,BELL RD,SIDNEY,MT,59270,RICHLAND,14,5, ,
Total, , , ,443,267,0,710,0,0.12,0,0.12, ,0,0,295829.22,296139.91,310.69,26.38,67.11, , , , , , , , , , , , , , , , , , , ,22,12,0,
Vehicle,Driver,Start Date,Start Time,Timezone,End Date,End Time,Timezone,Driving (hh:mm),Idle (hh:mm),PTO (hh:mm),Trip (hh:mm),Driving (gals),Idle (gals),PTO (gals),Trip (gals),Before Trip (gals), MPG,Driving MPG,Start,End,Trip Distance (miles),Average(MPH),Maximum(MPH),Maximum RPM(rpm),Date,Time,Timezone,Address,Street,City,State,Zip,County,Date,Time,Timezone,Address,Street,City,State,Zip,County,Stop Count,Fault Code,Hard Brake,
26035, ,01/23/08,03:59 AM,MST,01/23/08,09:05 AM,MST,222,85,0,306,50.87,1.5,0,52.37, ,3.32,3.42,285056.53,285230.5,173.98,34.07,68.97, ,01/23/08,03:24 AM,MST,756,BELL RD,SIDNEY,MT,59270,RICHLAND,01/23/08,09:00 AM,MST,0,COUNTY ROAD 338,SAVAGE,MT,59262,RICHLAND,7,3, ,
26035, ,01/23/08,09:06 AM,MST,01/23/08,03:37 PM,MST,273,118,0,391,68.5,1.5,0,70, ,3.24,3.31,285230.5,285457.3,226.8,34.8,69.59, ,01/23/08,09:00 AM,MST,0,COUNTY ROAD 338,SAVAGE,MT,59262,RICHLAND,01/23/08,03:37 PM,MST,756,BELL RD,SIDNEY,MT,59270,RICHLAND,11,3, ,
26035, ,01/23/08,03:45 PM,MST,01/23/08,04:38 PM,MST,32,21,0,53,7.37,0.25,0,7.62, ,3.42,3.54,285457.3,285483.4,26.1,29.73,67.73, ,01/23/08,03:37 PM,MST,756,BELL RD,SIDNEY,MT,59270,RICHLAND,01/23/08,04:33 PM,MST,0,COUNTY ROAD 108,SAVAGE,MT,59262,RICHLAND,2,3, ,
26035, ,01/23/08,04:49 PM,MST,01/23/08,10:22 PM,MST,256,76,0,332,52,1.25,0,53.25, ,3.71,3.8,285483.4,285681.0,197.6,35.67,64, ,01/23/08,04:33 PM,MST,0,COUNTY ROAD 108,SAVAGE,MT,59262,RICHLAND,01/23/08,10:17 PM,MST,0,COUNTY ROAD 108,SAVAGE,MT,59262,RICHLAND,9,3, ,
26035, ,01/23/08,10:25 PM,MST,01/23/08,10:35 PM,MST,4,5,0,10,0.37,0,0,0.37, ,1.68,1.68,285681.0,285681.62,0.62,3.9,24.85, ,01/23/08,10:17 PM,MST,0,COUNTY ROAD 108,SAVAGE,MT,59262,RICHLAND,01/23/08,10:35 PM,MST,0,COUNTY ROAD 338,SAVAGE,MT,59262,RICHLAND,1,5, ,
Total, , , ,787,305,0,1092,179.11,4.5,0,183.61, ,3.4,3.49,285056.53,285681.62,625.1,34.35,69.59, , , , , , , , , , , , , , , , , , , ,30,17,0,
    As you can see, I need to have the SQL Loader to ignore the header and summary lines that appear for each grouping. Your suggestions are welcome as I need to implement this in an automatic process, I can call with a batch script.

    blarman74 wrote:
    Can anyone suggest how to fix this problem? The log file shows this error:

    Record 8: Rejected - Error on table ROAD_DIAGS, column DATE2.
ORA-01858: a non-numeric character was found where a numeric was expected

    The error message is clear - you try to load a value "non-date" in a DATE column and, therefore, sql * loader berms.
    That points to a discrepancy between the list of columns in the control file and, in the data file.
    Note the position of the table of the "DATE2" column in the table "ROAD_DIAGS":

    test@XE>
test@XE> l
  1  select table_name,
  2         column_name,
  3         column_id
  4    from user_tab_columns
  5   where table_name='ROAD_DIAGS'
  6*    and column_name='DATE2'
test@XE> /

TABLE_NAME      COLUMN_NAME      COLUMN_ID
--------------- --------------- ----------
ROAD_DIAGS      DATE2                   34

test@XE>

    If your control file specifies that the 34th column in the data file must be "DATE2". However-

    test@XE>
test@XE> -- display the line number and the 34th column of the data file
test@XE>
test@XE> !perl -ne '{@a=split/,/; print $.,"\t",$a[33],"\n"}' Daily_Diagnostics_test.csv
1       County
2       RICHLAND
3       RICHLAND
4       RICHLAND
5
6       County
7       RICHLAND
8       RICHLAND
9       RICHLAND
10      RICHLAND
11      RICHLAND
12

test@XE>

    the 34th column in your data file is 'COUNTY', which is counted in your control file.

    Here is the list of the columns for 29 to 37 positions in the csv file:

    test@XE>
test@XE> !perl -ne '{if ($. == 1) {@a=split/,/; foreach $i (28..36) {print $i+1,"\t",$a[$i],"\n"}}}' Daily_Diagnostics_test.csv
29      Address
30      Street
31      City
32      State
33      Zip
34      County
35      Date
36      Time
37      Timezone

test@XE>
test@XE>

    And here are the columns specified in the control file:

      ...
  ...
  ADDRESS_NUM1,
  ADDRESS_STREET1,
  ADDRESS_ST1,
  ADDRESS_ZIP1,
  ADDRESS_COUNTY1,
  DATE2            DATE "MM/DD/YYYY",
  TM2              DATE "HH:MI AM",
  TZ2,
  ...
  ...

    (You can open the csv file in MS Excel to get a better picture of the data in there).

    As seen above, due to the incompatibility, sql * loader loads 'City' in ADDRESS_ST1, 'State' in ADDRESS_ZIP1, "Zip" in ADDRESS_COUNTY1 and DATE2 "County" - where it error.

    The solution would be-

    (a) If you want to load the 'City' information from the data file, and then add the "City" column in the table (if nonexistent) and specify in the control file.

    (b) If you do not want to load the 'City' information from the data file, you can specify a column to 'FILL' in the file of control as follows:

      ...
  ...
  ADDRESS_NUM1,
  ADDRESS_STREET1,
  city1            FILLER,
  ADDRESS_ST1,
  ADDRESS_ZIP1,
  ADDRESS_COUNTY1,
  DATE2            DATE "MM/DD/YYYY",
  TM2              DATE "HH:MI AM",
  TZ2,
  ADDRESS_NUM2,
  ADDRESS_STREET2,
  city2            FILLER,
  ADDRESS_ST2,
  ADDRESS_ZIP2,
  ADDRESS_COUNTY2,
  ...
  ...

    Hope that helps,
    isotope

  • something about sql loader

    Jin

    I want to load a txt file into a database table
    the contents of this file as txt

    a 15 123 k tt
    a 15 223 k tt
    a k 323 105 tt
    a 15 423 k tt
    ...

    I want to delimit each record in four fields by blanks
    but I don't want all these four fields loaded into the table,
    Instead, I want to just choose only two useful fields (for example, the second and third field) in a table with only two columns

    How do I make.

    also, I tried the toad sql * loader wizard, I map columns in the corresponding field in the field, mapping, but this does not work, can someone tell me why? can it work?

    Thank you.

    online waiting for your answer

    sqlldr

    use the FILL in the control file

  • Failed to connect to SQL Developer Data Modeling

    All,
    I use Oracle SQL Developer Data Modeler version 3.0.0.665 in an Oracle 10 g database environment. When I try to import CFDS, I get the following error message:
    NoSuchMethodError:
oracle.i18.ntext.converter.CharacterConverterOGS.getInstance(I)Loracle/i18n/text/converter/CharacterConverter;
    I use the same connection to Oracle SQL Developer version 1.5.0.53 profile and my connection works fine here, but the same credentials in the Modeler SQL Developer data fails with the above error message. All thoughts, what causes this connection problem?

    Thank you


    Seyed

    Seyed salvation,

    Here you can watch BOF solution http://forums.oracle.com/forums/thread.jspa?messageID=9497361�

    Philippe

  • Oracle SQL Developer with SQL Developer Data Modeler

    Friends,

    I'm trying to understand the differences between Oracle SQL Developer Data Modeler and the possibility to use the SQL Developer Data Modeler.

    I have looked at two home pages produced on OTN, http://www.oracle.com/technology/products/database/datamodeler/index.html & http://www.oracle.com/technology/products/database/sql_developer/index.html and would appreciate your comments on my findings so far

    (1) the Oracle SQL Developer Data Modeler is a paid product stand-a-lone while using the Data Modeler of SQL Developer feature is free

    (2) the Data Modeler in SQL Developer functionality is basically a "viewer" I create my existing table of DR. by dragging in the Modeler in the object browser, but that's all. I can't save the diagram for later use or use to change the structure of the or the tables

    (3) if I need to have 'Oracle Designer' as ERD capability so I need to use Oracle SQL Developer Data Modeler (after purchase!)

    (4) are there other differences that I missed?


    Thanks in advance

    Yes, you've nailed it!

    Have fun
    K.

  • Oracle SQL Developer Data Modeler start error

    Hi all

    I just downloaded Oracle SQL Developer Data Modeling (2.0.0.57.0) [released July 1, 2009] by selecting the option ' Oracle SQL Developer Data Modeler for Windows (this zip file contains the JRE).

    After that I installed the same and tried to start clicking on datamodeler.exe, I get the following error message

    "Failed to create an instance of the JVM on the path:... \jdk\jre\bin\client\jvm.dll.

    On the same machine, I can successfully run the SQL Oracle program without error. My machine is having Windows XP Professional SP3.

    Any ideas? If there is a problem with Java, that of the Oracle SQL Developer should also not run. But as I said it runs with any error, but the Oracle SQL Developer Data Modeler gives the above error.

    Kind regards

    JAA149

    Hey Jawad,

    you wrote you have 960 MB of RAM - how did you do a test with 1.5 MB of RAM and 2 MB of RAM?
    No doubt this is a problem of memory - memory change in the datamodeler.conf file option:
    -There 'AddVMOption-Xmx768M"by default; to take
    -"AddVMOption-Xmx512M" or never, making it suitable for your configuration - you can check the available free memory first.

    Philippe

  • Question to load data using sql loader in staging table, and then in the main tables!

    Hello

    I'm trying to load data into our main database table using SQL LOADER. data will be provided in separate pipes csv files.

    I have develop a shell script to load the data and it works fine except one thing.

    Here are the details of a data to re-create the problem.

    Staging of the structure of the table in which data will be filled using sql loader

    create table stg_cmts_data (cmts_token varchar2 (30), CMTS_IP varchar2 (20));

    create table stg_link_data (dhcp_token varchar2 (30), cmts_to_add varchar2 (200));

    create table stg_dhcp_data (dhcp_token varchar2 (30), DHCP_IP varchar2 (20));

    DATA in the csv file-

    for stg_cmts_data-

    cmts_map_03092015_1.csv

    WNLB-CMTS-01-1. 10.15.0.1

    WNLB-CMTS-02-2 | 10.15.16.1

    WNLB-CMTS-03-3. 10.15.48.1

    WNLB-CMTS-04-4. 10.15.80.1

    WNLB-CMTS-05-5. 10.15.96.1

    for stg_dhcp_data-

    dhcp_map_03092015_1.csv

    DHCP-1-1-1. 10.25.23.10, 25.26.14.01

    DHCP-1-1-2. 56.25.111.25, 100.25.2.01

    DHCP-1-1-3. 25.255.3.01, 89.20.147.258

    DHCP-1-1-4. 10.25.26.36, 200.32.58.69

    DHCP-1-1-5 | 80.25.47.369, 60.258.14.10

    for stg_link_data

    cmts_dhcp_link_map_0309151623_1.csv

    DHCP-1-1-1. WNLB-CMTS-01-1,WNLB-CMTS-02-2

    DHCP-1-1-2. WNLB-CMTS-03-3,WNLB-CMTS-04-4,WNLB-CMTS-05-5

    DHCP-1-1-3. WNLB-CMTS-01-1

    DHCP-1-1-4. WNLB-CMTS-05-8,WNLB-CMTS-05-6,WNLB-CMTS-05-0,WNLB-CMTS-03-3

    DHCP-1-1-5 | WNLB-CMTS-02-2,WNLB-CMTS-04-4,WNLB-CMTS-05-7

    WNLB-DHCP-1-13 | WNLB-CMTS-02-2

    Now, after loading these data in the staging of table I have to fill the main database table

    create table subntwk (subntwk_nm varchar2 (20), subntwk_ip varchar2 (30));

    create table link (link_nm varchar2 (50));

    SQL scripts that I created to load data is like.

    coil load_cmts.log

    Set serveroutput on

    DECLARE

    CURSOR c_stg_cmts IS SELECT *.

    OF stg_cmts_data;

    TYPE t_stg_cmts IS TABLE OF stg_cmts_data % ROWTYPE INDEX BY pls_integer;

    l_stg_cmts t_stg_cmts;

    l_cmts_cnt NUMBER;

    l_cnt NUMBER;

    NUMBER of l_cnt_1;

    BEGIN

    OPEN c_stg_cmts.

    Get the c_stg_cmts COLLECT in BULK IN l_stg_cmts;

    BECAUSE me IN l_stg_cmts. FIRST... l_stg_cmts. LAST

    LOOP

    SELECT COUNT (1)

    IN l_cmts_cnt

    OF subntwk

    WHERE subntwk_nm = l_stg_cmts (i) .cmts_token;

    IF l_cmts_cnt < 1 THEN

    INSERT

    IN SUBNTWK

    (

    subntwk_nm

    )

    VALUES

    (

    l_stg_cmts (i) .cmts_token

    );

    DBMS_OUTPUT. Put_line ("token has been added: ' |") l_stg_cmts (i) .cmts_token);

    ON THE OTHER

    DBMS_OUTPUT. Put_line ("token is already present'");

    END IF;

    WHEN l_stg_cmts EXIT. COUNT = 0;

    END LOOP;

    commit;

    EXCEPTION

    WHILE OTHERS THEN

    Dbms_output.put_line ('ERROR' |) SQLERRM);

    END;

    /

    output

    for dhcp


    coil load_dhcp.log

    Set serveroutput on

    DECLARE

    CURSOR c_stg_dhcp IS SELECT *.

    OF stg_dhcp_data;

    TYPE t_stg_dhcp IS TABLE OF stg_dhcp_data % ROWTYPE INDEX BY pls_integer;

    l_stg_dhcp t_stg_dhcp;

    l_dhcp_cnt NUMBER;

    l_cnt NUMBER;

    NUMBER of l_cnt_1;

    BEGIN

    OPEN c_stg_dhcp.

    Get the c_stg_dhcp COLLECT in BULK IN l_stg_dhcp;

    BECAUSE me IN l_stg_dhcp. FIRST... l_stg_dhcp. LAST

    LOOP

    SELECT COUNT (1)

    IN l_dhcp_cnt

    OF subntwk

    WHERE subntwk_nm = l_stg_dhcp (i) .dhcp_token;

    IF l_dhcp_cnt < 1 THEN

    INSERT

    IN SUBNTWK

    (

    subntwk_nm

    )

    VALUES

    (

    l_stg_dhcp (i) .dhcp_token

    );

    DBMS_OUTPUT. Put_line ("token has been added: ' |") l_stg_dhcp (i) .dhcp_token);

    ON THE OTHER

    DBMS_OUTPUT. Put_line ("token is already present'");

    END IF;

    WHEN l_stg_dhcp EXIT. COUNT = 0;

    END LOOP;

    commit;

    EXCEPTION

    WHILE OTHERS THEN

    Dbms_output.put_line ('ERROR' |) SQLERRM);

    END;

    /

    output

    for link -.

    coil load_link.log

    Set serveroutput on

    DECLARE

    l_cmts_1 VARCHAR2 (4000 CHAR);

    l_cmts_add VARCHAR2 (200 CHAR);

    l_dhcp_cnt NUMBER;

    l_cmts_cnt NUMBER;

    l_link_cnt NUMBER;

    l_add_link_nm VARCHAR2 (200 CHAR);

    BEGIN

    FOR (IN) r

    SELECT dhcp_token, cmts_to_add | ',' cmts_add

    OF stg_link_data

    )

    LOOP

    l_cmts_1: = r.cmts_add;

    l_cmts_add: = TRIM (SUBSTR (l_cmts_1, 1, INSTR (l_cmts_1, ',') - 1));

    SELECT COUNT (1)

    IN l_dhcp_cnt

    OF subntwk

    WHERE subntwk_nm = r.dhcp_token;

    IF l_dhcp_cnt = 0 THEN

    DBMS_OUTPUT. Put_line ("device not found: ' |") r.dhcp_token);

    ON THE OTHER

    While l_cmts_add IS NOT NULL

    LOOP

    l_add_link_nm: = r.dhcp_token |' _TO_' | l_cmts_add;

    SELECT COUNT (1)

    IN l_cmts_cnt

    OF subntwk

    WHERE subntwk_nm = TRIM (l_cmts_add);

    SELECT COUNT (1)

    IN l_link_cnt

    LINK

    WHERE link_nm = l_add_link_nm;

    IF l_cmts_cnt > 0 AND l_link_cnt = 0 THEN

    INSERT INTO link (link_nm)

    VALUES (l_add_link_nm);

    DBMS_OUTPUT. Put_line (l_add_link_nm |) » '||' Has been added. ") ;

    ELSIF l_link_cnt > 0 THEN

    DBMS_OUTPUT. Put_line (' link is already present: ' | l_add_link_nm);

    ELSIF l_cmts_cnt = 0 then

    DBMS_OUTPUT. Put_line (' no. CMTS FOUND for device to create the link: ' | l_cmts_add);

    END IF;

    l_cmts_1: = TRIM (SUBSTR (l_cmts_1, INSTR (l_cmts_1, ',') + 1));

    l_cmts_add: = TRIM (SUBSTR (l_cmts_1, 1, INSTR (l_cmts_1, ',') - 1));

    END LOOP;

    END IF;

    END LOOP;

    COMMIT;

    EXCEPTION

    WHILE OTHERS THEN

    Dbms_output.put_line ('ERROR' |) SQLERRM);

    END;

    /

    output

    control files -

    DOWNLOAD THE DATA

    INFILE 'cmts_data.csv '.

    ADD

    IN THE STG_CMTS_DATA TABLE

    When (cmts_token! = ") AND (cmts_token! = 'NULL') AND (cmts_token! = 'null')

    and (cmts_ip! = ") AND (cmts_ip! = 'NULL') AND (cmts_ip! = 'null')

    FIELDS TERMINATED BY ' |' SURROUNDED OF POSSIBLY "" "

    TRAILING NULLCOLS

    ('RTRIM (LTRIM (:cmts_token))' cmts_token,

    cmts_ip ' RTRIM (LTRIM(:cmts_ip)) ")". "

    for dhcp.


    DOWNLOAD THE DATA

    INFILE 'dhcp_data.csv '.

    ADD

    IN THE STG_DHCP_DATA TABLE

    When (dhcp_token! = ") AND (dhcp_token! = 'NULL') AND (dhcp_token! = 'null')

    and (dhcp_ip! = ") AND (dhcp_ip! = 'NULL') AND (dhcp_ip! = 'null')

    FIELDS TERMINATED BY ' |' SURROUNDED OF POSSIBLY "" "

    TRAILING NULLCOLS

    ('RTRIM (LTRIM (:dhcp_token))' dhcp_token,

    dhcp_ip ' RTRIM (LTRIM(:dhcp_ip)) ")". "

    for link -.

    DOWNLOAD THE DATA

    INFILE 'link_data.csv '.

    ADD

    IN THE STG_LINK_DATA TABLE

    When (dhcp_token! = ") AND (dhcp_token! = 'NULL') AND (dhcp_token! = 'null')

    and (cmts_to_add! = ") AND (cmts_to_add! = 'NULL') AND (cmts_to_add! = 'null')

    FIELDS TERMINATED BY ' |' SURROUNDED OF POSSIBLY "" "

    TRAILING NULLCOLS

    ('RTRIM (LTRIM (:dhcp_token))' dhcp_token,

    cmts_to_add TANK (4000) RTRIM (LTRIM(:cmts_to_add)) ")" ""

    SHELL SCRIPT-

    If [!-d / log]

    then

    Mkdir log

    FI

    If [!-d / finished]

    then

    mkdir makes

    FI

    If [!-d / bad]

    then

    bad mkdir

    FI

    nohup time sqlldr username/password@SID CONTROL = load_cmts_data.ctl LOG = log/ldr_cmts_data.log = log/ldr_cmts_data.bad DISCARD log/ldr_cmts_data.reject ERRORS = BAD = 100000 LIVE = TRUE PARALLEL = TRUE &

    nohup time username/password@SID @load_cmts.sql

    nohup time sqlldr username/password@SID CONTROL = load_dhcp_data.ctl LOG = log/ldr_dhcp_data.log = log/ldr_dhcp_data.bad DISCARD log/ldr_dhcp_data.reject ERRORS = BAD = 100000 LIVE = TRUE PARALLEL = TRUE &

    time nohup sqlplus username/password@SID @load_dhcp.sql

    nohup time sqlldr username/password@SID CONTROL = load_link_data.ctl LOG = log/ldr_link_data.log = log/ldr_link_data.bad DISCARD log/ldr_link_data.reject ERRORS = BAD = 100000 LIVE = TRUE PARALLEL = TRUE &

    time nohup sqlplus username/password@SID @load_link.sql

    MV *.log. / log

    If the problem I encounter is here for loading data in the link table that I check if DHCP is present in the subntwk table, then continue to another mistake of the newspaper. If CMTS then left create link to another error in the newspaper.

    Now that we can here multiple CMTS are associated with unique DHCP.

    So here in the table links to create the link, but for the last iteration of the loop, where I get separated by commas separate CMTS table stg_link_data it gives me log as not found CMTS.

    for example

    DHCP-1-1-1. WNLB-CMTS-01-1,WNLB-CMTS-02-2

    Here, I guess to link the dhcp-1-1-1 with balancing-CMTS-01-1 and wnlb-CMTS-02-2

    Theses all the data present in the subntwk table, but still it gives me journal wnlb-CMTS-02-2 could not be FOUND, but we have already loaded into the subntwk table.

    same thing is happening with all the CMTS table stg_link_data who are in the last (I think here you got what I'm trying to explain).

    But when I run the SQL scripts in the SQL Developer separately then it inserts all valid links in the table of links.

    Here, she should create 9 lines in the table of links, whereas now he creates only 5 rows.

    I use COMMIT in my script also but it only does not help me.

    Run these scripts in your machine let me know if you also get the same behavior I get.

    and please give me a solution I tried many thing from yesterday, but it's always the same.

    It is the table of link log

    link is already present: dhcp-1-1-1_TO_wnlb-cmts-01-1

    NOT FOUND CMTS for device to create the link: wnlb-CMTS-02-2

    link is already present: dhcp-1-1-2_TO_wnlb-cmts-03-3
    link is already present: dhcp-1-1-2_TO_wnlb-cmts-04-4

    NOT FOUND CMTS for device to create the link: wnlb-CMTS-05-5

    NOT FOUND CMTS for device to create the link: wnlb-CMTS-01-1

    NOT FOUND CMTS for device to create the link: wnlb-CMTS-05-8
    NOT FOUND CMTS for device to create the link: wnlb-CMTS-05-6
    NOT FOUND CMTS for device to create the link: wnlb-CMTS-05-0

    NOT FOUND CMTS for device to create the link: wnlb-CMTS-03-3

    link is already present: dhcp-1-1-5_TO_wnlb-cmts-02-2
    link is already present: dhcp-1-1-5_TO_wnlb-cmts-04-4

    NOT FOUND CMTS for device to create the link: wnlb-CMTS-05-7

    Device not found: wnlb-dhcp-1-13

    IF NEED MORE INFORMATION PLEASE LET ME KNOW

    Thank you

    I felt later in the night that during the loading in the staging table using UNIX machine he created the new line for each line. That is why the last CMTS is not found, for this I use the UNIX 2 BACK conversion and it starts to work perfectly.

    It was the dos2unix error!

    Thank you all for your interest and I may learn new things, as I have almost 10 months of experience in (PLSQL, SQL)

  • SQL Loader issue - CSV with commas and quotes IN the data

    Hello, I have a dataset for a simple table of 2 columns like this:

    Column 1, "it is given for"Column 2", with commas and quotes."

    Data are delimited by commas and may be surrounded by double quotes.  In ADDITION, it may include commas and quotation marks in the data fields.  I CANNOT manipulate data before sending it to SQL Loader.

    I set my file of control like this:

    DOWNLOAD THE DATA

    INFILE '. / TEST.dat'

    BADFILE '. / TEST. BAD'

    DISCARDFILE '. / TEST. DSC"

    REPLACE IN THE TEST TABLE

    Fields ended by ',' POSSIBLY BOX BY ' "' TRAILING NULLCOLS"

    (

    Col1 char (50),

    Col2 char (500)

    )


    Now when I run the present via SQLLDR, I get the following error in the log file:


    Sheet 1: Rejected - error on table TEST, column COL2.

    no terminator found after CLOSE and CLOSED field

    What are my options to get the loaded data as presented above?  I'm working on Oracle 11 g (11.2.0.3.0) 64-bit on AIX 6.1.0.0.

    Thank you!

    In this case, there is no way she can tell who's a delimiter or pregnant and which is part of the data.  As far as I know, there is no way that you can load it into the appropriate columns.

Maybe you are looking for