Error importing CSV files with "hidden" characters using the external Table

Similar Questions

• Cannot read Document in FileUtilities to search a file with french characters in the file name

  Hello

  I'm looking for a file with french characters in the filename as"ecran.doc" in my unix Server directory and write the file somewhere, say, in a list of documents, or write to files to another directory.

  I use to read Document in FileUtilities of the Foundation, but it cannot read the file due to the french character in its name. Although it can find another name of file without these french characters.

  I tried to rename but who also can not find the file with the name of the french character.

  Please provide any idea to solve using LC operation.

  Kind regards

  Raj of Rohan.

  Hi Thomas,

  Thanks for the post, but I already found a solution to this a month ago. Sorry for not posting the solution.

  Just put the '-Djava.property.file.encoding = ISO8859-1' in your server startup JVM argument and position the rebound on the back of the server to choose the new JVM arg . And now the reading service of Document in FileUtilities of the Foundation read characters all french.

  ISO8859-1refers 'no. 1 Latin alphabet,"consisting of 191 characters from the Latin alphabet. This character encoding scheme is used throughout the Americas, Western Europe, Oceania, and much of Africa. It is also commonly used in most of the Etymologies of the East Asian languages.

  Kind regards

  Raj of Rohan.

• Using the external table error

  Hi all

  This is the first time I use the external table for loading the data.

  I did these steps

  In the D:\sqlloade folder, I have my text file containing data example.txt

  I ran these queries

  CREATE OR REPLACE DIRECTORY  ext_tab_data AS 'D:\sqlloader';
  
  

  Directory created

  CREATE TABLE fxops.t_ext (
  t number
  )
  ORGANIZATION EXTERNAL (
    TYPE ORACLE_LOADER
    DEFAULT DIRECTORY ext_tab_data
    ACCESS PARAMETERS (
      RECORDS DELIMITED BY NEWLINE
      FIELDS TERMINATED BY ','
      MISSING FIELD VALUES ARE NULL
      (
       t number
      )
    )
    LOCATION ('sample.txt')
  )
  PARALLEL 5
  REJECT LIMIT UNLIMITED;
  
  

  create table

  When I tried to create a view on that table, he throws an error

  CREATE OR REPLACE VIEW fxops.t_view  AS
    SELECT *
    FROM   fxops.t_ext;
  
  SQL Error: ORA-06564: object EXT_TAB_DATA does not exist
  06564. 00000 -  "object %s does not exist"
  *Cause:    The named object could not be found.  Either it does not exist
             or you do not have permission to access it.
  *Action:   Create the object or get permission to access it.
  
  

  Sign in, and then run this:

  Select HOST_NAME from v$ instance;

  is this the same hostname that you are connected?

  Kind regards

  Harry

• Need help importing CSV file with commas in the rest of the data

  I have a linux script that gets the CSV files from a remote server and import the data into a table using SQLLoader.
  
  The problem I have is that the data in the CSV files sometimes contains commas in the text field that is interfering with the import and I have no control over what is in the file.
  
  my table:
  
  Create the table my_tab
  * (TIME_STAMP date, *)
  REQUEST_IP varchar2 (30),
  User_id varchar2 (30),
  FACILITY_ID VARCHAR2 (255),
  Number SUBFACILITY_ID,
  DETAIL VARCHAR2 (255)
  *);*
  
  example of a CSV file:
  
  * 27032011 story 232708,162.108.20.61,user123,cstmr_view_hlr_history,0,Viewed with 10 results per page for 1234567890 *.
  * 232737,162.108.20.61,user123,cstmr_view_customer,0,Facility 27032011 [display] selected *.
  * 232744,162.108.20.76,user123,cstmr_add_gprs,0,Facility 27032011 [Add x] selected *.
  * 232759,162.108.20.94,user456,cstmr_hlr_request,0,Facility 27032011 [x orders] selected *.
  * 27032011 added 232806,162.108.20.94,user123,cstmr_hlr_request,2,Customer note: note [x], MSISDN [1234567890] *.
  * 232806,162.108.22.96,user789,cstmr_hlr_request,0 27032011, queue update: Action [46], IMSI [1234567890] old [, MSISDN IMSI [1234567890] *.
  * 232815,162.108.20.67,user789,cstmr_view_customer,0,Facility 27032011 [display] selected *.
  * 27032011 232822,162.108.20.67,user123,cstmr_view_customer,5,Screen ' display customer details ': MSISDN [1234567890] *.
  * 27032011 232702,162.108.20.57,user456,cstmr_hlr_request,0,Queue update: Action [45], IMSI [1234567890] old [, MSISDN IMSI [12345678901], AFN [], [PDPREC] *.
  * 27032011 connection 232825,162.169.22.108,user456,adm_login,1,Successful: user [user_name] *.
  * 232829,162.169.22.108,user456,cstmr_view_customer,0,Facility 27032011 [display] selected *.
  
  How can I get around the commas in the text?
  
  Published by: cinnamon on April 4, 2011 08:07

  There is a way to do it, but it depends on a few assumptions, and (b) you have to jump through hoops to do it.

  First of all, this only works if there is no comma request_id, user_id or facility_id. (if there is, then you're stuck, because there is no way to determine which commas are part of data and those that are separators.)

  Second, he must know if there are any particular character - say | -who never appears in the detail.

  If you can do both of these assumptions, then what you have to do is:
  (1) create an array that consists of a single VARCHAR2 column large enough to hold longer line of your input data.
  (2) to import your data into this table.
  (3) replace the first five commas in each row of the table with. (or whatever your "unused" character).
  For example:

  UPDATE temp_data_table
  SET row_text = REPLACE(SUBSTR(row_text, 1, INSTR(row_text, ',', 1, 5)), ',', '|') || SUBSTR(row_text,  INSTR(row_text, ',', 1, 5) + 1);
  COMMIT;
  

  (4) write a SQL INSERT command to build your target table according to the values separated by the | characters.

  -Don

• empty data file with special characters in the name

  Hi all
  
  I'm having a problem with an empty data file which is in recovery mode.
  
  Before the following has been a success:
  ALTER database create datafile 'fully_qualified_file_name' as 'fully_qualified_file_name ';
  recover data file "fully_qualified_file_name";
  ALTER database datafile 'fully_qualified_file_name' online;
  
  But this time it does not work as I get an error:
  ORA-01516: file nonexistent journal, data file, or temporary file
  Using the file id does not work either.
  
  When I created a record of control file I see the data file has special characters in the name:
  ' / oracle/PRD/112_64/dbs/PSPR3I ^? ^?'
  
  Since it is empty, it would be possible to create the data file void anothr
  using the:
  ALTER database create datafile ' / oracle/PRD/112_64/dbs/PSPR3I ^? ^?'
  as ' / oracle/PRD/spdata2/r3i_8/r3i.data8';
  
  and then recover datafile 'fully_qualified_file_name ';
  ALTER database datafile 'fully_qualified_file_name' online;
  
  How orders oracle faced with special characters?
  
  Thanks for your advice

  user6737818 wrote:
  Hi all

  I'm having a problem with an empty data file which is in recovery mode.

  Before the following has been a success:
  ALTER database create datafile 'fully_qualified_file_name' as 'fully_qualified_file_name ';
  recover data file "fully_qualified_file_name";
  ALTER database datafile 'fully_qualified_file_name' online;

  But this time it does not work as I get an error:
  ORA-01516: file nonexistent journal, data file, or temporary file
  Using the file id does not work either.

  When I created a record of control file I see the data file has special characters in the name:
  ' / oracle/PRD/112_64/dbs/PSPR3I ^? ^?'

  It should work using a file as the definition of source file number and a new name for the new file. Works for me:

  orcl> alter database create datafile 7 as 'c:\tmp\newname.dbf';
  
  Database altered.
  
  orcl>
  

• mapping using the external table error.

  I'm trying to create a map using an external table.
  I'm testing a simple column to column mapping. As simple as you could possibly get.
  Can I deploy the mapping without errors, but when I try to run the map, I get a
  ORA-06564: EXT_DATA object does not exist
  
  I created my external to oracle using the ext_data directory. I can see the data without any problem in TOAD or SQLPlus.
  
  I tried to create a place called ext_data and it helps this creation, but I still get the same error.
  
  Any thoughts?
  
  Dan

  Hi Dan

  You must deploy the EXT_DATA connector (to be deployed as a Directory SQL) that is created under the location of Oracle used for your external table module is in.

  See you soon
  David

• Timestamp of reading using the external Table

  Hello
  I have a data file that looks like
  2011-08-15 00:00:00.000000 | 23:59:59.999999 2011-08-15
  
  I am trying to use an external table. But I always get an error trying to read the fractional part (ie the last 6 digits).
  The code is:
  create table ext_table_fdw)
  DW_Open TIMESTAMP
  DW_Close TIMESTAMP
  )
  (external) Organization
  type oracle_loader
  the default directory ftp_in
  (settings) access
  records delimited by newline
  nologfile
  fields ended by ' |'
  missing field values are null
  (
  DW_OPEN Char Date_Format Timestamp mask 'Mon dd yyyy Hh:Mi:Ss:Ff6'
  DW_Close Char Date_Format Timestamp mask 'Mon dd yyyy Hh:Mi:Ss:Ff6'
  )
  )
  location ("fdwextract_copy.txt")
  )
  reject limit unlimited;
  
  I get this error:
  ORA-29913: error in executing ODCIEXTTABLEFETCH legend
  ORA-29400: data cartridge error
  Error opening file /u02/lpremia/ftp-in/EXT_TABLE_FDW_11948.bad
  
  Please help as it is very important for us to be able to read the last part of the timestamp.
  Thank you.
  Best regards
  Brinda

  user6361157 wrote:
  Hello
  I have a data file that looks like
  2011-08-15 00:00:00.000000 | 23:59:59.999999 2011-08-15
  DW_OPEN Char Date_Format Timestamp mask 'Mon dd yyyy Hh:Mi:Ss:Ff6'
  DW_Close Char Date_Format Timestamp mask 'Mon dd yyyy Hh:Mi:Ss:Ff6'

  The MASK is NOT March data!
  'YYYY-MM-DD HH24:MI:SS:Ff6 ".

• Using the external Table

  Oracle 11.2.0.1
  Windows 7 Edition Home Premium 64-bit
  
  On every weekend, we get a file with the text like this:
  

  CUST_NAME:M/S ABC & COMPANY
  --------------------------------------------------------------------------------
  PROD_ID                              PR_05          STEAM_PIPE                              
  BILL_AMOUNT_ADVANCE                    1245.07          WITH VAT     
  BILL_DATE                         17-03-2011     
  GOODS_DELIVERED_ON                    20-03-2011     HAPPY CARGO
  
  
  CUST_NAME:M/S XYZ & COMPANY
  --------------------------------------------------------------------------------
  PROD_ID                              PR_09          PAPER_BAGS                              
  BILL_AMOUNT_CREDIT                    4000.07          WITHOUT VAT     
  BILL_DATE                         17-03-2011     
  GOODS_DELIVERED_ON                    20-03-2011     SHIVAM COURIER
  
  ...

  All ABOUT 100-125 billing information for our various regional staffs.
  
  We need an external table something like this:
  
  CREATE TABLE REGIONS_SALE
  (
  COL1 VARCHAR2 (40),
  COL2 VARCHAR2 (30),
  COL3 VARCHAR2 (15).
  COL4 VARCHAR2 (20)
  );
  
  And data should looks like this:
  
  Select * from REGIONS_SALE;

  COL1                    COL2                    COL3          COL4
  ABC & COMPANY               PROD_ID                    PR_05          STEAM_PIPE     
  ABC & COMPANY               BILL_AMOUNT_ADVANCE          1245.07          WITH VAT
  ABC & COMPANY               BILL_DATE               17-03-2011
  ABC & COMPANY               GOODS_DELIVERED_ON          20-03-2011     HAPPY CARGO
  XYZ & COMPANY               PROD_ID                    PR_09          PAPER_BAGS
  XYZ & COMPANY               BILL_AMOUNT_CREDIT          4000.07          WITHOUT VAT
  XYZ & COMPANY               BILL_DATE               17-03-2011     
  XYZ & COMPANY               GOOD_DELIVERED_ON          20-03-2011     SHIVAM COURIER

  Kindly help me, how to create the external table with the name of firms repeated in the COL1 of fixed length text file.
  
  Thanking you.

  One way is to create a view:

  SQL> CREATE TABLE TEST_EXT
    2  (
    3    str  VARCHAR2(4000 BYTE),
    4    RN         NUMBER
    5  )
    6  ORGANIZATION EXTERNAL
    7    (  TYPE ORACLE_LOADER
    8       DEFAULT DIRECTORY TESTDIR
    9       ACCESS PARAMETERS
   10         ( RECORDS DELIMITED BY NEWLINE
   11            BADFILE 'test.bad'
   12            LOGFILE 'test.log'
   13            NODISCARDFILE
   14            READSIZE 1048576
   15            FIELDS LDRTRIM
   16            REJECT ROWS WITH ALL NULL FIELDS
   17                ( str         (1:4000)     CHAR(4000),
   18                  rn recnum
   19         )                )
   20       LOCATION (TESTDIR:'test.txt')
   21    );
  
  Table created.
  
  SQL> create or replace view test_vu as
    2  select col1,col2,col3,col4
    3  from(
    4      select str,max(case when str like 'CUST_NAME:%' then replace(str,'CUST_NAME:') else null end) over(partition by sm) col1,
    5             regexp_substr(str,'[^   ]+') col2,regexp_substr(str,'[^ ]+',1,2) col3,regexp_substr(str,'[^     ]+',1,3) col4
    6      from(
    7          select sum(case when str like 'CUST_NAME:%' then 1 else 0  end ) over(order by rn) sm,t.*
    8          from TEST_EXT t
    9          where str like 'CUST_NAME%'
   10          or str like 'PROD_ID%'
   11          or str like 'BILL_AMOUNT%'
   12          or str like 'BILL_DATE%'
   13          or str like 'GOODS_DELIVERED%'
   14         )
   15       )
   16  where str not like 'CUST_NAME:%';
  
  View created.
  
  SQL> select * from test_vu;
  
  COL1                                     COL2                      COL3                           COL4
  ---------------------------------------- ------------------------- ------------------------------ ------------------------------
  M/S ABC & COMPANY                        PROD_ID                   PR_05                          STEAM_PIPE
  M/S ABC & COMPANY                        BILL_AMOUNT_ADVANCE       1245.07                        WITH VAT
  M/S ABC & COMPANY                        BILL_DATE                 17-03-2011
  M/S ABC & COMPANY                        GOODS_DELIVERED_ON        20-03-2011                     HAPPY CARGO
  M/S XYZ & COMPANY                        PROD_ID                   PR_09                          PAPER_BAGS
  M/S XYZ & COMPANY                        BILL_AMOUNT_CREDIT        4000.07                        WITHOUT VAT
  M/S XYZ & COMPANY                        BILL_DATE                 17-03-2011
  M/S XYZ & COMPANY                        GOODS_DELIVERED_ON        20-03-2011                     SHIVAM COURIER
  
  8 rows selected.
  

• The group using the external table name

  Hello
  
  I used the following link and I was able to create groups based on these steps. However I have a question, in the initialization block SQL SQL has a variable name,
  Select "GROUP «groupname...»» ». How is the presentation layer knows that the name of the variable 'GROUP' is attributed to membership in a group? Thank you.
  
  http://obieeblog.WordPress.com/2009/06/18/OBIEE-security-enforcement-%E2%80%93-external-database-table-authorization/

  Griup is a session of the system variable. As soon as you crush with block values init, same value will be overwritten at each location.

  Kind regards
  Sandeep

• Load the oracle database using the external table

  LKM:LKM file for oracle (external tables)
  Added command IKM:IKM sql
  Mapping: automatic mapping
  Error in step: insert new lines
  java.sql.SQLException: ORA-29913: error in executing ODCIEXTTABLEOPEN legend
  ORA-29400: data cartridge error
  opening the file in error... /Demo/file/test.txt_000.log
  
  Description:
  insert into TARGET_Schema.TARGET_Table
  (
  ColumnA,
  ColumnB,
  ColumnC
  )
  Select
  C1_ColumnA,
  C2_ColumnB,
  C3_ColumnC
  
  of ODI. C$ _0TARGET_Table
  where
  (1 = 1)
  answer please, how to solve this error.

  Hello

  GOTO topology and make sure that the database server has full path as D:\oracle\oracledi\oracledi\demo\file name

  and do not... file/demo /.

  After testing this issue, it should work fine.

  Thank you
  Fati

• Problem with the format of the external tables

  I am using the external Tables feature in Oracle.
  
  The code I am using currently to create my table is (I cut down the list of fields for clarity).

  create table EMP_EXTERNAL2
  (HIREDATE DATE )
  organization external
  ( default directory EXT_TABLES
  access parameters
  ( records delimited by newline
  fields terminated by ',')
  location ('EMP.CSV') )
  
  REJECT LIMIT 0
  ;

  But I get rejected folders when I select in the table due to the format of the HIRE date. The examples I've found suggest using a date format, change the code to:

  create table EMP_EXTERNAL2
  (HIREDATE DATE 'mm/dd/yyyy')
  organization external
  ( default directory EXT_TABLES
  access parameters
  ( records delimited by newline
  fields terminated by ',')
  location ('EMP.CSV') )
  
  REJECT LIMIT 0
  ;

  When I run this I get ORA-00907: lack the closing parenthesis. What I'm doing wrong here (I'm under the code in SQL Developer incidentally)?

  Hello

  Try this: -.

  create table EMP_EXTERNAL2
  (HIREDATE DATE )
  organization external
  ( default directory EXT_TABLES
  access parameters
  ( records delimited by newline
  fields terminated by ','
  (
  HIREDATE DATE 'MM/DD/YYYY'
  ))
  location ('EMP.CSV') )
  
  REJECT LIMIT 0
  

  -Pavan Kumar N

• UTL_FILE import csv file

  Hello

  I am writing a procdeure use the utl_file.get_line to read csv files and then proceed to a treatment before inserting data into a table.

  The csv file has four columns. The third column data have a newline when reading the file I get an error value.

  The procedure cannot detect the length of the 3rd field, it cannot detect the 3rd decimal point because this column has a line break.

  To differentiate the length of the column between each ',' part of the code I use is below.

  BEGIN

  Starts: = UTL_FILE. FOPEN ("D:\TEST','CT. CSV', 'R', 32765);

  loop

  UTL_FILE. GET_LINE (starts, input_String);

  delChar varchar2 (1): = ', ';

  -first field
  v_Pos: = instr (input_String, delChar, 1, 1);
  v_lenString: = v_Pos - 1;
  v_compname: = substr(input_String,1,v_lenString);

  v_startPos: = v_Pos + 1;

  -This will get the second field
  v_Pos: = instr (input_String, delChar, 1, 2);
  v_lenString: = v_Pos - v_startPos;
  v_comptype: = substr (input_String, v_startPos, v_lenString);
  
  v_startPos: = v_Pos + 1;
  
  -3rd field
  v_Pos: = instr (input_String, delChar, 1, 3);

  v_lenString: = v_Pos - v_startPos;
  v_notes: = substr (input_String, v_startPos, v_lenString);

  
  v_startPos: = v_Pos + 1;
  -the last land - there is no delimiter for last field
  v_Pos: = length (input_String) + 1;
  v_lenString: = v_Pos - v_startPos;
  v_comptypemodel: = substr (input_String, v_StartPos, v_lenString);

  My csv file looks like this when it is open with notepad ++,.

  CA COOLER VSM 89 E, MSM 89 E, 'EL. AC-MOTOR UNITS.

  ABB WADA 500L2L BSMH 11000V/1720KW/105 A/3584 RPM 60 HZ S1
  COOLING WATER FLOW 12.3 m3/h
  AMBIENT TEMP. 45 DEG", 123421
  ABB WADA 500L2L BSMH 11000, V/1720KW/105 A/3584 RPM 60 HZ, EL S1. AC-MOTOR UNITS. ABB WADA, EL. AC-MOTOR UNITS.

  The value of v_pos = 0 and v_lenSting =-30 for the 3rd field.

  Hope someone can point me in the right direction on how to solve this problem.

  Thanks in advance.

  OK, my apologies, I don't see the real problem you're trying to solve in the first post (it helps us if you format code / data using a font courier etc. in the Advanced Editor)

  Here is the response from Tom Kyte...

  https://asktom.Oracle.com/pls/Apex/f?p=100:11:0:P11_QUESTION_ID:2818047000346046084

  While there is an option in 11 GR 2 from an external preprocessor allows to process the file (which can be useful for files of decompression etc - see pretreatment: preprocess external Tables) it will probably not help in your case, because you will not easily be able to determine what 'new line' characters to substitute something else for you to make the difference when loading the data.

  Creating the CSV data can be modified to replace the line break characters in text by something else?  or use a different delimiter to the end of line?  That would be the ideal solution.

• How create the CSV file delimited by tabs using ORACLE utl_file?

  How create the CSV file delimited by tabs using ORACLE utl_file? Please provide the code sampl.

  This isn't a problem with Oracle, it is a problem with the way you open the data in Microsoft Excel.

  In Excel, you want (depending on your version may vary slightly)...

  Office 2010...

  1. go in the Ribbon "Data".

  2. click on 'text '.

  3. Locate and select your file, and then click "import."

  4 step 1 of the wizard - choose "Delimited", then click on "next >".

  5. step 2 of the wizard - choose "Tab" as the delimiter and click on "next >".

  6. step 3 of the wizard - define types of column as needed (if necessary) and click on "Finish".

  7. check where you want the data in the worksheet.

  Data is loading now in single cells as you expect.

  If you just double-click the CSV, Excel is apparently assuming that it will be separated by commas and does not recognize tabs as separators, unlike when you rename the file with a .xls extension where it examines the file, complains that it is not a content .xls and asks you to confirm that you want to continue loading and then intelligently recognizes the tabs and the format for you.

  As I said, not a problem with Oracle, just a problem with the MS Excel software.

• transfer with catalog will use the same file name

  Hello everyone, I evaluate the cloud of Lightroom version. I have a MacBook that I use on a trip to import and did a first job on my photos, so I have a home NAS at home where store pictures and do most of the work.

  I want to transfer RAW files, but all the work with the MacBook on the NAS, I tried to use "export as catalog" and then "import a catalog" with options to move the files, it works but it keep the old file name and it does not put them in a folder with a different name. For example, if on my Mac, I work with a folder named '2015' and then transfer into a folder with the same name on the NAS Server everything works fine, but if I made a folder called 'photos' to work on the MacBook, and then I want to transfer everything on my NAS under the folder "Hawaii", everything is put under Hawaii\photos\ instead of Hawaii\

  I'm having a hard time understand what you did. But the best thing to do would be to highlight all the images in the Hawaii\photos folder and drag them to the folder of Hawaii. You can simply delete the folder of pictures and everything should happen as you wish. Do all the going in Lightroom and it should be simple and quick.

• Not possible to export a list of virtual machines that are created in the past 7, 30, 120 and 180 days since an imported csv file containing the date of creation of virtual machine

  Not possible to export a list of virtual machines that are created in the past 7, 30, 120 and 180 days since an imported csv file containing the date of creation of virtual machine. My questions is the correct statement to the variable: $VmCreated7DaysAgo: $_CreatedOn "-lt" $CDate7.

  # #SCRIPT_START

  $file = "C:\Users\Admin\Documents\WindowsPowerShell\08-18-2014\VM-Repo.csv".

  $Import = import-csv $file

  $VMCreatedLast7RDayRepoFile = "C:\Users\Admin\Documents\WindowsPowerShell\08-18-2014\Last7Days.csv".

  $start7 = (get-Date). AddMonths(-1)

  $CDate7 = $start7. ToString('MM/dd/yyyy')

  $VmCreated7DaysAgo = $Import | Select-object - property name, Powerstate, vCenter, VMHost, Cluster, file, Application, CreatedBy, CreatedOn, NumCpu, MemoryGB | Where-Object {$_.} CreatedOn - lt $CDate7} | Sort-Object CreatedOn

  $TotalVmCreated7DaysAgo = $VmCreated7DaysAgo.count

  $VmCreated7DaysAgo | Export-Csv-path $VMCreatedLast7RDayRepoFile - NoTypeInformation - UseCulture

  Write-Host "$TotalVmCreated7DaysAgo VMs created in 7 days" - BackgroundColor Magenta

  Invoke-Item $VMCreatedLast7RDayRepoFile

  # #SCRIPT_END

  You can use the New-Timespan cmdlet in the Where clause, it returns the time difference between 2 DateTime objects.

  An example of this cmdley

  New-TimeSpan-start (Get-Date). AddDays(-7)-end (Get-Date). Select days - ExpandProperty

  In your case, you could do

  Where {(New Timespan-démarrer ([DateTime] $_.))} CreatedOn) - end $start7). {7 days - gt}

  But beware of negative numbers.