Need help importing CSV file with commas in the rest of the data

I have a linux script that gets the CSV files from a remote server and import the data into a table using SQLLoader.

The problem I have is that the data in the CSV files sometimes contains commas in the text field that is interfering with the import and I have no control over what is in the file.

my table:

Create the table my_tab
* (TIME_STAMP date, *)
REQUEST_IP varchar2 (30),
User_id varchar2 (30),
FACILITY_ID VARCHAR2 (255),
Number SUBFACILITY_ID,
DETAIL VARCHAR2 (255)
*);*

example of a CSV file:

* 27032011 story 232708,162.108.20.61,user123,cstmr_view_hlr_history,0,Viewed with 10 results per page for 1234567890 *.
* 232737,162.108.20.61,user123,cstmr_view_customer,0,Facility 27032011 [display] selected *.
* 232744,162.108.20.76,user123,cstmr_add_gprs,0,Facility 27032011 [Add x] selected *.
* 232759,162.108.20.94,user456,cstmr_hlr_request,0,Facility 27032011 [x orders] selected *.
* 27032011 added 232806,162.108.20.94,user123,cstmr_hlr_request,2,Customer note: note [x], MSISDN [1234567890] *.
* 232806,162.108.22.96,user789,cstmr_hlr_request,0 27032011, queue update: Action [46], IMSI [1234567890] old [, MSISDN IMSI [1234567890] *.
* 232815,162.108.20.67,user789,cstmr_view_customer,0,Facility 27032011 [display] selected *.
* 27032011 232822,162.108.20.67,user123,cstmr_view_customer,5,Screen ' display customer details ': MSISDN [1234567890] *.
* 27032011 232702,162.108.20.57,user456,cstmr_hlr_request,0,Queue update: Action [45], IMSI [1234567890] old [, MSISDN IMSI [12345678901], AFN [], [PDPREC] *.
* 27032011 connection 232825,162.169.22.108,user456,adm_login,1,Successful: user [user_name] *.
* 232829,162.169.22.108,user456,cstmr_view_customer,0,Facility 27032011 [display] selected *.

How can I get around the commas in the text?

Published by: cinnamon on April 4, 2011 08:07

There is a way to do it, but it depends on a few assumptions, and (b) you have to jump through hoops to do it.

First of all, this only works if there is no comma request_id, user_id or facility_id. (if there is, then you're stuck, because there is no way to determine which commas are part of data and those that are separators.)

Second, he must know if there are any particular character - say | -who never appears in the detail.

If you can do both of these assumptions, then what you have to do is:
(1) create an array that consists of a single VARCHAR2 column large enough to hold longer line of your input data.
(2) to import your data into this table.
(3) replace the first five commas in each row of the table with. (or whatever your "unused" character).
For example:

UPDATE temp_data_table
SET row_text = REPLACE(SUBSTR(row_text, 1, INSTR(row_text, ',', 1, 5)), ',', '|') || SUBSTR(row_text,  INSTR(row_text, ',', 1, 5) + 1);
COMMIT;

(4) write a SQL INSERT command to build your target table according to the values separated by the | characters.

-Don

Tags: Database

Similar Questions

  • Numbers with comma in the data Cube Viewer

    Hello

    I want to know what to do to display numbers with comma in the data Cube Viewer. The data type of the measure in the cube is NUMBER (4.3). The default aggregation is AVERAGE. When I load the cube with the numbers of the exactly NUMBER format (4.3) - in the data Cube Viewer I see only rounded integer values without decimal point.


    If I load the test instead of cube table - the numbers are loaded in the correct format.


    What should I do to load the cube with point numbers? Is this possible in OWB?

    Now, I decided to use a way to go around it - simply put 1453 instead of 1.453. Then it works (but this isn't a great solution). It is the only way?

    Thanks in advance for any help

    Peter

    Hi Peter

    The cube data viewer has a toolbar with a lot of options... formatting, you can add and remove the numbers after the decimal point. From memory, I think you have to select the cells you want, you can click and select all the cells in the table and then use the button Add digits after the decimal point.

    See you soon
    David

  • Error importing CSV files with "hidden" characters using the external Table

    Hi people

    Bit of a strange here.

    Well, we are accustomed to the use of the external Table method to load data from CSV files in the database, but a recent event presented us a problem.

    We have received some CSV files that "look like" regular CSV files, but Oracle will not load them.

    When we looked at the CSV using VIM on a UNIX machine, we saw the following characters 'hidden' between each regular character in the file.
    ^@
    If a string that looks like this when opened in Excel/Wordpad etc.
    "TEST","TEXT"
    Looks like this when exmained with VIM
    ^@"^@T^@E^@S^@T^@"^@,^@"^@T^@E^@X^@T^@"
    Has anyone encountered this before?

    Thank you very much

    Simon Gadd
    Oracle 11g 11.2.0.1.0

    Hi Simon,.

    ^ @ represents the ZERO character (0x00).
    So, most likely, you have a file encoded in Unicode.

    You need to specify the character set in the record specification (and if necessary the byte order mark), for example:

    CREATE TABLE ext_table
    (
      col1 VARCHAR2(10),
      col2 VARCHAR2(10)
    )
    ORGANIZATION EXTERNAL
    (
      TYPE ORACLE_LOADER
      DEFAULT DIRECTORY dump_dir
      ACCESS PARAMETERS
      (
       RECORDS DELIMITED BY '
    ' CHARACTERSET 'UTF16'
      FIELDS TERMINATED BY ','
      )
      LOCATION ('dump.csv')
    )
    REJECT LIMIT UNLIMITED;
    

    http://download.Oracle.com/docs/CD/E11882_01/server.112/e16536/et_params.htm#i1009499

  • I need help importing my old Windows contacts in the Mail application

    Windows mail for windows 8

    I just want my old windows mail contacts to people or post... I have saved the files to an expansion disc... but can't find how to put the files in the people... where can I find out how to do this?

    Hi Ellentavino.

    You can't import contacts of the PPP to Win 8 people directly.  However, if you go to http://people.live.com and choose manage it gives you the ability to import Outlook or a CSV file.

    Once these contacts are imported - it might take a while but they will eventually be replicated down to the people app in Windows 8.

    See you soon.

    Nick

  • Importing InDesign file with form in the new InDesign document fields

    I'm working on a multipage file. Then, I create a few versions where the majority of the pages are the pages of some, but some have slight differences. It fits my workflow to be able to import the same pages of new documents, so that I can just update the master file if changes are made.

    (Before I mixed just pages in Acrobat, but it now works better for this particular file)

    One of the pages is an interactive form, but interactivity seems to be lost when I place this page in a new InDesign file.

    Any ideas?

    Thank you

    Ruth

    PS InDesign CC

    You cannot place an interactive PDF in an InDesign file and retain the

    interactivity, only work is preserved.

  • Need help importing music mp3 file to my windows movie maker

    I've added music files to my films on windows, Director of music from cd that I ripped to my Windows Media Player, never had any problems.

    Today, I bought several songs from Amazon mp3 in order to add to my Windows Movie maker, and they will not import the Windows Movie Maker program.  They play on my Windows Media Player, however, will not import Windows Movie Maker.  The files say they are mp3 files and Windows Movie Maker says he takes these files, however, I get aRED X and the message 'could not be imported'.  Help!  What can I do to get these files to import to my movie on Windows Movie Maker.

    Thank you...

    I put the questions about music and its forum.  I really need help importing MP3 files that I just bought today from Amazon for the sole purpose to add to my movies in Windows Movie maker.

    In the past I have imported music music files that I ripped to CDs.  Now, today, I tried to import music files MP3 to Windows Movie Maker and it will not import files.  I get a message says"could not be imported" and a RED X.  Window Movie Maker, says it accepts MP3 files, but it does not accept these MP3 files.  Help!

    Thank you...

    ======================================
    Movie Maker may have problems of compatibility with MP3
    files... may be worth a try to convert MP3 files
    to WMA before you import into Movie Maker.

    There are many programs that can do conversions...
    The following freeware is an example...:

    (FWIW... it's always a good idea to create a system)
    Restore point before installing software or updates)

    Format Factory
    http://www.videohelp.com/tools/Format_Factory (the file you want to download is: > FFSetup260.zip
    (FWIW... installation..., you might want to
    Uncheck the box 'all' boxes on the last screen)

    First, you will need to decompress the file or just open the
    Drag FFSetup260.exe out of the folder
    and drop it on your desktop. To install left click.

    Next, after the download and installation of Format
    Factory... you can open the program and
    left click on the toolbar, the "Option" button and
    "Select an output folder to" / apply / OK.
    (this is where you find your files after they)
    are converted)

    Drag and drop your MP3 clips on the main screen...

    Select "while"WMA"/ OK...

    Click on... Beginning... in the toolbar...

    That should do it...

    Volunteer - MS - MVP - Digital Media Experience J - Notice_This is not tech support_I'm volunteer - Solutions that work for me may not work for you - * proceed at your own risk *.

  • the friends hyii UA HUD need help how to make a backup of the files from his deleted with (shift + delete) memory card

    the friends hyii UA HUD need help how to make a backup of the files from his deleted with (shift + delete) memory card

    Hello

    It seems that you want to restore the files removed from the memory card.

    I will definitely help you with this.

    I would say you can search the Internet for data recovery software by using your favorite search engine.

    Warning: Using third-party software, including hardware drivers can cause serious problems that may prevent your computer from starting properly. Microsoft cannot guarantee that problems resulting from the use of third-party software can be solved. Software using third party is at your own risk.

    It will be useful. If you have any other questions, feel free to let us know. We will be happy to help you.

    Thank you.

  • Import CSV file cell text in Memo field in FDM

    Hello world

    I try to import data in MDF Memo fields. I have a source CSV file that is imported via the batch loader. I suspect that I have to set a script but I don't know which. Can someone give nme (in a better sense of practice) some advice regarding:

    -What should be the layout of the source CSV file? (this is an extract a HFM prob app so not very flexible)

    -What script will need to be adjusted to import data in Memo fields?

    Some information:

    The source application is HFM which produces a CSV file for import of FDM. The data source contains the text amount and the cell for the amounts. FDM is used as an ETL to load data to another application of HFM. I understand theat LOADB Action script exports data of Memo fields to the text in the cell, provided that integration is enabled. So the second part of this load of data should work.

    Hello

    There are different options to achieve this.

    You can import your column with information about the cell to a dimension of the attribute. Then before the validation of data in the main table (script of the event ImportAction / sous-evenements PostWorkToMainProcess), you can create positions for memory based on this attribute and data loading. You can use SQL to insert the memo point tables or use fMemoAddItem (class ArchiveMgr)

    You must enable "Text the loading cell" of the adapter in the order that these memorandum items are loaded in the form of texts of cell in HFM.

    Hope that specifies

  • I need help importing audio from a reader audio tapes and then burn it to a cd or dvd. Thank you! __

    I need help importing audio from a reader audio tapes and then burn it to a cd or dvd. We have some old audio cassets with our children and other parents record and want to put them on a CD or DVD media. I can't understand this. Thank you!

    I need help importing audio from a reader audio tapes and then burn it to a cd or dvd. We have some old audio cassets with our children and other parents record and want to put them on a CD or DVD media. I can't understand this. Thank you!

    =================================
    The free Audacity software might be worth a visit:

    (FWIW... it's always a good idea to create a system)
    Restore point before installing software or updates)

    Audacity
    http://Audacity.sourceforge.NET/

    Features of Audacity
    http://Audacity.sourceforge.NET/about/features

    How to scan discs vinyls, audio tapes and minidiscs?
    http://Audacity.sourceforge.NET/Help/FAQ?s=recording&i=records-tapes

    Transferring tapes and records to computer or CD
    http://wiki.audacityteam.org/index.php?title=Transferring_tapes_and_records_to_computer_or_CD

    Volunteer - MS - MVP - Digital Media Experience J - Notice_This is not tech support_I'm volunteer - Solutions that work for me may not work for you - * proceed at your own risk *.

  • I also have a .csv file with the name of a jpeg file in a column and a text description of each jpeg format in a second column. Is it possible to automatically insert a jpeg (picture) and its corresponding text, each pair on one page, in an Indesign docum

    I also have a .csv file with the name of a jpeg file in a column and a text description of each jpeg format in a second column. Is it possible to automatically insert a jpeg (picture) and its corresponding text, each pair on one page, in an Indesign document?

    Read about to merge data in the help files.

  • Not possible to export a list of virtual machines that are created in the past 7, 30, 120 and 180 days since an imported csv file containing the date of creation of virtual machine

    Not possible to export a list of virtual machines that are created in the past 7, 30, 120 and 180 days since an imported csv file containing the date of creation of virtual machine. My questions is the correct statement to the variable: $VmCreated7DaysAgo: $_CreatedOn "-lt" $CDate7.

    # #SCRIPT_START

    $file = "C:\Users\Admin\Documents\WindowsPowerShell\08-18-2014\VM-Repo.csv".

    $Import = import-csv $file

    $VMCreatedLast7RDayRepoFile = "C:\Users\Admin\Documents\WindowsPowerShell\08-18-2014\Last7Days.csv".

    $start7 = (get-Date). AddMonths(-1)

    $CDate7 = $start7. ToString('MM/dd/yyyy')

    $VmCreated7DaysAgo = $Import | Select-object - property name, Powerstate, vCenter, VMHost, Cluster, file, Application, CreatedBy, CreatedOn, NumCpu, MemoryGB | Where-Object {$_.} CreatedOn - lt $CDate7} | Sort-Object CreatedOn

    $TotalVmCreated7DaysAgo = $VmCreated7DaysAgo.count

    $VmCreated7DaysAgo | Export-Csv-path $VMCreatedLast7RDayRepoFile - NoTypeInformation - UseCulture

    Write-Host "$TotalVmCreated7DaysAgo VMs created in 7 days" - BackgroundColor Magenta

    Invoke-Item $VMCreatedLast7RDayRepoFile

    # #SCRIPT_END

    You can use the New-Timespan cmdlet in the Where clause, it returns the time difference between 2 DateTime objects.

    An example of this cmdley

    New-TimeSpan-start (Get-Date). AddDays(-7)-end (Get-Date). Select days - ExpandProperty

    In your case, you could do

    Where {(New Timespan-démarrer ([DateTime] $_.))} CreatedOn) - end $start7). {7 days - gt}

    But beware of negative numbers.

  • I need help to write a script that detects the first instance of a paragraph style and then change

    I need help to write a script that detects the first instance of a paragraph style and then he goes to a different paragraph style.  I don't necessarily need someone to write all this, by the biggest problem is to find how to find just the first instance of the paragraph style.  Any help would be greatly appreciated, thank you!

    Hello

    then try this with your active doc:

    ....................

    myDoc var = app.activeDocument;

    mStyle var = myDoc.paragraphStyles.item ("PS_NameToFind"); change the name to paraStyle

    var mStyle_1 = myDoc.paragraphStyles.item ("PS_NameToChange"); change the name to paraStyle

    var mFrames = myDoc.pages.everyItem ().textFrames.everyItem () .getElements ();

    app.findTextPreferences = null;

    app.findTextPreferences.appliedParagraphStyle = mStyle;

    for (var k = 0; k)< mframes.length;="">

    {

    currFound = mFrames [k] .findText ();

    If (currFound.length > 0)

    currFound [0] .paragraphs [0] .appliedParagraphStyle = mStyle_1;

    }

    app.findTextPreferences = null;

    ................

    Rgds

  • I forgot to backup Outlook Express! However, the .dbx files are all in the data folder of Application that I have back up! How can I see what is in them to import to my chosen e-mail program?

    Original title: .dbx files

    I've updated computer laptop of my client from Windows XP to 7.  I forgot to backup Outlook Express!  However, the .dbx files are all in the data folder of Application that I have back up!  How can I see what is in them to import to my chosen e-mail program?

    This is how you would normally import into Windows Live Mail. If all goes well it will give you enough of a jump start.
     
     
     
    Copy the * ENTIRE * OE message store folder to a flash drive. (Folders.dbx must be included). Place it on the desktop or another location on the computer using WLMail.
     

    WLMail 2011/2012: folder button. Import Messages. Microsoft Outlook Express 6, and the point where it was saved.
     
     

     
     
     
     
     
     
     
     
  • Is there a file where are stored the data pause & rewind live TV in Windows Media Center?

    Is there a file where are stored the data pause & rewind live TV in Windows Media Center? More to the point - how can I save the time shift buffer in WMC?

    I would like to know if it is possible to find the data that Windows Media Center is to store during her watch TV Online. I don't speak data WMC stores by itself to allow the use of the function of pause & rewind live TV, how to find files that you have saved yourself by pressing the Red record button.

    For example, I often live TV playing on the computer in the background while I do something else. Can I see or hear a brief fact/clip/segment of interest that flashes by and I think, ' I wish I had recorded that. " If I go back to the bit I wanted to capture and hit record, it will not capture the part I want, it will start recording from the point, the program has achieved in real time.

    Does anyone know where the actual data of the TV are kept so that I can retrieve these clips that interest me, but were never recorded by me manually? I found the "TempRec" in the folder "Recorded TV" - anyone know how to convert this video (if indeed it is the location where you store temporary data for pause & rewind live TV)?

    If not, is it possible to enable a feature in WMC, which automatically starts the recording of TV channels as soon as you go to them? This way I can record everything, then just cut all the scenes of interest using Movie Maker. If not, are there other Win7 compatible programs that will do this?

    Thank you.

    Apparently, there is a way to achieve the goal.

    A nice gentleman from the Germany wrote dll that create a "time-shift buffer recording" add-in for WindowsMedia Center (for Vista and XP only) which can be found at:

    http://thegreenbutton.com/forums/p/3...83.aspx#162083

    For Windows 7, there would currently be via a 3rd party of program, i.e. PowerCinema 6, which records from any point in the time shift buffer zone - I don't know why Microsoft has not also incorporated this feature into WMC for have other programs third 3rd (including EyeTV for the Mac).

    Unfortunately, suggestions from another forum to change file extensions in .tmp .dvr-ms TempSBE and double-click on the file to read the file of time lag in WMC, does not work in Windows 7 x 64 (the message in this forum has been regarding the Vista x 64), however, I have not tried with other versions of Windows - I would like to know if anyone has had some luck using this method.

    Even if it had worked, I think I would have always preferred the usability and features of a program that does the job for me, like Power Cinema - damage, it could not have been WMC. What do you think?

    Thank you.

  • DIAdem - import programmatically .csv files with. STP file without dialog box

    Does anyone know how to import programmatically with a .csv files. STP file without opening the import box dialog box every time?

    I've been through the forum and seen for .xls files and solutions through the help file which is recommended to use the ExcelImport command, but these do not seem to work for .csv file types.  I am currently using the DataFileLoad command to import the files, which works very well to load the .csv files, but it forces me to go to the. STP file for each file.

    Does anyone know a better way to import a file .csv programmatically?

    Wow... Who had a fraction of the time that does the method of the dialog box.  I'll have to revise the data portion my DiAdem Advanced manual Plugin.

    Thank you, Brad!

    Steve

Maybe you are looking for

  • IMessage on ios 10

    Hello world. After upgrading ios 10, I use very correct imessage. But I remove the sim card and put it in iphone afterthat. Now, I can not active imassage and facetime when using the phone number. Please help me.

  • How to remove applications or programs to the macbook pro

    How to delete apps or programs of macbook pro!  Uninstalling microsoft worldwide feature is used.  I was told just drag to the trash can, it works sometimes, but not always!

  • software updates to iOS

    I've recently updated since an iPhone4s with iOS7 to an iPhone6s with iOS9.  I vaguely remember that in: Setting > general > software update There used to be an option to 'automatically' download updates (but not install).  From: Update the software

  • Recovery disk does not work - what to do?

    Hello I bought a genuine Toshiba recovery disc, start I press f12 as instructed, and then follow the instructions on the screen.He goes through several process then will be asked to press a button on any. The page reads as follows: WEIRD F:ready Driv

  • iPad 2 music recovery

    Noticed that all the music on the music app were stored in icloud for broadcast, with the exception of two music albums, which have been physically stored on the iPad.  To free storage capacity, I assumed removing albums in app Musci that the albums