Editing of duplicates in the csv columns
Hi all
This is a csv file that I need to handle:
VM, VI-SVC-VM009,0000e0, 0000e0
VM, VI-SVC-VM008,0000ea, 0000ea
VM, VI-SVC-VM007, 0000e9, 0000f1
VM, VI-SVC-VM005, 0000e7, 0000f4
VM, VI-SVC-VM004, 0000f6, 0000fa, 0000e6
VM, VI-SVC-VM003, 0000f5, 0000e5
VM, VI-SVC-VM002, 0000f0, 0000df
VM, VI-SVC-VM001, 0000ef, 0000de
VM, vi-SVC-vc3,0000e4, 0000e4
VM, NAS_OF, 0000f9, 0000f3
VM, CentOS with Prelert
VM, CentOS Desktop - Prelert,0000f6, 0000f6
LDEV, SVC PROD Cluster01, 0000fa, 0 x 29
LDEV, SVC PROD Cluster01, 0000f9, 0 x 15
LDEV, SVC PROD Cluster01, 0000f6, 0 x 14
LDEV, SVC PROD Cluster01, 0000f5, 0 x 13
LDEV, SVC PROD Cluster01, 0000f4, 0x0d
As you can see, there are several different columns with various inputs of length (about 3 rows, some 4, n). Some of the entries above:
VM, VI-SVC-VM009,0000e0, 0000e0
VM, VI-SVC-VM008,0000ea, 0000ea
Have the same entries. I need a script that will read the columns, look for an entry of duplicate column and remove the second double (almost like a get-unique, but with columns). The ideas people?
Best,
Rob.
Hi Rob,
You can still use the Get-Unique cmdlet to solve this problem by converting the first columns in rows using the Split method. Then, using Get-Unique. And finally convert the lines into columns using the Join method.
Get-Content -Path VMs.csv | ForEach-Object { [string]::Join(',',($_.Split(',') | Get-Unique)) } | Out-File -FilePath VmsNew.csv
Tags: VMware
Similar Questions
-
How to manage the pause embedded inside the CSV column line
Hello
I'm under pressure to make it work. I've already put this question on the forum of the APEX, but on reflection, I think that it relates more to PL/SQL rather than APEX APEX 4.1 having already utility to manage the Upload of CSV.
If you already read in the forum of the APEX, please ignore.
I'm sorry for that. Thanks for reading.
I need to develop an application that allows the user to upload a CSV file to a table of interface.
The APEX version on my workplace is 4.0.2.
I used the code of
http://dbswh.webhop.NET/HTMLDB/f?p=blog:read:0:article:11000346061523
It all works fine until recently I find
If a column in a CSV file cotain a break line (or line spacing) for example (the tester copy and paste this text which has a line break on a column in a spreadsheet)
It is the first sentence.
It is the second sentence.
It will break the 'it's the second sentence In a new column.
The contents of the CSV read next to Notepad as below
Date of assessment, Date, assessment provider, name of the assessor, Court, first name, middle name, last name, PRN person record, NHI number, assisted defendant Y/N number, is dependent on O/N, Notes, primary of ethnicity, "the ethnic origin other, please specify", gender, currently in treatment O/N, another Substance, Substance of concern 5 specified
22/09/2012, Co name of the provider, Warren Edgley, Wellington, sale, 2545554, dgsdf,, 'is the first sentence.
It's the second sentence. ', Japanese, woman, b.
This is the CSV UTIL code, please help me how can I replace the line break to a space so that the download process is correct.
In my applications, I save these lines in a table rather than a collection of APEX because the number of columns can contain greater than 50.CREATE OR REPLACE PACKAGE BODY "CSV_UTIL" AS --strip the beginning and the end quotes, then replace double quotation with single FUNCTION de_quote (p_str IN VARCHAR2, p_enc_by IN VARCHAR2) RETURN VARCHAR2 IS v_str VARCHAR2(32767) := p_str; BEGIN IF (p_enc_by IS NULL) THEN RETURN p_str; ELSE IF SUBSTR(p_str,-1) = p_enc_by THEN v_str := SUBSTR(p_str,1,LENGTH(p_str)-1); END IF; IF SUBSTR(p_str,1,1) = p_enc_by THEN v_str := SUBSTR(v_str,2); END IF; RETURN REPLACE (v_str, p_enc_by || p_enc_by, p_enc_by ); END IF; END de_quote; PROCEDURE parse (p_str IN VARCHAR2, p_enc_by IN VARCHAR2, p_sep IN VARCHAR2) IS l_n NUMBER DEFAULT 1; l_in_quote BOOLEAN DEFAULT FALSE; l_ch NCHAR (1); l_len NUMBER DEFAULT NVL (LENGTH (p_str), 0); BEGIN IF (l_len = 0) THEN RETURN; END IF; g_words := g_empty; g_words (1) := NULL; FOR i IN 1 .. l_len LOOP l_ch := SUBSTR (p_str, i, 1); IF (l_ch = p_enc_by) THEN l_in_quote := NOT l_in_quote; END IF; IF (l_ch = p_sep AND NOT l_in_quote) THEN l_n := l_n + 1; g_words (l_n) := NULL; ELSE g_words (l_n) := g_words (l_n) || l_ch; END IF; END LOOP; g_words (l_n) := de_quote (g_words (l_n), CHR(10)); g_words (l_n) := de_quote (g_words (l_n), CHR(13)); FOR i IN 1 .. l_n LOOP g_words (i) := de_quote (g_words (i), p_enc_by); END LOOP; END parse; /* Author: Oleg Lihvoinen Company: DbSWH Changes: 10.02.2011, There was a miscalculation of the file line last position in case it is the end of file */ PROCEDURE upload (p_file_name VARCHAR2, p_collection_name VARCHAR2, p_enc_by IN VARCHAR2, p_sep_by IN VARCHAR2, p_rows NUMBER) IS v_blob_data BLOB; v_clob_data CLOB; v_clob_len NUMBER; v_position NUMBER; v_char NCHAR (1); c_chunk_len NUMBER := 1; v_line VARCHAR2 (32767) := NULL; v_data_array vcarray; v_rows NUMBER := 0; n_seq NUMBER := 1; dest_offset NUMBER := 1; src_offset NUMBER := 1; amount INTEGER := DBMS_LOB.lobmaxsize; blob_csid NUMBER := DBMS_LOB.default_csid; lang_ctx INTEGER := DBMS_LOB.default_lang_ctx; warning INTEGER; l_sep VARCHAR2(100) := CASE WHEN p_sep_by = '\t' THEN chr(9) ELSE p_sep_by END; BEGIN htmldb_collection.create_or_truncate_collection (p_collection_name => p_collection_name); -- Read blob from wwv_flow_files SELECT blob_content INTO v_blob_data FROM wwv_flow_files WHERE NAME = p_file_name; v_position := 1; DBMS_LOB.createtemporary (lob_loc => v_clob_data, CACHE => TRUE, dur => DBMS_LOB.SESSION ); DBMS_LOB.converttoclob (v_clob_data, v_blob_data, amount, dest_offset, src_offset, blob_csid, lang_ctx, warning ); v_clob_len := DBMS_LOB.getlength (v_clob_data); IF v_clob_len = 0 THEN RETURN; END IF; WHILE (v_position <= v_clob_len + 1) LOOP v_char := DBMS_LOB.SUBSTR (v_clob_data, c_chunk_len, v_position); v_line := v_line || v_char; v_position := v_position + c_chunk_len; -- When the whole line is retrieved and not end of file or end of file IF v_char = CHR (10) AND v_position < v_clob_len OR v_position = v_clob_len + 1 THEN parse (p_str => v_line, p_enc_by => p_enc_by, p_sep => l_sep); v_data_array := g_words; FOR i IN 1..g_words.count LOOP IF i <= 50 THEN v_data_array(i) := g_words(i); ELSE exit; END IF; END LOOP; FOR i IN g_words.count + 1..50 LOOP v_data_array(i) := null; END LOOP; v_rows := v_rows + 1; -- exit if uploaded specified number of rows IF p_rows IS NOT NULL AND v_rows > p_rows THEN EXIT; END IF; -- Store data to collection n_seq := htmldb_collection.add_member (p_collection_name => p_collection_name, p_c001 => v_data_array (1), p_c002 => v_data_array (2), p_c003 => v_data_array (3), p_c004 => v_data_array (4), p_c005 => v_data_array (5), p_c006 => v_data_array (6), p_c007 => v_data_array (7), p_c008 => v_data_array (8), p_c009 => v_data_array (9), p_c010 => v_data_array (10), p_c011 => v_data_array (11), p_c012 => v_data_array (12), p_c013 => v_data_array (13), p_c014 => v_data_array (14), p_c015 => v_data_array (15), p_c016 => v_data_array (16), p_c017 => v_data_array (17), p_c018 => v_data_array (18), p_c019 => v_data_array (19), p_c020 => v_data_array (20), p_c021 => v_data_array (21), p_c022 => v_data_array (22), p_c023 => v_data_array (23), p_c024 => v_data_array (24), p_c025 => v_data_array (25), p_c026 => v_data_array (26), p_c027 => v_data_array (27), p_c028 => v_data_array (28), p_c029 => v_data_array (29), p_c030 => v_data_array (30), p_c031 => v_data_array (31), p_c032 => v_data_array (32), p_c033 => v_data_array (33), p_c034 => v_data_array (34), p_c035 => v_data_array (35), p_c036 => v_data_array (36), p_c037 => v_data_array (37), p_c038 => v_data_array (38), p_c039 => v_data_array (39), p_c040 => v_data_array (40), p_c041 => v_data_array (41), p_c042 => v_data_array (42), p_c043 => v_data_array (43), p_c044 => v_data_array (44), p_c045 => v_data_array (45), p_c046 => v_data_array (46), p_c047 => v_data_array (47), p_c048 => v_data_array (48), p_c049 => v_data_array (49), p_c050 => v_data_array (50) ); -- Clear the line v_line := NULL; END IF; END LOOP; END; END;
I want to know how can replace these newline within a column to a space.
If anyone has any ideas, please let me know.
Thank you very much in advance.
Ann
-
Why the location column is empty for all messages?
The location column header appears. but it has no content.
According to me, the location column is empty because it is not relevant when you view the Threads (message list) of a right folder pane - the location is highlighted in the folders pane. But if you select a message that for example is part of a thread, with items in the files box of received and sent, then Message/Open in the Conversation, the location column displays the name of the folder for each message. Similarly, if you search for edit/search/search messages, the location column displays the folder where the search result is stored.
-
How to find duplicates in the table
I have a table with 3 columns
name of the table - used
empcode firstname lastname
XYZ 123 pk
yzz 456 pk
101 kkk jk
ALTER TABLE employee
ADD (CONSTRAINT PRIMARY KEY employee_PK
(empcode, firstname, lastname))
all the three columns are as key to porimary, we are migrating the data, there are problems with the data as the cobination of all three, resulting in duplicate, in the last column is supposed to be duplicates but first two columns will not have the duplicate and a complete line of the table (combination have no duplicates)
a query need to find duplicates to validate the whole lines(B-)
select empcode,firstname,lastname,count(*) from employee group by empcode,firstname,lastname having count(*)>1;
-
the csv for excellent data is concatenated to a column
I have a 2D chart, I would like to put in a CSV so it can open in excel.
The data which could be the 2nd column is to be straight with the first.
I'm not sure why this is happening, I've done this before without a problem.
There must be a hidden character that is missing.
Anyone?
I use table build twice.
the first takes a singular double value and adds it to a table 1 d
There are two 1 d tables that are built for two differnet settings.
Then a 2nd build table combines two tables 1 d and this is what should go to Excel.
I tried to save it to excel.vi that someone posted earlier.
I wasn't able to run it as he opened lack of new worksheet.vi.
-
adding multiple files *.csv while retaining the original file name in the first column
Hi guys Cela made some time.
I am trying to add several *.csv files while retaining the original file name in the first column, the real data set is about 40 cases.
file a.csv contains:
1, line an a.csv
2, line 2 in a.csvfile b.csv contains:
1, line a b.csv
2, line 2 in b.csvresult output.csv is:
I would like this:
a.csv, 1, the line in a.csv
a.csv, 2, line 2 in a.csv
b.csv, 1, the line in b.csv
b.csv, 2, line 2 in b.csvAny suggestions to speed up my hobbling attempts would be aprieciated
Thank you
-SS
What you could do is given in attachment.
Started with 2 files:
a.csv
copy of a.csv
Both with data:
-
Remove duplicates from the oracle table using 2 columns
Hello
I need to remove the duplicates of an oracle table based on 2 columns in the table.i tried to remove duplicates using the join, but get the error like sql error ora-00933
Thank you
Hello
Here's one way:
DELETE FROM table_x
WHERE ROWID NOT IN)
SELECT MIN (ROWID)
FROM table_x
Col_1, col_2
);
I hope that answers your question.
If this isn't the case, please post a small example data (CREATE TABLE and only relevant columns, INSERT statements), and the results you want from this data.
In the case of a DML operation (for example, REMOVE) the sample data should show what look like the paintings before the DML, and results will be the content of the or the tables changed after the DML.
Explain, using specific examples, how you get these results from these data.
Always say what version of Oracle you are using (for example, 11.2.0.2.0).
See the FAQ forum: Re: 2. How can I ask a question on the forums?
-
How to concatenate values based on multiple columns of data duplicate in the loop
Hi all
Select '100' col_1, '111111' col2 of double
Union of all the
Select '100' col_1, '222222' col2 of double
Union of all the
Select '101' col_1, '333333' col2 of double
Union of all the
Select '102' col_1, '333333' col2 of double
I need to write by using cursor loop (configurable) so, it must concatenate col_1 or col_2
If,
In the table
col_1, col_2
100 111111
100 222222
101 333333
102 333333
Then
col_1, col_2
100 111111,222222
333333 101 102
Is it possible to compare and concatenate values in the loop
I have more than 4,000 bytes in the table of col_1,col_2 data values
Oracle Database 11 g Enterprise Edition Release 11.2.0.1.0 - 64 bit Production
Ok then:
create a type that makes "grouped" clob
CREATE or REPLACE TYPE clob_obj IS OBJECT
(
CLOB theClob
ORDER MEMBER FUNCTION equals (in_clob_obj IN clob_obj) RETURN NUMBER
member function getClobVal return clob)
/CREATE or REPLACE TYPE clob_obj BODY
IS
FUNCTION MEMBER ORDER is equal to (in_clob_obj IN clob_obj)
RETURN NUMBER
IS
BEGIN
RETURN sys.dbms_lob.compare (SELF.theClob, in_clob_obj.theClob);
END;
Member return clob getClobVal function
is
BEGIN
Return self.theClob;
end;
END;
/WITH conc_table AS (SELECT 1111 cust_id, cust_no 'ABC', '101010' ordr_no OF DOUBLE UNION ALL
SELECT 2222 cust_id, cust_no 'ABC', '101010' ordr_no OF DOUBLE UNION ALL
SELECT 3333 cust_id, cust_no 'ABC', '101010' ordr_no OF DOUBLE UNION ALL
SELECT cust_id 4444, "CCC" cust_no, ordr_no ' 123456' OF DOUBLE UNION ALL
SELECT cust_id 5555, "CCC" cust_no, '454545' ordr_no OF DOUBLE UNION ALL
Select cust_id 66666, "CCC" cust_no, ordr_no ' 77777' DOUBLE UNION ALL
SELECT cust_id, "CCC" cust_no, ordr_no ' 77777 66666' FROM DUAL)
SELECT rtrim (XMLAGG (XMLELEMENT (E, cust_id_no, ',')). EXTRACT ('customer_id //text()').getClobVal (),',')
clob_obj (ord_number) .getClobVal)
cust_num
FROM (SELECT cust_id_no
cust_num
rtrim (XMLAGG (XMLELEMENT (E, ord_num, ",")). EXTRACT ('//text ()') .getClobVal (), ',') ord_number
FROM (SELECT DISTINCT cust_id_no, cust_num, ordr_no AS conc_table FROM ord_num cust_no cust_id)
GROUP BY cust_id_no
cust_num)
GROUP BY clob_obj (ord_number)
cust_num;
I got the distinct in the select more intimate for no duplicate of the basetable is only sent to reunification of...
HTH
-
OBIEE 11.1.1.7 CSV export order to the wrong column
When you export a view with PDF/PPT/XLSX the column order is correct.
If I use CSV, TAB separated or XML almost always order is correct.
But now it happened that the order is just the wrong way autour.
View:
Column A column B column C
XSLX export:
Column A column B column C
Export CSV (false):
C column B column A
Is there a workaround/solution option to influence that?
The CSV does not export the view like the fact of XLS/XLSX, CSV taking the columns in the criteria tab, so it can also export columns excluded according to your table.
-
How to select the csv data stored in a BLOB column as if it were an external table?
Hi all
(Happy to be back after a while! )
Currently I am working on a site where users should be able to load the data of csv (comma is the separator) of their client machines (APEX 3.2 application) in the Oracle 11.2.0.4.0 EE database.
My problem is:
I can't use an external table (for the first time in my life ) so I'm a little clueless what to do as the csv data is stored by the application of the APEX in a BLOB column, and I'm looking for an elegant way (at least SQL PL/SQL/maximization) to insert the data into the destination table (run validations by a MERGER would be the most effective way to do the job).
I found a few examples, but I think they are too heavy and there could be a more elegant way in Oracle DB 11.2.
Simple unit test:
drop table CBC purge;
drop table dst serving;
create table src
(myblob blob
);
create table dst
(num number
, varchar2 (6) str
);
Insert in src
Select utl_raw.cast_to_raw (1; AAAAAA ;'|| Chr (10) |
2; BATH; »
)
Double;
Desired (of course) output based on the data in table SRC:
SQL > select * DST;
NUM STR
---------- ------
1 ABDELKRIM
2 BATH
Does anybody know a solution for this?
Any ideas/pointers/links/examples are welcome!
/ * WARNING: I was 'off' for about 3 months, then the Oracle - the part of my brain has become a bit "rusty and I feel it should not be so complicated as the examples I've found sofar ' * /"
Haha, wonder about regexp is like the blind leading the blind!
However, it's my mistake: I forgot to put the starting position setting (so 1, 2, 3,... was in fact the starting position, not the nth occurrence. duh!)
So, it should actually be:
select x.* , regexp_substr(x.col1, '[^;]+', 1, 1) , regexp_substr(x.col1, '[^;]+', 1, 2) , regexp_substr(x.col1, '[^;]+', 1, 3) , regexp_substr(x.col1, '[^;]+', 1, 4) , regexp_substr(x.col1, '[^;]+', 1, 5) , regexp_substr(x.col1, '[^;]+', 1, 6) from src , xmltable('/a/b' passing xmltype(''||replace(conv_to_clob(src.myblob), chr(10), '')||'') columns col1 varchar2(100) path '.') x;
Note: that's assuming that all the "columns" passed in the string won't be lame.
If one of them might be null, then:
select x.* , regexp_substr(ltrim(x.col1, ';'), '[^;]+', 1, 1) , regexp_substr(ltrim(x.col1, ';'), '[^;]+', 1, 2) , regexp_substr(ltrim(x.col1, ';'), '[^;]+', 1, 3) , regexp_substr(ltrim(x.col1, ';'), '[^;]+', 1, 4) , regexp_substr(ltrim(x.col1, ';'), '[^;]+', 1, 5) , regexp_substr(ltrim(x.col1, ';'), '[^;]+', 1, 6) from src , xmltable('/a/b' passing xmltype(replace(';'||replace(conv_to_clob(src.myblob), chr(10), ';')||'', ';;', '; ;')) columns col1 varchar2(100) path '.') x;
-
Given «example.csv» csv file
I want to load the data in:
the C7 cell in column X
cell D8 in column Y
cell F7 in column Z
Oracle table "exampletable" and so on. The csv file has not ordered rows and columns, so I can't just load the csv file into a new table itself. So what I'm asking is how to map to the columns in table cells load this file into Oracle (XE). Can someone point me to a tutorial?
I know it's quite elementary, please let me know if I am sous-prescrit anything in the question.
I ended up doing in Excel, through a very laborious process that will probably be repeated by others.
What a pity that there is no functionality in Oracle for this kind of data import more flexible. Analysts of data often throw himself spreadsheets that look like this (or 485 of them) who must enter data one way or another.
-
Delete the name of the column in the CSV file generated by ODI
Hello
I want to remove the column names in the CSV file that I create the database.
I tried to put "towards zero in the data store ', that I created for the csv file, but it does not work.
Please help.
Thank you
Sudeep NathalieSudeep salvation,
Ok.. If you don't want the column to be generated and then in your IKM (IKM SQL to THE grit file) choose the option GENERATE_HEADER to 'no '.
Then run the interface. The target csv file will not now column header.
Thank you
Fati -
Versistand version is 2013 sp1.
I use stimulus CSV expressions in my sequence in real time and want a channel to keep its value current (last) until a certain time. Is there a way to do this?
Stimulation CSV file example:
timestamp, chan1, chan2
0,0,
10, 1,
20,1,5
30,2,10
In this example, chan2 would retain its current value until the timestamp 20.
Is it possible to implement this feature?
Kind regards
Rick Howard
Thank you! It is a valuable feedback. I can't think of a way to do natively with the functionality of reading CSV for sequences in real time. Some thoughts on how this can be done:
1. create your own utility to sequences of script based on a CSV file in real-time. Extend the functionality of having a special marker in a cell which designates not only to change the value for the channel as timestep. Do not start from scratch. This tool open source and this one the two script of sequences in real time and stimulation of reading profiles based on data sets (although in different ways).
2. with the help of software without insertion to the fault of the string value that you want to keep the initial value for the first X seconds of reading the CSV file. For example, you could play another sequence with your CSV file at fault the channel at its current value so that reading CSV does not replace the value.
-
How to download the Csv file with column headers
Hi all
This is pavan, using Apex version 4.2.3
I am trying to download the csv file I followed this link , and I'm able to download excel with headers, when I try to download with headers of this error "'ORA-01858: a non-digit character was found here where was waiting for a digital" I searched in google but could not find the right solution, "
can anyone help on this please.
Thanks in advance,
Kind regards
Pavan
This article is 6 years old.
You should study the solutions that are available for APEX 4.2: data loader or the 'Excel2Collection' plugin (which also manages the CSV files).
Data Loader
It is a wizard that generates an Assistant for your application.
Excel2Collection
You will use the Excel2Collection (in a single process) to convert the BLOB in a Collection
Then, in a 2nd address), you just do a "INSERT...". SELECT statement. Add ' where seq_id > 1 "for files with a header.
MK
PS - Use the "EXECUTE IMMEDIATE" article is not necessary.
-
Remove duplicate for a single column table
Hi all
So I try to work on a problem here. I searched the forums but I did find something that works for me. Here's my problem... I have a chart (conversion to a .csv file) with a lot of lines and columns. Each line is a log for a specific serial number file. The amount of columns is fixed. But, I must create a vi to go through the table and remove the duplicates under the original and create a new table. For example, my table might look like this (I added a header desc for clarity):
Job, Tech, SN
1827, SJ, 23827
1825, SJ, 23827
1827, SJ, 47384
1827, SJ, 57483
1827, SJ, 37473
1825, SJ, 37473
But I want a table attached like this:
Job, Tech, SN
1827, SJ, 23827
1827, SJ, 47384
1827, SJ, 57483
1827, SJ, 37473
Can someone help me?
Thank you
Ryan
Here's a version that works if duplicates are not necessarily adjacent. As a side effect, the output array is also sorted by serial number.
Maybe you are looking for
-
Hi all. I have an OLD NI instrument Simulator. It is so old, that I'm not still able to use the NI instrument Simulator Wizard. I called for the support and the technician was more useful however, whenever I run the wizard, it opens and alarm that re
-
How to import audio as two lanes. More Bonus Question!
I left my client's Office to my home editing suite and now when I want to import new audio files (mp3 music in this case), they come as a single stereo clip. In the interests of consistency, I would like to have them come as a two-lane. Clearly this
-
Wireless LAN does not start automatically
Hey,. I have a HP-Compaq CQ60-100ED and my phone does not start automatically. If I boot my laptop it comes on when Windows starts, but then stops again. I need to push my wireless button 2 times and it works perfectly. Also if I close my cover, and
-
Change the page size - grayed out of boxes
I'm changing the page size of the pages within a document. Apparently, you're supposed to select the pages in the Pages panel and then change the height and width of the Panel settings. My height and width boxes are grayed out. What have I done wrong
-
I should probably start by saying that I am a novice user of PS. Yesterday, I was using the pen tool, for a school assignment, to draw around the areas of an image to a vector illustration. It worked fine for awhile, but now when I click to start a