New Line within CSV/Tab delimited files

[Adam105]

Hello all,

I have successfully created an OnRowImported script that can import to EA a CSV/tab delimited file correctly. The purpose of this script is to import connectors from a CSV/Tab delimited file. I have noticed that some of my 'Notes' fields are causing issues due to a new line being used within this field. I can't really edit the input data, so I am looking for a work around

I did a CSV export of a test object from EA with new lines within the description, and the CSV file export makes the second line of a field, the first value of a new row.

I was having issues with commas within my Notes fields, so switched to tab delimited. Does anyone know of a way I can include commas and new lines within my description of the connector (Notes field)?

Thanks a lot

[qwerty]

Most CSV implementations are incorrect. Text which contains newline or a separator needs to be quoted ("). In that case each quote inside the text needs to be doubled. E.g. [highlight]Some<nl>text[/highlight] becomes [highlight]"Some<nl>text"[/highlight] (where <nl> is a new line char) or [highlight]a"b[/highlight] becomes [highlight]"a""b"[/highlight].

q.

[Adam105]

It seems if I save the following in one field;

Description
With
New
Lines

From an excel spreadsheet to a CSV file, then quote marks appear automatically (If I open the CSV in notepad). In EA, the quotes disappear, but the only word read is Description. If I manually add quote marks to this field in my CSV file, then I get ""Description imported.

If the description field is at the last coloumn of my CSV file, then it will always start a new row with only one field; a line from the description. Otherwise it fails.

I followed the example CSV import process at the following;

http://www.sparxsystems.com/cgi-bin/yabb/YaBB.cgi?num=1396913036/0#1

Any ideas?

Thanks

[qwerty]

As I said: the implementation is plain wrong. It first splits by newline and then removes the quotes. This is simply wrong. The parser needs to be a state machine which reads the file char by char.

Report a bug.

I have no idea whether Geerts implementation is better in that respect. He's a smart guy, but I have seen so many failed CSV implementations :-/

q.

[Adam105]

Ahh bugger. Ok, I have reported it as a bug, so hopefully I will find some answers..

Thanks a lot for your help

[qwerty]

Actually it's not too difficult to correct the code if you ever have written a parser. So you might to try it yourself. If Sparx would pay a bounty for the correction I would have done it  

q.

[Eve]

I can understand why you would like to see added features in the csv import, but a different implementation, with different available features on a function where there is no formal specification doesn't equal a bug (or plain wrong.)

[qwerty]

Well, see here: https://tools.ietf.org/html/rfc4180

Shame on you.

q.

[Eve]

Well, see here: https://tools.ietf.org/html/rfc4180

Exactly. The document supports my statement. Did you actually read it?

The document is categorized as "Informational", the second sentence reads:

It does not specify an Internet standard of any kind.

Other quotes that support my statement:

This section documents the format that seems to be followed by most implementations

Each field may or may not be enclosed in double quotes (however some programs, such as Microsoft Excel, do not use double quotes at all).

Due to lack of a single specification, there are considerable differences among implementations.

[qwerty]

RFCs are quasi-standards. The linked RFC explains very well and concise how CSV files should be implemented. Of course, everybody (obviously) is free to implement it to their own wish. The customers are grateful as you can see (following your last citation).

I can only shake my head.

q.