Merge branch 'master' of github.com:Keyang/node-csvtojson

Author: Keyang

docs/csvtojson-v2.md

@@ -222,15 +222,15 @@ const jsonArr=await csv().fromFile(myFile);
 
 ```js
 csv({
-  ignoreColumn:["gender","age"]
+  ignoreColumns:["gender","age"]
 })
 ```
 
 **Now**
 
 ```js
 csv({
-  ignoreColumn: /gender|age/
+  ignoreColumns: /gender|age/
 })
 ```
 

readme.md

@@ -26,7 +26,7 @@
 
 `csvtojson` has released version `2.0.0`. 
 * To upgrade to v2, please follow [upgrading guide](https://github.com/Keyang/node-csvtojson/blob/master/docs/csvtojson-v2.md)
-* If you are looking for documentation for `v1`, open [this page](https://github.com/Keyang/node-csvtojson/blob/master/docs/readme.v1.md).
+* If you are looking for documentation for `v1`, open [this page](https://github.com/Keyang/node-csvtojson/blob/master/docs/readme.v1.md)
 
 It is still able to use v1 with `csvtojson@2.0.0`
 
@@ -195,20 +195,19 @@ $ csvtojson
 # API
 
 * [Parameters](#parameters)
-* [Asynchronouse Result Process](#asynchronouse-result-process)
+* [Asynchronous Result Process](#asynchronous-result-process)
 * [Events](#events)
 * [Hook / Transform](#hook--transform)
 * [Nested JSON Structure](#nested-json-structure)
 * [Header Row](#header-row)
-* [Multi CPU Core Support(experimental) ](#multi-cpu-core-support)
 * [Column Parser](#column-parser)
 
 
 ## Parameters
 
 `require('csvtojson')` returns a constructor function which takes 2 arguments:
 
-1. parser parameters
+1. Parser parameters
 2. Stream options
 
 ```js
@@ -230,16 +229,16 @@ const converter=csv({
 Following parameters are supported:
 
 * **output**: The format to be converted to. "json" (default) -- convert csv to json. "csv" -- convert csv to csv row array. "line" -- convert csv to csv line string  
-* **delimiter**: delimiter used for seperating columns. Use "auto" if delimiter is unknown in advance, in this case, delimiter will be auto-detected (by best attempt). Use an array to give a list of potential delimiters e.g. [",","|","$"]. default: ","
-* **quote**: If a column contains delimiter, it is able to use quote character to surround the column content. e.g. "hello, world" wont be split into two columns while parsing. Set to "off" will ignore all quotes. default: " (double quote)
+* **delimiter**: delimiter used for separating columns. Use "auto" if delimiter is unknown in advance, in this case, delimiter will be auto-detected (by best attempt). Use an array to give a list of potential delimiters e.g. [",","|","$"]. default: ","
+* **quote**: If a column contains delimiter, it is able to use quote character to surround the column content. e.g. "hello, world" won't be split into two columns while parsing. Set to "off" will ignore all quotes. default: " (double quote)
 * **trim**: Indicate if parser trim off spaces surrounding column content. e.g. "  content  " will be trimmed to "content". Default: true
 * **checkType**: This parameter turns on and off whether check field type. Default is false. (The default is `true` if version < 1.1.4)
 * **ignoreEmpty**: Ignore the empty value in CSV columns. If a column value is not given, set this to true to skip them. Default: false.
 * **fork (experimental)**: Fork another process to parse the CSV stream. It is effective if many concurrent parsing sessions for large csv files. Default: false
 * **noheader**:Indicating csv data has no header row and first row is data row. Default is false. See [header row](#header-row)
 * **headers**: An array to specify the headers of CSV data. If --noheader is false, this value will override CSV header row. Default: null. Example: ["my field","name"]. See [header row](#header-row)
 * **flatKeys**: Don't interpret dots (.) and square brackets in header fields as nested object or array identifiers at all (treat them like regular characters for JSON field identifiers). Default: false.
-* **maxRowLength**: the max character a csv row could have. 0 means infinite. If max number exceeded, parser will emit "error" of "row_exceed". if a possibly corrupted csv data provided, give it a number like 65535 so the parser wont consume memory. default: 0
+* **maxRowLength**: the max character a csv row could have. 0 means infinite. If max number exceeded, parser will emit "error" of "row_exceed". if a possibly corrupted csv data provided, give it a number like 65535 so the parser won't consume memory. default: 0
 * **checkColumn**: whether check column number of a row is the same as headers. If column number mismatched headers number, an error of "mismatched_column" will be emitted.. default: false
 * **eol**: End of line character. If omitted, parser will attempt to retrieve it from the first chunks of CSV data.
 * **escape**: escape character used in quoted column. Default is double quote (") according to RFC4108. Change to back slash (\\) or other chars for your own case.
@@ -252,18 +251,18 @@ Following parameters are supported:
 * **needEmitAll**: Parser will build JSON result is `.then` is called (or await is used). If this is not desired, set this to false. Default is true. 
 All parameters can be used in Command Line tool.
 
-## Asynchronouse Result Process
+## Asynchronous Result Process
 
-Since `v2.0.0`, asynchronouse processing has been fully supported.
+Since `v2.0.0`, asynchronous processing has been fully supported.
 
-e.g. Process each JSON result asynchronousely.
+e.g. Process each JSON result asynchronously.
 
 ```js
 csv().fromFile(csvFile)
 .subscribe((json)=>{
 	return new Promise((resolve,reject)=>{
 		// Async operation on the json
-		// dont forget to call resolve and reject
+		// don't forget to call resolve and reject
 	})
 })
 ```
@@ -294,7 +293,7 @@ csv()
 
 ### data
 
-`data` event is emitted for each parsed CSV line. It passes buffer of strigified JSON in [ndjson format](http://ndjson.org/) unless `objectMode` is set true in stream option.
+`data` event is emitted for each parsed CSV line. It passes buffer of stringified JSON in [ndjson format](http://ndjson.org/) unless `objectMode` is set true in stream option.
 
 ```js
 const csv=require('csvtojson')
@@ -306,7 +305,7 @@ csv()
 ```
 
 ### error
-`error` event is emitted if there is any errors happened during parsing.
+`error` event is emitted if any errors happened during parsing.
 
 ```js
 const csv=require('csvtojson')
@@ -350,7 +349,7 @@ csv()
 	return newData;
 })
 
-// asynchronouse
+// asynchronous
 csv()
 .preRawData((csvRawData)=>{
 	return new Promise((resolve,reject)=>{
@@ -363,7 +362,7 @@ csv()
 
 ### CSV File Line Hook
 
-the function is called each time a file line has been parsed in csv stream. the `lineIdx` is the file line number in the file starting with 0. 
+The function is called each time a file line has been parsed in csv stream. The `lineIdx` is the file line number in the file starting with 0. 
 
 ```js
 const csv=require('csvtojson')
@@ -376,7 +375,7 @@ csv()
 	return fileLineString
 })
 
-// asynchronouse
+// asynchronous
 csv()
 .preFileLine((fileLineString, lineIdx)=>{
 	return new Promise((resolve,reject)=>{
@@ -398,7 +397,7 @@ const csv=require('csvtojson')
 csv()
 .subscribe((jsonObj,index)=>{
 	jsonObj.myNewKey='some value'
-	// OR asynchronousely
+	// OR asynchronously
 	return new Promise((resolve,reject)=>{
 		jsonObj.myNewKey='some value';
 		resolve();
@@ -489,7 +488,7 @@ csv({flatKeys:true})
 1. First row of csv source. Use first row of csv source as header row. This is default.
 2. If first row of csv source is header row but it is incorrect and need to be replaced. Use `headers:[]` and `noheader:false` parameters.
 3. If original csv source has no header row but the header definition can be defined. Use `headers:[]` and `noheader:true` parameters.
-4. If original csv source has no header row and the header definition is unknow. Use `noheader:true`. This will automatically add `fieldN` header to csv cells
+4. If original csv source has no header row and the header definition is unknown. Use `noheader:true`. This will automatically add `fieldN` header to csv cells
 
 
 ### Example
@@ -581,7 +580,7 @@ csv({
 
 Above example will convert `birthday` column into a js `Date` object.
 
-the returned value will be used in result JSON object. returning `undefined` will not change result JSON object. 
+The returned value will be used in result JSON object. Returning `undefined` will not change result JSON object. 
 
 ### Flat key column
 
@@ -617,7 +616,7 @@ Very much appreciate any types of donation and support.
 
 1. Fork the repo to your github account
 2. Checkout code from your github repo to your local machine.
-3. Make code changes and dont forget add related tests.
+3. Make code changes and don't forget add related tests.
 4. Run `npm test` locally before pushing code back.
 5. Create a [Pull Request](https://help.github.com/articles/creating-a-pull-request/) on github.
 6. Code review and merge
@@ -675,7 +674,7 @@ If a module packager is preferred, just simply `require("csvtojson")`:
 var csv=require("csvtojson");
 
 // or with import
-import * as csv from "csvtojson");
+import * as csv from "csvtojson";
 
 //then use csv as normal
 ```

src/rowSplit.ts

@@ -75,9 +75,15 @@ export class RowSplit {
             continue;
           } else if (e.indexOf(quote) !== -1) {
             let count = 0;
+            let prev = "";
             for (const c of e) {
-              if (c === quote) {
+              // count quotes only if previous character is not escape char
+              if (c === quote && prev !== this.escape) {
                 count++;
+                prev = "";
+              } else {
+                // save previous char to temp variable
+                prev = c;
               }
             }
             if (count % 2 === 1) {

test/data/dataWithSlashEscapeAndDelimiterBetweenQuotes

@@ -0,0 +1,2 @@
+id,raw
+0,"\"hello,\"world\""

test/testCSVConverter2.ts

@@ -294,7 +294,22 @@ describe("testCSVConverter2", function () {
     rs.pipe(test_converter);
   });
 
-  it("should output ndjson format", function (done) {
+  it("should process escape chars when delimiter is between escaped quotes", function(done) {
+    var test_converter = new Converter({
+      escape: "\\"
+    });
+
+    var testData =
+      __dirname + "/data/dataWithSlashEscapeAndDelimiterBetweenQuotes";
+    var rs = fs.createReadStream(testData);
+    test_converter.then(function(res) {
+      assert.equal(res[0].raw, '"hello,"world"');
+      done();
+    });
+    rs.pipe(test_converter);
+  });
+
+  it("should output ndjson format", function(done) {
     var conv = new Converter();
     conv.fromString("a,b,c\n1,2,3\n4,5,6")
       .on("data", function (d) {

v1/core/workerMgr.js

@@ -4,7 +4,6 @@ var eom1 = "\x0e";
 var eom2 = "\x0f";
 var CSVError = require('./CSVError');
 function workerMgr() {
-  var spawn = require("child_process").spawn;
   var exports = {
     initWorker: initWorker,
     sendWorker: sendWorker,