kaue / jsonexport

{} → :page_facing_up: it's easy to convert JSON to CSV

Home Page:http://kaue.github.io/jsonexport/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

speed question/issue

m-prof opened this issue · comments

This is a follow-up to: #64

Thanks so much kaue! This works now for me, for all files I checked.

I did have a follow-up on speed: things slow down for files 1.5M or higher. For me (Win 10 Pro, Surface Pro, 16G RAM, Intel i7-7660U) when I tried a 3.9M file it took 42 minutes (command: jsonexport <json_filename> <output_filename>. Is this sort of time to complete expected?

commented

@m-prof can you try running the benchmark/benchmark.js?

Sorry for being a noob but I am not getting this to work: I am trying ~\jsonexport\benchmark>node benchmark.js from a command prompt and I get error messages:

internal/modules/cjs/loader.js:976
  throw err;
  ^

Error: Cannot find module 'benchmark'
Require stack:
- \npm\node_modules\jsonexport-master\benchmark\benchmark.js
�[90m    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:973:15)�[39m
�[90m    at Function.Module._load (internal/modules/cjs/loader.js:855:27)�[39m
�[90m    at Module.require (internal/modules/cjs/loader.js:1033:19)�[39m
�[90m    at require (internal/modules/cjs/helpers.js:72:18)�[39m
    at Object.<anonymous> (\npm\node_modules\�[4mjsonexport-master�[24m\benchmark\benchmark.js:3:17)
�[90m    at Module._compile (internal/modules/cjs/loader.js:1144:30)�[39m
�[90m    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1164:10)�[39m
�[90m    at Module.load (internal/modules/cjs/loader.js:993:32)�[39m
�[90m    at Function.Module._load (internal/modules/cjs/loader.js:892:14)�[39m
�[90m    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)�[39m {
  code: �[32m'MODULE_NOT_FOUND'�[39m,
  requireStack: [
    �[32m'\\npm\\node_modules\\jsonexport-master\\benchmark\\benchmark.js'�[39m
  ]
}

I assume I am not executing this correctly!

commented

@m-prof you are almost there :)
try running npm install first

Thanks. Not sure if this is sufficient but here is the output (there was a bunch of error messages also as well, let me know if I should post):

npm\node_modules\jsonexport\benchmark>node benchmark.js
Executed benchmark against node module: "jsonexport"
Count (1), Cycles (1), Elapsed (512.53 sec), Hz (0.00975936285335975 ops/sec)

Executed benchmark against node module: "jsonexport-stream"
Count (1), Cycles (1), Elapsed (10.388 sec), Hz (0.8795467729800702 ops/sec)

PS There were two output files (out.csv and out-no-streams.csv), the converted version of data.json. I can post them though they look fine!