Google and Firebase - Unexpected end of JSON input

Question

Google and Firebase - Unexpected end of JSON input

gastonmorixe opened this issue 6 years ago · comments

Gaston Morixe commented 6 years ago

This is a (Bug Report / Feature Proposal)

Bug

Description

For bug reports:

What went wrong?

⟩ serverless deploy

{}
Serverless: Bundling with Webpack...
Time: 637ms
   Asset     Size  Chunks             Chunk Names
index.js  4.32 kB       0  [emitted]  main
   [0] ./handler.js 1.57 kB {0} [built]
   [1] external "firebase-admin" 42 bytes {0} [not cacheable]
   [2] external "firebase-functions" 42 bytes {0} [not cacheable]

  Syntax Error -------------------------------------------

  Unexpected end of JSON input

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           9.3.0
     Serverless Version:     1.25.0

What did you expect should have happened?

Deployed correctly

What was the config you used?

service: wrud

provider:
  name: google
  runtime: nodejs
  project: XXXX
  credentials: ~/.gcloud/XXXX.json

plugins:
  - serverless-google-cloudfunctions
  - serverless-webpack

custom:
  webpackIncludeModules:
    packagePath: './package.json'    

# needs more granular excluding in production as only the serverless provider npm
# package should be excluded (and not the whole node_modules directory)
package:
  exclude:
    - node_modules/**
    - .gitignore
    - .git/**

functions:
  hello:
    handler: helloWorld
    events:
      - http: path

What stacktrace or error message from your provider did you see?

  Stack Trace --------------------------------------------

SyntaxError: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at BbPromise.try (/Users/work1/Projects/wrud/wrud/node_modules/serverless-webpack/lib/packExternalModules.js:236:47)
    at tryCatcher (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/util.js:16:23)
    at Function.Promise.attempt.Promise.try (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/method.js:39:29)
    at BbPromise.fromCallback.then.depJson (/Users/work1/Projects/wrud/wrud/node_modules/serverless-webpack/lib/packExternalModules.js:236:35)
    at tryCatcher (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/promise.js:693:18)
    at Promise._fulfill (/Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/promise.js:638:18)
    at /Users/work1/Projects/wrud/wrud/node_modules/bluebird/js/release/nodeback.js:42:21
    at childProcess.exec (/Users/work1/Projects/wrud/wrud/node_modules/serverless-webpack/lib/packExternalModules.js:233:16)
    at ChildProcess.exithandler (child_process.js:279:5)
    at ChildProcess.emit (events.js:159:13)
    at maybeClose (internal/child_process.js:943:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:220:5)
From previous event:
    at PluginManager.invoke (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:368:22)
    at PluginManager.spawn (/usr/local/lib/node_modules/serverless/lib/classes/PluginManager.js:386:17)
    at ServerlessWebpack.BbPromise.bind.then.then.then (/Users/work1/Projects/wrud/wrud/node_modules/serverless-webpack/index.js:90:51)
    at runCallback (timers.js:773:18)
    at tryOnImmediate (timers.js:734:5)
    at processImmediate [as _immediateCallback] (timers.js:711:5)

Additional Data

package.json

{
  "name": "google-nodejs",
  "version": "0.1.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "serverless.com",
  "license": "MIT",
  "dependencies": {
    "babel-runtime": "^6.26.0",
    "firebase-admin": "^5.8.0",
    "firebase-functions": "^0.8.0",
    "jsonschema": "^1.2.2",
    "serverless-google-cloudfunctions": "^1.1.1"
  },
  "devDependencies": {
    "babel-core": "^6.26.0",
    "babel-loader": "^7.1.2",
    "babel-plugin-transform-runtime": "^6.23.0",
    "babel-preset-env": "^1.6.1",
    "babel-preset-stage-3": "^6.24.1",
    "eslint": "^4.15.0",
    "eslint-config-standard": "^11.0.0-beta.0",
    "eslint-plugin-import": "^2.8.0",
    "eslint-plugin-node": "^5.2.1",
    "eslint-plugin-promise": "^3.6.0",
    "eslint-plugin-standard": "^3.0.1",
    "serverless-webpack": "^4.2.0",
    "webpack": "^3.10.0",
    "webpack-node-externals": "^1.6.0"
  }
}

Serverless-Webpack Version you're using: 4.2.0
Webpack version you're using: 3.10.0
Serverless Framework Version you're using: 1.25.0
Operating System: OSX 10.13.2 (17C88) High Sierra
Stack Trace (if available):

Frank Schmid · Answer 1 · Thu Jan 18 2018 03:03:00 GMT+0800 (China Standard Time)

Hi @gastonmorixe, thanks for reporting.

According to the stacktrace, the plugin crashed, because it got an invalid output from the npm ls -prod -json -depth=1 command execution.

Can you try to execute it manually and post the output, so that we can check what exactly is going wrong there?

Gaston Morixe · Answer 2 · Thu Jan 18 2018 04:54:26 GMT+0800 (China Standard Time)

@HyperBrain sure ! https://transfer.sh/LX0S2/serverless_error.log

Gaston Morixe · Answer 3 · Thu Jan 18 2018 04:57:48 GMT+0800 (China Standard Time)

@HyperBrain also


npm ERR! peer dep missing: firebase-admin@~5.6.0, required by firebase-functions@0.8.0
npm ERR! extraneous: node-pre-gyp@0.6.39 /Users/work1/Projects/wrud/wrud/node_modules/node-pre-gyp

Gaston Morixe · Answer 4 · Thu Jan 18 2018 05:02:26 GMT+0800 (China Standard Time)

@HyperBrain sorry to add more comments, it seems that with npm install instead of yarn I get this

⟩ npm install
npm WARN rm not removing /Users/work1/Projects/wrud/wrud/node_modules/.bin/pbjs as it wasn't installed by /Users/work1/Projects/wrud/wrud/node_modules/protobufjs
npm WARN rm not removing /Users/work1/Projects/wrud/wrud/node_modules/google-gax/node_modules/grpc/node_modules/.bin/pbjs as it wasn't installed by /Users/work1/Projects/wrud/wrud/node_modules/google-gax/node_modules/grpc/node_modules/protobufjs
npm WARN rm not removing /Users/work1/Projects/wrud/wrud/node_modules/.bin/semver as it wasn't installed by /Users/work1/Projects/wrud/wrud/node_modules/semver
npm WARN rm not removing /Users/work1/Projects/wrud/wrud/node_modules/@google-cloud/common-grpc/node_modules/grpc/node_modules/.bin/node-pre-gyp as it wasn't installed by /Users/work1/Projects/wrud/wrud/node_modules/@google-cloud/common-grpc/node_modules/grpc/node_modules/node-pre-gyp

> fsevents@1.1.3 install /Users/work1/Projects/wrud/wrud/node_modules/fsevents
> node install

[fsevents] Success: "/Users/work1/Projects/wrud/wrud/node_modules/fsevents/lib/binding/Release/node-v59-darwin-x64/fse.node" is installed via remote

> grpc@1.7.3 install /Users/work1/Projects/wrud/wrud/node_modules/google-gax/node_modules/grpc
> node-pre-gyp install --fallback-to-build --library=static_library

sh: node-pre-gyp: command not found
npm WARN firebase-functions@0.8.0 requires a peer of firebase-admin@~5.6.0 but none is installed. You must install peer dependencies yourself.
npm WARN google-nodejs@0.1.0 No description
npm WARN google-nodejs@0.1.0 No repository field.

npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! grpc@1.7.3 install: `node-pre-gyp install --fallback-to-build --library=static_library`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the grpc@1.7.3 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/work1/.npm/_logs/2018-01-17T21_00_21_979Z-debug.log
2018-01-17 18:00:22.277 osascript[75656:3819841] ApplePersistence=NO

Frank Schmid · Answer 5 · Thu Jan 18 2018 05:48:49 GMT+0800 (China Standard Time)

Thanks a lot. It seems that exactly the same happens during execution of the plugin (there might be a bug in the plugin that prevents noticing the npm error and so the parser fails in the end).

I will try if I get some time tomorrow, so that I can prepare exactly the same package.json, so that I can try to find out why npm fails here. (currently the plugin is based entirely on npm internally). I'll let you know if I found out anything.

Gaston Morixe · Answer 6 · Thu Jan 18 2018 23:16:49 GMT+0800 (China Standard Time)

I moved to firebase-cli tool. It seems even with all the dependencies solved, it still not working.

António Lopes · Answer 7 · Sun Jan 21 2018 09:49:58 GMT+0800 (China Standard Time)

Hey, I believe I might be having the same issue. Works fine running locally with $sls offline start but once I try to deploy it I get that same error. Any ideas on how to tackle this issue?
Thanks

➜  SLS_DEBUG=* serverless deploy
Serverless: Load command run
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command create
Serverless: Load command install
Serverless: Load command package
Serverless: Load command deploy
Serverless: Load command deploy:function
Serverless: Load command deploy:list
Serverless: Load command deploy:list:functions
Serverless: Load command invoke
Serverless: Load command invoke:local
Serverless: Load command info
Serverless: Load command logs
Serverless: Load command login
Serverless: Load command logout
Serverless: Load command metrics
Serverless: Load command print
Serverless: Load command remove
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command slstats
Serverless: Load command plugin
Serverless: Load command plugin
Serverless: Load command plugin:install
Serverless: Load command plugin
Serverless: Load command plugin:uninstall
Serverless: Load command plugin
Serverless: Load command plugin:list
Serverless: Load command plugin
Serverless: Load command plugin:search
Serverless: Load command emit
Serverless: Load command config
Serverless: Load command config:credentials
Serverless: Load command rollback
Serverless: Load command rollback:function
Serverless: Load command webpack
Serverless: Load command dynamodb
Serverless: Load command dynamodb:migrate
Serverless: Load command dynamodb:seed
Serverless: Load command dynamodb:start
Serverless: Load command dynamodb:noStart
Serverless: Load command dynamodb:remove
Serverless: Load command dynamodb:install
Serverless: Load command offline
Serverless: Load command offline:start
Serverless: Invoke deploy
Serverless: Invoke package
Serverless: Invoke aws:common:validate
Serverless: Invoke aws:common:cleanupTempDir
Serverless: Invoke webpack:validate
Serverless: Invoke webpack:compile
Serverless: Bundling with Webpack...
Time: 874ms
     Asset     Size  Chunks             Chunk Names
handler.js  6.49 kB       0  [emitted]  handler
   [0] ./handler.js 3.65 kB {0} [built]
   [1] external "jwks-rsa" 42 bytes {0} [not cacheable]
   [2] external "jsonwebtoken" 42 bytes {0} [not cacheable]
   [3] external "babel-polyfill" 42 bytes {0} [not cacheable]
Time: 862ms
     Asset     Size  Chunks             Chunk Names
handler.js  6.49 kB       0  [emitted]  handler
   [0] ./handler.js 3.65 kB {0} [built]
   [1] external "jwks-rsa" 42 bytes {0} [not cacheable]
   [2] external "jsonwebtoken" 42 bytes {0} [not cacheable]
   [3] external "babel-polyfill" 42 bytes {0} [not cacheable]
Time: 860ms
     Asset     Size  Chunks             Chunk Names
handler.js  6.49 kB       0  [emitted]  handler
   [0] ./handler.js 3.65 kB {0} [built]
   [1] external "jwks-rsa" 42 bytes {0} [not cacheable]
   [2] external "jsonwebtoken" 42 bytes {0} [not cacheable]
   [3] external "babel-polyfill" 42 bytes {0} [not cacheable]
Serverless: Invoke webpack:package

  Syntax Error -------------------------------------------

  Unexpected end of JSON input

     For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

  Stack Trace --------------------------------------------

SyntaxError: Unexpected end of JSON input
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at BbPromise.try (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/lib/packExternalModules.js:236:47)
    at tryCatcher (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/util.js:16:23)
    at Function.Promise.attempt.Promise.try (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/method.js:39:29)
    at BbPromise.fromCallback.then.depJson (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/lib/packExternalModules.js:236:35)
    at tryCatcher (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/promise.js:693:18)
    at Promise._fulfill (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/promise.js:638:18)
    at /Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/node_modules/bluebird/js/release/nodeback.js:42:21
    at childProcess.exec (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/lib/packExternalModules.js:233:16)
    at ChildProcess.exithandler (child_process.js:278:5)
    at emitTwo (events.js:125:13)
    at ChildProcess.emit (events.js:213:7)
    at maybeClose (internal/child_process.js:927:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
From previous event:
    at PluginManager.invoke (/Users/alopes/.nvm/versions/node/v8.6.0/lib/node_modules/serverless/lib/classes/PluginManager.js:368:22)
    at PluginManager.spawn (/Users/alopes/.nvm/versions/node/v8.6.0/lib/node_modules/serverless/lib/classes/PluginManager.js:386:17)
    at ServerlessWebpack.BbPromise.bind.then.then.then (/Users/alopes/Code/Clients/xx-redacted-xx/services/job-user-profiles/node_modules/serverless-webpack/index.js:90:51)
    at runCallback (timers.js:781:20)
    at tryOnImmediate (timers.js:743:5)
    at processImmediate [as _immediateCallback] (timers.js:714:5)

  Get Support --------------------------------------------
     Docs:          docs.serverless.com
     Bugs:          github.com/serverless/serverless/issues
     Forums:        forum.serverless.com
     Chat:          gitter.im/serverless/serverless

  Your Environment Information -----------------------------
     OS:                     darwin
     Node Version:           8.6.0
     Serverless Version:     1.25.0

Offline version

✗ sls offline start
Serverless: Bundling with Webpack...
Time: 762ms
     Asset     Size  Chunks             Chunk Names
handler.js  6.49 kB       0  [emitted]  handler
   [0] ./handler.js 3.65 kB {0} [built]
   [1] external "jwks-rsa" 42 bytes {0} [not cacheable]
   [2] external "jsonwebtoken" 42 bytes {0} [not cacheable]
   [3] external "babel-polyfill" 42 bytes {0} [not cacheable]
Serverless: Watching for changes...
Dynamodb Local Started, Visit: http://localhost:8000/shell
Serverless: DynamoDB - created table jobs-user-profile-dev
Serverless: Starting Offline: dev/eu-west-2.

Serverless: Routes for customAuthorizer:
Serverless: (none)

Serverless: Routes for public:
Serverless: GET /public

Serverless: Routes for private:
Serverless: GET /private
Serverless: Configuring Authorization: private customAuthorizer

Serverless: Offline listening on http://localhost:3051

package.json

  "dependencies": {
    "app-root-path": "^2.0.1",
    "auth0": "^2.5.0",
    "babel-polyfill": "^6.26.0",
    "bluebird": "^3.4.6",
    "globule": "^1.2.0",
    "jsonwebtoken": "^7.4.1",
    "jwks-rsa": "^1.2.1",
    "require-dir": "^0.3.2",
    "serverless-dynamodb-local": "^0.2.27",
    "serverless-offline": "^3.16.0",
    "systemjs": "^0.20.19",
    "webpack": "^3.8.1",
    "webpack-node-externals": "^1.6.0"
  },
  "devDependencies": {
    "babel-cli": "^6.26.0",
    "babel-core": "^6.26.0",
    "babel-loader": "^7.1.2",
    "babel-plugin-istanbul": "^4.1.5",
    "babel-plugin-transform-async-to-generator": "^6.24.1",
    "babel-plugin-transform-class-properties": "^6.24.1",
    "babel-preset-es2015": "^6.24.1",
    "babel-preset-stage-0": "^6.24.1",
    "babel-register": "^6.26.0",
    "chai": "^4.1.2",
    "commitizen": "^2.9.6",
    "cross-env": "^5.1.1",
    "cz-conventional-changelog": "^2.1.0",
    "dotenv": "^4.0.0",
    "eslint": "^4.12.0",
    "format-json": "^1.0.3",
    "mocha": "^4.0.1",
    "nyc": "^11.3.0",
    "rimraf": "^2.6.2",
    "semantic-release": "^8.2.0",
    "serverless": "^1.24.1",
    "serverless-dotenv-plugin": "^1.0.1",
    "serverless-webpack": "^4.1.0"
  }

António Lopes · Answer 8 · Sun Jan 21 2018 09:59:42 GMT+0800 (China Standard Time)

Looks like this is caused by using yarn.
By deleting node_modules and then using npm, it works fine.

Frank Schmid · Answer 9 · Sun Jan 21 2018 21:23:43 GMT+0800 (China Standard Time)

Hi @alopes . Thanks for the information and verification.

I think, having #286 done would make these cases very rare, as the plugin can then be configured to the packager that the user normally uses in his environment.

Shawn Dempsey · Answer 10 · Wed Feb 07 2018 07:27:54 GMT+0800 (China Standard Time)

Just got caught by this while using yarn, there is section in the README about yarn that I missed.

Frank Schmid · Answer 11 · Wed Feb 07 2018 18:06:52 GMT+0800 (China Standard Time)

@shwanton The section has been added just a few days ago ;-)

samwiseduzer · Answer 12 · Wed Feb 21 2018 05:23:41 GMT+0800 (China Standard Time)

I"m having this problem without using yarn, with identical results to @alopes

Frank Schmid · Answer 13 · Tue Feb 27 2018 20:49:47 GMT+0800 (China Standard Time)

@samwiseduzer Can you check your npm version and try to delete the npm cache (see npm cache --help)

samwiseduzer · Answer 14 · Wed Feb 28 2018 01:29:02 GMT+0800 (China Standard Time)

Deleting and reinstalling my packages seemed to take care of this

samwiseduzer · Answer 15 · Sat Mar 03 2018 21:24:21 GMT+0800 (China Standard Time)

I ran into this problem again because of unmet peer dependencies (which I discovered after running "npm ls -prod -json -depth=1"). Fixing the peer dependency took care of the issue for me.

Frank Schmid · Answer 16 · Wed Mar 14 2018 20:06:43 GMT+0800 (China Standard Time)

@alopes Yarn support is available in master now and will be part of the next release. Please give it a try and check if it now correctly handles yarn.lock and works with yarn as expected. The README in master contains the exact steps on how to configure it - and adjust the webpack plugin configuration in serverless.yml properly to the new format.

Frank Schmid · Answer 17 · Thu Mar 15 2018 03:08:02 GMT+0800 (China Standard Time)

Update: Yarn support is just released with 5.1.0 💥

Frank Schmid · Answer 18 · Thu Mar 15 2018 03:10:12 GMT+0800 (China Standard Time)

@gastonmorixe Is there any update for this issue on your side. In case you use yarn, you could try plugin version 5.1.0 and configure it to use yarn as packager (if you don't use it, you can try that too ;-) ).

Gaston Morixe · Answer 19 · Thu Mar 15 2018 03:45:37 GMT+0800 (China Standard Time)

Sorry guys, not using it anymore. Just using firebase directly.

On Wednesday, March 14, 2018, Frank Schmid ***@***.***> wrote: @gastonmorixe Is there any update for this issue on your side. In case

you use yarn, you could try plugin version 5.1.0 (if you don't use it, you can try that too ;-) ).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.<

https://ci5.googleusercontent.com/proxy/X6b_ssklWbCx7raD-6I2AdNF9Mf5VFymOM5KtDag6pd1xB_06SfHjtCXvmCKyCCrysuMHRNdf-Lit0TFk0WEMtGafL8bl5ZI9UYeDdAxoQosOlWPx8x0xvUCrmXI62F4WdGHeerFdfeTq0IwCpihf_4Y9RDONg=s0-d-e1-ft#https://github.com/notifications/beacon/AAm5KXfTdEXy3c3nxNJkm92bCQ2VW5_Mks5teWsVgaJpZM4Rhw8D.gif

…

-- Sent from Gmail Mobile

Frank Schmid · Answer 20 · Fri Mar 16 2018 04:56:06 GMT+0800 (China Standard Time)

Ok. I'm closing the issue now. Feel free to continue here if needed 😄

Frank Schmid · Answer 21 · Thu Apr 26 2018 00:43:28 GMT+0800 (China Standard Time)

@gastonmorixe @alopes I finally found the root cause for the original Unexpected end of JSON error. It was an improper error messaging and internally a buffer overflow of the stdio buffers when using child_process.exec. With #373 I switched the packager invocations to use child_process.spawn now. This function uses streams to transfer the stdio stream data to the parent process as opposed to exec which uses fixed buffers (that are broken in some environments).

The fix will be released soon with version 5.1.2. If you have some time, you can check out if the current master branch works for you.