VSCode extension fails in large workspace

Question

VSCode extension fails in large workspace

jnobile opened this issue 2 years ago · comments

When loading a large workspace (260+ .tf files) in order to estimate a subset of it, breakdown fails with stdout maxBuffer length exceeded. Ideally, I could breakdown just the main.tf in the workspace I'm interested in evaluating and lazily load dependent modules, but it seems infracost is trying to load everything proactively.

debug: initializing workspace
debug: running Infracost in project: c:\Code\terraform-configuration
debug: running Infracost cmd cmd /C "set INFRACOST_CLI_PLATFORM=vscode && infracost breakdown --path "c:\Code\terraform-configuration" --format json --log-level info"
error: Infracost cmd error trace RangeError [ERR_CHILD_PROCESS_STDIO_MAXBUFFER]: stdout maxBuffer length exceeded
debug: providing codelens for file /c:/Code/terraform-configuration/.../main.tf

Hugo Rut · Answer 1 · Fri Aug 12 2022 16:05:04 GMT+0800 (China Standard Time)

Thanks @jnobile,

Seems like something we can fix by configuring the max output settings with node exec. Unfortunately, lazily loading modules isn't really something that we can do at the moment because to find out which modules to load, we need to evaluate the whole context of the module. i.e. module inputs or counts can depend on other blocks or even other module outputs, which need to be evaluated. A simple example is as follows:

Within the root module, web.tf has module call web_app that depends on a variable environment
which is then used in the module to build several resource instances
which is then used to build an output instance_count
This output is then used in the root module main.tf resource eip count attribute
Infracost shows you a price based on this count

Alex-Vinet · Answer 2 · Wed Sep 14 2022 20:13:31 GMT+0800 (China Standard Time)

I'm having the same issue.

Hugo Rut · Answer 3 · Mon Dec 05 2022 23:03:59 GMT+0800 (China Standard Time)

Hey @jnobile & @Alex-Vinet, I've released a potential fix for this in the 0.2.1 version of the extension. Let me know if this resolves your workspace issues.

John Nobile · Answer 4 · Tue Dec 06 2022 02:52:31 GMT+0800 (China Standard Time)

Good news: new error
Bad news: no idea what file is the problem

debug: setting context infracost:loggedIn to true
debug: initializing workspace
debug: running Infracost in project: c:\Code\terraform-configuration
debug: running Infracost breakdown
error: Infracost cmd error trace SyntaxError: Unexpected end of JSON input

More context: I'm able to run a breakdown from the command line so the Unexpected end of JSON input in vscode is probably a remaining bug in the extension. The output from the command line is 1.3MB well formed json. I have infracost-debug-report.json but it is 1.8 GB, so you'll have to let me know if there's something interesting you'd like to see in there... but this was a success from the command line.

cc @hugorut

Hugo Rut · Answer 5 · Tue Dec 06 2022 16:41:55 GMT+0800 (China Standard Time)

ok thanks @jnobile, could I ask what windows machine you are running on? I'll try and boot up a VM and see if I can replicate it. I'll ping you if I need the debug report, but as you say, I don't think the problem is with the CLI here.

A further question - if you open the extension in a smaller project on your machine (https://github.com/infracost/infracost/tree/master/examples/terraform as an example) - do you still get this error?

John Nobile · Answer 6 · Wed Dec 07 2022 01:57:56 GMT+0800 (China Standard Time)

The extension runs fine in the linked example workspace; I can see monthly costs in main.tf.

Here's select machine specs:

Processor	Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz   2.40 GHz
Installed RAM	32.0 GB (31.7 GB usable)
System type	64-bit operating system, x64-based processor

Edition		Windows 11 Pro
Version		22H2
Installed on	‎11/‎11/‎2022
OS build	22621.819

Hugo Rut · Answer 7 · Wed Dec 07 2022 20:50:29 GMT+0800 (China Standard Time)

ok thanks @jnobile i'll have a play around with this and see if I can replicate localy ❤️

Hugo Rut · Answer 8 · Sun Dec 11 2022 17:00:30 GMT+0800 (China Standard Time)

hey @jnobile, I think I've got a fix for this in PR now, but it would be great if you test it on your machine. Here's the steps to install the build:

Head over to https://github.com/infracost/vscode-infracost/actions/runs/3668220937 and download the win-x64 asset
After downloading unzip the asset - this should produce a file called infracost-windows-amd64-0.2.3.vsix
Head over to vs code and uninstall the infracost extension
Then in the extensions browser click the three dots on the top right corner and select install from vsix
Select the vsix file from the downloaded asset this will prompt a vs code Refresh
Test this out on the workspace you were having problems with

John Nobile · Answer 9 · Mon Dec 12 2022 08:18:32 GMT+0800 (China Standard Time)

@hugorut the pr build version works on my machine. :) Thanks for the fix! Do you know when I can expect a release build with the fix?

Hugo Rut · Answer 10 · Mon Dec 12 2022 16:47:04 GMT+0800 (China Standard Time)

@jnobile I'll release today

Hugo Rut · Answer 11 · Mon Dec 12 2022 20:45:00 GMT+0800 (China Standard Time)

hey @jnobile this has been released in the 0.2.3 version, thanks so much for your help tracking this down 😍