`dart run <package>` can fail if a `dart pub get` operation is running concurrently
andrewkolos opened this issue · comments
As documented in Running a program that's a package, dart run
can be used to invoke a dart program that exists in a package dependency. However, this can fail with an filesystem-related error if dart pub get
is running concurrently (this can be happen frequently, as the Dart VSCode extension watches for changes on pubspec.yaml and will run dart pub get
automatically).
Minimal repro
To demonstrate what I mean, here is a bash script that can repro this issue:
dart create myproject
cd myproject
dart pub add vector_graphics_compiler
dart pub get & dart run vector_graphics_compiler
If this doesn't repro, try running dart pub get & dart run vector_graphics_compiler
once more. There might be a race-condition here. Regardless, here is the kind of error this should print out:
Unhandled exception:
PathNotFoundException: Cannot open file, path = '/Users/andrewkolos/Desktop/deleteme/pubgetrun/.dart_tool/pub/bin/vector_graphics_compiler/tmpXaiYq6/vector_graphics_compiler.dart-3.4.0-190.0.dev.snapshot.incremental.temp' (OS Error: No such file or directory, errno = 2)
#0 _checkForErrorResponse (dart:io/common.dart:55)
dart-lang/sdk#1 _File.open.<anonymous closure> (dart:io/file_impl.dart:381)
<asynchronous suspension>
dart-lang/sdk#2 _FileStreamConsumer.addStream.<anonymous closure> (dart:io/file_impl.dart:205)
<asynchronous suspension>
Compilation did not produce any result. Expected file at `/Users/andrewkolos/Desktop/deleteme/pubgetrun/.dart_tool/pub/bin/vector_graphics_compiler/tmpXaiYq6/vector_graphics_compiler.dart-3.4.0-190.0.dev.snapshot.incremental.temp`
Built vector_graphics_compiler:vector_graphics_compiler.
Cannot rename file to '/Users/andrewkolos/Desktop/deleteme/pubgetrun/.dart_tool/pub/bin/vector_graphics_compiler/vector_graphics_compiler.dart-3.4.0-190.0.dev.snapshot', path = '/Users/andrewkolos/Desktop/deleteme/pubgetrun/.dart_tool/pub/bin/vector_graphics_compiler/tmpXaiYq6/vector_graphics_compiler.dart-3.4.0-190.0.dev.snapshot.incremental.temp' (OS Error: No such file or directory, errno = 2)
PathNotFoundException: Cannot rename file to '/Users/andrewkolos/Desktop/deleteme/pubgetrun/.dart_tool/pub/bin/vector_graphics_compiler/vector_graphics_compiler.dart-3.4.0-190.0.dev.snapshot', path = '/Users/andrewkolos/Desktop/deleteme/pubgetrun/.dart_tool/pub/bin/vector_graphics_compiler/tmpXaiYq6/vector_graphics_compiler.dart-3.4.0-190.0.dev.snapshot.incremental.temp' (OS Error: No such file or directory, errno = 2)
`dart info` output
#### General info
- Dart 3.4.0-190.0.dev (dev) (Thu Feb 29 08:14:33 2024 -0800) on "macos_arm64"
- on macos / Version 14.3.1 (Build 23D60)
- locale is en
#### Project info
- sdk constraint: '>=3.0.0 <4.0.0'
- dependencies: cupertino_icons, flutter, vector_graphics, vector_graphics_compiler
- dev_dependencies: flutter_lints, flutter_test
- elided dependencies: 4
#### Process info
| Memory | CPU | Elapsed time | Command line |
| -----: | ---: | -----------: | ------------------------------------------------------------------------------- |
| 4 MB | 0.0% | 06-03:09:32 | dart devtools --machine --allow-embedding |
| 7 MB | 0.0% | 06-01:39:55 | dart devtools --machine --allow-embedding |
| 6 MB | 0.0% | 06-01:38:20 | dart devtools --machine --allow-embedding |
| 6 MB | 0.0% | 06-03:08:18 | dart devtools --machine --allow-embedding |
| 4 MB | 0.0% | 06-03:07:08 | dart devtools --machine --allow-embedding |
| 4 MB | 0.0% | 06-03:05:38 | dart devtools --machine --allow-embedding |
| 4 MB | 0.0% | 06-03:02:59 | dart devtools --machine --allow-embedding |
| 4 MB | 0.0% | 06-02:59:53 | dart devtools --machine --allow-embedding |
| 6 MB | 0.0% | 06-02:52:31 | dart devtools --machine --allow-embedding |
| 4 MB | 0.0% | 06-02:47:30 | dart devtools --machine --allow-embedding |
| 20 MB | 0.0% | 01:07:20 | dart devtools --machine --allow-embedding --dtd-uri ws:<path>/ |
| 20 MB | 0.0% | 01:07:20 | dart devtools --machine --allow-embedding --dtd-uri ws:<path>/ |
| 5 MB | 0.0% | 01-22:35:28 | dart devtools --no-launch-browser |
| 5 MB | 0.0% | 01-22:35:27 | dart devtools --no-launch-browser |
| 40 MB | 0.0% | 01:07:21 | dart language-server --protocol=lsp --client-id=VS-Code --client-version=3.84.0 |
| 121 MB | 0.0% | 01:07:20 | dart language-server --protocol=lsp --client-id=VS-Code --client-version=3.84.0 |
| 18 MB | 0.0% | 01:07:21 | dart tooling-daemon --machine |
| 18 MB | 0.0% | 01:07:20 | dart tooling-daemon --machine |
| 6 MB | 0.0% | 02-01:34:17 | dart tooling-daemon --machine |
| 34 MB | 0.0% | 01:07:21 | flutter_tools.snapshot daemon |
| 34 MB | 0.0% | 01:07:20 | flutter_tools.snapshot daemon |
Impact/Use case (added March 6)
This was encountered while working on a new feature for Flutter: flutter/flutter#143348. This issue is preventing this new feature from being compatible with hot reload.
The idea of this feature is that Flutter app developers can specify pub packages to be used to transform assets as they are copied to the device.
Here is how a user would could run into this issue:
- In VSCode, the user edits their pubspec.yaml to include a new asset that is configured to have a package transform it:
flutter:
# ...
assets:
- path: my-new-asset.svg # new
transformers:
- package: vector_graphics_compiler
- The user saves their file, VSCode automatically runs a
flutter pub get
- Within the new few seconds, the user initiates a hot reload
- The
flutter
tool invokesdart run vector_graphics_compiler --input=<path to my-new-asset.svg> --output=<path to output assets dir>
. If thepub get
from step 2 has not already completed, then the issue will be reproduced here.
Move to https://github.com/dart-lang/pub if it's entirely a pub
issue, or keep here if it should be treated as an interaction between dart run
and dart pub
.
Transferring since snapshots inside .dart_tool/pub/bin/
are created by pub
still.
For context:
- It's not necessary to run
pub get
afterpub add
--pub add
will resolved dependencies - It's not necessary to run
pub get
beforedart run
--dart run
will runpub get
if necessary.
That said: I think we should aim to avoid a race condition like this.
@sigurdm my two cents is that perhaps pub get
deletes .dart_tool/pub/bin/
or deletes temporary files in there? Thus, renaming it becomes a problem.
For context:
- It's not necessary to run
pub get
afterpub add
--pub add
will resolved dependencies- It's not necessary to run
pub get
beforedart run
--dart run
will runpub get
if necessary.
Right, this is just a minimal repro, the actual issue Andrew's hitting is when saving pubspec.yaml and Dart-Code auto-triggers a flutter pub get on it, and then concurrently he triggers a hot reload, which now will invoke dart run <ASSET_TRANSFORMER>
as part of the build for assets that specify one in their pubspec.yaml: flutter/flutter#143348
We probably delete the .dart_tool/pub/bin/ folder as a cleanup, while we have the temp-folder in there in the other process.
Perhaps it is enough to create the temp-folder at a higher directory level, so that it doesn't get removed in the clean-up process.
Is there any planned work for this issue? This will soon (within X days) be blocking flutter/flutter#143348 barring a hacky workaround
Sorry for stalling on this.
#4170 should fix this.