kevin-hanselman / dud

A lightweight CLI tool for versioning data alongside source code and building data pipelines.

https://kevin-hanselman.github.io/dud/

kevin-hanselman/dud Issues

[Bug] Inaccurate transfer progress if stage contains over 10k files
Closed 2 months ago1
Verify remote authentication before attempting push and fetch
Updated 23 days ago1
`fsspec` filesystem support
Updated 4 months ago3
`dud run` look for previous results
Closed 4 months ago1
Wrong symlinks when fetch / checkout is done from symlinked folder
Closed 6 months ago10
dud run: explain why a stage is being run
Closed 6 months ago
Print out the command before running it
Closed 6 months ago
Multiple stages per file
Closed 6 months ago6
Don't mess with yaml formatting while checksumming
Closed 6 months ago2
Getting started suggests gitignoring symlinks to cached files, why?
Closed 6 months ago1
Add CLI for adding new data to existing stage
Updated 6 months ago1
Deleting files
Closed 6 months ago1
status: show progress when doing byte-level checks
Updated 8 months ago1
add diff command to summarize the differences between two cached artifacts
Updated a year ago1
can't copy - source file is being updated
Closed a year ago1
Add ability to configure rclone parameters
Updated a year ago
Add command to list all committed artifacts (i.e. checksums) for a given stage
Updated a year ago
Support operation on a sub-tree of stages
Updated a year ago
add a `--leaves-only` option to only operate on stages without downstream dependencies
Updated a year ago
Incorrect file type: directory
Updated a year ago2
commit does not work properly in certain situation: 'expected regular file'
Updated a year ago1
How do you add a directory?
Closed a year ago2
checkout: add flag to delete artifacts prior to checkout
Updated a year ago
data is missing from workspace
Closed a year ago1
Support cloud usage patterns
Closed 2 years ago1
Improve data integrity for misbehaving users
Closed 2 years ago4
Support stage definitions as JSON
Closed 2 years ago
status: add flag to skip byte-level equality checks
Updated 2 years ago
add flag to dry-run impactful commands
Updated 2 years ago
add command to move a stage
Updated 2 years ago
empty sub-directories not included in directory status output
Closed 2 years ago
"comitting stage..." can appear after inputs are checksummed
Closed 3 years ago
Improve status speed
Closed 3 years ago1
Improve error message around tracking a stage file
Closed 3 years ago
Invalidate stages with only inputs and no command
Closed 3 years ago
Improve fetch speed on large recursive directories
Closed 3 years ago
document stage file format
Closed 4 years ago
add pull command
Closed 4 years ago
add more context to errors when loading index
Closed 4 years ago
throw a warning or error when generating a stage with no inputs and no outputs
Closed 4 years ago
Benchmarks page uses dev version of Dud
Closed 4 years ago
Parallelize checkout
Closed 4 years ago
Add CONTRIBUTING.md
Closed 4 years ago
Add benchmarks to docs
Closed 4 years ago
Automate Github releases
Closed 4 years ago
`stage gen -w` is always resolved relative to the project root
Closed 4 years ago
benchmark datasets slow down go commands
Closed 4 years ago
Index.Run doesn't check Stage locked status
Closed 4 years ago
dud help errors out when not in project
Closed 4 years ago
prevent Artifacts being both a dependency and an output of the same Stage
Closed 4 years ago