databrickslabs / dbldatagen

Generate relevant synthetic data quickly for your projects. The Databricks Labs synthetic data generator (aka `dbldatagen`) may be used to generate large simulated / synthetic data sets for test, POCs, and other uses in Databricks environments including in Delta Live Tables pipelines

https://databrickslabs.github.io/dbldatagen

databrickslabs/dbldatagen Issues

DateRange Timestamp conversion on windows fails with timestamps close to EPOCH
Updated 2 months ago3
Support generation of test data from spark schema with nested structures
Updated 3 months ago
Support use of Dbldatagen from Databricks Serverless environment
Updated 3 months ago
Doesn't work with Databricks Serverless Instances.
Updated 3 months ago2
Upgrade this lib to be compatible with Spark Connect / DB Connect
Updated 3 months ago1
Support use of non string columns in Faker integration
Updated 3 months ago
Support integration with Mimesis library
Updated 3 months ago
Add support for constraints
Closed 3 months ago
Compatibility Issue: dbldatagen DataAnalyzer Not Accepting Spark Connect DataFrame
Updated 5 months ago4
License issue
Closed 5 months ago1
AnalysisException/DATATYPE_MISMATCH error when generating summary dataframe from sources that have a column named "summary"
Closed 5 months ago2
Github pages for docs fails to load
Updated 5 months ago2
Copy options picks up extra % in pip install command code snippet.
Closed 6 months ago
Random number generation not generating random data unless `maxValue` is specified or is implied from other options
Updated 6 months ago
Create some pre-canned standard datasets like the h20 groupby dataset
Updated 6 months ago2
Using Databricks Labs Data Generator on Databricks Runtime 14.x
Closed 7 months ago
DBLDatagen broken on 14.3LTS Shared Cluster
Closed 7 months ago2
Document recommended Databricks runtime releases for Unity Catalog
Closed a year ago
How to set template and min,max value for a nested schema attribute
Updated a year ago1
Reference Previous Rows
Updated a year ago3
Add support for structured columns
Closed a year ago1
ArrayType(StringType()) columns result in Null column, doesn't take values from `values` argument
Closed a year ago2
DblDatagenerator causes global logger to issue messages twice in some circumstances
Closed 2 years ago9
Improve generation of data generation specification from existing schema or dataframe
Closed 2 years ago2
When generating array valued column generation spec, use different random seed for each element
Closed 2 years ago
Support range of values for generating array valued columns
Closed 2 years ago
Distribution functions (and perhaps others) not compatible with Databricks UC Clusters operating in `shared` mode
Closed 2 years ago4
Allow precision and scale options to apply to any numeric type
Updated 2 years ago
Generate data based on estimated Delta table size
Updated 2 years ago2
Setuptools need to include required packages for working with the library locally (outside of the databricks environment)
Closed 2 years ago2
Add enhanced options and documentation for streaming data generation
Updated 2 years ago
Improve build ordering dependencies
Closed 2 years ago
Changed interim build labelling to comply with PEP 440
Closed 2 years ago1
Generating text with baseColumn not consistent
Closed 2 years ago6
Cast exception when nested schema passed as input
Closed 2 years ago6
Error in creating ArrayType cols
Closed 2 years ago3
Problem with column which is named "ID"
Closed 2 years ago3
Improve template text generator
Closed 2 years ago
Build fails due to changes to build runner 'ubuntu-latest'
Closed 2 years ago
Enhancement: Generate standard data sets
Closed 2 years ago4
Improve speed and coverage of unit tests
Closed 2 years ago1
Typo in readme
Closed 2 years ago
Add support for Spark 3.1.2
Closed 2 years ago3
time string for interval should support both `seconds` and `second`
Closed 2 years ago
Update docs to include new install options
Closed 2 years ago1
Uninstalling and reinstalling wheel on cluster running DBR 8.3 may fail
Closed 2 years ago2
Document %pip based install
Closed 3 years ago
Expose __version__ attribute
Closed 3 years ago
Docuementation issues
Closed 3 years ago
Unable to create columns containing lists of strings or integers
Closed 3 years ago1