walmartlabs / LinearGenerator

Reworked data generator for LinearRoad streaming benchmark that no longer needs mitsim or any database.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool


NOTICE:

This repository has been archived and is not supported.

No Maintenance Intended


NOTICE: SUPPORT FOR THIS PROJECT HAS ENDED

This projected was owned and maintained by Walmart. This project has reached its end of life and Walmart no longer supports this project.

We will no longer be monitoring the issues for this project or reviewing pull requests. You are free to continue using this project under the license terms or forks of this project at your own risk. This project is no longer subject to Walmart's bug bounty program or other security monitoring.

Actions you can take

We recommend you take the following action:

  • Review any configuration files used for build automation and make appropriate updates to remove or replace this project
  • Notify other members of your team and/or organization of this change
  • Notify your security team to help you evaluate alternative options

Forking and transition of ownership

For security reasons, Walmart does not transfer the ownership of our primary repos on Github or other platforms to other individuals/organizations. Further, we do not transfer ownership of packages for public package management systems.

If you would like to fork this package and continue development, you should choose a new name for the project and create your own packages, build automation, etc.

Please review the licensing terms of this project, which continue to be in effect even after decommission.

LinearGenerator

Reworked data generator for LinearRoad streaming benchmark that no longer needs mitsim.

To use the new generator compile the files and run: java com.walmart.linearroad.generator.LinearGen [-o <output file>] [-x <number of xways>] [-m <dummy value to activate multi-threading>]

The default settings in Environment.java create:

  • ~1 GB per xway files with
  • ~25M records per file
  • ~370K unique cars per xway

Each xway takes roughly 5m30s to create.

About

Reworked data generator for LinearRoad streaming benchmark that no longer needs mitsim or any database.


Languages

Language:Java 100.0%