Providing git-friendly file format support to VAST
Report a defect
|
Request feature
Tonel is a text-based file format used to store Smalltalk source code on disk, and was designed from scratch to be version control system (VCS) friendly. It is widely accepted by the Smalltalk community at this point, so Instantiations saw an opportunity to enhance ENVY (the standard VCS in VAST) by adding Tonel support for greater functionality and flexibility.
Development on Tonel Tools for VAST began in September 2019, and official support started when it shipped as a feature within VAST Platform 2021 (10.0.0) in March 2021.
Open source projects using Tonel for VAST can be found in the VAST Community Hub or in Instantiations' GitHub respositories.
- The code is licensed under MIT.
- The documentation is licensed under CC BY-SA 4.0.
We continue to improve the Tonel support by fixing bugs and adding new features to it, if you want to use the latest version you can do it by downloading the latest version in this repository into the latest version of VAST.
Tonel comes as a Feature inside the product. That means it can be easily installed from the Transcript -> Load/Unload Features.... You will find two features you can install: ST: Tonel Support and ST:Tonel Support, Testing.
Since there were significant changes in the codebase, the update must be split in sub steps, and do intermediate updates.
- Load Tonel VAST Features using
Transcript->Load/Unload Features.... There you will find two features you can install:ST: Tonel SupportandST:Tonel Support, Testing. - Clone this repository locally using your preferred git tool.
- Checkout the commit tagged as
v1.4.2 - Open Configurations Maps Browser
- Right click in the
All nameslist, selectImport->Load Configuration Maps from Tonel repository... - Select the
.projectfile of the directory you cloned in the step above - Select
ENVY/Image Toneland/orTest ENVY/Image Tonel
You might get an error during the update, since Tonel is modifying itself as it gets loaded. If that happens, retry the import.
- Checkout the latest commit (master branch)
- Open Application Manager
- Right click in the Applications list, select
Import/Export->Load Applications from Tonel packages... - Select the
.projectfile of the directory you cloned in the step above - Select
TonelSystemExtensionsand import it - Open Configurations Maps Browser
- Right click in the
All nameslist, selectImport->Load Configuration Maps from Tonel repository... - Select the
.projectfile of the directory you cloned in the step above - Select
ENVY/Image Toneland/orTest ENVY/Image Tonel
- Clone this repository locally
- Download VAST from Instantiations website.
- If you're using 9.2.1 checkout the v1.1.0 tag
- If you're using 9.2.2 checkout the v1.2.0 tag
- From the Configuration Map Browser, do an
Import->All Most Recent Versions...-> and select theenvy/Tonel.datlocated in the root folder of your local repository clone. - Select the map
Tonel(and optionallyTest Tonelif you want to run its SUnit tests) and do aLoad With Required Mapsthe latest version of it. - Optionally run the SUnit tests included in the map
Test Tonelto ensure correct installation. One easy way is to right-click on theTest Tonelmap name in the Name pane (as opposed to version pane ) and then selectTest Loaded Applications. - Explore the documentation.
You can load individual Tonel packages as Applications or export VAST Applications as Tonel packages via Application Manager's Import/Export.
When loading apps using this GUI the loader will attempt to detect whether the files are in a Git repository, and if a git repository is detected it will configure itself to use git versioning, otherwise it will use the default settings (or leave all editions open without version).
You can also export and load Configuration Maps together with their applications from the Configuration Maps Browser.
Select the Configuration Maps you want to export in the names list, and then open the contextual menu and choose Export, and then Export Configuration Maps to Tonel.
This will ask you which version of the config maps you want to export (only one version per map is allowed), where do you want to store it, as well as a few other settings.
To load a Configuration Map in a Tonel repository into VAST, you have to choose the Import option in the names list, and then Load Configuration Maps from Tonel....
Once you select the available Configuration Maps, they will be loaded together with their required maps. If a required map is within the Tonel repository, then this map version will take precedence over the version specified.
As with the GUI options, you can also export independent Applications or whole Configuration Maps to Tonel.
"Exporting to Tonel"
TonelWriter new
clearSourcesDirectory; "deletes everything in the target directory"
writeProjectIncluding: (Array with: MyAppCore)
into: (CfsPath named: 'my-tonel-demos')."Loading from Tonel"
(TonelLoader readFromPath: (CfsPath named: 'my-tonel-demos'))
loadApplicationNamed: 'MyAppCore'.
"or you can load by Tonel package name"
(TonelLoader readFromPath: (CfsPath named: 'my-tonel-demos'))
loadApplicationsForPackagesNamed: #('MyApp-Core' 'MyApp-Tests')."Exporting to Tonel"
TonelWriter new
addLatestConfigurationMapNamed: 'My Tonel Demo';
addLatestConfigurationMapNamed: 'My Tonel Demo Tests';
addApplicationsFromConfigurationMaps;
writeProjectInto: (CfsPath named: 'my-tonel-demos')."Loading from Tonel"
(TonelLoader readFromPath: (CfsPath named: 'my-tonel-demos')) loadAllMapsWithRequiredMaps.You can specify other options like versioning, prerequisite resolution, and others using the different strategies for each case, read the strategies documentation to learn more about them.
Since Tonel wasn't designed with VAST use in mind, some features are not supported by the spec and we needed to extend it to support them.
You can read document of VAST specific features to know more about them.
If you want to make your code fully compatible and interoperable with other Smalltalk dialects there is a specific document with the recommendations for compatibility.
VAST's way of declaring and/or initializing Shared Pools is different from other Smalltalk dialects, and to explain how this work with Tonel, there is a specific document explaining how VAST Tonel handles shared pools.
Read the versioning, prerequisites and base editions strategies documentation to learn how to configure the loader to work interactively, unattended, read the version from a git repository, etc.
Although dependency management is outside of the scope of Tonel, VAST Platform's Tonel tools provide a way to write them to disk and later read and load them.
Read the Configuration Maps documentation to learn how to use it.
There are four main building blocks that work in layers to support Tonel in VAST Platform:
Parses the Tonel specific files following the same rules as the canonical version.
Relies on the parser to create first-class objects representing each abstraction from Tonel. E.g. Package, Class, Method Definition, Method Extension
Uses the reader to compile classes, methods, extensions, etc. and then create the required editions and versions in the ENVY Library.
Writes to disk, in a Tonel compatible format (plus the VAST specific features described below) the Applications, SubApplications, Classes, Shared Pools, Methods, Extensions, etc.
If the Application has a class side #tonelPackageName selector it will then honor it when creating the package name.
The Reader, Parser and Writer tests can be run out of the box once loaded, but the Loader tests need a little setup in order to run.
Once you clone this repository you have to configure TonelLoaderTest to point to the tests directory within the repository, since it contains the repositories used for testing the loader.
TonelLoaderTest testRepositoriesPath: (CfsPath named: 'repo-path\tests')The Loader tests will attempt to leave the ENVY Library (aka "the library") in the same logical state as it was before running them by means of reloading the applications that were loaded before starting and purging newly created editions, but since this can the access to the library isn't atomic (as in a single enclosing transaction) there could be leftovers if something goes wrong. So it isn't adviced to run the tests against your production library.
The versioning of this particular project is somewhat special because there are several versions going on in parallel: the version that comes with the VAST release, the config map version of each commit and and release version of Github releases. Because of that, the versioning strategy has been changing since the beginning of the project, looking for the optimal one.
Starting with VAST 10.0.0 we decided to adopt a versioning strategy that might eliminate the multiple version naming, and help in mapping the different versions together.
Guidelines:
- The
Tonel Supportfeature (and/or its Test variant) must be loaded before loading this project into a VAST image - The
ENVY/Image Tonelversion that comes as a feature in a VAST release (currently v 10.0.2) is considered the "trunk" - The code from this repository is loaded on top of the configurations maps loaded in the previous points
- When versioning configuration maps both the "trunk" version identifier and a sequential version will be used as the version name, e.g.
v 10.0.0 [v1.3.6]. - Once a new Github release is published, the
v1.3.6part above will likely be re-published asv1.4.0(the version of the release). - When there is a VAST release that integrates one of the Github releases as a built-in feature, it will rename
v 10.0.1 [v1.4.0]to something likev 10.0.2 [543](being the543the VAST build number), but both versions will likely share the same internal timestamp.
- The webinar "Getting Started with Tonel Tools In VAST", which discussed how to use the new Tonel Tools features with step-by-step instructions and practical examples.
See the roadmap document for further details.
- Mercap Software for their first draft on the Tonel writer
- Github repository layout was generated with Ba-St Github-setup project.
Check the Contribution Guidelines