PocketNerdIO / ctran

Recreation of CTRAN.EXE, the Psion OO C preprocessor from the SIBO C SDK.

CTRAN

An attempt to reverse-engineer and recreate CTRAN.EXE, the Psion Object Oriented C preprocessor. In Object Pascal.

Aims

• A drop-in replacement for the original CTRAN.EXE in the Psion SIBO C SDK, replicating both the input switches and the generated files.

• Run on modern OSes and DOS.

• Produce identical files to the original (3.43 or 5.02 — see below).

• Allow multiple include sources.

Targets for v1.00

• ✓ Psion Category file lexer

• ✓ Psion Category file parser/AST builder

• ✓ Command line parser [basic, but works well enough!]

• ✓ Build class dependency tree

• ✓ Generate .ext files

• ✓ Generate .c files

• ✓ Generate .g files

• ✓ Handle sub-category (.CL) files

• ✓ Generate files designed to only work with the SDK

• ✓ Generate skeleton method files

• ✓ Generate .asm (TASM assembly) files

• ❏ Generate .ing (TASM assembly header) files [mostly working; PROPERTY and TYPES currently broken]

• ✓ Generate .lis (class listing) files

What is (or was) CTRAN?

CTRAN.EXE is a pre-processor for the object-oriented part of Psion’s SIBO C SDK, targeting the SIBO/EPOC16 range of portable devices. It takes a .cat category file containing class definitions (methods, properties, etc) and generates the following files:

• A .c file

• A .g header file, to be `#include`d with the class’s methods.

• A .ext file with a summary of the class that can be reused by CTRAN

• A .asm file

• A .ing file (assembly header)

• A .lis file

In regular software development, only the first three are generated.

The original CTRAN.EXE can also generate alternative versions of files for use only with the SDK, plus skeleton source files on which methods can be built.

The problem

My main gripe with the original CTRAN.EXE is that it can only include files from one folder. So, if you want to include the ROOT class in OLIB (the main OO library) and one of your own classes, they both have to be in the same location. This means that you have to store every generated file inside the SDK folder.

I did not like this.

State of the project

What the new CTRAN can currently do

CTRAN in its current form is mostly feature complete (see the list above). Most people only need the ability to generate .C and .G files, which it does. It can be (and has been) used as a drop-in replacement for the original Psion CTRAN.EXE.

In addition, it can act as a Psion SIBO category file validation tool. It extends the original -v (verbose) switch by adding the following options:

• -vt Show token list

• -vl Show lexer process

• -vp Show parser process

• -va Show abstract syntax tables

• -vr Reconstruct the file using the abstract syntax tables

Finally, the -e switch can take multiple paths, separated by a semicolon.

What it can’t do

The main missing feature is the ability to handle sub-category files. This is the current priority and should be ready in the next few weeks.

There is also a small issue when processing classes that only have DEFER`red methods. The original way of handling this is to completely skip these classes, which the new `CTRAN doesn’t do. This shouldn’t impact existing code, but it will be fixed in the future.

TASM file generation, specifically .ING ASM include files, is currently broken. It is currently unable to convert the C in category file PROPERTY and TYPES sections to 8086 assembly. Generated files will include the original C code, commented out. If you want to use .ING files, you will need to manually convert the C to ASM. Fixing this will require some major work, requiring a miniature C compiler to be written and added to the code.

Finally, it can’t generate files for use with Psion OVAL (see below).

Different versions of CTRAN.EXE

There are two main versions of Psion’s CTRAN:

• 3.43: This came with the SIBO C SDK 2.20 and is (probably) the version that most people using the SDK would have.

• 5.02: This came with Psion OVAL 1.21, an IDE based on Visual Basic 6 that could be used to create new controls for use in EPOC16 applications, as well as entire EPOC16 applications. This version of CTRAN generates cleaner code than the previous version, and fixes some (but not all) bugs in 3.43. It adds the ability to generate files for OVAL controls. It also adds a (broken) option for generating code for EPOC32.

The original aim was to reach feature-parity with 3.43. However, if 5.02 doesn’t introduce any breaking changes, then the new CTRAN will aim to mimic the code that it generates. OVAL-related features won’t be in v1.00, but might be added in a later release. As the new version of CTRAN is only designed to be used for making EPOC16 apps, the EPOC32 "feature" will not be added.

Why is this project written in (Object) Pascal?

I want this to run on all modern operating systems, as well as DOS. I didn’t want to write it in C or C++, so I went hunting for another language/compiler.

The only compiler that I could find that was still maintained and stable was Free Pascal. It targets pretty much everything, including Linux, Windows, macOS, Haiku, *BSD…​ and both 16-bit and 32-bit DOS.

Luckily, Object Pascal is just high-level enough, taking away some of the worries of memory management and how to handle dictionaries/trees, while letting me go low-level when I want to.

Unfortunately, some of Free Pascal’s units (e.g. classes, generics.collections) are too big to fit into 64 KB code blocks. Because of this, the new CTRAN will target 32-bit DOS using a DPMI extender. As most people will be running the Psion SIBO C SDK in DOSBox, it shouldn’t affect anyone using CTRAN directly, or if it is run from a TopSpeed project (.pr) file. However, it has been known to clash with Borland Make, as its 16-bit DPMI extender prevents CTRAN from running. In such situations, it is recommended that you switch your project to using GNU Make from the DJGPP project. Alternatively, TopSpeed project files provide a good alternative without needing an external Make app.

The future

At some point the entire SDK needs to be moved away from DOSBox. This will require rewriting every single tool that was included in the SDK, plus the TopSpeed C compiler.

There is a lot of work to do, but rewriting CTRAN as a FOSS project is the first step.