Potential Messaging System Refactor

[Eric Edlund]

Problems this would solve:

• Messages aren't type safe
• Senders can make up arbitrary addresses #3
• There's no enforcement of bijection with mailboxes and modules #2
• Possible to provide garbage message content
• Receivers have to watch out for messages they can't receive
• Mailboxes have arbitrary size constraints
• Missing formal analysis

I think all of these problems could be fixed with better use information hiding and encapsulation. I may have some misunderstandings about the capabilities of Ada/Spark in OO programming. I think that this design should be allowed with the newest pointer rules in SPARK.

Changes to Modules

We force the "message" component of modules to inherit from an abstract MessageComponent class which provides a MailBox object for the module with custom instantiation for that module. Every mailbox will then be naturally associated with one module and meet the module's requirements.

Stricter Message Class

Create an abstract class Message with a sender and receiver address mutable to the user and generated in a safe way.

[Eric Edlund]

@pchapin Is something in this ballpark feasible?

[Peter Chapin]

Here are a few initial comments...

• I think it's great to consider alternate design possibilities. Especially given that SPARK is an evolving technology, we may be able to come up with something significantly better than the current system using SPARK's newer features.

• I feel like the diagram would benefit from a more detailed prose description of the design. Diagrams make good supplementary material to prose, but often do a lackluster job at replacing prose. I suggest that you commit your diagram and notes to the CubedOS repository, perhaps in a new folder under the doc folder. That way we can mutually evolve the ideas and explore their ramifications in an environment that has more permanence and long-term visibility than as an issue on the issues list. Even if we don't pursue changes like this, I would like to have them documented along with reasons why not so future generations of developers don't have to rethink everything from scratch.

• I'm a little unclear about what a MessageComponent is. Do you mean a component of a message, or do you mean a component of a module? It seems like MessageComponents have mailboxes, which suggests the latter interpretation, but it would be helpful (to me) if that was clarified.

• You talk about a module extending MessageComponent to a type that is specialized for that module. What would these specialized MessageComponent objects do, exactly?

• SPARK only allows pointers to dynamically allocated objects (and I think dynamic allocation of memory might be disallowed by Ravenscar, which we need for tasking... but also check the Jorvik profile). So I assume you are visualizing that mailboxes now hold access values (aka pointers) to messages rather than the messages themselves. But doesn't that still open up the possibility that a mailbox might get unknown or invalid message subtype? Or are you suggesting that each module define it's own abstract message type for use by that module alone?

• Because of CubedOS's distributed nature, the problem of serializing and de-serializing messages remains. This means the machinery we currently have would remain, in some form, although perhaps it could be more hidden from CubedOS developers. That would be good.

• I wonder if it would be better to start with something more incremental. The type-safety issue is mitigated somewhat by Merc and could be seen as a lesser priority to fix. However, it would be nice if a module could control the size of its mailbox and tune the size of the messages that could fit into its mailbox to match the longest message it would ever legitimately receive (rather than the longest message any module in the domain might receive, as is the case now). This would entail dynamically allocating message arrays inside the mailboxes. While this won't solve everything, it would give us experience with adding pointers and dynamic allocation to the system in a smallish way. It could be that we can't even do that (due to Ravenscar, etc.), but it would at least be a relatively simple change to make experimentally.

[Eric Edlund]

Should we first discuss the proposal here while we figure out what subset of it may be useful and make an action plan, then I will commit a summary of our discussions to the repo? Seeing as it's just you and me for the time being, I'd rather file fewer merge requests.

Also, do we have licenses to any real UML software? I'm using an open source tool Gaphor which produces Gaphor specific files that ( I think) only it can read. I'd rather use something standard we all have.

[Eric Edlund]

Some prose:

1. MessageComponent

Every module currently has a child Module.Message package. In the proposed change, that package would contain an extended MessageComponent object instance. Every module has 1 MessageComponent which abstracts some details of the messaging system.

Specifically, on construction, you supply info about the mailbox your module needs (like message count, message size, the module's domain, etc.), then the object would create and store the configured mailbox and do whatever setup we deem necessary without the implementing module knowing. This allows us to have mailboxes tailored to each module, not to a domain. The second reason I want this is so that we can get more safety using OO methods in part 2.

2. Message Objects

My original thought was to force messages to be constructed by Mailbox objects by referencing other mailboxes. If module A wanted to send something to module B, module B might first define a custom message type:

-- module_b-API.ads

package module_b.API is

    type Custom_Message is private;
    function Make_Custom_Message ( Message_Base: Message, ... contents...) return Custom_Message;
    --  getter/setter functions for custom message...
    ... to and from byte array overrides...

private

    type Custom_Message is new Message with
        record
             ...custom data...
        end record;

end;

Module B's Mailbox would include info saying that it is allowed to receive messages of type Custom_Message.

Then Module A would send the message:

-- module_a.adb
with module_b.Public_Mail_Box;

procedure Send_To_B(info) is
     Message_Base : Message
     Full_Message : Custom_Message
begin
    -- Module A code has access to the full module_a mailbox 
    Message_Base := module_a.Mailbox   
                      .Build_Message( module_b.Public_Mailbox )

    -- Now Message_Base has the source and dest addresses

    -- Create the full message using it
    Full_Message := module_a.API.Make_Custom_Message ( Message_Base, ... other info ...);
    
    -- Send the message
    module_a.Mailbox.Send (Full_Message); 

end;

By forcing modules to use their private mailbox to send messages, they can't lie about where a message is coming from, we can check that the receiving mailbox is allowed to receive the message that we send, and modules can't directly access message content.

But thinking about distributed systems, I'm not sure we could still get that level of clean? I would still prefer to use OO messages because calling Message_Type_Encode() makes the user think about encoding/decoding, something the messaging system could handle behind the scenes. As for building messages with mailboxes, I think we could still do a version of this. There could be two functions:

• one for intra-system messages where you give the module mailbox as the parameter
• another for external or internal communications where the destination domain and module ids are given directly

This version still has the added benefit of guaranteed correct source addresses on messages.

3. Domain objects and message manager singleton

-- fill this in once I understand Sevan's implementation --

[Eric Edlund]

#43

I ended up doing a more reserved version of this.