Interest in non-async drivers in libtock-rs

[Guillaume E]

In OpenSK, we've successfully used the pre-async version of libtock-rs drivers. Although there have been substantial changes in libtock-rs since the introduction of the new async runtime, we've migrated the drivers to work on top of the current libtock-rs/core.

These libtock-drivers are derived from the pre-async libtock-rs, now adapted to work with changes that have affected libtock-rs/core (e.g. the new TockResult, the callback subscription/consumer APIs).

Is there any interest in having these forked drivers back in libtock-rs, i.e. having support for both async and non-async drivers in mainline libtock-rs? I think this would be easier to maintain by avoiding duplicate work, and would benefit users of libtock-rs for whom the async runtime doesn't fit (e.g. is too bulky).

Also, how would this interplay with "libtock-platform" (#217)?

[torfmaster]

The support for synchronuous subscriptions is still there, or am I missing something?

[Guillaume E]

It seems that some drivers have migrated to the async style, while others are indeed still synchronous, which maybe drove the confusion.

For example, the RNG driver only has an async API:

libtock-rs/src/rng.rs

Lines 26 to 41 in 05ac22a

 pub async fn fill_buffer(&mut self, buf: &mut [u8]) -> TockResult<()> { 
     let buf_len = buf.len(); 
     let shared_memory = syscalls::allow(DRIVER_NUMBER, allow_nr::SHARE_BUFFER, buf)?; 
     let is_filled = Cell::new(false); 
     let mut is_filled_alarm = || is_filled.set(true); 
     let subscription = syscalls::subscribe::<Identity0Consumer, _>( 
         DRIVER_NUMBER, 
         subscribe_nr::BUFFER_FILLED, 
         &mut is_filled_alarm, 
     )?; 
     syscalls::command(DRIVER_NUMBER, command_nr::REQUEST_RNG, buf_len, 0)?; 
     futures::wait_until(|| is_filled.get()).await; 
     mem::drop(subscription); 
     mem::drop(shared_memory); 
     Ok(()) 
 } 

whereas the buttons driver is only synchronous:

libtock-rs/src/buttons.rs

Lines 65 to 75 in 05ac22a

 pub fn subscribe<CB: Fn(usize, ButtonState)>( 
     &self, 
     callback: &'a mut CB, 
 ) -> TockResult<CallbackSubscription> { 
     syscalls::subscribe::<ButtonsEventConsumer, _>( 
         DRIVER_NUMBER, 
         subscribe_nr::SUBSCRIBE_CALLBACK, 
         callback, 
     ) 
     .map_err(Into::into) 
 } 

Also, a main blocker for building applications without async runtime is that yieldk_for was removed, as mentioned in the CHANGELOG.

So I guess my question is more:

• Should there be separate folders for sync/ drivers and async/ drivers?
• Should all drivers implement both styles (in the same file) - or be able to implement both styles depending on how much resources contributors are willing to provide?
• Likewise, should there be separate folders for examples/sync/ and examples/async/? (I assume there are no more synchronous examples now that yieldk_for has been removed)

[Johnathan Van Why]

Also, how would this interplay with "libtock-platform" (#217)?

For libtock_platform, I intend for all drivers to be written in an asynchronous manner. I am developing traits to represent the asynchronous implementation. There are a lot of details and complications in flux (such as support for calling through zero-sized pointer types and mitigations against accidental recursion), but conceptually the traits are as follows:

// AsyncCall is implemented by types that provide asynchronous APIs.
pub trait AsyncCall<Request, SyncResponse> {
    // Starts an asynchronous operation. Callbacks will be delivered to a `Callback`.
    // Simple asynchronous operations ("do a thing and tell me when it is done") will
    // receive one callback, while more complex operations ("start a recurring timer")
    // may cause multiple callbacks. Therefore, implementers of `AsyncCall` must
    // document when they will make callbacks after `start` has been called.
    fn start(&self, request: Request) -> SyncResponse;
}

// Callback is implemented by clients of asynchronous APIs.
pub trait Callback<AsyncResponse> {
    // Called when an asynchronous operation completes.
    fn callback(&self, response: AsyncResponse);
}

Based on these traits, we can then develop a generic synchronous adapter to the asynchronous APIs:

pub struct SyncAdapter<Request, SyncResponse, AsyncResponse, Driver: AsyncCall<Request, SyncResponse>> {
   ...
}

impl<...> Callback<AsyncResponse> for SyncAdapter<...> { ... }

impl<...> SyncAdapter<...> {
    // Calls into the driver to start the operation. This will not block.
    pub fn start(&self, request: Request) -> SyncResponse { ... }

    // Waits until a callback is received from the driver.
    pub fn wait(&self) -> AsyncResponse { ... }
}

It may be possible to provide a single method that does start and wait if SyncResponse is a Result, which would be more convenient for working with simple asynchronous APIs.

Using this design, we can avoid maintaining parallel async + sync driver implementations. We may want to provide nicer wrappers around SyncAdapter for each API, but it's unclear yet whether doing so is feasible (dependency injection and interoperability between async and sync code make this a nontrivial problem).

[torfmaster]

I guess the a sync version of the rng can be easily restored. Moreover, yieldk_for is no syscall on its own but is just yield + waiting for a condition (use yield + a cell). While yield is now declared unsafe for good reasons it can still be used.

[Pat Pannuto]

FWIW, this is basically the libtock-c approach. It provides either raw yield and APIs built on top (async) or yield_for and APIs built on top (sync).

Things tend to light on fire if people try to mix them (as sync APIs assume that nothing (but other sync routine flag-setting) happens while they wait. In Rust, it'd probably be nicer to expose these as more formally mutually exclusive if you go that route.

[Johnathan Van Why]

Things tend to light on fire if people try to mix them (as sync APIs assume that nothing (but other sync routine flag-setting) happens while they wait. In Rust, it'd probably be nicer to expose these as more formally mutually exclusive if you go that route.

Ultimately, I think it would be good to allow mixing sync and async code in a single app, both to improve library reusability (you can write an async library and use it in a sync app), and to allow migrations between API styles (e.g. if an app that was originally synchronous suddenly needs to become asynchronous). There need to be clear boundaries, however. Async callbacks should not call into synchronous code, as doing so is going to cause surprising reentrancy.

Ultimately, in libtock_platform, main() will be synchronous. Synchronous routines will be able to call into other synchronous routines without problems, as well as start or wait for asynchronous operations. The boundaries will need to be clear. We can probably use some zero-sized types to help with this. For example, functions that can only be called in synchronous context can take a SyncContext type to make programmers second-guess calling them from async code. In my libtock_platform prototype, asynchronous callbacks already require a CallbackContext instance that can only be obtained by callback functions.

I'll have to think a bit more about how Futures work in that model.

[Guillaume E]

Ok, then let's wait for some progress on the libtock_platform prototype.

[Hudson Ayers]

Given that all of the libtock-rs drivers are currently being rewritten as part of the transition to Tock 2.0, and that libtock-rs does not currently include futures support, I think that this issue is no longer relevant.