GSConnect / gnome-shell-extension-gsconnect

Describe the bug

When playing a file on computer, pressing the rewind/forward button on mobile app should go 5 seconds back/forward.
Instead, pressing the rewind button cause the file to restart and pressing the forward one makes the player stop, or play the next file.

I'm trying to use it with mpv and with kdeconnect was working fine, so I guess it is something related to gsconnect.

Steps to reproduce

Play a file on computer
Press rewind/forward button on KDE Connect mobile

Expected behavior

The player going 5 seconds back/forward.

GSConnect version

50

Installed from

OS package manager

GNOME Shell version

42.9

Linux distribution/release

Pop!_OS 22.04 LTS

Paired device(s)

Moto G32

KDE Connect app version

1.29.0

Plugin(s)

No response

Support log

No response

Screenshots

No response

Notes

No response

Sorry, but we don't support old versions of GNOME Shell.

Please reach out to your distribution for support, or even better, ask if they would consider sponsoring a developer with some of the income they get for LTS subscriptions.

Hmmm. With GSConnect 56, controlling VLC, I can confirm that pressing the "Rewind" button restarts the file from the beginning, same as "Skip Backwards".

Pressing "Fast Forward" seems to skip forward approximately 1 second, not 5, but otherwise works as expected.

(There is, of course, also the position slider to perform seeking within the current file. But the rewind/ff buttons should ideally work properly as well.)

AFAICT, Rewind sends:

{
    "id": 1712175014041,
    "type": "kdeconnect.mpris.request",
    "body": {
        "player": "VLC media player",
        "Seek": -10000000
    }
}

and FF sends "Seek": 10000000. (Which actually should be a 10-second jump, by default. Tho that's configurable in KDEConnect Android's plugin settings.)

It looks like we're interpreting both of those wrong.

Playing with VLC's MPRIS interface in d-spy, sending it Seek((10000000,)) skips 10 seconds forward, and Seek((-10000000,)) skips 10 seconds backwards. So if nothing else, the fact that we're multiplying/dividing offset and position values appears to be wrong:

gnome-shell-extension-gsconnect/src/service/plugins/mpris.js

Lines 244 to 250 in 9d27463

    
           if (packet.body.hasOwnProperty('Seek')) 
        
               await player.Seek(packet.body.Seek * 1000); 
        
           if (packet.body.hasOwnProperty('SetPosition')) { 
        
               const offset = (packet.body.SetPosition * 1000) - player.Position; 
        
               await player.Seek(offset); 
        
           }

gnome-shell-extension-gsconnect/src/service/plugins/mpris.js

Lines 351 to 362 in 9d27463

    
           _onPlayerSeeked(mpris, player, offset) { 
        
               // TODO: although we can handle full seeked signals, kdeconnect-android 
        
               //       does not, and expects a position update instead 
        
               this.device.sendPacket({ 
        
                   type: 'kdeconnect.mpris', 
        
                   body: { 
        
                       player: player.Identity, 
        
                       pos: Math.floor(player.Position / 1000), 
        
                       // Seek: Math.floor(offset / 1000), 
        
                   }, 
        
               }); 
        
           }

Interestingly, it appears that VLC's Position property is an int32, but Seek and SetPosition take int64 arguments. That makes as close to zero sense as possible. (Other MPRIS interfaces publish Position as an int64 as well, VLC is just weird.)

But they don't appear to be scaled in relation to each other. Pausing VLC at 10 seconds in results in a Position value of 10150203. KDEConnect appears to use the same microsecond scale for its values.

Playing with VLC's MPRIS interface in d-spy, sending it Seek((10000000,)) skips 10 seconds forward, and Seek((-10000000,)) skips 10 seconds backwards.

Weirdly enough, ~~Chrome's~~ the org.mpris.MediaPlayer2.playerctld interface from playerctl DOES skip 5 seconds back/forward in YouTube videos (playing in Chrome), in response to Seek((-10000000,)) / Seek((10000000,)) (respectively). But its Position values are still time_in_seconds * 1000000.

(Edit: That's clearly a playerctl bug, though, because playerctl position 10+ — which should seek forward 10 seconds — also moves the position value forward five seconds.)

Playing with VLC's MPRIS interface in d-spy, sending it Seek((10000000,)) skips 10 seconds forward, and Seek((-10000000,)) skips 10 seconds backwards. So if nothing else, the fact that we're multiplying/dividing offset and position values appears to be wrong:

gnome-shell-extension-gsconnect/src/service/plugins/mpris.js

Lines 244 to 250 in 9d27463

if (packet.body.hasOwnProperty('Seek'))

await player.Seek(packet.body.Seek * 1000);

if (packet.body.hasOwnProperty('SetPosition')) {

const offset = (packet.body.SetPosition * 1000) - player.Position;

await player.Seek(offset);

}

Seek should indeed¹ not be multiplied in the first call; the value of this field is in microseconds. The second call is correct, however, since the value of SetPosition field is in milliseconds and the player.Position is coming from MPRIS which always deals in microseconds.

See kdeconnect.mpris.request.

¹ I'm now unsure about Seek, since the Android method takes an Int, but it was recently refactored to Kotlin and I haven't the time currently to dig through the classes and find the code reference.

gnome-shell-extension-gsconnect/src/service/plugins/mpris.js

Lines 351 to 362 in 9d27463

_onPlayerSeeked(mpris, player, offset) {

// TODO: although we can handle full seeked signals, kdeconnect-android

// does not, and expects a position update instead

this.device.sendPacket({

type: 'kdeconnect.mpris',

body: {

player: player.Identity,

pos: Math.floor(player.Position / 1000),

// Seek: Math.floor(offset / 1000),

},

});

}

I believe this call is also correct, since the the pos field is in milliseconds, while the MPRIS Position property is in microseconds.

See kdeconnect.mpris.

@andyholmes Yeah, I've since confirmed that (#ForSomeReason) KDEConnect's MPRIS proxying uses microsecond positions, but nanosecond seek offsets. Oh, and their SetPosition values are also in microseconds, confoundingly. MPRIS2 uses nanoseconds everywhere, so why they'd depart from that in such odd ways is a mystery.

Anyway, I now have seeking, skipping, and progress-slider needle-drops all working rock-solid reliably, at least with VLC, so that's a major step in the right direction.

I've also added passing of mpris:trackid in the player metadata, and use that to implement KDEConnect SetPosition requests as real MPRIS SetPosition(trackId, position) calls if available.

The method the KDEConnect code uses to fake SetPosition with Seek (by computing a delta from the current Position and seeking that amount) doesn't actually work very well in practice. Some players seem to ignore everything but the sign of the Seek() argument, and seek by a fixed distance.

(That's why @niels0n was reporting 5-second seeks, which should be impossible as it's not one of the options for KDEConnect's seek distance — but some players, including Chrome, have a fixed 5-second seek distance.) So, real SetPosition() time jumps are a win overall.

There are still a few more things I need to sort out:

MPRIS' Identity isn't guaranteed to be unique across player instances, since (for example) two Chrome windows both playing video will have the same Identity property. KDEConnect does the whole "scan for collisions and append an integer counter" thing in their code, which I guess we'll have to do as well. So I'm still going through the code to add a player.UniqueName property, and making sure that replaces Identity everywhere it should.
Fixing that still won't help with #1780, though, which is something else entirely. Still not sure what's up with that.
^ It could be related to this: I'm seeing crazy amounts of duplicate MPRIS updates going over the bus, seemingly due to duplicated signals in the plugin.

I added some debug logs, and for every notify signal the PlayerManager gets, the player-changed signal handler is fired like 4-5 times — each one sending a full update packet to the device. Even something as simple as a Seek always generates two position updates in a row, one in response to the seek and then a second a few milliseconds later, with a slightly higher pos number. That makes me think there are too many signals-triggering-signals, which end up being redundant with the property watchers.
I'm also seeing tons of 100% identical audio device updates being sent, unrequested and totally unnecessary, which I have to think is another symptom of the same condition.

So I'm trying to pare down the packet spam going out from the GSConnect side, and hoping that maybe helps reveal how GSConnect ends up getting confused between multiple players sometimes.

I've also added passing of mpris:trackid in the player metadata, and use that to implement KDEConnect SetPosition requests as real MPRIS SetPosition(trackId, position) calls if available.

(Not that KDEConnect Android makes any use of mpris:trackid directly. But now that it's in the local player metadata, I can pick it up and include it in an MPRIS SetPosition() call when a KDEConnect SetPosition request packet is received.)

Isn't all of MPRIS in microseconds? I assumed that was someone deciding to stuff a time_t or whatever into the spec.

Isn't all of MPRIS in microseconds? I assumed that was someone deciding to stuff a time_t or whatever into the spec.

For the record: Yes, it is. I was confused by three orders of magnitude, above.

The real issue (fixed in #1783) was that KDE Connect uses milliseconds for position values, but microseconds for seek amounts. (For some reason!)

	if (packet.body.hasOwnProperty('Seek'))
	await player.Seek(packet.body.Seek * 1000);

	if (packet.body.hasOwnProperty('SetPosition')) {
	const offset = (packet.body.SetPosition * 1000) - player.Position;
	await player.Seek(offset);
	}

	_onPlayerSeeked(mpris, player, offset) {
	// TODO: although we can handle full seeked signals, kdeconnect-android
	// does not, and expects a position update instead
	this.device.sendPacket({
	type: 'kdeconnect.mpris',
	body: {
	player: player.Identity,
	pos: Math.floor(player.Position / 1000),
	// Seek: Math.floor(offset / 1000),
	},
	});
	}

Rewind/forward buttons on mobile cause the file playing to restart/finish

Describe the bug

Steps to reproduce

Expected behavior

GSConnect version

Installed from

GNOME Shell version

Linux distribution/release

Paired device(s)

KDE Connect app version

Plugin(s)

Support log

Screenshots

Notes