Hello!

I would like to use Gio to display a video read with libvlc on macOS 12.6
(Intel). Basically, I would like to adapt the following tutorial of the
libvlc-go bindings, replacing gotk3 by Gio:

https://github.com/adrg/libvlc-go-examples/tree/master/v3/gtk3_player

A libvlc player has a Darwin-specific SetNSObject method that allow to pass a
pointer to a window following a certain protocol such that the VLC player
handles everything from there. Using gtk3, this is done in the example using
some CGo:

https://github.com/adrg/libvlc-go-examples/blob/5abfcdeb80373adeb43c4075ee8ac6f33f972215/v3/gtk3_player/player_darwin.go#L19

How do I achieve the same thing using Gio? I suspect the question can be
precised as: How do I create a handle to a drawable area of a Gio window that
follows the desired protocol?

(Except for 2 hours playing with the learning examples, I have no experience
with Gio.)

I understand I could make libvlc generate a stream of frames and use a Gio
event loop to display them, but then I would have to deal with syncing,
framerate, audio, … while libvlc could do the whole thing by itself (much
better).

Thanks,
Louis

> Hello!

Hi Louis!

> I would like to use Gio to display a video read with libvlc on macOS 12.6
> (Intel). Basically, I would like to adapt the following tutorial of the
> libvlc-go bindings, replacing gotk3 by Gio:
>
> https://github.com/adrg/libvlc-go-examples/tree/master/v3/gtk3_player
>
> A libvlc player has a Darwin-specific SetNSObject method that allow to pass a
> pointer to a window following a certain protocol such that the VLC player
> handles everything from there. Using gtk3, this is done in the example using
> some CGo:
>
> https://github.com/adrg/libvlc-go-examples/blob/5abfcdeb80373adeb43c4075ee8ac6f33f972215/v3/gtk3_player/player_darwin.go#L19
>
> How do I achieve the same thing using Gio? I suspect the question can be
> precised as: How do I create a handle to a drawable area of a Gio window that
> follows the desired protocol?
>
> (Except for 2 hours playing with the learning examples, I have no experience
> with Gio.)
>
> I understand I could make libvlc generate a stream of frames and use a Gio
> event loop to display them, but then I would have to deal with syncing,
> framerate, audio, … while libvlc could do the whole thing by itself (much
> better).

You are correct on all counts. Sadly, you've stumbled into a feature
gap. Gio doesn't currently have a means to export a region of the
screen for external use, nor does it have a means to reference a GPU
texture drawn by external code within a Gio layout. This gap makes
use-cases like yours extremely challenging.

To manually manage the stream of frames, you'd have to copy the frame
data, and that adds a ton of overhead.

We would love to change this. If you are interested, we could talk
about how to extend Gio to support use-cases of this kind.

I can offer you one ray of hope with Gio's current API, but it's more
cumbersome that I'd like:

You can create a Gio window with a "custom renderer", which basically
means that Gio will not automatically initialize a graphics context
for the window, but will provide the handles necessary to do so for
you to use. You could then build a metal/opengl/whatever context
manually, use that to satisfy libvlc's API, and thus draw the VLC
content into the window. You can then layer Gio interface code on top
by rendering Gio into your custom graphics context. With some work,
you can make your Gio layout code track the coordinates at which you
want the video graphics to appear so that your custom graphics code
can render to the right place. There is an example of doing this using
ANGLE (at least on macOS) here:

https://git.sr.ht/~eliasnaur/gio-example/tree/main/item/opengl

This is a much more awkward way to achieve the goal, but it is in use
in production by some Gio applications to layer a Gio interface atop
games and other custom graphics (including 3D content).

Cheers,
Chris