Facetracker
Facetracker is a wrapper UI for OpenSeeFace written in python using libadwaita and GTK4.
It's primary use case is for VTubbing on Linux as most VTubbing Software is only supported on Windows. While those often times work fine by using Wine or Proton their integrated Webcam tracking is often times broken due to Wine and Proton not properly supporting Webcams. Therefore the facial tracking data are to be fetched by an external application and then send over the local network to the VTubbing application.
One prominent tool to do this is called OpenSeeFace developed by Emiliana. What Facetracker does is to put the setup burden to flatpak and flathub so users can just download the application and start using the OpenSeeFace included Facetracker using a handy user interface.
For users making use of their phones or other tracking hardware Facetracker is not required.
Facetracker can be installed on any Linux Distribution supporting flatpak. Simply search your software center for Facetracker and it should show up right away.
In case flathub is not setup by default please see the flathub setup guide for your Linux distribution.
Facetracker flatpak: Flathub
Facetracker source code: Codeberg
Integrate Facetracker with VTubbing applications
To use Facetracker with popular VTubbing applications depends on the application in use.
Portals
Facetracker does not use Portals to access webcam devices from within the sandbox because it is simply not possible to do so. The underlying issue is OpenSeeFace itself which relies on OpenVC which is a python video API which only can make use of devices in /dev/video* via the video4linux2 interface. However, using portals will not expose any video device at this path and pipewire is strictly required to then access cameras. OpenVC does not speak Pipewire, therefore does not OpenSeeFace and ultimately Facetracker can not integrate Portals as well.
See more in the (closed) issue here #22.
Hand tracking
Currently there is no hand tracking support in Facetracker as OpenSeeFace only offers face tracking capabilities.
GPU Acceleration
OpenSeeFace does not support GPU accelerated face tracking. Hence, this features is not supported.
Video:
Most things on this page is also explained in this video we've made:
Sources
- Flathub: https://flathub.org/apps/de.z_ray.Facetracker
- Source Code: https://codeberg.org/ZRayEntertainment/Facetracker