Commit graph

4,237 commits

Author SHA1 Message Date
Goffi
5b60aaf83c core: implement SCRAM-SHA auth algorithm:
Twisted auth mechanism are outdated, and as a result, Libervia was not supporting the
mandatory SCRAM-SHA auth mechanism. This patch implements it for SCRAM-SHA-1,
SCRAM-SHA-256 and SCRAM-SHA-512 variants.
2024-04-08 12:29:40 +02:00
Goffi
f1cc67fd46 test (unit/cli): Add a file send/receive test for WebRTC:
fix 442
2024-04-06 15:21:00 +02:00
Goffi
6634c917bd install (pyproject): update lxml and add lxml_html_clean which is now a separate project. 2024-04-06 15:04:07 +02:00
Goffi
586b1f1440 docker (e2e): add GStreamer dependencies to test WebRTC stack:
rel 424
2024-04-06 15:04:01 +02:00
Goffi
728b01d236 cli (call, file), frontends: implement webRTC data channel transfer:
- file send/receive commands now supports webRTC transfer. In `send` command, the
  `--webrtc` flags is currenty used to activate it.

- WebRTC related code have been factorized and moved to `libervia.frontends.tools.webrtc*`
  modules.

rel 442
2024-04-06 13:43:09 +02:00
Goffi
27b66b3fb2 tests (unit/webrtc,XEP-0176, XEP-0234): Fix tests and add webrtc file transfer tests:
fix 441
2024-04-06 12:59:50 +02:00
Goffi
e8fb0a8a28 plugin XEP-0353, XEP-0234, jingle: WebRTC data channel signaling implementation:
Implement XEP-0343: Signaling WebRTC Data Channels in Jingle. The current version of the
XEP (0.3.1) has no implementation and contains some flaws. After discussing this on xsf@,
Daniel (from Conversations) mentioned that they had a sprint with Larma (from Dino) to
work on another version and provided me with this link:
https://gist.github.com/iNPUTmice/6c56f3e948cca517c5fb129016d99e74 . I have used it for my
implementation.

This implementation reuses work done on Jingle A/V call (notably XEP-0176 and XEP-0167
plugins), with adaptations. When used, XEP-0234 will not handle the file itself as it
normally does. This is because WebRTC has several implementations (browser for web
interface, GStreamer for others), and file/data must be handled directly by the frontend.
This is particularly important for web frontends, as the file is not sent from the backend
but from the end-user's browser device.

Among the changes, there are:

- XEP-0343 implementation.
- `file_send` bridge method now use serialised dict as output.
- New `BaseTransportHandler.is_usable` method which get content data and returns a boolean
  (default to `True`) to tell if this transport can actually be used in this context (when
  we are initiator). Used in webRTC case to see if call data are available.
- Support of `application` media type, and everything necessary to handle data channels.
- Better confirmation message, with file name, size and description when available.
- When file is accepted in preflight, it is specified in following `action_new` signal for
  actual file transfer. This way, frontend can avoid the display or 2 confirmation
  messages.
- XEP-0166: when not specified, default `content` name is now its index number instead of
  a UUID. This follows the behaviour of browsers.
- XEP-0353: better handling of events such as call taken by another device.
- various other updates.

rel 441
2024-04-06 12:57:23 +02:00
Goffi
ca7c061986 core (xmpp): Add a timeout for messages processing to avoid blocking the queue. 2024-04-06 12:21:04 +02:00
Goffi
0ee6a64e3b core (constants): use bool_type:
Using `bool` for type hint in `Const.bool_const` if conflicting with `Const.bool` method.
To avoid this, `bool_type` is used as an alias to `bool`.
2024-04-06 12:14:10 +02:00
Goffi
188c2c97ba migration: fix migration when XEP-0373 or XEP-0384 can't be imported:
Those plugins are using optional modules, and if their importation fails, there is
probably no data to migrate either.
2024-03-09 15:13:14 +01:00
Goffi
48201780de tests: fix and update tests:
- update code to newer version of `sh`
- update code to newer version of `selenium`
- update names of OMEMO algorithms
- fix paths and names used to mount source to the test container
- skip modules using `Gst` and `PyQt` when they are not available.
2024-03-05 17:31:56 +01:00
Goffi
fa55bc6fb9 cli (pubsub): use PEP when service is not specified in error message. 2024-03-05 17:31:56 +01:00
Goffi
dec2fa48bc cli (file/receive): handle CONFIRM dialog received in pre-flight. 2024-03-05 17:31:56 +01:00
Goffi
8db12deaff cli (base): fix exit code transmission when SystemExit is catched. 2024-03-05 17:31:56 +01:00
Goffi
25aedf57d7 plugin XEP-0391: remove the need to have a OLDMEMO session started to decrypt:
It should not be mandatory to have an OLDMEMO session started to decrypt JET files, and
this was actually making a test failing.
2024-03-05 17:31:56 +01:00
Goffi
bb57b12329 plugin XEP-0313: fix value used in msg_from when from is not explicitely specified. 2024-03-05 17:31:56 +01:00
Goffi
b6383c557e plugin XEP-0060: fix incorrect type hint. 2024-03-05 17:31:56 +01:00
Goffi
a804e6a9dc plugin XEP-0234: fix action type when pre-flight file confirmation is confirmed:
the `C.META_TYPE_FILE` can't be used in this case as it is a preflight and we don't have
yet data such as progress ID. `C.META_TYPE_FILE` is used as a sub-type to help with dialog
management in frontend.
2024-03-05 17:31:49 +01:00
Goffi
f20e5d6103 plugin XEP-0384, XEP-0391: download missing devices list:
when a peer jid was not in our roster, devices list was not retrieved, resulting in failed
en/decryption. This patch does check it and download missing devices list in necessary.

There is no subscription managed yet, so the list won't be updated in case of new devices,
this should be addressed at some point.
2024-03-05 17:31:36 +01:00
Goffi
23e7b2720d plugin XEP-0384: be sure to have open access model for created nodes. 2024-03-05 17:31:36 +01:00
Goffi
0548f443ca plugin XEP-0373: fix serialisation of public_key_list 2024-03-05 17:31:36 +01:00
Goffi
3694bb3991 core (memory/migration):
Update XEP-0384 and `fe3a02cb4bec_convert_legacypickle_columns_to_json.py` migration to
properly handle (de)serialisation of `TrustMessageCacheEntry`.
2024-03-05 17:31:12 +01:00
Goffi
d50ca26d12 docker: modernisation and fixes:
- fix geckodriver version used following `backend-dev-e2e` base image update
- use Prosody image `prosodyim/prosody:0.12` instead of a deprecated one
- update `pubsub` image with base image update and use of new repos names
2024-03-05 16:43:45 +01:00
Goffi
146dd6022f doc (contributing/testing): typo 2024-02-23 15:47:53 +01:00
Goffi
039a9b5830 tests (unit): fix tests 2024-02-23 15:42:02 +01:00
Goffi
a8348576ae core: get rid of pickle:
Use of `pickle` to serialise data was a technical legacy that was causing trouble to store
in database, to update (if a class was serialised, a change could break update), and to
security (pickle can lead to code execution).

This patch remove all use of Pickle in favour in JSON, notably:
- for caching data, a Pydantic model is now used instead
- for SQLAlchemy model, the LegacyPickle is replaced by JSON serialisation
- in XEP-0373 a class `PublicKeyMetadata` was serialised. New method `from_dict` and
  `to_dict` method have been implemented to do serialisation.
- new methods to (de)serialise data can now be specified with Identity data types. It is
  notably used to (de)serialise `path` of avatars.

A migration script has been created to convert data (for upgrade or downgrade), with
special care for XEP-0373 case. Depending of size of database, this migration script can
be long to run.

rel 443
2024-02-23 13:31:04 +01:00
Goffi
672f967080 doc (cli/call): add `tui` output, and document options:
fix 428
2024-02-16 18:46:06 +01:00
Goffi
430f2d9354 cli (call): tui output implementation:
- Moved original UI to a separated class, and use if with the `simple` output
- By default, best output is automatically selected. For now `gui` is selected if possible,
  and `simple` is used as fallback.
- The new `tui` output can be used to have the videos directly embedded in the terminal,
  either with real videos for compatible terminal emulators, or with Unicode blocks.
- Text contrôls are used for both `simple` and `tui` outputs
- several options can be used with `--oo` (will be documented in next commit).

rel 428
2024-02-16 18:46:06 +01:00
Goffi
ec3e037a1a frontends (tools/webrtc): add a options to merge video for PiP and to specify size:
- the `merge_pip` option is now used to indicate if local feedback and remote video
  streams must be merged with local feedback being a PiP of remote video. By default, it's
  done for `SINKS_AUTO`, but it can be manually specified to use it with `SINKS_APP` or to
  split ``SINKS_AUTO`` in 2 windows.
- target size of the compositor used with `merge_pip` can be specified. By default, 720p
  is used.

rel 428
2024-02-16 18:46:02 +01:00
Goffi
40ababcf9c doc (cli/call): add outputs documentation:
outputs are now documentated, and notably the new `gui` one.

fix 427
2024-02-11 23:22:18 +01:00
Goffi
ce2d217c64 tests (unit/cli/calls): unit tests for CLI call GUI:
rel 427
2024-02-11 23:21:07 +01:00
Goffi
3d38bbd155 cli (call): implement GUI output:
``call`` commands now handle various output. Beside the original one (now named
``simple``), a new ``gui`` one display a full featured GUI (make with Qt).

PyQt 6 or more needs to be installed.

rel 427
2024-02-11 23:20:24 +01:00
Goffi
04ed0e2bd6 frontends (tools/webrtc): implement screensharing for Wayland + bug fixes:
- Freedesktop Desktop Screenshare port is now used when Wayland is detected (needs
  `xdg-desktop-portal` with the implementation corresponding to desktop environment).
- Add a binding feature to feedback state to application (e.g. if desktop sharing is
  cancelled from desktop environment, or at portal's permission request level).
- fix misnaming of video source (was wrongly named `camera` instead of `video`).
- fix desktop sharing pad selection in `input-selector` when it has been added once, then
  removed, then added again.

rel 434
2024-01-18 23:29:25 +01:00
Goffi
27ae8655d7 frontends (tools/webrtc): implement X11 desktop sharing:
rel 433
2024-01-16 10:42:00 +01:00
Goffi
3fecc07a7e frontends (tools): module to handle display servers:
rel 433
2024-01-16 10:41:58 +01:00
Goffi
dc8078fe4c core: use a user specific directory for PID file:
default location of pid file is now specific to logged user, this allow to run several
instances of Libervia by different users on the same machine without PID conflicts.
2024-01-14 17:48:02 +01:00
Goffi
00de7e0de0 docker (demo): create file-sharing and demo account in init script 2023-12-14 09:19:32 +01:00
Goffi
133fe9c6fe plugin XEP-0444: fix emoji check and ID used 2023-12-13 22:00:25 +01:00
Goffi
b571b0d6e2 plugin XEP-0424: fix ID used and recipients 2023-12-13 22:00:25 +01:00
Goffi
817d7b6daf plugin XEP-0308: fix ID used, recipients and author check. 2023-12-13 22:00:25 +01:00
Goffi
5470bbf7b9 plugin XEP-0359: if no origin ID is found, use <message> ID instead:
This way, we always have the right ID to check for non groupchat message in
`History.origin_id`.
2023-12-13 22:00:25 +01:00
Goffi
79fd3d7c13 plugin XEP-0045: avoir crash if subject is not set when _get_room_joined_args is called. 2023-12-13 22:00:25 +01:00
Goffi
b12d7a48e1 docker (demo): fix starting + better UX:
- Use the new `wait_for_service` option to be sure that Prosody is launched.
- Use the new `init_script_path` option to initiate forums and bookmark a demo room.
- Activate MUC service.
- Change log formats to make them more readable.
2023-12-13 22:00:25 +01:00
Goffi
6fa46e20fd core (launcher): new wait_for_service option:
if `wait_for_service` is used in `[DEFAULT]`, the service at the specified host/port is
waited for before starting the backend. This is useful in the situation where we need to
be sure that something is started before running, e.g. in a container.
2023-12-13 22:00:25 +01:00
Goffi
e14dd9da8b core: add new init_script_path option:
`init_script_path` option can be used in `[DEFAULTS]` to run a script at the end of
backend initialisation. A new `init_pre_script` method is used to wait for backend to
reach this stage (designed to be used mostly by CLI frontend), then the usual `ready_get`
method is finished once the script is finished.
2023-12-13 22:00:22 +01:00
Goffi
258c99b3c5 plugin forums: parsing fix + formatting:
- reformatted with black
- fix parsing of forums
- minor improvements
2023-12-12 12:17:15 +01:00
Goffi
b634e41ac0 core, doc, cli (forums): new forums set commands + doc:
- document the fact that if an empty `uri` is used, the forum node is created
  automatically
- new `forums/set` CLI commands and its documentation
2023-12-11 18:10:27 +01:00
Goffi
05bdfe122a component AP gateway: be sure to have a string for error message. 2023-12-11 18:07:58 +01:00
Goffi
690c79b956 docker (backend-dev-demo): command renaming + add a bookmark to avoid empty chat 2023-12-11 00:54:01 +01:00
Goffi
c31b33c01d docker (backend-dev): update Debian version + getting OMEMO related packages from git is not necessary anymore 2023-12-11 00:53:59 +01:00