Commit graph libervia-backend/libervia/backend/plugins
Author SHA1 Message Date
Goffi
08cd3fd587 component email gateway: Add support for XEP-0131 headers:
Some email headers (`Keywords` and `Importance` for now) are converted between XMPP and
email.

rel 451
2024-09-28 15:59:12 +02:00
Goffi
80ec968740 plugin XEP-0131: Stanza Headers and Internet Metadata implementation:
- SHIM is now supported and put in `msg_data["extra"]["headers"]`.
- `Keywords` are converted from and to list of string in `msg_data["extra"]["keywords"]`
field (if present in headers on message sending, values are merged).
- Python minimal version upgraded to 3.11 due to use of `StrEnum`.

rel 451
2024-09-28 15:56:04 +02:00
Goffi
bf0515378a plugin XEP-0033: formatting. 2024-09-28 15:52:14 +02:00
Goffi
15b29444a7 component email gateway: add multicasting:
XEP-0033 multicasting is now supported both for incoming and outgoing messages. XEP-0033
metadata are converted to suitable Email headers and vice versa.
Email address and JID are both supported, and delivery is done by the gateway when
suitable on incoming messages.

rel 450
2024-09-26 16:12:01 +02:00
Goffi
489eed2540 plugin XEP-0033: code modernisation, improve delivery, data validation:
- Code has been rewritten using Pydantic models and `async` coroutines for data validation
  and cleaner element parsing/generation.
- Delivery has been completely rewritten. It now works even if server doesn't support
  multicast, and send to local multicast service first. Delivering to local multicast
  service first is due to bad support of XEP-0033 in server (notably Prosody which has an
  incomplete implementation), and the current impossibility to detect if a sub-domain
  service handles fully multicast or only for local domains. This is a workaround to have
  a good balance between backward compatilibity and use of bandwith, and to make it work
  with the incoming email gateway implementation (the gateway will only deliver to
  entities of its own domain).
- disco feature checking now uses `async` corountines. `host` implementation still use
  Deferred return values for compatibility with legacy code.

rel 450
2024-09-26 16:12:01 +02:00
Goffi
73a08f5c91 component email-gateway: first draft:
Initial implementation of the Email Gateway.

This component uses XEP-0100 for registration. Upon registration and subsequent startups,
a connection is made to registered IMAP services, and incoming emails (in `INBOX`
mailboxes) are immediately forwarded as XMPP messages.

In the opposite direction, an SMTP connection is established to send emails on incoming
XMPP messages.

rel 449
2024-09-06 18:07:17 +02:00
Goffi
c0a9fa611a plugin XEP-0077, XEP-0100: Adapt to component, and modernize:
- Plugin XEP-0077 can now be used with component, allowing to register methods to return
  registration form, and to (un)register.
- Plugin XEP-0077 now advertises its feature in disco.
- Plugin XEP-0100 has been modernized a bit: it is one of the older plugin in Libervia,
  and it has now some type hints, models and async methods.
- Plugin XEP-0100's bridge method `gateways_find` now returns a serialised dict with
  relevant data. Former XMLUI version has been moved to `gateways_find_xmlui`.

rel 449
2024-09-06 18:01:31 +02:00
Goffi
a4f494ac70 plugin XEP-0106, tools (common/email): type hints. 2024-09-06 17:44:08 +02:00
Goffi
5513da8792 core (xmpp): move is_local to SatXMPPEntity + type hints + some log level changes. 2024-09-06 17:42:07 +02:00
Goffi
08e1409925 component conferences, plugin XEP-0167, XEP-0298: add stream user metadata:
A/V conference now adds user metadata about the stream it is forwarding through XEP-0298.
This is parsed and added to metadata during confirmation on client side.

rel 448
2024-08-06 23:43:11 +02:00
Goffi
e357f36f32 plugin XEP-0298: Delivering Conference Information to Jingle Participants (Coin) implementation:
This is the first draft of XEP-0298 implementation. The focus is to implement elements
needed for A/V Conferences protoXEP.

rel 447
2024-07-29 03:31:13 +02:00
Goffi
96a1f4435c plugin XEP-0167: handle conferences:
- SDP can now be answered by component instead of frontend. This is useful for A/V
  conferences component to handle A/V call jingle sessions.
- new `call_update` and method, and `content-add` action preparation. This is not yet used
  by A/V conference, but it's a preparation for a potential future use.
- Add NS_AV_CONFERENCES to features as required by the newly proposed A/V Conferences
  protoXEP.

rel 447
2024-07-29 03:31:09 +02:00
Goffi
01ee77b785 plugin XEP-0176: add debug logs:
rel 447
2024-07-29 03:31:06 +02:00
Goffi
f637be1e4f plugin XEP-0167: fix session level SDP attributes parsing:
rel 447
2024-07-29 03:31:02 +02:00
Goffi
31c80d3626 plugin XEP-0166: handle content-add action + expose get_transport:
- `content-add` is now handled at this plugin level (implementation needs to be done in
  apps and transports plugins).
- `get_transport` is now exposed.

rel 447
2024-07-29 03:30:58 +02:00
Goffi
6a4133b32c plugin identity: disco identity is now added in identity metadata:
rel 447
2024-07-29 03:30:53 +02:00
Goffi
90e81c9fc6 component conferences: fix session management and init:
- jingle session management is now correctly handled, with id of the Galène client used as
  prefix when necessary.
- Galène is now run only when component session is started, avoiding systematic run on
  __init__.
- Galène ICE logs are displayed with high verbosity.
- Fix disco identity handling

rel 447
2024-07-29 03:29:14 +02:00
Goffi
c9ab0cd9dd black reformatting 2024-07-14 16:47:45 +02:00
Goffi
01e695cebb plugin XEP-0060: add a force options for publish options in send_items:
The new `force` option can be used to force the update of configuration is
`publish-option` fails.

Use this new `force` option in XEP-0384.
2024-07-14 16:45:17 +02:00
Goffi
ba170701f0 component Conferences: implementation of SFU component to make multi-party A/V conferences:
This component wrap `Galène` SFU and translate its signaling to XMPP Jingle in both
direction.

rel 445
2024-07-05 17:18:37 +02:00
Goffi
63b50a2d27 plugin XEP-0167: Let use a call_setup_cb method in session:
when set, the `call_setup_cb` method will be used instead of calling the `call_setup`
bridge signal. This will be useful to incoming `Conferences` component to handle the
signaling internally.

rel 445
2024-07-05 17:18:37 +02:00
Goffi
6afc342875 plugin XEP-0167: move ICE candidates parsing/SDP line building in separate methods:
These methods will be used by incoming "Conferences" component.

rel 445
2024-07-05 17:18:37 +02:00
Goffi
afbeeb2d1a plugin XEP-0176: Send ICE candidates buffer for initiator:
- ICE buffer is now sent also for initiator.
- If `ice_candidates_new_cb` is set in `session`, it will be used instead of using the
  `ice_candidates_new` signal. This is useful to handle internally the workflow, and will
  be used in incoming `Conferences` component.


rel 445
2024-07-05 17:18:32 +02:00
Goffi
f7c8ebea06 plugin misc list: Comments node creation can now be specified with comments field in extra. 2024-06-20 14:36:09 +02:00
Goffi
53fdaa9d09 Reformatted code base using black. 2024-06-19 18:44:57 +02:00
Goffi
1c1f617cdf plugin XEP-0048: don't fail if a type is missing in bookmark data. 2024-06-12 22:36:15 +02:00
Goffi
a6059161bb plugin XEP-470: Don't file configuration on feature-not-implemented. 2024-06-12 22:35:13 +02:00
Goffi
f97ed65a7a component AP Gateway: log invalid account in case of error. 2024-06-12 22:34:19 +02:00
Goffi
b402be2a7a component AP Gateway: implement HTTP GET signature. 2024-06-05 22:34:09 +02:00
Goffi
99b218fca4 plugin text syntaxes: fix empty string handling in _remove_markups. 2024-06-05 22:33:37 +02:00
Goffi
3d30449a93 app (weblate): use application weblate section 2024-05-31 15:15:42 +02:00
Goffi
47a29aa5ce app (peertube): environment variable must be strings (bis) 2024-05-31 11:24:12 +02:00
Goffi
187c09204c app (peertube): environment variable must be strings 2024-05-31 11:18:12 +02:00
Goffi
6fc76d1a0b plugin app manager: Add an app configuration file for PeerTube. 2024-05-31 11:08:22 +02:00
Goffi
4c7d4de032 doc: style/typos/URL fixes 2024-05-31 11:08:20 +02:00
Goffi
da80cae625 plugin app manager: various improvements:
- Generated password must now be named and are stored, so they are re-used on following
  restarts. Password size can now be specified.
- New `not` filter for `!libervia_param` to inverse a boolean value.
- Former `front_url` field has been renamed to `web_url_path` as it is the URL path used
  for web frontend. All Web frontend related field are prefixed with `web_`.
- `front_url` is now used to specify a whole front URL (notably useful if an app uses its
  own domain). A list can be used to retrieve a key, like for `url_prefix`, and `https`
  scheme is added if no scheme is specified.
- An abstract class is now used for App Managers.
- Last application start time is stored in persistent data.
2024-05-31 11:08:14 +02:00
Goffi
0e23b8c1b0 plugin XEP-0272: implement XEP-0272: Multiparty Jingle (Muji)
rel 429
2024-05-15 17:34:46 +02:00
Goffi
ed7a2fa132 core: Jingle Remote Control implementation:
This is an implementation of the protoXEP that will be submitted to XSF. It handle
establishment of a remote control session, and the management of A/V calls with XEP-0167.

rel 436
2024-05-11 13:52:43 +02:00
Goffi
de7260bfb9 backend, frontend: prepare remote control:
This is a series of changes necessary to prepare the implementation of remote control
feature:

- XEP-0166: add a `priority` attribute to `ApplicationData`: this is needed when several
  applications are working in a same session, to know which one must be handled first.
  Will be used to make Remote Control have precedence over Call content.
- XEP-0166: `_call_plugins` is now async and is not used with `DeferredList` anymore: the
  benefit to have methods called in parallels is very low, and it cause a lot of trouble
  as we can't predict order. Methods are now called sequentially so workflow can be
  predicted.
- XEP-0167: fix `senders` XMPP attribute <=> SDP mapping
- XEP-0234: preflight acceptance key is now `pre-accepted` instead of `file-accepted`, so
  the same key can be used with other jingle applications.
- XEP-0167, XEP-0343: move some method to XEP-0167
- XEP-0353: use new `priority` feature to call preflight methods of applications according
  to it.
- frontend (webrtc): refactor the sources/sink handling with a more flexible mechanism
  based on Pydantic models. It is now possible to have has many Data Channel as necessary,
  to have them in addition to A/V streams, to specify manually GStreamer sources and
  sinks, etc.
- frontend (webrtc): rework of the pipeline to reduce latency.
- frontend: new `portal_desktop` method. Screenshare portal handling has been moved there,
  and RemoteDesktop portal has been added.
- frontend (webrtc): fix `extract_ufrag_pwd` method.

rel 436
2024-05-11 13:52:41 +02:00
Goffi
e8fb0a8a28 plugin XEP-0353, XEP-0234, jingle: WebRTC data channel signaling implementation:
Implement XEP-0343: Signaling WebRTC Data Channels in Jingle. The current version of the
XEP (0.3.1) has no implementation and contains some flaws. After discussing this on xsf@,
Daniel (from Conversations) mentioned that they had a sprint with Larma (from Dino) to
work on another version and provided me with this link:
https://gist.github.com/iNPUTmice/6c56f3e948cca517c5fb129016d99e74 . I have used it for my
implementation.

This implementation reuses work done on Jingle A/V call (notably XEP-0176 and XEP-0167
plugins), with adaptations. When used, XEP-0234 will not handle the file itself as it
normally does. This is because WebRTC has several implementations (browser for web
interface, GStreamer for others), and file/data must be handled directly by the frontend.
This is particularly important for web frontends, as the file is not sent from the backend
but from the end-user's browser device.

Among the changes, there are:

- XEP-0343 implementation.
- `file_send` bridge method now use serialised dict as output.
- New `BaseTransportHandler.is_usable` method which get content data and returns a boolean
  (default to `True`) to tell if this transport can actually be used in this context (when
  we are initiator). Used in webRTC case to see if call data are available.
- Support of `application` media type, and everything necessary to handle data channels.
- Better confirmation message, with file name, size and description when available.
- When file is accepted in preflight, it is specified in following `action_new` signal for
  actual file transfer. This way, frontend can avoid the display or 2 confirmation
  messages.
- XEP-0166: when not specified, default `content` name is now its index number instead of
  a UUID. This follows the behaviour of browsers.
- XEP-0353: better handling of events such as call taken by another device.
- various other updates.

rel 441
2024-04-06 12:57:23 +02:00
Goffi
25aedf57d7 plugin XEP-0391: remove the need to have a OLDMEMO session started to decrypt:
It should not be mandatory to have an OLDMEMO session started to decrypt JET files, and
this was actually making a test failing.
2024-03-05 17:31:56 +01:00
Goffi
bb57b12329 plugin XEP-0313: fix value used in msg_from when from is not explicitely specified. 2024-03-05 17:31:56 +01:00
Goffi
b6383c557e plugin XEP-0060: fix incorrect type hint. 2024-03-05 17:31:56 +01:00
Goffi
a804e6a9dc plugin XEP-0234: fix action type when pre-flight file confirmation is confirmed:
the `C.META_TYPE_FILE` can't be used in this case as it is a preflight and we don't have
yet data such as progress ID. `C.META_TYPE_FILE` is used as a sub-type to help with dialog
management in frontend.
2024-03-05 17:31:49 +01:00
Goffi
f20e5d6103 plugin XEP-0384, XEP-0391: download missing devices list:
when a peer jid was not in our roster, devices list was not retrieved, resulting in failed
en/decryption. This patch does check it and download missing devices list in necessary.

There is no subscription managed yet, so the list won't be updated in case of new devices,
this should be addressed at some point.
2024-03-05 17:31:36 +01:00
Goffi
23e7b2720d plugin XEP-0384: be sure to have open access model for created nodes. 2024-03-05 17:31:36 +01:00
Goffi
0548f443ca plugin XEP-0373: fix serialisation of public_key_list 2024-03-05 17:31:36 +01:00
Goffi
3694bb3991 core (memory/migration):
Update XEP-0384 and `fe3a02cb4bec_convert_legacypickle_columns_to_json.py` migration to
properly handle (de)serialisation of `TrustMessageCacheEntry`.
2024-03-05 17:31:12 +01:00
Goffi
a8348576ae core: get rid of pickle:
Use of `pickle` to serialise data was a technical legacy that was causing trouble to store
in database, to update (if a class was serialised, a change could break update), and to
security (pickle can lead to code execution).

This patch remove all use of Pickle in favour in JSON, notably:
- for caching data, a Pydantic model is now used instead
- for SQLAlchemy model, the LegacyPickle is replaced by JSON serialisation
- in XEP-0373 a class `PublicKeyMetadata` was serialised. New method `from_dict` and
  `to_dict` method have been implemented to do serialisation.
- new methods to (de)serialise data can now be specified with Identity data types. It is
  notably used to (de)serialise `path` of avatars.

A migration script has been created to convert data (for upgrade or downgrade), with
special care for XEP-0373 case. Depending of size of database, this migration script can
be long to run.

rel 443
2024-02-23 13:31:04 +01:00
Goffi
133fe9c6fe plugin XEP-0444: fix emoji check and ID used 2023-12-13 22:00:25 +01:00