Commit graph

3,935 commits

Author SHA1 Message Date
Goffi
b91bc6fa2d plugin XEP-0374: exluce openpgp element from XEP-0373 on send trigger:
the `openpgp` element is excluded to avoid useless double encryption.

rel 380
2022-10-15 20:37:00 +02:00
Goffi
da9725479b plugin OXPS: OpenPGP for XMPP Pubsub implementation:
OpenPGP for XMPP Pubsub (https://xmpp.org/extensions/inbox/pubsub-encryption.html,
currently a protoXEP) is implemented and activated when `encrypted` is set to `True` in
pubsub's `extra` data.

On item retrieval, the decryption is transparent if the key is known, except if the
`decrypt` key in `extra` is set to `False` (notably useful when one wants to checks that
data is well encrypted).

Methods and corresponding bridge methods have been implemented to manage shared secrets
(to share, revoke or rotate the secrets).

plugin XEP-0060's `XEP-0060_publish` trigger point as been move before actual publish so
item can be modified (here e2ee) by the triggers. A new `XEP-0060_items` trigger point has
also been added.

`encrypted` flag can be used with plugin XEP-0277's microblog data

rel 380
2022-10-15 20:36:53 +02:00
Syndace
6d1e942f43 plugin XEP-0373 and XEP-0374: Implementation of OX and OXIM:
GPGME is used as the GPG provider.

rel 374
2022-09-20 16:22:18 +02:00
Goffi
f160577a65 core: fix and renamed getLocalPath -> get_local_path:
- renaming is following global methods renaming to use snake_case
- for reasons lost in the mists of time, a `profile` argument was used while checking if
  `client` is set is enough, and `client.profile` was escaped but actually not used in the
  path.
2022-10-10 15:23:59 +02:00
Goffi
0cd676115c plugins (forums, lists, import, XEP-0277): remove invalid max_items values 2022-10-07 22:22:13 +02:00
Goffi
8414475285 plugin lists: fix double deserialisation 2022-10-06 17:50:54 +02:00
Syndace
8daec13fbb plugin XEP-0384: proper handling of exceptions in prepare_for_profile 2022-10-06 17:25:37 +02:00
Goffi
3896e091fd plugin lists: fix deserialisation on listSet 2022-10-06 17:22:21 +02:00
Goffi
8f204aba8f plugin XEP-0448: Encryption for stateless file sharing implementation:
- registered as a source handle for XEP-0447
- can be used as an attachment
- registered as a download handler
- only usable when OMEMO2 is active, as we been SCE to hide encryption data

fix 379
2022-10-06 16:02:05 +02:00
Goffi
cb44cb1991 cli (file/get): handle full JSON attachment following change in backend:
rel 379
2022-10-06 16:02:05 +02:00
Goffi
d557e06fbd cli (message/send): new --attach argument to send an attachment:
rel 379
2022-10-06 16:02:05 +02:00
Goffi
0f20af88f1 tools (stream): log error when a SatFile is closed:
error where only sent on `ProgressError`, but it is necessary to log the error as it may
happen before any progression is started.
2022-10-06 16:02:05 +02:00
Goffi
aea64e7b87 plugin XEP-0334: avoid adding twice the same hint:
rel 379
2022-10-06 16:02:05 +02:00
Goffi
a8fc70fca6 plugin XEP-0447: handle attachment and download:
- plugin XEP-0447 can now be used in message attachments and to retrieve an attachment
- plugin attach: `attachment` being processed is added to `extra` so the handler can inspect it
- plugin attach: `size` is added to attachment
- plugin download: a whole attachment dict is now used in `download` and
  `file_download`/`file_download_complete`. `download_uri` can be used as a shortcut when
  just a URI is used. In addition to URI scheme handler, whole attachment handlers can now
  be registered with `register_download_handler`
- plugin XEP-0363: `file_http_upload` `XEP-0363_upload_size` triggers have been renamed to
  `XEP-0363_upload_pre_slot` and is now using a dict with arguments, allowing for the size
  but also the filename to be modified, which is necessary for encryption (filename may
  be hidden from URL this way).
- plugin XEP-0446: fix wrong element name
- plugin XEP-0447: source handler can now be registered (`url-data` is registered by
  default)
- plugin XEP-0447: source parsing has been put in a separated `parse_sources_elt` method,
  as it may be useful to do it independently (notably with XEP-0448)
- plugin XEP-0447: parse received message and complete attachments when suitable
- plugin XEP-0447: can now be used with message attachments
- plugin XEP-0447: can now be used with attachments download
- renamed `options` arguments to `extra` for consistency
- some style change (progressive move from legacy camelCase to PEP8 snake_case)
- some typing

rel 379
2022-10-06 16:02:05 +02:00
Goffi
11f16fc2d0 core (memory/encryption): helper method to get currently utilised e2ee algorithm's namespace:
rel 379
2022-10-06 16:02:05 +02:00
Goffi
5b40d71f40 core (xmpp): check presence of some keys in attachment before using them:
in `completeAttachments` `url` was used to complete a missing name, but it may be missing
with some attachments, thus its presence is now checked.
2022-10-06 16:02:05 +02:00
Goffi
4b498cffe0 plugin XEP-0384: temporarily disable <iq> stanza encryption:
`<iq>` stanza encryption is implemented, but the `send` trigger doesn't catch all `iq`
stanza send (which are often sent directly with `xmlstream.send`), and some stanza
encryption may cause trouble (happened with disco on a bare jid).

It is temporarily disabled but should be reactivated before `0.9` release.
2022-10-06 16:02:05 +02:00
Goffi
7d415c705d plugin XEP-0384: expose OLDMEMO and TWOMEMO namespaces:
rel 379
2022-10-06 16:02:05 +02:00
Goffi
a68564e68f plugin XEP-0420: fix exception on missing from or to:
fix exception raised on missing `from` or `to` attribute when the policy doesn't require
them.

Patch made by Syndace <me@syndace.dev>
2022-10-06 16:02:00 +02:00
Goffi
1fd553db40 plugin XEP-0300: type hints 2022-10-06 15:19:08 +02:00
Goffi
131096ed53 plugin events: fix bad import 2022-10-06 14:59:49 +02:00
Goffi
8444115262 plugin XEP-0384, xml_tools: avoid getItems timeout + fix empty node crash + parsing:
- use `max_items` in `getItems` calls for bundles, as otherwise some pubsub service may
  return full nodes, which may be huge is `max_items=1` is not set on the node, possibly
  resulting in timeouts.

- the plugin was crashing when TWOMEMO devices list node has no items at all. This is not
  the case anymore.

- a naive parsing method has been implemented in `xml_tools` to replace the
  serialisation/deserialisation method. This should be more efficient and will avoid
  annoying `ns0:` prefixes in XML logs.
2022-09-24 16:37:46 +02:00
Goffi
ee5784811e core (xmpp): make send a blocking method, fix sendMessageData calls:
original `send` method is blocking, and it is used as such by Wokkel and thus can't be
changed to an async method easily. However, an Async method is necessary to have an async
trigger at the very end of the send workflow for end-to-end encryption. To workaround
that, `send` is an async method which call `a_send`, an async method which actually does
the sending. This way legacy method can still call `send` while `a_send` can be await
otherwise.

Fix calls to `sendMessageData`: the method now being an `async` one, `ensureDeferred` had
to be used in some calls.
2022-09-24 16:31:39 +02:00
Goffi
d9a04a36ca core (main): OrderedDict is not necesasry here anymore, as dict are now ordered 2022-09-24 16:08:26 +02:00
Syndace
75ae8160ae plugin XEP-0384: Fully reworked to adjust to the reworked python-omemo:
- support for both (modern) OMEMO under the `urn:xmpp:omemo:2` namespace and (legacy) OMEMO under the `eu.siacs.conversations.axolotl` namespace
- maintains one identity across both versions of OMEMO
- migrates data from the old plugin
- includes more features for protocol stability
- uses SCE for modern OMEMO
- fully type-checked, linted and format-checked
- added type hints to various pieces of backend code used by the plugin
- added stubs for some Twisted APIs used by the plugin under stubs/ (use `export MYPYPATH=stubs/` before running mypy)
- core (xmpp): enabled `send` trigger and made it an asyncPoint

fix 375
2022-08-23 21:06:24 +02:00
Goffi
6eac59dc2d doc (components/AP Gateway): Ad-Hoc and Events:
New Ad-Hoc command is explained, and `Events` section specify how the conversion is done
between the 2 protocols.

fix 372
2022-09-22 12:03:12 +02:00
Goffi
b06f5cad04 doc (cli/event): update doc following changes:
rel 372
2022-09-22 12:01:44 +02:00
Goffi
c2e2f0c332 tests (unit/AP gateway): add tests for events:
rel 372
2022-09-22 00:01:48 +02:00
Goffi
a206196e9d tests (unit/AP gateway): fix tests:
rel 372
2022-09-22 00:01:48 +02:00
Goffi
4bae534e0f cli (event): update commands following changes in events:
- commands have been update to match changes in bridge method.
- arguments have been completely reworked for `create` and `modify` commands, making it
  possible to fine tune events.
- `list` has been removed in favor of `get` which can now display several events at once

rel 372
2022-09-22 00:01:48 +02:00
Goffi
093b9d83bc tools (common/date_utils) handle timestamp and in + delta2human:
regex used to parse datetimes has been improved to handle a unix time (which can be used
with `+ <delta>` or `- <delta>`), and the `in <delta>` (e.g. `in 6 days`).

`DEFAULT_DATETIME` has been removed as dateutil use current date by default, which is the
expected behaviour.

Add `delta2human` method which convert the difference of 2 unix times to a human friendly
approximate text.

rel 372
2022-09-22 00:01:48 +02:00
Goffi
adea3770de component AP gateway: events:
- XMPP Events <=> AP Events conversion
- `Join`/`Leave` activities are converted to RSVP attachments and vice versa
- fix caching/notification on item published on a virtual pubsub node
- add Ad-Hoc command to convert XMPP Jid/Node to virtual AP Account
- handle `Update` activity
- on `convertAndPostItems`, `Update` activity is used instead of `Create` if a version of
  the item is already present in cache
- `events` field is added to actor data (and to `endpoints`), it links the `outbox` of the
  actor mapping the same JID with the Events node (i.e. it links to the Events node of the
  entity)
- fix subscription to nodes which are not the microblog one

rel 372
2022-09-22 00:01:41 +02:00
Goffi
97c0d6a0e5 tools (xml_tools): handle required field in dataDict2dataForm:
rel 372
2022-09-21 22:43:55 +02:00
Goffi
7793a482a2 plugin events: update following Events protoXEP submission:
update the plugin to follow the specification proposed at
https://github.com/xsf/xeps/pull/1206

Events are internally converted to a dict following a format described in
`event_data_2_event_elt` docstring.

RSVP mechanism now uses Pubsub Attachments (XEP-0470), with a user extensible data form.

Bridge methods signatures have changed.

rel 372
2022-09-21 22:41:49 +02:00
Goffi
cfbbe66358 plugin XEP-0470: add create_attachments_node method:
this method create the attachments node if it doesn't already exists, and use same access
model as target item, and an `open` publish model.

rel 372
2022-09-21 22:36:30 +02:00
Goffi
2389f235f1 plugin XEP-0050: use coroutine + minor code readability improvments:
rel 372
2022-09-21 22:33:59 +02:00
Goffi
2502e5062a plugin XEP-0060: raise an NotFound exception instead of StanzaError in subscribe:
`exceptions.NotFound` is now raised in stead of `item-not-found` `StanzaError` for
consistency with `getItems` and because it's easier to catch (condition has to be checked
when `StanzaError` is received).

rel 372
2022-09-21 22:32:36 +02:00
Goffi
1e4fc213f6 tools (utils): helper parse_xmpp_date method:
This method converts an XMPP date(time) to a unix time

rel 372
2022-09-21 22:29:54 +02:00
Goffi
e80df24bb1 plugin XEP-0447: Stateless File Sharing implementation:
rel 379
2022-09-21 22:28:20 +02:00
Goffi
a884331ed2 plugin XEP-0446: File Metadata implementation:
rel 379
2022-09-21 22:27:28 +02:00
Goffi
d30b63b617 plugin XEP-0103: URL Address Information implementation:
rel 379
2022-09-21 22:26:48 +02:00
Goffi
fe1deb10cd plugin XEP-0080: User Location implementation:
rel 372
2022-09-21 22:23:57 +02:00
Goffi
a74f9327f2 core, CLI (base), tools (common/data_format): typing/core readability 2022-09-21 22:20:43 +02:00
Goffi
84e6d9ceb2 doc (components/AP gateway): "reactions":
fix 371
2022-08-31 17:07:03 +02:00
Goffi
4d2263f54b tests (AP gateway): reactions tests:
2 new tests check XMPP <=> AP reactions conversion

rel 371
2022-08-31 17:07:03 +02:00
Goffi
85afb52968 tests (AP gateway): fix tests following changes in gateway:
- `shareInbox` is now used when available, thus tests now check for that
- `client._ap_storage` is used, thus is added to the mockClient
  `ap_const.NS_AP_PUBLIC` is now in `cc` instead of `to` field

rel 371
2022-08-31 17:07:03 +02:00
Goffi
68fb304032 cli (pubsub/attachment/set): update reactions handling:
- reactions data has been updated following changes in XEP-0470 plugin
- it is now possible to remove a reaction using uppercase `-R`
- short option from `--replace` has been removed, as it is more handy to use `-R` to
  remove reactions

rel 371
2022-08-31 17:07:03 +02:00
Goffi
5e3d4ca581 component AP gateway: AP <=> XMPP reactions conversions:
- Pubsub Attachments plugin has been renamed to XEP-0470 following publication
- XEP-0470 has been updated to follow 0.2 changes
- AP reactions (as implemented in Pleroma) are converted to XEP-0470
- XEP-0470 events are converted to AP reactions (again, using "EmojiReact" from Pleroma)
- AP activities related to attachments (like/reactions) are cached in Libervia because
  it's not possible to retrieve them from Pleroma instances once they have been emitted
  (doing an HTTP get on their ID returns a 404). For now those cache are not flushed, this
  should be improved in the future.
- `sharedInbox` is used when available. Pleroma returns a 500 HTTP error when ``to`` or
  ``cc`` are used in a direct inbox.
- reactions and like are not currently used for direct messages, because they can't be
  emitted from Pleroma in this case, thus there is no point in implementing them for the
  moment.

rel 371
2022-08-31 17:07:03 +02:00
Goffi
7ddf0241c6 plugin XEP-0424: rename bridge method:
``messRetract`` is renamed to ``messageRetract`` for consistency with other message bridge
methods.
2022-08-31 17:07:03 +02:00
Goffi
943ef29e90 plugin XEP-0444: Message Reactions implementation:
rel 371
2022-08-31 17:07:03 +02:00