Commit graph

4,399 commits

Author SHA1 Message Date
Goffi
63edb309f8 component email message: Mailing list messages cleaning:
For mailing-list messages:
    - convert all text-only messages to XHTML for consistency.
    - detect and flag "noisy" content, i.e. content which are not useful and annoying in
      forum-like view (full-lenght quoted previous messages, mailing-list signature)

rel 464
2025-09-11 21:17:47 +02:00
Goffi
412db6a216 component email (imap): fetch all recent emails instead of only the last one. 2025-09-11 21:10:35 +02:00
Goffi
81f63ef93e plugin XEP-0277: add mentions to extra data:
rel 463
2025-09-06 16:31:24 +02:00
Goffi
5c9c57588c plugin XEP-0376: Fix type hint for node which is optional in subscriptions. 2025-09-05 19:32:47 +02:00
Goffi
0a42dd3da3 core (main): renamed hasFeature to has_feature to fix inconsistency. 2025-09-05 19:31:50 +02:00
Goffi
b816dc5e54 cli (pubsub): List support first draft + doc:
Basic support for listing hierarchy of nodes with pubsub extended discovery. Nodes are
displayed as a tree, depth can be specified.

rel 463
2025-08-31 12:34:37 +02:00
Goffi
57e71c326d component email gateway: implement pubsub relationships and pubsub extended discovery:
- Pubsub relationships are used to set node hierarchy (comments nodes are set as children of
corresponding pubsub nodes).
- XEP-0499 Extended Pubsub Discovery is used to return hierarchy of pubsub nodes and
  items.

rel 463
2025-08-31 12:34:37 +02:00
Goffi
70e76dd190 plugin XEP-0499 Pubsub Extended Discovery implementation:
- Implement Pubsub Extended Discovery, and Pydantic models to handle the data easily.
- `client.discoHandler` initialisation has been moved in `core.xmpp` at an earlier stage
  of the workflow as it has to be initialised so XEP-0499 could remove the default disco
  handler to handle disco itself.
- Removed `Disco handled` trigger point as it was not used.

rel 463
2025-08-31 12:34:30 +02:00
Goffi
5b18ba7313 plugin XEP-0060: Add a model to handle node metadata:
rel 463
2025-08-29 18:53:03 +02:00
Goffi
edb649eff7 memory (sqla): update set_pubsub_node and get_pubsub_nodes to handle pubsub relationships:
- `parent_node` and `linked_node` can now be specified in `set_pubsub_node`.
- `get_pubsub_nodes` has been rewritten to recursively traverse a tree and retrieve nodes
  matching criteria and their linking nodes/children/items.

rel 463
2025-08-29 18:52:58 +02:00
Goffi
e593f32f09 memory (sqla): add "parent_node_id" and "linked_mode_id" to PubsubNode:
This is to prepare for the implementation of XEP-0496 and XEP-0499.

rel 463
2025-08-29 17:59:12 +02:00
Goffi
f5a4b4b599 add .gitignore to .hgignore:
Some tools don't support Mercurial, and use `.gitignore`. Adding it here allow to have a
local `.gitignore` for dev local tools.
2025-08-29 11:44:43 +02:00
Goffi
009ef1aa4b doc (components): Document the handling of mailing lists in Email Gateway:
fix 462
2025-08-03 23:45:48 +02:00
Goffi
e1828b3672 component Email gateway: Convert mailing list to pubsub nodes:
- `Credentials` is now a Pydantic based model.
- Mailing list related emails are now detected hand saved as pubsub items, using XEP-0277
  blog items, and creating suitable nodes. The ID of the mailing list is used for root
  node.
- Mailing list items are currently restricted to recipient associated JID.
- Words in square bracket in title `[Like][That]` are removed and converted to blog
  categories.
- Method to convert between MbData and email (in both directions) have been created.

rel 462
2025-08-03 23:45:48 +02:00
Goffi
26002870a8 memory (sqla): Add columns and tables to handles permissions:
Add columns to handle `access_model` and `publish_model` and a table for `affiliations`.
With those new data, components handling pubsub can now manage permissions correctly.

rel 462
2025-08-03 23:45:45 +02:00
Goffi
3dbd990902 backend: various fixes, and type hints improvments. 2025-08-03 23:36:22 +02:00
Goffi
cab460152c plugin XEP-0277: Make MbData a Pydantic model:
- MbData is now a Pydantic based model.
- Bridge method has been updated to use `service` and `node` from MbData instead of using
  separate fields.
- Added `In-Reply-To` to MbData.
- Adapted code using XEP-0277 to work with the changes.

rel 462
2025-08-03 23:35:21 +02:00
Goffi
b1e0e336c2 plugin XEP-0297: Reworked forward method and add bridge method:
`Forward` method has been reworked and now includes a fallback. XEP-0297 ask to not use
fallback, but following a discussion on xsf@, we agreed that this is a legacy thing and a
fallback should nowadays be used, I'll propose a patch to the specification.

A `message_forward` has been added to bridge.

rel 461
2025-07-04 12:33:42 +02:00
Goffi
030be64c45 plugin XEP-0428: Add missing "for" attribute:
The "for" attribute was missing from <fallback> element.

rel 461
2025-07-04 12:30:20 +02:00
Goffi
16f9e07ce1 minor type update and unused import removal. 2025-07-04 12:28:40 +02:00
Goffi
0df0b0a5d0 core (types): Renamed libervia.backend.core.core_types.MessageData to MessageDataLegacy:
There are 2 concurrent models, this one is an historical legacy. The type to use the one
from `libervia.backend.models.core`, and it will progressively be used everywhere and
fully based on Pydantic.
2025-07-04 12:28:00 +02:00
Goffi
8302d09cf6 plugin data policy: Data Policy implementation:
This plugin implement data policy parsing and an algorithm to calculate a score based on
them.

rel 460
2025-06-26 17:02:33 +02:00
Goffi
6c95b3bec8 component email gateway: Add basic disco policy:
rel 460
2025-06-26 17:01:41 +02:00
Goffi
328e0ce775 memory (sqla): Add a filter for origin in history_get:
rel 459
2025-06-08 17:16:58 +02:00
Goffi
ceb2484a37 core (xmpp): add origin and origin_label to message data:
If a message comes from a gateway, its type is used as `origin` in `extra`, and it has a
name, it's used as `origin_label`.
`is-bot` is also set when the sending entity is advertised as a bot.

rel 459
2025-06-08 17:16:56 +02:00
Goffi
14f50afa64 tests (unit/sqla): Add test for keyword filtering:
rel 458
2025-06-06 10:45:54 +02:00
Goffi
660dd6d538 memory (sqla): Add filter for keyword:
A `keyword` filter can now be used to retrieve messages matching given keyword.

rel 458
2025-06-04 17:52:18 +02:00
Goffi
331e7dc5c4 plugin identity: add a timeout when getting identities. 2025-05-15 17:45:19 +02:00
Goffi
4a0d106b90 test (XEP-0461): some tests for XEP-0461:
rel 457
2025-05-06 00:34:01 +02:00
Goffi
51c40dd5f0 plugin XEP-0461: Message Replies implementation:
Implement message replies. Thread ID are always added when a reply is initiated from
Libervia, so a thread can continue the reply.

rel 457
2025-05-06 00:34:01 +02:00
Goffi
cfc4041b1d plugin XEP-0045: Fix _message_parse_trigger which was incorrectly breaking the trigger workflow:
`None` was returned in some case instead of `True`, breaking the trigger workflow.
2025-05-06 00:34:01 +02:00
Goffi
7642c7b1db storage: Add a add_thread method:
This method is used to add a thread on an existing History record.

rel 457
2025-05-06 00:34:01 +02:00
Goffi
b2e64a9f03 storage: Add a thread_id filter to history_get:
rel 457
2025-05-06 00:34:01 +02:00
Goffi
931052f40c plugin XEP-0444, storage: move get_history_from_xmpp_id to storage as it is generally useful. 2025-05-06 00:33:57 +02:00
Goffi
f2075750ed storage (mapping): Fix nullable for thread ID, and add a is_retroactive field:
`is_retroactive` is used to link a thread in a parent message when it didn't have the
thread initially.

rel 457
2025-05-06 00:21:24 +02:00
Goffi
764352afc7 storage: Add missing migration script to use JSON type instead of TEXT in History and private binary data. 2025-05-06 00:19:43 +02:00
Goffi
57f879f2d3 core (xmpp): parse <thread> element:
<thread> were generated from message data, but not parsed for incoming messages.

rel 457
2025-05-06 00:18:30 +02:00
Goffi
c008b65e63 core: Add a "Global" variable with a LiberviaBackend singleton to libervia.backend. 2025-05-06 00:16:15 +02:00
Goffi
58a0e168e2 core (xmpp): Add missing f-string prefix 2025-04-11 21:49:46 +02:00
Goffi
618fbcd750 plugin XEP-0433: Extended Channel Search implementation:
Implements client part of XEP-0433, and add its results to plugin JID Search.
2025-04-11 18:19:28 +02:00
Goffi
65c6aefd68 plugin XEP-0045: add MUC search to plugin JID search results. 2025-04-11 18:19:28 +02:00
Goffi
7f6081b54f plugin JID Search: Use Pydantic models + better handling of partial search. 2025-04-11 18:19:28 +02:00
Goffi
b6928f9b96 various minor fixes. 2025-04-11 18:19:28 +02:00
Goffi
174bf8de1a plugin XEP-0059: Use Pydantic models for RSM. 2025-04-11 18:19:28 +02:00
Goffi
0d40feedc5 types: Add SatXMPPClient to core types, add roster and improve type hints. 2025-04-11 18:19:28 +02:00
Goffi
0f42421804 template: Add initials filter. 2025-04-11 18:19:17 +02:00
Goffi
ce5ba7123b template: fix name handling for icons. 2025-04-11 18:18:50 +02:00
Goffi
2346dfb461 doc (components, encryption): add documentation on autocrypt for client and email gateway:
fix 456
2025-02-28 09:23:35 +01:00
Goffi
7eb5d8f371 plugin autocrypt: Autocrypt protocol implementation:
Implementation of autocrypt: `autocrypt` header is checked, and if present and no public
key is known for the peer, the key is imported.

`autocrypt` header is also added to outgoing message (only if an email gateway is
detected).

For the moment, the JID is use as identifier, but the real email used by gateway should be
used in the future.

rel 456
2025-02-28 09:23:35 +01:00
Goffi
2a0f4e28f8 component email gateway: convert autocrypt header:
Autocrypt header must be transmitted in both directions to allow opportunistic end-to-end
encryption with this protocol.

Moved email validation regex to `tools/common/regex.py`, as it can be used in other
locations.

rel 456
2025-02-28 09:23:35 +01:00