Commit graph libervia-backend/libervia/backend/memory/sqla.py
Author SHA1 Message Date
Goffi
50ffc8c3b4 core, memory (sqla), plugins XEP-0277, XEP-0346: various fixes:
- `is_local`: now use `server_jid`.
- `aio_func`: pass the missing `self` argument (`aio_fund` is unbound).
- Declare `pubsub_service` in `core_types.SatXMPPEntity`
- plugin XEP-0277: fix handling of `MbData.from_element` when node is not specified (so we
  use default node).
- plugin XEP-0346: `IdentityData`: fix field access, following move to Pydantic model.
2026-10-27 16:04:50 +01:00
Goffi
2be5d62dfc memory (sqla): add after_uid filter in history_get 2026-08-26 17:55:01 +02:00
Goffi
b88d335d92 memory (sqla): Get original message in thread in history_get:
When using `thread_id` filter, original message could be missing (notably with a reply).
We now include it with `History.uid == filters["thread_id"]`.
2026-07-21 19:13:37 +02:00
Goffi
e241075a34 core: Unseen message counter:
This patches add handling for unseen messages counter, and last message received. The
counters are retrieved from DB on startup, they updated in-memory. Frontend are expected
to have their own in-memory counter too, the backend one is used for re-sync (e.g.; on page
reload).

Bridge methods are added to retrieve the state, and indicate which message has been seen
last by user. A signal is sent on counter reset to all frontends can synchronise.

This is currently local only, but it should be compatible with a future XEP-0490 (Message
Displayed Synchronization) implementation.
2026-06-20 15:11:06 +02:00
Goffi
23cbd8ff9d memory (sqla), bridge: New history_last_messages_metadata_get method:
This method is used to retrieve last messages and their timestamps. It's useful on frontend
startup to display data on last discussions.
2026-05-25 17:52:26 +02:00
Goffi
e5646c9a9d memory: add WAL checkpoint and unload:
- Add `wal_checkpoint()` method to manually checkpoint Write-Ahead logs.
- Do the checkpoint on startup to start clean and avoid delay on first login.
- Add an `unload` method to clean DB on exit, and do a checkpoint there.
2026-05-02 15:55:18 +02:00
Goffi
b7b5797262 tools (utils): Add aio_func attribute to @aio decorated methods:
We sometimes to need to call another method in asyncio context when we already are in this
context. With the `@aio` decorator which makes the method return a `Deferred`, it was
necessary to use a ping pong between `Deferred` and `Future`.
This patch adds a `aio_func` attribute which directly returns the wrapped asyncio method,
so it can now be awaited directly.
2026-05-02 15:55:18 +02:00
Goffi
610eac0718 Formatting + type hints 2026-05-02 15:55:18 +02:00
Goffi
42c2f88b30 component pubsub: PEP presence access model and auto-subscription
- Implement `presence` access model for PEP nodes.
- Add PEP auto-subscription (`+notify` handling)
- New `get_pep_last_items` storage method to retrieve in a single query items to send to
  auto-subscribers.
- Online resource tracking.

This completes the PEP support.

rel 476
2026-04-15 20:51:51 +02:00
Goffi
3aa58e73dc memory (sqla): new pep column in PubsubNode:
This column can be used for quick filtering on PEP nodes.

rel 476
2026-04-14 23:10:19 +02:00
Goffi
da2c2c49ea component pubsub: Implement "max_items":
A new `max_items` field has been added to `PubsubNode`. `0` means `max`. If any >0 value
is used, items will be automatically deleted by `cache_pubsub_items`. Items are currently
discarded (i.e. no notification is sent), this is the same behaviour as in legacy Libervia
Pubsub (formerly "SàT Pubsub").

rel 476
2026-04-11 12:47:40 +02:00
Goffi
f1ceb7db8e memory (sqla): Add with_parent and with_link to get_pubsub_nodes():
Added `with_parent` and `with_link` parameters for eager loading of parent and linked_node
relationships.

Fixed CTE to include all children (not just base nodes) and use `selectinload` for parent
to avoid cartesian products.

rel 476
2026-04-11 02:08:15 +02:00
Goffi
0c6921aad0 plugin XEP-0496: Pubsub Relationships implementation:
Add methods to handles the relationships which were already present in storage.
This plugins add methods to handle validation and rules from the specs.

`get_pubsub_node_children` helper method has been added to storage.
2026-04-11 01:55:15 +02:00
Goffi
4b91ea873e memory (sqla): New node_id, with_parent and with_link argument for get_pubsub_node:
- A node can now be retrieve directly by its ID.
- `with_parent` and `with_link` let load `parent_node` and `linked_node`.

rel 476
2026-04-10 22:02:53 +02:00
Goffi
378c6e3fef memory (sqla): docstrings/type hints improvements 2026-04-09 21:47:06 +02:00
Goffi
0a0735ec4c memory (sqla): Add methods to handle pubsub node subscriptions:
Add `get_pubsub_subscriptions` and `set_pubsub_subscriptions`.

rel 476
2026-04-09 10:11:29 +02:00
Goffi
34d6032cf5 component pubsub: Handle affiliations setting/retrieval:
Let retrieve affiliations for the whole service or a specific node, and set affiliations
for a node.

rel 476
2026-04-08 16:16:20 +02:00
Goffi
57d3d8e07e memory (sqla): Improve handling of RSM in get_items:
- update queries to handle correctly the `before=""` case and RSM order (older to newer)
  while having correct order when `max_items` without RSM is used (then order it newer to
  older).
- put RSM code path in a separated method for better readability.
- check values of `max_items` and `from_index` and raise exception is they are not
  positive
- when using `before` or `after`, check that the item ID used as reference exists, and
  raise an exception otherwise.
- updated docstring to reflect the new behaviour.
- update pubsub cache use of `get_items` with RSM, to adapt to new behaviour.

rel 476
2026-04-07 14:58:42 +02:00
Goffi
9cf71f2119 component Pubsub: initial implementation:
Base component implementation, handle node creation, listing, configuration and metadata.

rel 476
2026-04-07 11:48:53 +02:00
Goffi
68da01a043 Dates update. 2026-03-31 12:57:17 +02:00
Goffi
a405d6c8f7 memory (storage): add get_profile_by_name method 2026-03-17 17:55:03 +01:00
Goffi
82e2c6efac apply black to the whole codebase. 2026-03-09 09:22:04 +01:00
Goffi
e4e879d871 core (memory/sqla): Improve file_update by using raw value:
ORM was used, resulting in deserialisation/reserialisation of value checked for race
condition during update. In addition to the overhead, this could lead to modification even
if the value didn't change (e.g., because of JSON key order changed).

The raw value is not used to check if the row has not change between `select` and
`update`, making the method more robust.
2025-10-10 17:44:04 +02:00
Goffi
96bf3af324 Global reformatting with black. 2025-09-13 10:46:10 +02:00
Goffi
edb649eff7 memory (sqla): update set_pubsub_node and get_pubsub_nodes to handle pubsub relationships:
- `parent_node` and `linked_node` can now be specified in `set_pubsub_node`.
- `get_pubsub_nodes` has been rewritten to recursively traverse a tree and retrieve nodes
  matching criteria and their linking nodes/children/items.

rel 463
2025-08-29 18:52:58 +02:00
Goffi
26002870a8 memory (sqla): Add columns and tables to handles permissions:
Add columns to handle `access_model` and `publish_model` and a table for `affiliations`.
With those new data, components handling pubsub can now manage permissions correctly.

rel 462
2025-08-03 23:45:45 +02:00
Goffi
328e0ce775 memory (sqla): Add a filter for origin in history_get:
rel 459
2025-06-08 17:16:58 +02:00
Goffi
660dd6d538 memory (sqla): Add filter for keyword:
A `keyword` filter can now be used to retrieve messages matching given keyword.

rel 458
2025-06-04 17:52:18 +02:00
Goffi
7642c7b1db storage: Add a add_thread method:
This method is used to add a thread on an existing History record.

rel 457
2025-05-06 00:34:01 +02:00
Goffi
b2e64a9f03 storage: Add a thread_id filter to history_get:
rel 457
2025-05-06 00:34:01 +02:00
Goffi
931052f40c plugin XEP-0444, storage: move get_history_from_xmpp_id to storage as it is generally useful. 2025-05-06 00:33:57 +02:00
Goffi
53fdaa9d09 Reformatted code base using black. 2024-06-19 18:44:57 +02:00
Goffi
a8348576ae core: get rid of pickle:
Use of `pickle` to serialise data was a technical legacy that was causing trouble to store
in database, to update (if a class was serialised, a change could break update), and to
security (pickle can lead to code execution).

This patch remove all use of Pickle in favour in JSON, notably:
- for caching data, a Pydantic model is now used instead
- for SQLAlchemy model, the LegacyPickle is replaced by JSON serialisation
- in XEP-0373 a class `PublicKeyMetadata` was serialised. New method `from_dict` and
  `to_dict` method have been implemented to do serialisation.
- new methods to (de)serialise data can now be specified with Identity data types. It is
  notably used to (de)serialise `path` of avatars.

A migration script has been created to convert data (for upgrade or downgrade), with
special care for XEP-0373 case. Depending of size of database, this migration script can
be long to run.

rel 443
2024-02-23 13:31:04 +01:00
Goffi
191d06126e core (memory/sqla): add id filter in history_get 2023-11-28 17:26:45 +01:00
Goffi
b74f55d73d core: notifications implementation, first draft:
add a new table for notifications, and methods/bridge methods to manipulate them.
2023-10-16 17:29:31 +02:00
Goffi
1cd7d44e2e core (memory/sqla): fix row to dict conversion in get_files 2023-06-09 12:25:51 +02:00
Goffi
d2685ba11b refactoring: renamed sat package to libervia.backend 2023-06-02 11:49:51 +02:00
Renamed from sat/memory/sqla.py (Previous history)