Commit graph libervia-backend/sat/memory
Author SHA1 Message Date
Goffi
11f16fc2d0 core (memory/encryption): helper method to get currently utilised e2ee algorithm's namespace:
rel 379
2022-10-06 16:02:05 +02:00
Syndace
75ae8160ae plugin XEP-0384: Fully reworked to adjust to the reworked python-omemo:
- support for both (modern) OMEMO under the `urn:xmpp:omemo:2` namespace and (legacy) OMEMO under the `eu.siacs.conversations.axolotl` namespace
- maintains one identity across both versions of OMEMO
- migrates data from the old plugin
- includes more features for protocol stability
- uses SCE for modern OMEMO
- fully type-checked, linted and format-checked
- added type hints to various pieces of backend code used by the plugin
- added stubs for some Twisted APIs used by the plugin under stubs/ (use `export MYPYPATH=stubs/` before running mypy)
- core (xmpp): enabled `send` trigger and made it an asyncPoint

fix 375
2022-08-23 21:06:24 +02:00
Goffi
a74f9327f2 core, CLI (base), tools (common/data_format): typing/core readability 2022-09-21 22:20:43 +02:00
Goffi
8bae5e3d7e core (memory/sqla): add argument to getPubsubNode to auto-create a node:
if the node is not in cache, the new `create` and `create_kwargs` arguments can be used to
automatically call `setPubsubNode`

rel 370
2022-07-20 17:41:44 +02:00
Goffi
f2e11805b2 core (memory/cache): type hints + filename fix:
cached file filename is now always UID + extension to avoid collision. `filename` argument
as been renamed to `original_filename`, it is store if present, and used to guess media
type when necessary.
2022-06-29 11:54:53 +02:00
Goffi
ead62068ce core (memory/sqla_config): set a timeout for SQLite:
with default timeout, we may have database lock errors from time to time. To avoid this, a
timeout of 30s is now set.
2022-06-29 10:17:33 +02:00
Goffi
a2c409a4a2 core (memory/sqla): session_add must have a default value in delete 2022-06-29 10:16:03 +02:00
Goffi
0bd68d0e62 core (memory/sqla): generic get + session_add in delete:
new generic `get` allow to retrieve an ORM object with given column and value, and
optionally join load attributes.

In `delete`, it is now possible to specify instances to add to session, this allow, to
update them in the same transaction. It is also now possible to specify a list of object
to delete them all in the same transaction.

rel 367
2022-06-17 14:15:23 +02:00
Goffi
bb3e7ffa9b core (memory/sqla), plugin XEP-0359: always add origin-id, and store:
`origin-id` is now always added to messages, and it is stored to database in the new
column instead of `extra` when present.
2022-06-17 14:15:23 +02:00
Goffi
53ca566a9d core (memory/mapping): add origin_id column to History + constraints update:
- `origin_id` is added as a column instead of being just in extra, as it an important data
to filter on.
- Add some constraints.
- Add `serialise` method to Message and Subject.
2022-06-17 14:15:23 +02:00
Goffi
c8df7104eb core (memory): fileDelete is now async, fix coroutine issue 2022-05-16 14:37:58 +02:00
Goffi
c8025a52eb core: method to know if a profile/entity is an admin:
the new `is_admin` SatXMPPEntity property tell if it's an administrator.

Admin JIDs are retrieve on init, so they can be looked after when profle is not available
(notably when a component handle a request and has only a JID available). The new
`memory.isAdminJID` method is then used.
2022-05-15 14:14:52 +02:00
Goffi
c678f1f993 core (memory/sqla): searchPubsubItems can now filter on item names 2022-05-13 18:02:37 +02:00
Goffi
75684953be core (memory/sqla): add attribute to filter on item_ids in getItems 2022-05-13 17:58:51 +02:00
Goffi
033e18d996 core (memory/param): fix unproper use of Failure 2022-05-13 17:57:24 +02:00
Goffi
4a201bc0f2 core (memory/disco): fix hard encoded value in findServiceEntity 2022-05-13 17:56:40 +02:00
Goffi
608944d9d9 core (disco): use new client.server_jid property in getItems 2022-05-13 17:55:51 +02:00
Goffi
317e8a6a94 core (memory/sqla): new table/mapping to handle Pubsub node subscriptions:
node subscriptions can now be cached, this can be useful for components which must keep
track of subscibers.

rel 364
2022-03-22 17:00:42 +01:00
Goffi
f90d76aa18 core (memory/migration): ignore FTS table when autogenerating script for migration:
SQLite Full-Text Search stable are not associated to Python object and can't be detected
by Alembic. To avoid the generation of unwanted drop commands, they are now ignored when
autogenerating migration scripts.

rel 364
2022-03-22 17:00:42 +01:00
Goffi
5c6bac6ade memory (sqla): fix complete setting when index is None in getItems 2022-01-25 17:09:54 +01:00
Goffi
d7945138f7 memory (sqla): don't set keys with None as value for RSM in getItems 2022-01-25 17:09:11 +01:00
Goffi
7ed5c0ff9d memory (disco), core (main): new hasIdentity method 2022-01-25 17:07:15 +01:00
Goffi
7b05d97229 merge bookmark 0.9 2021-12-01 16:13:31 +01:00
Goffi
97f6d1010a core (memory/param): don't crash when getting unset int value 2021-11-30 17:28:03 +01:00
Goffi
4ca6c043ac core (memory/sqla): fix fileUpdate 2021-09-08 17:58:48 +02:00
Goffi
247a5f1ea8 core (memory/slqa): implement getProfileById
rel 361
2021-09-08 17:58:48 +02:00
Goffi
6c400f14c6 memory (sqla): implement searchPubsubItems:
`searchPubsubItems` is a high level method to handle Full-Text Search queries on Pubsub
cache.

rel 361
2021-09-08 17:58:48 +02:00
Goffi
39bdd34d60 db (schema): create virtual table for FTS + migration
rel 361
2021-09-08 17:58:48 +02:00
Goffi
50d7cb2a51 merge bookmark @ 2021-09-08 11:20:37 +02:00
Goffi
5cf43b2e2e core (memory/persistent): implement __repr__ (via __str__) 2021-09-08 11:17:04 +02:00
Goffi
a6267ac1fa core (memory/sql_mapping): remove autoincrement:
SQLite specific behaviour with autoincrement (https://sqlite.org/autoinc.html) is actually
already handled by SQLAlchemy, and unsetting autoincrement cause troubles when storing new
objects.
2021-09-01 15:25:02 +02:00
Goffi
efcd128062 core (memory/sqla): fix key/value inversion in self.profiles 2021-09-01 15:24:59 +02:00
Goffi
cb0223b5ab core (memory/sqla): minor docstring line lenght fix 2021-08-02 21:52:17 +02:00
Goffi
53a77a4c71 core (memory/sqla): generic methods to add and delete ORM objects directly 2021-07-29 22:51:01 +02:00
Goffi
85ea69c3c3 core (memory/sqla): methods to manipulate pubsub tables 2021-07-29 22:51:01 +02:00
Goffi
fdc44cdd49 migration: script to create pubsub caching tables 2021-07-29 22:50:57 +02:00
Goffi
4fb32ced31 core (memory/sqla_mapping): mapping for PubsubNode and PubsubItem (will be used for caching) 2021-07-29 21:38:37 +02:00
Goffi
a3e6425eb2 core (memory/sqla_mapping): fix some technical debt:
a first Alembic revision file is joined to migrate existing databases to new mapping.

This patch:
    - set `nullable=False` in various places
    - drop legacy `message_types` table, and use `Enum` instead
    - use naming convention, and get rid of anonymous constraint (this is the main reason
      why the revision file is so long)
2021-06-27 00:15:40 +02:00
Goffi
8ae38a1131 core (memory/sqla): database migration using Alembic:
Alembic database migration tool, which is the recommended one for SQLAlchemy has been
integrated. When a database is created, it will be used to stamp to current (head)
revision, otherwise, DB will be checked to see if it needs to be updated, and upgrade will
be triggered if necessary.
2021-06-25 17:55:23 +02:00
Goffi
71db80fc4d memory (storage): adjustements to SQLAlchemy mapping to match current state of database 2021-06-25 10:17:34 +02:00
Goffi
4421eefbf3 merge changes from main branch 2021-06-17 13:05:58 +02:00
Goffi
211c9c8079 core (memory/cache): don't crash on EOFError in getMetadata 2021-06-17 10:35:42 +02:00
Goffi
7adbf3ead8 core, frontends: fix invalid translatable strings.
Some f-strings where used in translatable text, this has been fixed by using explicit
`format()` call (using a script based on `tokenize`).

As tokenize messes with spaces, a reformating tool (`black`) has been applied to some
files afterwards.
2021-06-14 18:35:12 +02:00
Goffi
d22bd6c96d memory(sqlite): fix encoding when loading pickled data 2021-06-09 10:01:56 +02:00
Goffi
fd4f70ac6c memory (sqlite): work around encoding issues with pickle 2021-06-04 14:07:58 +02:00
Goffi
97df5f8be3 core: various changes and fixes to work with new storage and D-Bus bridge:
- fixes coroutines handling in various places
- fixes types which are not serialised by Tx DBus
- XEP-0384: call storage methods in main thread in XEP: Python OMEMO's Promise use thread
  which prevent the use of AsyncIO loop. To work around that, callLater is used to launch
  storage method in main thread. This is a temporary workaround, as Python OMEMO should
  get rid of Promise implementation and threads soon.
2021-06-03 15:21:43 +02:00
Goffi
a5f225e1ea core (memory/storage): backend storage is now based on SQLAlchemy 2021-06-03 15:20:47 +02:00
Goffi
838b4a7947 core (memory/sqlite): return 0 instead of None when user has no file yet 2021-05-06 21:39:08 +02:00
Goffi
06054ceeaf core (memory, sqlite): new fileGetUsedSpace method 2021-05-05 15:37:21 +02:00
Goffi
18cbbee1a5 memory: new getFilesOwner method:
this method is used by `getFiles` and `setFile` and fill `owner` when it's missing to
avoid getting accidentally files from other users.

`owner` is not filled in some case (no client, `file_id` or `parent` specified) and an
exception is raised if a component has neither `owner` nor `peer_jid` specified.
2021-04-16 18:32:37 +02:00