Commit graph libervia-backend/sat/memory
Author SHA1 Message Date
Goffi
4ca6c043ac core (memory/sqla): fix fileUpdate 2021-09-08 17:58:48 +02:00
Goffi
247a5f1ea8 core (memory/slqa): implement getProfileById
rel 361
2021-09-08 17:58:48 +02:00
Goffi
6c400f14c6 memory (sqla): implement searchPubsubItems:
`searchPubsubItems` is a high level method to handle Full-Text Search queries on Pubsub
cache.

rel 361
2021-09-08 17:58:48 +02:00
Goffi
39bdd34d60 db (schema): create virtual table for FTS + migration
rel 361
2021-09-08 17:58:48 +02:00
Goffi
50d7cb2a51 merge bookmark @ 2021-09-08 11:20:37 +02:00
Goffi
5cf43b2e2e core (memory/persistent): implement __repr__ (via __str__) 2021-09-08 11:17:04 +02:00
Goffi
a6267ac1fa core (memory/sql_mapping): remove autoincrement:
SQLite specific behaviour with autoincrement (https://sqlite.org/autoinc.html) is actually
already handled by SQLAlchemy, and unsetting autoincrement cause troubles when storing new
objects.
2021-09-01 15:25:02 +02:00
Goffi
efcd128062 core (memory/sqla): fix key/value inversion in self.profiles 2021-09-01 15:24:59 +02:00
Goffi
cb0223b5ab core (memory/sqla): minor docstring line lenght fix 2021-08-02 21:52:17 +02:00
Goffi
53a77a4c71 core (memory/sqla): generic methods to add and delete ORM objects directly 2021-07-29 22:51:01 +02:00
Goffi
85ea69c3c3 core (memory/sqla): methods to manipulate pubsub tables 2021-07-29 22:51:01 +02:00
Goffi
fdc44cdd49 migration: script to create pubsub caching tables 2021-07-29 22:50:57 +02:00
Goffi
4fb32ced31 core (memory/sqla_mapping): mapping for PubsubNode and PubsubItem (will be used for caching) 2021-07-29 21:38:37 +02:00
Goffi
a3e6425eb2 core (memory/sqla_mapping): fix some technical debt:
a first Alembic revision file is joined to migrate existing databases to new mapping.

This patch:
    - set `nullable=False` in various places
    - drop legacy `message_types` table, and use `Enum` instead
    - use naming convention, and get rid of anonymous constraint (this is the main reason
      why the revision file is so long)
2021-06-27 00:15:40 +02:00
Goffi
8ae38a1131 core (memory/sqla): database migration using Alembic:
Alembic database migration tool, which is the recommended one for SQLAlchemy has been
integrated. When a database is created, it will be used to stamp to current (head)
revision, otherwise, DB will be checked to see if it needs to be updated, and upgrade will
be triggered if necessary.
2021-06-25 17:55:23 +02:00
Goffi
71db80fc4d memory (storage): adjustements to SQLAlchemy mapping to match current state of database 2021-06-25 10:17:34 +02:00
Goffi
4421eefbf3 merge changes from main branch 2021-06-17 13:05:58 +02:00
Goffi
211c9c8079 core (memory/cache): don't crash on EOFError in getMetadata 2021-06-17 10:35:42 +02:00
Goffi
7adbf3ead8 core, frontends: fix invalid translatable strings.
Some f-strings where used in translatable text, this has been fixed by using explicit
`format()` call (using a script based on `tokenize`).

As tokenize messes with spaces, a reformating tool (`black`) has been applied to some
files afterwards.
2021-06-14 18:35:12 +02:00
Goffi
d22bd6c96d memory(sqlite): fix encoding when loading pickled data 2021-06-09 10:01:56 +02:00
Goffi
fd4f70ac6c memory (sqlite): work around encoding issues with pickle 2021-06-04 14:07:58 +02:00
Goffi
97df5f8be3 core: various changes and fixes to work with new storage and D-Bus bridge:
- fixes coroutines handling in various places
- fixes types which are not serialised by Tx DBus
- XEP-0384: call storage methods in main thread in XEP: Python OMEMO's Promise use thread
  which prevent the use of AsyncIO loop. To work around that, callLater is used to launch
  storage method in main thread. This is a temporary workaround, as Python OMEMO should
  get rid of Promise implementation and threads soon.
2021-06-03 15:21:43 +02:00
Goffi
a5f225e1ea core (memory/storage): backend storage is now based on SQLAlchemy 2021-06-03 15:20:47 +02:00
Goffi
838b4a7947 core (memory/sqlite): return 0 instead of None when user has no file yet 2021-05-06 21:39:08 +02:00
Goffi
06054ceeaf core (memory, sqlite): new fileGetUsedSpace method 2021-05-05 15:37:21 +02:00
Goffi
18cbbee1a5 memory: new getFilesOwner method:
this method is used by `getFiles` and `setFile` and fill `owner` when it's missing to
avoid getting accidentally files from other users.

`owner` is not filled in some case (no client, `file_id` or `parent` specified) and an
exception is raised if a component has neither `owner` nor `peer_jid` specified.
2021-04-16 18:32:37 +02:00
Goffi
018d09caaa Renamed the project from "Salut à Toi" to "Libervia":
This renaming follows the decision taken at the General Assemblee of January 27th 2021 (cf.
https://repos.goffi.org/sat_docs/raw-file/tip/association/2021-01-27%20-%20compte%20rendu%20de%20l%27AG%20extraordinaire.pdf)
2021-03-20 20:42:04 +01:00
Goffi
dc86957043 date update 2021-03-19 15:45:14 +01:00
Goffi
b6d26f3a67 core (memory/params): removed legacy "Register new account" label, it's not used anymore 2020-11-12 14:53:15 +01:00
Goffi
0250053e19 core (memory/memory): added getCachePath method to get a cache dir from a namespace. 2020-09-28 17:00:14 +02:00
Goffi
f3bb26f9d9 core (memory/memory): public_id should not be used when creating parent dirs 2020-09-17 23:07:12 +02:00
Goffi
280f86881a core (memory/memory): fixed directory creation in setFile 2020-09-17 21:40:23 +02:00
Goffi
a82f7108a2 core (memory/memory): use str for path in getFiles and setFile to allow use of Path instances 2020-09-17 16:34:22 +02:00
Goffi
35920d53af core (memory/memory): fixed assignations 2020-09-05 20:20:05 +02:00
Goffi
ba8496ae36 plugin XEP-0264: only log "fix orientation" when the option is used 2020-08-25 08:52:46 +02:00
Goffi
ea5e0a46f5 core (memory): setFileAccessModel implementation:
this method let change read permission using access model similar to what is used with
Pubsub.
2020-08-01 16:07:39 +02:00
Goffi
97375ee1db core (memory): getFileAffiliations and setFileAffiliations implementations:
those methods allow to change files permissions using affiliations similar to the ones
used with PubSub.
2020-08-01 16:06:12 +02:00
Goffi
800fbd3f7c core (memory): files are now public by default except for those in root directory:
using public permissions by default makes permissions change more easy as we don't have to
recursively change all permissions of sub files when changing directory. Root directories
are private by default.
2020-08-01 16:02:41 +02:00
Goffi
cb49365811 memory: new "affiliation" metadata for files:
this affiliation is similar to pubsub affiliation, for now only `owner` is used.
Affiliation is set when possible (peer_jid is set notably).
2020-07-17 12:58:57 +02:00
Goffi
4e48a889c0 core (memory): fixed recursive file deletion + log an error and continue when deleting a missing file 2020-06-19 14:55:30 +02:00
Goffi
d1521dde4b core (memory/sqlite): new database schema (v9):
this new schema brings the new `public_id` column, which will be used to serve files via
HTTP, splits `mime_type` in 2 (`media_type` and `media_subtype`) to make indexing/media
filtering easier, and fixes indexes for `files` table.
2020-05-29 21:50:49 +02:00
Goffi
a1eb68c1e7 core: replaced sat shell script by a python script:
the backend is now launched by `sat.core.launcher`, and the script is generated during
installation with a new entry point in `setup.py`.
2020-05-25 15:50:01 +02:00
Goffi
88072e0453 core, frontends: avatar refactoring:
/!\ huge commit

Avatar logic has been reworked around the IDENTITY plugin: plugins able to handle avatar
or other identity related metadata (like nicknames) register to IDENTITY plugin in the
same way as for other features like download/upload. Once registered, IDENTITY plugin will
call them when suitable in order of priority, and handle caching.

Methods to manage those metadata from frontend now use serialised data.

For now `avatar` and `nicknames` are handled:

- `avatar` is now a dict with `path` + metadata like `media_type`, instead of just a string
  path
- `nicknames` is now a list of nicknames in order of priority. This list is never empty,
  and `nicknames[0]` should be the preferred nickname to use by frontends in most cases.
  In addition to contact specified nicknames, user set nickname (the one set in roster) is
  used in priority when available.

Among the side changes done with this commit, there are:

- a new `contactGet` bridge method to get roster metadata for a single contact
- SatPresenceProtocol.send returns a Deferred to check when it has actually been sent
- memory's methods to handle entities data now use `client` as first argument
- metadata filter can be specified with `getIdentity`
- `getAvatar` and `setAvatar` are now part of the IDENTITY plugin instead of XEP-0054 (and
  there signature has changed)
- `isRoom` and `getBareOrFull` are now part of XEP-0045 plugin
- jp avatar/get command uses `xdg-open` first when available for `--show` flag
- `--no-cache` has been added to jp avatar/get and identity/get
- jp identity/set has been simplified, explicit options (`--nickname` only for now) are
  used instead of `--field`. `--field` may come back in the future if necessary for extra
  data.
- QuickContactList `SetContact` now handle None as a value, and doesn't use it to delete the
  metadata anymore
- improved cache handling for `metadata` and `nicknames` in quick frontend
- new `default` argument in QuickContactList `getCache`
2020-04-14 21:00:33 +02:00
Goffi
68a4d8ea98 core (memory/persistent): fixed "items", added "all" + "adel" to LazyPersistentBinaryDict:
- `items` was returning the dict with all values, it now returns a Deferred with actual dict items()
- `all` new method does return the dict with all values (in a Deferred)
- `adel` has been added to LazyPersistentBinaryDict
2020-04-14 20:36:24 +02:00
Goffi
83260b827a core (memory/disco): fixed types of disco extensions:
typeCheck() is not automatically called on reception by Wokkel, as a result extensions
fields values may be strings instead of the field type. TypeCheck is now explicitly
called in memory.disco to avoid that. It is not called immediately on reception as the
string value is needed to calculate the capability hash
2020-04-14 20:25:05 +02:00
Goffi
b878c3327f core (memory/encryption): use utils.asDeferred for trust UI, so async coroutines can be used 2020-03-27 09:53:59 +01:00
Goffi
973beb4220 core (memory/persistent): added aset method in LazyPersistentBinaryDict:
this method does the same as `force` but is there to be consistent with PersistentDict.
`force` should be removed in the future.
2020-03-27 09:53:00 +01:00
Goffi
2f8902e0d6 core (memory/encryption): automatic start encryption if peer send encrypted message:
If peer sends encrypted message and we have no encryption activated, we automatically
start encryption to avoid sending plain text message when answering.

markAsEncrypted now needs the encryption algorithm namespace as mandatory argument.
2020-03-23 17:52:18 +01:00
Goffi
8e9905363e core (memory/encryption): set encrypted flag also for outgoing messages and put it in extra:
- `encrypted` flag is now put directly in extra, so that there is nothing to change when
  sending the message to bridge for frontends to be aware of it.
- this flag is now also set when markAsEncrypted is used, so frontend can display
  appropriate indicator to show that outgoing message is encrypted or not.
2020-03-23 16:53:03 +01:00
Goffi
900daa7ad9 core (memory/encryption): encryption session are now restored on client connection 2020-03-22 18:39:12 +01:00