Commit graph libervia-backend/src/memory/sqlite.py
Author SHA1 Message Date
Goffi
f5f25e8ab5 misc: date update (yes it's a bit late :p ) 2017-11-03 14:31:42 +01:00
Goffi
3d6cac0403 core (memory/sqlite): typo 2017-09-22 18:04:14 +02:00
Goffi
ab2e625607 core (memory/sqlite): better stability:
- timeout increased to 15 s
    - if a query fail, it's retried up to 5 times
    - foreign key PRAGMA is executed on every new connection object (should fix non working foreign keys)

those are Q&D workarounds for issues with Sqlite, before switching to a higher level module to handle it correctly.
2017-09-21 21:53:28 +02:00
Goffi
06fd34fe5e memory (persistent, sqlite): fixed getPrivates bug with keys arguments and its use in LazyPersistentBinaryDict 2017-03-29 19:27:52 +02:00
Goffi
ceab0c6bdc memory (persistent, sqlite): better private values handling + new LazyPersistentBinaryDict:
- merged private values method handling in sqlite, and added a keys arguments to get only some keys
    - LazyPersistentBinaryDict allow to get values in database only when they are needed, saving memory for big data
2017-03-12 19:33:17 +01:00
Goffi
97337202cb core, jp: component handling + client handling refactoring:
- SàT can now handle components
    - plugin have now a "modes" key in PLUGIN_INFO where they declare if they can be used with clients and or components. They default to be client only.
    - components are really similar to clients, but with some changes in behaviour:
        * component has "entry point", which is a special plugin with a componentStart method, which is called just after component is connected
        * trigger end with a different suffixes (e.g. profileConnected vs profileConnectedComponent), so a plugin which manage both clients and components can have different workflow
        * for clients, only triggers of plugins handling client mode are launched
        * for components, only triggers of plugins needed in dependencies are launched. They all must handle component mode.
        * component have a sendHistory attribute (False by default) which can be set to True to allow saving sent messages into history
        * for convenience, "client" is still used in method even if it can now be a component
        * a new "component" boolean attribute tells if we have a component or a client
        * components have to add themselve Message protocol
        * roster and presence protocols are not added for components
        * component default port is 5347 (which is Prosody's default port)
    - asyncCreateProfile has been renamed for profileCreate, both to follow new naming convention and to prepare the transition to fully asynchronous bridge
    - createProfile has a new "component" attribute. When used to create a component, it must be set to a component entry point
    - jp: added --component argument to profile/create
    - disconnect bridge method is now asynchronous, this way frontends can know when disconnection is finished
    - new PI_* constants for PLUGIN_INFO values (not used everywhere yet)
    - client/component connection workflow has been moved to their classes instead of being a host methods
    - host.messageSend is now client.sendMessage, and former client.sendMessage is now client.sendMessageData.
    - identities are now handled in client.identities list, so it can be updated dynamically by plugins (in the future, frontends should be able to update them too through bridge)
    - profileConnecting* profileConnected* profileDisconnected* and getHandler now all use client instead of profile
2017-02-12 17:55:43 +01:00
Goffi
9b2923f161 memory (sqlite): introduced component table, schema updated:
components are special profiles, entry_point will be the plugin to launch when they are connected.
2017-02-07 00:15:03 +01:00
Goffi
9647e90943 backend (memory), frontends: improved history filtering:
a "filters" dictionnary is now use to filter, it can have, for now, filtering on:
        - "body": filter only on the body (equivalent to former "search" parameter, but not case sensitive)
        - "search": fitler on body + source resource
        - "types": allowed types
        - "not_types": forbidden types

    primitivus now do searching using "search", i.e. source resource is now taken into account (and search is now case insensitive)
2016-07-18 00:52:02 +02:00
Goffi
feadc8c89e memory (sqlite): fixed handling of extra (pickled data) by using sqlite3.Binary 2016-06-19 22:22:13 +02:00
Goffi
74ebd4cb66 memory (sqlite): better handling of IntegrityError 2016-06-19 22:22:12 +02:00
Goffi
f12803fa65 backend, frontend: messages refactoring (huge commit, not finished):
/!\ database schema has been modified, do a backup before updating

    message have been refactored, here are the main changes:

    - languages are now handled
    - all messages have an uid (internal to SàT)
    - message updating is anticipated
    - subject is now first class
    - new naming scheme is used newMessage => messageNew, getHistory => historyGet, sendMessage => messageSend
    - minimal compatibility refactoring in quick_frontend/Primitivus, better refactoring should follow
    - threads handling
    - delayed messages are saved into history
    - info messages may also be saved in history (e.g. to keep track of people joining/leaving a room)
    - duplicate messages should be avoided
    - historyGet return messages in right order, no need to sort again
    - plugins have been updated to follow new features, some of them need to be reworked (e.g. OTR)
    - XEP-0203 (Delayed Delivery) is now fully handled in core, the plugin just handle disco and creation of a delay element
    - /!\ jp and Libervia are currently broken, as some features of Primitivus

    It has been put in one huge commit to avoid breaking messaging between changes.
    This is the main part of message refactoring, other commits will follow to take profit of the new features/behaviour.
2016-05-24 22:11:04 +02:00
Goffi
6bbcbaf522 use of /usr/bin/env instead of /usr/bin/python in shebang 2016-04-17 14:45:26 +02:00
Goffi
348be27934 copyright update 2016-01-03 16:28:30 +01:00
Goffi
1a16f085d0 core (sqlite storage): do explicit delete to profile linked tables for deleteProfile 2015-11-30 20:42:14 +01:00
Goffi
32592acdf5 tmp (wokkel.rsm): use of super instead of direct call in PubSubRequest 2015-08-15 22:24:40 +02:00
souliane
67ff77e5de massive (preventive) addition of 'u' (unicode) before the strings passed to logging functions 2015-04-16 14:57:57 +02:00
Goffi
fbec02d396 2015 copyright dates update 2015-03-30 10:26:26 +02:00
souliane
767f3d50dc bridge, memory: add "search" attribute to getHistory (filter the results with a unix globbing pattern) 2014-10-04 10:25:32 +02:00
souliane
4f339272b7 memory: add Parameter "Chat history limit" 2014-09-24 13:49:43 +02:00
souliane
4c61bf8fad memory: getHistory with limit=0 now returns an empty list, use limit=None to return all messages 2014-10-03 12:43:59 +02:00
souliane
bb92c9fe74 memory (sqlite): fixes upgrade Deferred issue 2014-07-06 21:54:07 +02:00
souliane
4e2366a3d5 memory (sqlite): fixes upgrade to database v2 2014-06-12 15:56:41 +02:00
souliane
50503332d2 memory: synchronize the upgrade to database v2 to be sure libervia's profile password is read to be written in the config file before its encryption 2014-05-23 13:00:35 +02:00
souliane
c68b147398 memory: auto-update configuration file with libervia's passphrase when migrating the database 2014-05-23 09:59:15 +02:00
souliane
0d28c03d27 core, memory, bridge: added profile password + password encryption:
/!\ This changeset updates the database version to 2 and modify the database content!

Description:
    - new parameter General / Password to store the profile password
    - profile password is initialized with XMPP password value, it is stored hashed
    - bridge methods asyncCreateProfile/asyncConnect takes a new argument "password" (default = "")
    - bridge method asyncConnect returns a boolean (True = connection already established, False = connection initiated)
    - profile password is checked before initializing the XMPP connection

    - new private individual parameter to store the personal encryption key of each profile
    - personal key is randomly generated and encrypted with the profile password
    - personal key is decrypted after profile authentification and stored in a Sessions instance
    - personal key is used to encrypt/decrypt other passwords when they need to be retrieved/modified

    - modifying the profile password re-encrypt the personal key
    - Memory.setParam now returns a Deferred (the bridge method "setParam" is unchanged)
    - Memory.asyncGetParamA eventually decrypts the password, Memory.getParamA would fail on a password parameter

TODO:
    - if profile authentication is OK but XMPP authentication is KO, prompt the user for another XMPP password
    - fix the method "registerNewAccount" (and move it to a plugin)
    - remove bridge method "connect", sole "asyncConnect" should be used
2014-05-07 16:02:23 +02:00
souliane
1eeddf5900 better PEP-8 compliance 2014-05-07 15:44:31 +02:00
Goffi
501bc92528 core: use of the new core.log module:
/!\ this is a massive refactoring and was largely automated, it probably did bring some bugs /!\
2014-04-19 19:19:19 +02:00
souliane
fceb06836f memory: be sure that local_dir exists before creating a new database 2014-03-25 14:57:38 +01:00
souliane
03d4dcf029 plugin account: bug fix profile deletion from the database + unsubscribe the contacts 2014-03-24 16:21:14 +01:00
Goffi
a280080563 core (sqlite): automatic database update:
- new Updater class check database consistency (by calculating a hash on the .schema), and updates base if necessary
    - database now has a version (1 for current, 0 will be for 0.3's database), for each change this version will be increased
    - creation statements and update statements are in the form of dict of dict with tuples. There is a help text at the top of the module to explain how it works
    - if we are on a development version, the updater try to update the database automaticaly (without deleting table or columns). The Updater.generateUpdateData method can be used to ease the creation of update data (i.e. the dictionary at the top, see the one for the key 1 for an example).
    - if there is an inconsistency, an exception is raised, and a message indicate the SQL statements that should fix the situation.
    - well... this is rather complicated, a KISS method would maybe have been better. The future will say if we need to simplify it :-/
    - new DatabaseError exception
2014-02-23 23:30:32 +01:00
souliane
e63752db76 memory (sqlite): the result of pickle.dumps in addToHistory must be encoded for the special characters to be stored. 2014-02-12 21:46:48 +01:00
Goffi
2ca6efd8cc dates update 2014-02-04 18:32:49 +01:00
Goffi
5a00dc9503 core: i18n refactoring:
- _() is no more installed in __builtin__
    - instead, there is a new sat.core.i18n module
    - added D_() method for deferred translation
    - languageSwitch method allow to dynamically change translation language
    - import gettext is tested against ImportError, and dummy methods are used when not available (mainly useful for Libervia)
2013-12-29 17:06:01 +01:00
souliane
63ba8ab235 memory: fixed getHistory behavior when parameter between=True:
- the previous behavior returns the messages between X and Y + between X and X
2013-10-23 00:28:00 +02:00
Goffi
8624c3c547 core, bridge: extra parameter is saved in history:
- added extra data in getHistory return value
    - extra data is saved in database
    /!\ WARNING: DATABASE SCHEMA HAS CHANGED /!\
to update sqlite database: ALTER TABLE history ADD COLUMN extra BLOB;
2013-11-07 15:29:39 +01:00
Goffi
4d08b7f72e core: fixed bad database creation on first run in sqlite storage 2013-06-20 17:17:35 +02:00
Goffi
1cd98e8dea fixed licences in docstrings (they are now in comments) 2013-03-08 00:36:22 +01:00
Goffi
e51a8a650b core: added forgotten errback in sqlite's deleteProfile 2013-01-21 00:59:50 +01:00
Emmanuel Gil Peyrot
45488f486c Fix pep8 support in src/memory. 2013-01-18 17:55:35 +01:00
Emmanuel Gil Peyrot
88074cd284 Remove every old-style class. 2013-01-18 17:55:35 +01:00
Emmanuel Gil Peyrot
27b3592e16 Remove trailing whitespaces. 2013-01-18 17:55:34 +01:00
Goffi
def79e44cd dates update 2013-01-07 23:30:13 +01:00
Goffi
252e4ccf6f core: sqlite storage constraint fix 2013-01-07 00:57:50 +01:00
Goffi
fa38394401 core, frontends, bridgen plugins: fixed methods which were unproperly managing multi-profiles
- added profile argument to askConfirmation, actionResult, actionResultExt, entityDataUpdated, confirmationAnswer, getProgress
    - core, frontends: fixed calls/signals according to new bridge API
    - user of proper profile namespace for progression indicators and dialogs
    - memory: getParam* now return bool when param type is bool
    - memory: added getStringParam* to return string instead of typed value
    - core, memory, storage, quick_frontend: getHistory now manage properly multi-profiles
    - plugins XEP-0047, XEP-0054, XEP-0065, XEP-0077, XEP-0096; multi-profiles proper handling
2012-11-10 16:38:16 +01:00
Goffi
fef2b1a620 core, bridge, quick_frontend: MUC private messages history management:
- history now store message type
    - sqlite3 storage: fixed resource management in getHistory
    - bridge: added message type in getHistory return value
    - quick_chats: (ugly) hacks to manage private history filtering/nickname printing
2012-10-17 00:35:48 +02:00
Goffi
c5a930b627 Licence modification: the full project is now under AGPL v3+ instead of GPL v3+ 2012-08-01 22:53:02 +02:00
Goffi
21dbb4473a copyleft date update 2012-03-06 09:07:24 +01:00
Goffi
53f1b5fcce core: - history management improved
- better timestamp precision for history
bridge + core: history signature change (now return a list instead of a dict, and timestamp is now float)
quick_frontend: - use of new history API
				- removed deprecated keep_last argument in getHistory (was only used by Sortilège)
2011-12-04 16:18:56 +01:00
Goffi
383f58333f core: fixed bad query in Sqlite storage 2011-12-03 22:21:27 +01:00
Goffi
75b502fff5 core: sqlite persistentBinaryDict storage fix 2011-11-30 00:23:50 +01:00