erebion/libervia-backend
Watch
1
0
Fork
You've already forked libervia-backend
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

core: Replace legacy INI config system with Pydantic-based TOML configuration.

Browse source 
This is a huge configuration refactoring.

The configuration system now uses Pydantic settings, with better validation, better
support for environment variables and other goodies.

The main config files now use TOML instead of INI, this allow for better handling of
complex types, notably dict.

Plugins, components and frontends can declare their config settings in a class which will
be imported dynamically.
This commit is contained in:
Goffi 2025-11-30 12:27:35 +01:00
commit 6a7b7bc36a
59 changed files with 1392 additions and 1011 deletions
Download patch file Download diff file Expand all files Collapse all files

145
doc/components.rst
View file

@ -104,7 +104,7 @@ entities (JIDs). Permissions can be set through Ad-Hoc Commands.
Configuration
~~~~~~~~~~~~~
All options are to be set in ``[component file-sharing]`` section.
All options are to be set in ``[components.file_sharing]`` section.
``http_upload_port``
port to use for HTTP File Upload
@ -181,148 +181,7 @@ documentation to find how this must be done.
Configuration
~~~~~~~~~~~~~
All options are to be set in ``[component ap-gateway]`` section.
``public_url``
Main user-facing domain of the HTTP server, this will be used to construct all AP URLs
**default**: if not set, ``xmpp_domain`` is used. If ``xmpp_domain`` is not set either,
an error is raised.
``http_port``
port where the HTTP server should listen. Port ``80`` is not used directly as it would
require root privileges, and it is strongly recommended against launching Libervia under
a privileged account. An HTTP Proxy or a port redirection should be set to redirect the
``80`` port to the port specified here.
**default**: ``8123``
``http_connection_type``
either ``http`` or ``https``. If you have a HTTP proxy such as Apache or NGINX which
already handles HTTPS, you may want to use ``http``.
**default**: ``https``
Note that the HTTP server should always use ``https`` with end-user, the ``http`` option
is only to be used with an HTTPS proxy.
``local_only``
A boolean value indicating if the gateway is allowed to convert pubsub node from
external XMPP service or not. A JID is considered external if its domain part doesn't
end with the gateway's server. For instance, if a gateway ``ap.example.org`` is set on
the server ``example.org``, the JIDs ``pierre@example.org`` or
``some-node@pubsub.example.org`` will be considered local, but
``buenaventura@example.net`` won't (note the different domain).
Most of time, ``local_only`` should be used.
**default**: ``true``
``ap_path``
Path prefix to use for ActivityPub request. It's usually not necessary to change the
default value.
**default**: ``_ap``
``comments_max_depth``
An integer value indicating the maximum number of comment nodes that can be created. See
:ref:`ap-xmpp-threads-conversion`
``auto_mentions``
A boolean value indicating if received XMPP pubsub blog items bodies must be scanned to
find ``@user@server.tld`` type mentions. If mentions are found, they'll be added to the
resulting AP items.
**default**: ``true``
``http_sign_get``
A boolean value indicating if HTTP ``GET`` request made to AP implementations must be
signed. Some AP implementations require those requests to be signed, notably Mastodon
when "secure mode" is activated.
**default**: ``true``
``html_redirect_dict``
A dictionary used to redirect HTTP requests when it's not an ActivityPub request (i.e.
when the ``Accept`` header is not set to ``application/json``). Several ActivityPub
implementations link to original server when clicking on some links such as post author
or item ID, and this result in a request to the corresponding ActivityPub endpoint, but
made to retrieve HTML instead of ActivityPub JSON data.
By default, this request is showing the corresponding AP JSON data, but if you set HTML
redirection, you can redirect the page to your XMPP client web frontend (or anything
making sense), which results in better user experience.
The expected dictionary is a mapping URL request types to destination URL.
The URL request types is the first path element after ``ap_path``, it can be:
``actor``
Actor information. It should link to the profile page of the corresponding XMPP entity
``item``
A specific publication. It should link to the rendered item
``followers``
Entities publicly subscribed to the actor. It is usually not used for redirection.
``following``
Entities that the actor is publicly subscribed to. It is usually not used for
redirection.
In the target URL you can uses template values in curly braces (something like
``{item}``). The values that you can use are:
``jid``
Full JID corresponding to the AP actor.
``jid_user``
Only the user part of the JID (what is before the ``@``)
``node``
Name of the pubsub node corresponding to the AP actor.
``item``
ID of the XMPP pubsub item.
You can use a slash ``/`` followed by a part of a node name to filter only on specific
nodes. For instance, if you want to redirect microblog items to a different URL that
event items, you can use ``item/urn:xmpp:microblog:0`` and ``item/urn:xmpp:events:0``,
the first redirection to match will be used. Note that anything containing the filter
value will match, thus you can also use ``item/microblog`` (but it then would match an
event which has ``microblog`` in is node name).
You can also use a dictionary as value instead of the target URL. This can be useful to
filter on an other value than the node name, if you want to make specific redirection
for a particular user for instance. If you use a dictionary, you must use a ``url`` key
with the target URL (which may have template values as usual), and you may use a
``filters`` key mapping to an other dictionary where each key is a filter on a specific
template value (in other words: key is the template value, like ``jid`` or ``node``, and
value is the filter to match).
*examples*
Redirect actor page to ``/profile/<JID user part>``, blogs items to ``/blog/<full
JID>/<node name>/<item id>`` and events items to ``/event/<full JID>/<node name>/<item
id>`` ::
html_redirect_dict = {
"actor": "/profile/{jid_user}",
"item/microblog": "/blog/{jid}/{node}/{item}",
"item/events": "/event/{jid}/{node}/{item}"
}
Redirect items of user ``louise@example.org`` to ``/b/<item>``::
html_redirect_dict = {
"item": {
"url": "/b/{item}",
filters: {
"jid": "louise@example.org"
}
}
}
All options are to be set in ``[components.ap_gateway]`` section.
.. _ap-actor-from-xmpp:

5
libervia/backend/__init__.py
View file

@ -26,6 +26,7 @@ if not wokkel.installed:
if TYPE_CHECKING:
from libervia.backend.core.main import LiberviaBackend
from libervia.backend.memory.sqla import Storage
from libervia.backend.tools.config import Settings
class Global:
@ -47,5 +48,9 @@ class Global:
def storage(self) -> "Storage":
return self.host.memory.storage
@property
def config(self) -> "Settings":
return self.host.memory.config
G = Global()

6
libervia/backend/bridge/bridge_constructor/constructors/dbus/dbus_core_template.py
View file

@ -30,9 +30,9 @@ from txdbus.interface import DBusInterface, Method, Signal
log = getLogger(__name__)
# Interface prefix
const_INT_PREFIX = config.config_get(
config.parse_main_conf(), "", "bridge_dbus_int_prefix", "org.libervia.Libervia"
)
conf = config.parse_main_conf()
bridge_conf = conf.common.get_bridge_conf(config.DBusBridgeSettings)
const_INT_PREFIX = bridge_conf.int_prefix
const_ERROR_PREFIX = const_INT_PREFIX + ".error"
const_OBJ_PATH = "/org/libervia/Libervia/bridge"
const_CORE_SUFFIX = ".core"

6
libervia/backend/bridge/bridge_constructor/constructors/dbus/dbus_frontend_template.py
View file

@ -32,9 +32,9 @@ log = getLogger(__name__)
# Interface prefix
const_INT_PREFIX = config.config_get(
config.parse_main_conf(), "", "bridge_dbus_int_prefix", "org.libervia.Libervia"
)
conf = config.parse_main_conf()
bridge_conf = conf.common.get_bridge_conf(config.DBusBridgeSettings)
const_INT_PREFIX = bridge_conf.int_prefix
const_ERROR_PREFIX = const_INT_PREFIX + ".error"
const_OBJ_PATH = "/org/libervia/Libervia/bridge"
const_CORE_SUFFIX = ".core"

9
libervia/backend/bridge/bridge_constructor/constructors/pb/pb_core_template.py
View file

@ -108,17 +108,16 @@ class bridge(object):
log.info("Init Perspective Broker...")
self.root = PBRoot()
conf = config.parse_main_conf()
get_conf = partial(config.get_conf, conf, "bridge_pb", "")
conn_type = get_conf("connection_type", "unix_socket")
bridge_conf = conf.common.get_bridge_conf(config.PbBridgePbSettings)
conn_type = bridge_conf.connection_type
if conn_type == "unix_socket":
local_dir = Path(config.config_get(conf, "", "local_dir")).resolve()
socket_path = local_dir / "bridge_pb"
socket_path = bridge_conf.socket_dir / "bridge_pb"
log.info(f"using UNIX Socket at {socket_path}")
reactor.listenUNIX(
str(socket_path), pb.PBServerFactory(self.root), mode=0o600
)
elif conn_type == "socket":
port = int(get_conf("port", 8789))
port = bridge_conf.port
log.info(f"using TCP Socket at port {port}")
reactor.listenTCP(port, pb.PBServerFactory(self.root))
else:

11
libervia/backend/bridge/bridge_constructor/constructors/pb/pb_frontend_template.py
View file

@ -137,16 +137,13 @@ class bridge(object):
def bridge_connect(self, callback, errback):
factory = pb.PBClientFactory()
conf = config.parse_main_conf()
get_conf = partial(config.get_conf, conf, "bridge_pb", "")
conn_type = get_conf("connection_type", "unix_socket")
bridge_conf = conf.common.get_bridge_conf(config.PbBridgePbSettings)
conn_type = bridge_conf.connection_type
if conn_type == "unix_socket":
local_dir = Path(config.config_get(conf, "", "local_dir")).resolve()
socket_path = local_dir / "bridge_pb"
socket_path = bridge_conf.socket_dir / "bridge_pb"
reactor.connectUNIX(str(socket_path), factory)
elif conn_type == "socket":
host = get_conf("host", "localhost")
port = int(get_conf("port", 8789))
reactor.connectTCP(host, port, factory)
reactor.connectTCP(bridge_conf.host, bridge_conf.port, factory)
else:
raise ValueError(f"Unknown pb connection type: {conn_type!r}")
d = factory.getRootObject()

6
libervia/backend/bridge/dbus_bridge.py
View file

@ -30,9 +30,9 @@ from txdbus.interface import DBusInterface, Method, Signal
log = getLogger(__name__)
# Interface prefix
const_INT_PREFIX = config.config_get(
config.parse_main_conf(), "", "bridge_dbus_int_prefix", "org.libervia.Libervia"
)
conf = config.parse_main_conf()
bridge_conf = conf.common.get_bridge_conf(config.DBusBridgeSettings)
const_INT_PREFIX = bridge_conf.int_prefix
const_ERROR_PREFIX = const_INT_PREFIX + ".error"
const_OBJ_PATH = "/org/libervia/Libervia/bridge"
const_CORE_SUFFIX = ".core"

9
libervia/backend/bridge/pb.py
View file

@ -108,17 +108,16 @@ class bridge(object):
log.info("Init Perspective Broker...")
self.root = PBRoot()
conf = config.parse_main_conf()
get_conf = partial(config.get_conf, conf, "bridge_pb", "")
conn_type = get_conf("connection_type", "unix_socket")
bridge_conf = conf.common.get_bridge_conf(config.PbBridgePbSettings)
conn_type = bridge_conf.connection_type
if conn_type == "unix_socket":
local_dir = Path(config.config_get(conf, "", "local_dir")).resolve()
socket_path = local_dir / "bridge_pb"
socket_path = bridge_conf.socket_dir / "bridge_pb"
log.info(f"using UNIX Socket at {socket_path}")
reactor.listenUNIX(
str(socket_path), pb.PBServerFactory(self.root), mode=0o600
)
elif conn_type == "socket":
port = int(get_conf("port", 8789))
port = bridge_conf.port
log.info(f"using TCP Socket at port {port}")
reactor.listenTCP(port, pb.PBServerFactory(self.root))
else:

168
libervia/backend/core/constants.py
View file

@ -16,12 +16,8 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
try:
from xdg import BaseDirectory
from os.path import expanduser, realpath
except ImportError:
BaseDirectory = None
from os.path import dirname, join
import os
from pathlib import Path
import tempfile
from typing import Final
@ -29,16 +25,22 @@ from libervia import backend
try:
from getpass import getuser
except ImportError:
logged_user = "unkonwn"
else:
logged_user = getuser()
except Exception as e:
logged_user = "unknown"
bool_type = bool
xdg_config_home = Path(os.environ.get("XDG_CONFIG_HOME") or Path.home() / ".config")
xdg_config_dirs = [
d for d in os.environ.get("XDG_CONFIG_DIRS", "/etc/xdg").split(":") if d
]
xdg_data_home = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share")
class Const(object):
class Const:
## Application ##
APP_NAME = "Libervia"
@ -50,6 +52,36 @@ class Const(object):
APP_RELEASE_NAME = "La Ruche"
APP_URL = "https://libervia.org"
## Configuration ##
CONFIG_PATHS = [
# /etc/_ is used for system-related settings (e.g. when media_dir is set by the
# distribution and has not reason to change, or in a Docker image)
Path("/etc/_"),
Path("/etc/"),
Path.home(),
Path.home() / ".",
xdg_config_home / APP_NAME,
# We use backend root as last config path, this is useful for per instance
# configurations.
Path(backend.__file__).parent.parent,
]
for d in xdg_config_dirs:
CONFIG_PATHS.append(Path(d) / APP_NAME)
# on recent versions of Flatpak, FLATPAK_ID is set at run time
# it seems that this is not the case on older versions,
# but FLATPAK_SANDBOX_DIR seems set then
if os.getenv("FLATPAK_ID") or os.getenv("FLATPAK_SANDBOX_DIR"):
# for Flatpak, the conf can't be set in /etc or $HOME, so we have
# to add /app
CONFIG_PATHS.append(Path("/app/"))
CONFIG_FILES: list[Path] = []
for p in CONFIG_PATHS:
CONFIG_FILES.append(p / f"{APP_NAME_FILE}.toml")
TEMP_DIR = Path(tempfile.gettempdir(), f"{APP_NAME_FILE}_{logged_user}")
## Runtime ##
PLUGIN_EXT = "py"
HISTORY_SKIP = "skip"
@ -222,6 +254,7 @@ class Const(object):
# to make the header more readable
PI_NAME = "name"
PI_IMPORT_NAME = "import_name"
PI_CONFIG_KEY = "config_key"
PI_MAIN = "main"
PI_HANDLER = "handler"
PI_TYPE = "type" #  FIXME: should be types, and should handle single unicode type or tuple of types (e.g. "blog" and "import")
@ -319,7 +352,7 @@ class Const(object):
LOG_BACKEND_CUSTOM = "custom"
LOG_BASE_LOGGER = "root"
LOG_TWISTED_LOGGER = "twisted"
LOG_OPT_SECTION = "DEFAULT" # section of sat.conf where log options should be
LOG_OPT_SECTION = "" # section of libervia.toml where log options should be
LOG_OPT_PREFIX = "log_"
# (option_name, default_value) tuples
LOG_OPT_COLORS = (
@ -435,7 +468,6 @@ class Const(object):
## Misc ##
SAVEFILE_DATABASE = APP_NAME_FILE + ".db"
IQ_SET = '/iq[@type="set"]'
ENV_PREFIX = "SAT_" # Prefix used for environment variables
IGNORE = "ignore"
NO_LIMIT = -1 # used in bridge when a integer value is expected
DEFAULT_MAX_AGE = 1209600 # default max age of cached files, in seconds
@ -472,90 +504,38 @@ class Const(object):
## Configuration ##
if (
BaseDirectory
): # skipped when xdg module is not available (should not happen in backend)
if "org.libervia.cagou" in BaseDirectory.__file__:
# FIXME: hack to make config read from the right location on Android
# TODO: fix it in a more proper way
# if "org.libervia.cagou" in BaseDirectory.__file__:
# # FIXME: hack to make config read from the right location on Android
# # TODO: fix it in a more proper way
# we need to use Android API to get downloads directory
import os.path
from jnius import autoclass
# # we need to use Android API to get downloads directory
# import os.path
# from jnius import autoclass
# we don't want the very verbose jnius log when we are in DEBUG level
import logging
# # we don't want the very verbose jnius log when we are in DEBUG level
# import logging
logging.getLogger("jnius").setLevel(logging.WARNING)
logging.getLogger("jnius.reflect").setLevel(logging.WARNING)
# logging.getLogger("jnius").setLevel(logging.WARNING)
# logging.getLogger("jnius.reflect").setLevel(logging.WARNING)
Environment = autoclass("android.os.Environment")
# Environment = autoclass("android.os.Environment")
BaseDirectory = None
Const.DEFAULT_CONFIG = {
"local_dir": "/data/data/org.libervia.cagou/app",
"media_dir": "/data/data/org.libervia.cagou/files/app/media",
# FIXME: temporary location for downloads, need to call API properly
"downloads_dir": os.path.join(
Environment.getExternalStoragePublicDirectory(
Environment.DIRECTORY_DOWNLOADS
).getAbsolutePath(),
Const.APP_NAME_FILE,
),
"pid_dir": join(tempfile.gettempdir(), Const.APP_NAME_FILE, logged_user),
"log_dir": "%(local_dir)s",
}
Const.CONFIG_FILES = [
"/data/data/org.libervia.cagou/files/app/android/"
+ Const.APP_NAME_FILE
+ ".conf"
]
else:
import os
# we use parent of "sat" module dir as last config path, this is useful for
# per instance configurations (e.g. a dev instance and a main instance)
root_dir = dirname(dirname(backend.__file__)) + "/"
Const.CONFIG_PATHS = (
# /etc/_sat.conf is used for system-related settings (e.g. when media_dir
# is set by the distribution and has not reason to change, or in a Docker
# image)
["/etc/_", "/etc/", "~/", "~/."]
+ [
"{}/".format(path)
for path in list(BaseDirectory.load_config_paths(Const.APP_NAME_FILE))
]
# this is to handle legacy sat.conf
+ [
"{}/".format(path)
for path in list(BaseDirectory.load_config_paths("sat"))
]
+ [root_dir]
)
# on recent versions of Flatpak, FLATPAK_ID is set at run time
# it seems that this is not the case on older versions,
# but FLATPAK_SANDBOX_DIR seems set then
if os.getenv("FLATPAK_ID") or os.getenv("FLATPAK_SANDBOX_DIR"):
# for Flatpak, the conf can't be set in /etc or $HOME, so we have
# to add /app
Const.CONFIG_PATHS.append("/app/")
## Configuration ##
Const.DEFAULT_CONFIG = {
"media_dir": "/usr/share/" + Const.APP_NAME_FILE + "/media",
"local_dir": BaseDirectory.save_data_path(Const.APP_NAME_FILE),
"downloads_dir": "~/Downloads/" + Const.APP_NAME_FILE,
"pid_dir": join(tempfile.gettempdir(), Const.APP_NAME_FILE, logged_user),
"log_dir": "%(local_dir)s",
}
# List of the configuration filenames sorted by ascending priority
Const.CONFIG_FILES = [
realpath(expanduser(path) + Const.APP_NAME_FILE + ".conf")
for path in Const.CONFIG_PATHS
] + [
# legacy sat.conf
realpath(expanduser(path) + "sat.conf")
for path in Const.CONFIG_PATHS
]
# BaseDirectory = None
# Const.DEFAULT_CONFIG = {
# "local_dir": "/data/data/org.libervia.cagou/app",
# "media_dir": "/data/data/org.libervia.cagou/files/app/media",
# # FIXME: temporary location for downloads, need to call API properly
# "downloads_dir": os.path.join(
# Environment.getExternalStoragePublicDirectory(
# Environment.DIRECTORY_DOWNLOADS
# ).getAbsolutePath(),
# Const.APP_NAME_FILE,
# ),
# "pid_dir": join(tempfile.gettempdir(), Const.APP_NAME_FILE, logged_user),
# "log_dir": "%(local_dir)s",
# }
# Const.CONFIG_FILES = [
# "/data/data/org.libervia.cagou/files/app/android/"
# + Const.APP_NAME_FILE
# + ".toml"
# ]

45
libervia/backend/core/launcher.py
View file

@ -1,7 +1,7 @@
#!/usr/bin/env python3
# Libervia: an XMPP client
# Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
# Copyright (C) 2009-2025 Jérôme Poisson (goffi@goffi.org)
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
@ -16,16 +16,16 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Script launching SàT backend"""
"""Script launching Libervia backend"""
import sys
import os
import argparse
from pathlib import Path
from configparser import ConfigParser
from twisted.application import app
from twisted.python import usage
from libervia.backend.core.constants import Const as C
from libervia.backend.tools.config import parse_main_conf
class LiberviaLogger(app.AppLogger):
@ -46,6 +46,13 @@ class Launcher:
def NOT_RUNNING_MSG(self):
return f"{self.APP_NAME} is *NOT* running"
def __init__(self):
try:
self.config = parse_main_conf()
except Exception as e:
print(rf"/!\ Can't read main config! {e}")
sys.exit(1)
def cmd_no_subparser(self, args):
"""Command launched by default"""
args.extra_args = []
@ -64,8 +71,7 @@ class Launcher:
import signal
import time
config = self.get_config()
pid_file = self.get_pid_file(config)
pid_file = self.get_pid_file()
if not pid_file.is_file():
print(self.NOT_RUNNING_MSG)
sys.exit(0)
@ -110,8 +116,7 @@ class Launcher:
sys.exit(0)
def cmd_status(self, args):
config = self.get_config()
pid_file = self.get_pid_file(config)
pid_file = self.get_pid_file()
if pid_file.is_file():
import errno
@ -186,19 +191,8 @@ class Launcher:
return parser.parse_args()
def get_config(self):
config = ConfigParser(defaults=C.DEFAULT_CONFIG)
try:
config.read(C.CONFIG_FILES)
except Exception as e:
print(rf"/!\ Can't read main config! {e}")
sys.exit(1)
return config
def get_pid_file(self, config):
pid_dir = Path(config.get("DEFAULT", "pid_dir")).expanduser()
pid_dir.mkdir(parents=True, exist_ok=True)
return pid_dir / f"{self.APP_NAME_FILE}.pid"
def get_pid_file(self) -> Path:
return self.config.common.pid_dir / f"{self.APP_NAME_FILE}.pid"
def wait_for_service(
self, service_host: str, service_port: int, timeout: int, service_name: str
@ -252,12 +246,9 @@ class Launcher:
app_runner.loggerFactory = LiberviaLogger
server_options = ServerOptions()
config = self.get_config()
# wait for a service (e.g. XMPP server)
wait_for_service_value = config.get(
"DEFAULT", "init_wait_for_service", fallback=None
)
wait_for_service_value = self.config.common.init_wait_for_service
if wait_for_service_value is not None:
try:
@ -291,8 +282,8 @@ class Launcher:
else:
self.wait_for_service(host, port, timeout, service_name)
pid_file = self.get_pid_file(config)
log_dir = Path(config.get("DEFAULT", "log_dir")).expanduser()
pid_file = self.get_pid_file()
log_dir = self.config.common.log_dir
log_file = log_dir / f"{self.APP_NAME_FILE}.log"
server_opts = [
"--no_save",
@ -322,7 +313,7 @@ class Launcher:
if runner._exitSignal is not None:
app._exitWithSignal(runner._exitSignal)
try:
sys.exit(app._exitCode)
sys.exit(app._exit_code)
except AttributeError:
pass

23
libervia/backend/core/log_config.py
View file

@ -20,8 +20,10 @@
"""High level logging functions"""
# XXX: this module use standard logging module when possible, but as SàT can work in different cases where logging is not the best choice (twisted, pyjamas, etc), it is necessary to have a dedicated module. Additional feature like environment variables and colors are also managed.
from libervia.backend import G
from libervia.backend.core.constants import Const as C
from libervia.backend.core import log
from libervia.backend.tools.config import parse_main_conf
class TwistedLogger(log.Logger):
@ -391,26 +393,25 @@ def libervia_configure(backend=C.LOG_BACKEND_STANDARD, const=None, backend_data=
- C.LOG_BACKEND_BASIC: print based backend
- C.LOG_BACKEND_TWISTED: Twisted logging backend
- C.LOG_BACKEND_STANDARD: standard logging backend
@param const: Const class to use instead of sat.core.constants.Const (mainly used to change default values)
@param const: Const class to use instead of sat.core.constants.Const (mainly used to
change default values)
"""
if const is not None:
global C
C = const
log.C = const
from libervia.backend.tools import config
import os
log_conf = {}
sat_conf = config.parse_main_conf()
# FIXME: Q&D adaptation following configuration refactoring. All this code should be
# removed in close future to use standard logging directly.
for opt_name, opt_default in C.LOG_OPTIONS():
config = parse_main_conf()
try:
log_conf[opt_name] = os.environ[
"".join((C.ENV_PREFIX, C.LOG_OPT_PREFIX.upper(), opt_name.upper()))
]
except KeyError:
log_conf[opt_name] = config.config_get(
sat_conf, C.LOG_OPT_SECTION, C.LOG_OPT_PREFIX + opt_name, opt_default
)
value = getattr(config.common, f"{C.LOG_OPT_PREFIX}{opt_name}")
log_conf[opt_name] = value if value else opt_default
except AttributeError:
log_conf[opt_name] = opt_default
log.debug(f"Using default log option {opt_name!r}: {opt_default!r}")
_parse_options(log_conf)
configure(backend, backend_data=backend_data, **log_conf)

90
libervia/backend/core/main.py
View file

@ -23,14 +23,17 @@ import hashlib
import copy
from pathlib import Path
import shutil
from typing import Optional, List, Tuple, Dict, cast
from typing import Optional, List, Tuple, Dict
from pydantic import ValidationError
from libervia import backend
from libervia.backend.core.i18n import _, D_, language_switch
from libervia.backend.core import patches
from libervia.backend.tools.config import DBusBridgeSettings
patches.apply()
from twisted.application import service
from twisted.application import app, service
from twisted.internet import defer
from twisted.words.protocols.jabber import jid
from twisted.internet import reactor
@ -85,14 +88,15 @@ class LiberviaBackend(service.Service):
}
self.memory = memory.Memory(self)
config = self.memory.config
# trigger are used to change Libervia behaviour
self.trigger = trigger.TriggerManager()
bridge_name = os.getenv("LIBERVIA_BRIDGE_NAME") or self.memory.config_get(
"", "bridge", "dbus"
)
bridges = config.common.bridges
# FIXME: We only handle first bridge for now.
bridge = next(iter(bridges))
bridge_name: str = bridge.type
bridge_module = dynamic_import.bridge(bridge_name)
if bridge_module is None:
log.error(f"Can't find bridge module of name {bridge_name}")
@ -104,13 +108,8 @@ class LiberviaBackend(service.Service):
log.exception("bridge can't be initialised, can't start Libervia Backend")
sys.exit(1)
# If set, a temporary dir in cache will be used to share files between backend and
# frontends, useful when they are separated (e.g. when using containers). If
# unset, a temporary dir will be automatically created in os-relevant location.
use_local_shared_tmp: bool = C.bool(
self.memory.config_get("", "use_local_shared_tmp", C.BOOL_FALSE)
)
if use_local_shared_tmp:
# FIXME: Seems not used anymore, maybe delete?
if config.common.use_local_shared_tmp:
self.local_shared_path = self.get_local_path(
None, C.CACHE_DIR, C.LOCAL_SHARED_DIR
)
@ -280,31 +279,24 @@ class LiberviaBackend(service.Service):
self._import_plugins()
ui_contact_list.ContactList(self)
ui_profile_manager.ProfileManager(self)
except Exception as e:
log.error(f"Could not initialize backend: {e}")
sys.exit(1)
except exceptions.ConfigError as e:
log.error(f"Configuration error, please check and fix your configuration: {e}")
self.stop(exit_code=C.EXIT_BAD_ARG)
except Exception:
log.exception(f"Could not initialize backend")
self.stop(exit_code=C.EXIT_ERROR)
self._add_base_menus()
self.init_pre_script.callback(None)
init_script = self.memory.config_get(None, "init_script_path")
if init_script is not None:
init_script = cast(str, init_script)
script_path = Path(init_script)
if not script_path.exists():
log.error(f"Init script doesn't exist: {init_script}")
sys.exit(C.EXIT_BAD_ARG)
elif not script_path.is_file():
log.error(f"Init script is not a file: {init_script}")
sys.exit(C.EXIT_BAD_ARG)
else:
log.info(f"Running init script {init_script!r}.")
try:
await async_process.run(str(init_script), verbose=True)
except RuntimeError as e:
log.error(f"Init script failed: {e}")
self.stopService()
sys.exit(C.EXIT_ERROR)
if (init_script := G.config.common.init_script) is not None:
log.info(f"Running init script {init_script!r}.")
try:
await async_process.run(str(init_script), verbose=True)
except RuntimeError as e:
log.error(f"Init script failed: {e}")
self.stopService()
sys.exit(C.EXIT_ERROR)
self.initialised.callback(None)
log.info(_("Backend is ready"))
@ -444,6 +436,10 @@ class LiberviaBackend(service.Service):
)
continue
plugins_to_import[import_name] = (plugin_path, mod, plugin_info)
# We reparse configuration to handle plugin settings before initialising them.
self.memory.reparse_configuration()
while True:
try:
self._import_plugins_from_dict(plugins_to_import)
@ -701,8 +697,10 @@ class LiberviaBackend(service.Service):
log.debug(_("running app"))
reactor.run()
def stop(self):
def stop(self, exit_code: int|None = None):
log.debug(_("stopping app"))
if exit_code:
app._exit_code = exit_code
reactor.stop()
## Misc methods ##
@ -751,14 +749,22 @@ class LiberviaBackend(service.Service):
raise exceptions.ProfileKeyUnknown
return [self.profiles[profile]]
def _get_config(self, section, name):
def _get_config(self, section, name) -> str:
"""Get the main configuration option
@param section: section of the config file (None or '' for DEFAULT)
@param section: section of the config
@param name: name of the option
@return: unicode representation of the option
@return: Serialised otpion value
"""
return str(self.memory.config_get(section, name, ""))
# FIXME: This is a quick adaptation of old code.
if not section or section == "DEFAULT":
section = "common"
try:
value = getattr(getattr(G.config, section), name)
except:
raise exceptions.InternalError(f"Can't retrieve value {section=} {name=}.")
else:
return data_format.serialise(value)
def log_errback(self, failure_, msg=_("Unexpected error: {failure_}")):
"""Generic errback logging
@ -977,10 +983,8 @@ class LiberviaBackend(service.Service):
@param component: if True, path will be prefixed with C.COMPONENTS_DIR
@return: path
"""
local_dir = self.memory.config_get("", "local_dir")
if not local_dir:
raise exceptions.InternalError("local_dir must be set")
path_elts = [local_dir]
local_dir = G.config.common.local_dir
path_elts: list[Path | str] = [local_dir]
if component:
path_elts.append(C.COMPONENTS_DIR)
path_elts.append(regex.path_escape(dir_name))

5
libervia/backend/core/xmpp.py
View file

@ -24,7 +24,7 @@ import mimetypes
from pathlib import Path
import sys
import time
from typing import Callable, Dict, Tuple, Optional, cast
from typing import Callable, Dict, Optional, Tuple, cast
from urllib.parse import unquote, urlparse
import uuid
@ -42,6 +42,7 @@ from wokkel import component
from wokkel import delay
from zope.interface import implementer
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core import core_types
from libervia.backend.core.constants import Const as C
@ -964,7 +965,7 @@ class SatXMPPClient(SatXMPPEntity, core_types.SatXMPPClient, wokkel_client.XMPPC
# for now we consider Android devices to be always phones
self.identities = [disco.DiscoIdentity("client", "phone", C.APP_NAME)]
hosts_map = host_app.memory.config_get(None, "hosts_dict", {})
hosts_map = G.config.common.hosts_dict
if host is None and user_jid.host in hosts_map:
host_data = hosts_map[user_jid.host]
if isinstance(host_data, str):

3
libervia/backend/memory/cache.py
View file

@ -24,6 +24,7 @@ from typing import Any
from pydantic import BaseModel, ValidationError
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.i18n import _
@ -58,7 +59,7 @@ class Cache:
if None, the cache will be common for all profiles
"""
self.profile = profile
path_elts = [host.memory.config_get("", "local_dir"), C.CACHE_DIR]
path_elts = [G.config.common.local_dir, C.CACHE_DIR]
if profile:
path_elts.extend(["profiles", regex.path_escape(profile)])
else:

27
libervia/backend/memory/memory.py
View file

@ -261,12 +261,21 @@ class Memory:
self.subscriptions = {}
self.auth_sessions = PasswordSessions() # remember the authenticated profiles
self.disco = Discovery(host)
self.config = tools_config.parse_main_conf(log_filenames=True)
self._cache_path = Path(self.config_get("", "local_dir"), C.CACHE_DIR)
self.admins = self.config_get("", "admins_list", [])
self.config: tools_config.Settings = tools_config.parse_main_conf(
log_filenames=True
)
self._cache_path = self.config.common.local_dir / C.CACHE_DIR
self.admins = self.config.common.admins_list
self.admin_jids = set()
self._file_path_lock = defer.DeferredLock()
def reparse_configuration(self) -> None:
"""Reparse configuration.
This is needed when dynamic settings has been added (after importing plugins).
"""
self.config = tools_config.parse_main_conf(ignore_cache=True)
async def initialise(self):
self.storage = Storage()
await self.storage.initialise()
@ -296,18 +305,6 @@ class Memory:
## Configuration ##
def config_get(
self, section: str | None, name: str, default: Any = None
) -> str | list | dict:
"""Get the main configuration option
@param section: section of the config file (None or '' for DEFAULT)
@param name: name of the option
@param default: value to use if not found
@return: str, list or dict
"""
return tools_config.config_get(self.config, section, name, default)
def load_xml(self, filename):
"""Load parameters template from xml file

4
libervia/backend/memory/sqla_config.py
View file

@ -16,7 +16,6 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from pathlib import Path
from urllib.parse import quote
from libervia.backend.core.constants import Const as C
from libervia.backend.tools import config
@ -30,8 +29,7 @@ def get_db_config() -> dict:
- path: path to the sqlite DB
"""
main_conf = config.parse_main_conf()
local_dir = Path(config.config_get(main_conf, "", "local_dir"))
database_path = (local_dir / C.SAVEFILE_DATABASE).expanduser()
database_path = main_conf.common.local_dir / C.SAVEFILE_DATABASE
url = f"sqlite+aiosqlite:///{quote(str(database_path))}?timeout=30"
return {
"type": "sqlite",

12
libervia/backend/plugins/plugin_app_manager_docker/libervia_app_peertube.yaml
View file

@ -54,14 +54,14 @@ override:
PEERTUBE_WEBSERVER_HOSTNAME: !libervia_conf ["application peertube", "hostname"]
PEERTUBE_TRUST_PROXY: '["127.0.0.1", "loopback", "172.18.0.0/16"]'
PEERTUBE_SECRET: !libervia_generate_pwd {name: secret}
PEERTUBE_SMTP_HOSTNAME: !libervia_conf ["", "email_server"]
PEERTUBE_SMTP_USERNAME: !libervia_conf ["", "email_username"]
PEERTUBE_SMTP_PASSWORD: !libervia_conf ["", "email_password"]
PEERTUBE_SMTP_DISABLE_STARTTLS: !libervia_conf ["", "email_starttls", "false", "not"]
PEERTUBE_SMTP_PORT: !libervia_conf ["", "email_port"]
PEERTUBE_SMTP_HOSTNAME: !libervia_conf ["email", "server"]
PEERTUBE_SMTP_USERNAME: !libervia_conf ["email", "username"]
PEERTUBE_SMTP_PASSWORD: !libervia_conf ["email", "password"]
PEERTUBE_SMTP_DISABLE_STARTTLS: !libervia_conf ["email", "starttls", "false", "not"]
PEERTUBE_SMTP_PORT: !libervia_conf ["email", "port"]
PEERTUBE_SMTP_FROM: !libervia_conf ["application peertube", "email_from", "peertube@example.com"]
PEERTUBE_SMTP_TLS: "false"
PEERTUBE_ADMIN_EMAIL: !libervia_conf ["", "email_admins_list", "", "first"]
PEERTUBE_ADMIN_EMAIL: !libervia_conf ["common", "email_admins_list", "", "first"]
PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PUBLIC: "public-read"
PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PRIVATE: "private"

2
libervia/backend/plugins/plugin_app_manager_docker/libervia_app_weblate.yaml
View file

@ -21,7 +21,7 @@ override:
WEBLATE_DEFAULT_FROM_EMAIL: !libervia_conf ["application weblate", "email_from", "weblate@example.com"]
WEBLATE_SITE_DOMAIN: !libervia_conf ["application weblate", "public_url"]
WEBLATE_ADMIN_PASSWORD: !libervia_generate_pwd {name: admin}
WEBLATE_ADMIN_EMAIL: !libervia_conf ["", "email_admins_list", "", "first"]
WEBLATE_ADMIN_EMAIL: !libervia_conf ["common", "email_admins_list", "", "first"]
WEBLATE_ENABLE_HTTPS: !libervia_conf ["application weblate", "enable_https", "1"]
volumes:
- ./settings-override.py:/app/data/settings-override.py:ro

121
libervia/backend/plugins/plugin_comp_ap_gateway/__init__.py
View file

@ -53,6 +53,7 @@ from twisted.words.protocols.jabber import error, jid
from twisted.words.xish import domish
from wokkel import pubsub, rsm
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.core_types import SatXMPPEntity
@ -66,14 +67,15 @@ from libervia.backend.tools.common import data_format, tls, uri
from libervia.backend.tools.common.async_utils import async_lru
from .ad_hoc import APAdHocService
from .events import APEvents
# We import PLUGIN_INFO as it's needed for the plugin, and we put it in .constants because
# it's necessary to create the ComponentSettings subclass.
from .constants import PLUGIN_INFO
from .constants import (
ACTIVITY_OBJECT_MANDATORY,
ACTIVITY_TARGET_MANDATORY,
ACTIVITY_TYPES,
ACTIVITY_TYPES_LOWER,
COMMENTS_MAX_PARENTS,
CONF_SECTION,
IMPORT_NAME,
LRU_MAX_SIZE,
MEDIA_TYPE_AP,
@ -84,55 +86,23 @@ from .constants import (
TYPE_EVENT,
TYPE_FOLLOWERS,
TYPE_ITEM,
TYPE_JOIN,
TYPE_LEAVE,
TYPE_LIKE,
TYPE_MENTION,
TYPE_REACTION,
TYPE_TOMBSTONE,
TYPE_JOIN,
TYPE_LEAVE,
)
from .events import APEvents
from .http_server import HTTPServer
from .pubsub_service import APPubsubService
from .regex import RE_MENTION
from .settings import APGatewaySettings
log = getLogger(__name__)
IMPORT_NAME = "ap-gateway"
PLUGIN_INFO = {
C.PI_NAME: "ActivityPub Gateway component",
C.PI_IMPORT_NAME: IMPORT_NAME,
C.PI_MODES: [C.PLUG_MODE_COMPONENT],
C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
C.PI_PROTOCOLS: [],
C.PI_DEPENDENCIES: [
"XEP-0050",
"XEP-0054",
"XEP-0060",
"XEP-0084",
"XEP-0106",
"XEP-0277",
"XEP-0292",
"XEP-0329",
"XEP-0372",
"XEP-0424",
"XEP-0465",
"XEP-0470",
"XEP-0447",
"XEP-0471",
"PUBSUB_CACHE",
"TEXT_SYNTAXES",
"IDENTITY",
],
C.PI_RECOMMENDATIONS: [],
C.PI_MAIN: "APGateway",
C.PI_HANDLER: C.BOOL_TRUE,
C.PI_DESCRIPTION: _(
"Gateway for bidirectional communication between XMPP and ActivityPub."
),
}
HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})"
RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}")
RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}")
@ -181,6 +151,7 @@ class APGateway:
self.pubsub_service = APPubsubService(self)
self.ad_hoc = APAdHocService(self)
self.ap_events = APEvents(self)
self.html_redirect: HTMLRedirectDict = {}
host.trigger.add_with_check(
"message_received", self, self._message_received_trigger, priority=-1000
)
@ -242,14 +213,19 @@ class APGateway:
# params
# URL and port
self.public_url = self.host.memory.config_get(
CONF_SECTION, "public_url"
) or self.host.memory.config_get(CONF_SECTION, "xmpp_domain")
# Get component settings
ap_settings = G.config.get_settings(APGatewaySettings)
self.public_url = (
ap_settings.public_url
or ap_settings.xmpp_domain
or G.config.common.xmpp_domain
)
if self.public_url is None:
log.error(
'"public_url" not set in configuration, this is mandatory to have'
"ActivityPub Gateway running. Please set this option it to public facing "
f"url in {CONF_SECTION!r} configuration section."
f"url in {IMPORT_NAME!r} configuration section."
)
return
if parse.urlparse(self.public_url).scheme:
@ -258,65 +234,34 @@ class APGateway:
'"public_url" configuration option. ActivityPub Gateway won\'t be run.'
)
return
self.http_port = int(self.host.memory.config_get(CONF_SECTION, "http_port", 8123))
connection_type = self.host.memory.config_get(
CONF_SECTION, "http_connection_type", "https"
)
self.http_port = ap_settings.http_port
connection_type = ap_settings.http_connection_type
if connection_type not in ("http", "https"):
raise exceptions.ConfigError(
'bad ap-gateay http_connection_type, you must use one of "http" or '
'"https"'
)
self.http_sign_get = C.bool(
self.host.memory.config_get(CONF_SECTION, "http_sign_get", C.BOOL_TRUE)
)
self.max_items = int(
self.host.memory.config_get(CONF_SECTION, "new_node_max_items", 50)
)
self.comments_max_depth = int(
self.host.memory.config_get(CONF_SECTION, "comments_max_depth", 0)
)
self.ap_path = self.host.memory.config_get(CONF_SECTION, "ap_path", "_ap")
self.http_sign_get = ap_settings.http_sign_get
self.max_items = ap_settings.new_node_max_items
self.comments_max_depth = ap_settings.comments_max_depth
self.ap_path = ap_settings.ap_path
self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/")
# True (default) if we provide gateway only to entities/services from our server
self.local_only = C.bool(
self.host.memory.config_get(CONF_SECTION, "local_only", C.BOOL_TRUE)
)
self.local_only = ap_settings.local_only
# if True (default), mention will be parsed in non-private content coming from
# XMPP. This is necessary as XEP-0372 are coming separately from item where the
# mention is done, which is hard to impossible to translate to ActivityPub (where
# mention specified inside the item directly). See documentation for details.
self.auto_mentions = C.bool(
self.host.memory.config_get(CONF_SECTION, "auto_mentions", C.BOOL_TRUE)
)
self.auto_mentions = ap_settings.auto_mentions
html_redirect: Dict[str, Union[str, dict]] = self.host.memory.config_get(
CONF_SECTION, "html_redirect_dict", {}
)
self.html_redirect: Dict[str, List[dict]] = {}
for url_type, target in html_redirect.items():
if isinstance(target, str):
target = {"url": target}
elif not isinstance(target, dict):
raise exceptions.ConfigError(
f"html_redirect target must be a URL or a dict, not {target!r}"
)
filters = target.setdefault("filters", {})
if "url" not in target:
log.warning(f"invalid HTML redirection, missing target URL: {target}")
continue
# a slash in the url_type is a syntactic shortcut to have a node filter
if "/" in url_type:
url_type, node_filter = url_type.split("/", 1)
filters["node"] = node_filter
self.html_redirect.setdefault(url_type, []).append(target)
self.html_redirect = ap_settings.html_redirect
# HTTP server launch
self.server = HTTPServer(self)
if connection_type == "http":
reactor.listenTCP(self.http_port, self.server)
else:
options = tls.get_options_from_config(self.host.memory.config, CONF_SECTION)
options = tls.get_options_from_config(ap_settings.tls)
tls.tls_options_check(options)
context_factory = tls.get_tls_context_factory(options)
reactor.listenSSL(self.http_port, self.server, context_factory)
@ -1596,8 +1541,10 @@ class APGateway:
elif start_index > 5000:
raise error.StanzaError(
"feature-not-implemented",
text="Maximum limit for previous_index has been reached, this limit"
"is set to avoid DoS",
text=(
"Maximum limit for previous_index has been reached, this limit"
"is set to avoid DoS"
),
)
else:
# we'll convert "start_index" to "after_id", thus we need the item just
@ -1972,7 +1919,7 @@ class APGateway:
except StopIteration:
# no item corresponding to this author, we use ap_account
log.warning(
"Can't find a single cached item for parent item " f"{parent_item!r}"
f"Can't find a single cached item for parent item {parent_item!r}"
)
parent_ap_account = ap_account
else:
@ -2419,7 +2366,7 @@ class APGateway:
if parsed_anchor["type"] != "pubsub":
log.warning(
f"ignoring reference with non pubsub anchor, this is not supported: "
"{reference_data!r}"
f"{{reference_data!r}}"
)
return False

37
libervia/backend/plugins/plugin_comp_ap_gateway/constants.py
View file

@ -17,8 +17,10 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from libervia.backend.core.constants import Const as C
from libervia.backend.core.i18n import _
IMPORT_NAME = "ap-gateway"
CONF_SECTION = f"component {IMPORT_NAME}"
CONTENT_TYPE_WEBFINGER = "application/jrd+json; charset=utf-8"
CONTENT_TYPE_AP = "application/activity+json; charset=utf-8"
TYPE_ACTOR = "actor"
@ -129,3 +131,36 @@ MAX_AVATAR_SIZE = 1024 * 1024 * 5
# storage prefixes
ST_AVATAR = "[avatar]"
ST_AP_CACHE = "[AP_item_cache]"
PLUGIN_INFO = {
C.PI_NAME: "ActivityPub Gateway component",
C.PI_IMPORT_NAME: IMPORT_NAME,
C.PI_MODES: [C.PLUG_MODE_COMPONENT],
C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
C.PI_PROTOCOLS: [],
C.PI_DEPENDENCIES: [
"XEP-0050",
"XEP-0054",
"XEP-0060",
"XEP-0084",
"XEP-0106",
"XEP-0277",
"XEP-0292",
"XEP-0329",
"XEP-0372",
"XEP-0424",
"XEP-0465",
"XEP-0470",
"XEP-0447",
"XEP-0471",
"PUBSUB_CACHE",
"TEXT_SYNTAXES",
"IDENTITY",
],
C.PI_RECOMMENDATIONS: [],
C.PI_MAIN: "APGateway",
C.PI_HANDLER: C.BOOL_TRUE,
C.PI_DESCRIPTION: _(
"Gateway for bidirectional communication between XMPP and ActivityPub."
),
}

11
libervia/backend/plugins/plugin_comp_ap_gateway/http_server.py
View file

@ -18,7 +18,7 @@
import time
import html
from typing import Optional, Dict, List, Any
from typing import TYPE_CHECKING, Optional, Dict, List, Any
import json
from urllib import parse
from collections import deque
@ -40,6 +40,9 @@ from libervia.backend.core.log import getLogger
from libervia.backend.tools.common import date_utils, uri
from libervia.backend.memory.sqla_mapping import SubscriptionState
if TYPE_CHECKING:
from . import APGateway
from .constants import (
NS_AP,
MEDIA_TYPE_AP,
@ -80,7 +83,7 @@ class HTTPAPGServer(web_resource.Resource):
isLeaf = True
def __init__(self, ap_gateway):
def __init__(self, ap_gateway: "APGateway"):
self.apg = ap_gateway
self._seen_digest = deque(maxlen=50)
super().__init__()
@ -1064,7 +1067,7 @@ class HTTPAPGServer(web_resource.Resource):
redirections = self.apg.html_redirect[request_type]
for redirection in redirections:
filters = redirection["filters"]
filters = redirection.filters
if not filters:
break
# if we have filter, they must all match
@ -1076,7 +1079,7 @@ class HTTPAPGServer(web_resource.Resource):
if redirection is not None:
kw = {k: parse.quote(str(v), safe="") for k, v in kw.items()}
target_url = redirection["url"].format(**kw)
target_url = redirection.url.format(**kw)
content = web_util.redirectTo(target_url.encode(), request)
request.write(content)
request.finish()

226
libervia/backend/plugins/plugin_comp_ap_gateway/settings.py Normal file
View file

@ -0,0 +1,226 @@
#!/usr/bin/env python3
# Libervia ActivityPub Gateway
# Copyright (C) 2009-2025 Jérôme Poisson (goffi@goffi.org)
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from typing import Any, Literal
from pydantic import BaseModel, Field, field_validator
from libervia.backend.tools.config import ComponentSettings, TLSSettings
from .constants import PLUGIN_INFO
HTMLRedirectDict = dict[
Literal["actor", "item", "followers", "following"], list["HTMLRedictData"]
]
class HTMLRedictData(BaseModel):
url: str = Field(description="Target URL for the redirect")
filters: dict[Literal["jid", "jid_user", "node", "item"], str] = Field(
default_factory=dict,
description=("Filters to apply when matching redirection rules."),
)
class APGatewaySettings(ComponentSettings, plugin_info=PLUGIN_INFO):
# HTTP server configuration
http_port: int = Field(
default=8123,
description="""
port where the HTTP server should listen. Port ``80`` is not used directly as it would
require root privileges, and it is strongly recommended against launching Libervia
under a privileged account. An HTTP Proxy or a port redirection should be set to
redirect the ``80`` port to the port specified here.
""",
)
http_connection_type: Literal["http", "https"] = Field(
default="https",
description="""
either ``http`` or ``https``. If you have a HTTP proxy such as Apache or NGINX which
already handles HTTPS, you may want to use ``http``.
Note that the HTTP server should always use ``https`` with end-user, the ``http``
option is only to be used with an HTTPS proxy.
""",
)
http_sign_get: bool = Field(
default=True,
description="""
A boolean value indicating if HTTP ``GET`` request made to AP implementations must be
signed. Some AP implementations require those requests to be signed, notably Mastodon
when "secure mode" is activated.
""",
)
# TLS
tls: TLSSettings | None = None
# ActivityPub configuration
public_url: str | None = Field(
default=None,
description=""""
Main user-facing domain of the HTTP server, this will be used to construct all AP URLs
**default**: if not set, ``xmpp_domain`` is used. If ``xmpp_domain`` is not set
either, an error is raised.
""",
)
xmpp_domain: str | None = Field(
default=None, description="XMPP domain. Default to ``common.xmpp_domain``."
)
ap_path: str = Field(
default="_ap",
description="""
Path prefix to use for ActivityPub request. It's usually not necessary to change the
default value.
""",
)
local_only: bool = Field(
default=True,
description="""
A boolean value indicating if the gateway is allowed to convert pubsub node from
external XMPP service or not. A JID is considered external if its domain part doesn't
end with the gateway's server. For instance, if a gateway ``ap.example.org`` is set on
the server ``example.org``, the JIDs ``pierre@example.org`` or
``some-node@pubsub.example.org`` will be considered local, but
``buenaventura@example.net`` won't (note the different domain).
Most of time, ``local_only`` should be used.
""",
)
auto_mentions: bool = Field(
default=True,
description="""
A boolean value indicating if received XMPP pubsub blog items bodies must be scanned
to find ``@user@server.tld`` type mentions. If mentions are found, they'll be added to
the resulting AP items.
""",
)
# PubSub configuration
new_node_max_items: int = Field(
default=50, description="Maximum items for new pubsub nodes"
)
comments_max_depth: int = Field(
default=0,
description="""
An integer value indicating the maximum number of comment nodes that can be created.
See :ref:`ap-xmpp-threads-conversion`
""",
)
# HTML redirection configuration
html_redirect: HTMLRedirectDict = Field(
default_factory=HTMLRedirectDict,
description="""
A dictionary used to redirect HTTP requests when it's not an ActivityPub request (i.e.
when the ``Accept`` header is not set to ``application/json``). Several ActivityPub
implementations link to original server when clicking on some links such as post
author or item ID, and this result in a request to the corresponding ActivityPub
endpoint, but made to retrieve HTML instead of ActivityPub JSON data.
By default, this request is showing the corresponding AP JSON data, but if you set
HTML redirection, you can redirect the page to your XMPP client web frontend (or
anything making sense), which results in better user experience.
The expected dictionary is a mapping URL request types to destination URL.
The URL request types is the first path element after ``ap_path``, it can be:
``actor``
Actor information. It should link to the profile page of the corresponding XMPP
entity
``item``
A specific publication. It should link to the rendered item
``followers``
Entities publicly subscribed to the actor. It is usually not used for redirection.
``following``
Entities that the actor is publicly subscribed to. It is usually not used for
redirection.
In the target URL you can uses template values in curly braces (something like
``{item}``). The values that you can use are:
``jid``
Full JID corresponding to the AP actor.
``jid_user``
Only the user part of the JID (what is before the ``@``)
``node``
Name of the pubsub node corresponding to the AP actor.
``item``
ID of the XMPP pubsub item.
You can use a slash ``/`` followed by a part of a node name to filter only on specific
nodes. For instance, if you want to redirect microblog items to a different URL that
event items, you can use ``item/urn:xmpp:microblog:0`` and ``item/urn:xmpp:events:0``,
the first redirection to match will be used. Note that anything containing the filter
value will match, thus you can also use ``item/microblog`` (but it then would match an
event which has ``microblog`` in is node name).
You can also use a dictionary as value instead of the target URL. This can be useful
to filter on an other value than the node name, if you want to make specific
redirection for a particular user for instance. If you use a dictionary, you must use
a ``url`` key with the target URL (which may have template values as usual), and you
may use a ``filters`` key mapping to an other dictionary where each key is a filter on
a specific template value (in other words: key is the template value, like ``jid`` or
``node``, and value is the filter to match).
*examples*
Redirect actor page to ``/profile/<JID user part>``, blogs items to ``/blog/<full
JID>/<node name>/<item id>`` and events items to ``/event/<full JID>/<node name>/<item
id>`` ::
[components.ap_gateway]
html_redirect.actor = "/profile/{jid_user}"
html_redirect.item = [
{node="microblog", url="/blog/{jid}/{node}/{item}"},
{node="events", url="/event/{jid}/{node}/{item}"},
]
Redirect items of user ``louise@example.org`` to ``/b/<item>``::
[components.ap_gateway]
html_redirect.item = {url = "/b/{item}", filters = { jid = "louise@example.org" }}
""",
)
@field_validator(
"html_redirect",
mode="before",
json_schema_input_type=str | HTMLRedictData | list[HTMLRedictData],
)
@classmethod
def html_redirect_parse_simple_syntax(cls, html_redirect: Any) -> Any:
if isinstance(html_redirect, dict):
for key, value in html_redirect.items():
if isinstance(value, str):
html_redirect[key] = [HTMLRedictData(url=value)]
elif isinstance(value, HTMLRedictData):
html_redirect[key] = [value]
elif isinstance(value, dict):
html_redirect[key] = [HTMLRedictData(**value)]
return html_redirect

42
libervia/backend/plugins/plugin_comp_conferences/__init__.py
View file

@ -19,9 +19,11 @@
import json
import os
from pathlib import Path
from typing import Self
from urllib.parse import quote
from autobahn.twisted.websocket import WebSocketClientFactory, WebSocketClientProtocol
from pydantic import Field
from shortuuid import uuid
import treq
from twisted.internet import defer, reactor
@ -36,6 +38,7 @@ from twisted.words.xish import domish
from wokkel import disco, iwokkel
from zope.interface import implementer
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.core_types import SatXMPPEntity
@ -47,7 +50,7 @@ from libervia.backend.plugins.plugin_xep_0167 import XEP_0167, mapping, NS_AV_CO
from libervia.backend.plugins.plugin_xep_0176 import XEP_0176
from libervia.backend.plugins.plugin_xep_0298 import XEP_0298
from libervia.backend.tools.common import async_process, regex
from libervia.backend.tools.config import ComponentSettings, ExecutableFilePath
log = getLogger(__name__)
@ -70,7 +73,21 @@ PLUGIN_INFO = {
),
}
CONF_SECTION = f"component {IMPORT_NAME}"
class ConferenceSettings(ComponentSettings, plugin_info=PLUGIN_INFO):
galene_path: ExecutableFilePath | None = Field(
default_factory=lambda: ConferenceSettings._find_galene_exec()
)
http_port: int = Field(
default=9443, description="Port to use for Galène HTTP server."
)
@staticmethod
def _find_galene_exec() -> Path | None:
try:
return Path(which("galene")[0])
except IndexError:
return None
class Conference:
@ -386,21 +403,16 @@ class ConferencesComponent:
self.client.identities.append(
disco.DiscoIdentity("conference", "audio-video", NAME)
)
settings = G.config.get_settings(ConferenceSettings)
galene_path = settings.galene_path
try:
galene_path = Path(
self.host.memory.config_get(CONF_SECTION, "galene_path")
or which("galene")[0]
)
except IndexError:
if galene_path is None:
raise exceptions.NotFound(
'Can\'t find "galene" executable, "conferences" component can\'t be '
"started without it. Please install it in location accessible in PATH, "
'or use "galene_path" setting.'
f'Can\'t find "galene" executable, "PLUGIN_INFO[C.PI_NAME]" component'
f" can't be started without it. Please install it in location accessible"
f' in PATH, or use "galene_path" setting.'
)
self.galene_http_port = self.host.memory.config_get(
CONF_SECTION, "http_port", "9443"
)
self.galene_http_port = settings.http_port
galene_data_path = self.host.memory.get_cache_path(IMPORT_NAME, "galene")
galene_static_path = galene_path.parent / "static"
self.galene_group_path = galene_data_path / "groups"
@ -460,7 +472,7 @@ class ConferencesComponent:
}
with data_file.open("w") as f:
json.dump(group_data, f)
log.debug(f"Conference data for {group_name!r} created at " f"{data_file} .")
log.debug(f"Conference data for {group_name!r} created at {data_file} .")
url = f"http://localhost:{self.galene_http_port}/group/{quote(stem)}"
status_url = f"{url}/.status"

126
libervia/backend/plugins/plugin_comp_file_sharing.py
View file

@ -21,29 +21,33 @@ import os.path
import mimetypes
import tempfile
from functools import partial
from typing import Any
from pydantic import Field, field_validator
import shortuuid
import unicodedata
from urllib.parse import urljoin, urlparse, quote, unquote
from pathlib import Path
from libervia.backend import G
from libervia.backend.core.i18n import _, D_
from libervia.backend.core.constants import Const as C
from libervia.backend.core import exceptions
from libervia.backend.core.log import getLogger
from libervia.backend.tools import stream
from libervia.backend.tools import video
from libervia.backend.tools.config import ComponentSettings, ConfigOptions, TLSSettings
from libervia.backend.tools.utils import ensure_deferred
from libervia.backend.tools.common import regex
from libervia.backend.tools.common import uri
from libervia.backend.tools.common import files_utils
from libervia.backend.tools.common import utils
from libervia.backend.tools.common import tls
from libervia.backend.models.types import JIDType
from twisted.internet import defer, reactor
from twisted.words.protocols.jabber import error
from twisted.words.protocols.jabber import error, jid
from twisted.web import server, resource, static, http
from wokkel import pubsub
from wokkel import generic
log = getLogger(__name__)
@ -85,6 +89,66 @@ OVER_QUOTA_TXT = D_(
HTTP_VERSION = unicodedata.normalize("NFKD", f"{C.APP_NAME} file sharing {C.APP_VERSION}")
class QuotasSettings(ConfigOptions):
"""Configuration settings for file sharing quotas.
All quota values should be specified in bytes or using human-readable format
like "100MB", "2GB", etc.
"""
admins: int | None = Field(
default=None,
description="Quota for administrators. If no set, no limit applies.",
)
users: int | None = Field(
default=None,
description="Default quota for regular users. If not set, no limit applies.",
)
jids: dict[JIDType, int] = Field(
default_factory=dict,
description="Mapping of specific JIDs to their quotas.",
)
@field_validator(
"admins", "users", mode="before", json_schema_input_type=int | str | None
)
@classmethod
def parse_quotas(cls, value: Any) -> Any:
"""Parse string sizes to int"""
if isinstance(value, str):
return utils.parse_size(value)
return value
@field_validator(
"jids", mode="before", json_schema_input_type=dict[str, int | str]
)
@classmethod
def parse_jid_quotas(cls, value: Any) -> Any:
"""Parse string sizes to int for ``jids``."""
if isinstance(value, dict):
for k, v in value.items():
if isinstance(v, str):
value[k] = utils.parse_size(v)
return value
class FileSharingSettings(ComponentSettings, plugin_info=PLUGIN_INFO):
public_url: str | None = Field(default=None, description="")
http_upload_port: int = Field(default=8888, description="Port for HTTP file upload")
http_upload_connection_type: str = Field(
default="https", description="Connection type for HTTP upload (http or https)"
)
quotas: QuotasSettings = QuotasSettings()
http_upload_allowed_hosts: list[str] = Field(
default_factory=list,
description=(
"List of XMPP domain which can upload on this component. Default to"
" component's host."
),
)
tls: TLSSettings | None = None
class HTTPFileServer(resource.Resource):
isLeaf = True
@ -310,8 +374,9 @@ class FileSharingRequest(server.Request):
if upload_request.filename != filename:
log.warning(
f"invalid filename for PUT (upload id: {upload_id!r}, URL: {self.channel._path.decode()}). Original "
f"{upload_request.filename!r} doesn't match {filename!r}"
f"invalid filename for PUT (upload id: {upload_id!r}, URL:"
f" {self.channel._path.decode()}). Original"
f" {upload_request.filename!r} doesn't match {filename!r}"
)
return self.refuse_request()
@ -356,6 +421,7 @@ class FileSharing:
if self.initialised:
return
self.initialised = True
self.settings = G.config.get_settings(FileSharingSettings)
log.info(_("File Sharing initialization"))
self._f = self.host.plugins["FILE"]
self._jf = self.host.plugins["XEP-0234"]
@ -390,14 +456,8 @@ class FileSharing:
"XEP-0329_parseResult_directory", self, self._get_directory_metadata_elts
)
self.files_path = self.host.get_local_path(None, C.FILES_DIR)
self.http_port = int(
self.host.memory.config_get(
"component file-sharing", "http_upload_port", 8888
)
)
connection_type = self.host.memory.config_get(
"component file-sharing", "http_upload_connection_type", "https"
)
self.http_port = self.settings.http_upload_port
connection_type = self.settings.http_upload_connection_type
if connection_type not in ("http", "https"):
raise exceptions.ConfigError(
'bad http_upload_connection_type, you must use one of "http" or "https"'
@ -407,9 +467,7 @@ class FileSharing:
if connection_type == "http":
reactor.listenTCP(self.http_port, self.server)
else:
options = tls.get_options_from_config(
self.host.memory.config, "component file-sharing"
)
options = tls.get_options_from_config(self.settings.tls)
tls.tls_options_check(options)
context_factory = tls.get_tls_context_factory(options)
reactor.listenSSL(self.http_port, self.server, context_factory)
@ -422,32 +480,32 @@ class FileSharing:
client.enabled_features.add("XEP-0363")
self.init()
public_base_url = self.host.memory.config_get(
"component file-sharing", "http_upload_public_facing_url"
)
public_base_url = self.settings.public_url
if public_base_url is None:
client._file_sharing_base_url = f"https://{client.host}:{self.http_port}"
else:
client._file_sharing_base_url = public_base_url
path = client.file_tmp_dir = os.path.join(
self.host.memory.config_get("", "local_dir"),
G.config.common.local_dir,
C.FILES_TMP_DIR,
regex.path_escape(client.profile),
)
if not os.path.exists(path):
os.makedirs(path)
def get_quota(self, client, entity):
"""Return maximum size allowed for all files for entity"""
quotas = self.host.memory.config_get("component file-sharing", "quotas_json", {})
if self.host.memory.is_admin_jid(entity):
quota = quotas.get("admins")
else:
try:
quota = quotas["jids"][entity.userhost()]
except KeyError:
quota = quotas.get("users")
return None if quota is None else utils.parse_size(quota)
def get_quota(self, entity: jid.JID) -> int | None:
"""Get the maximum file storage quota for a given entity.
@param entity: The JID to check quota for.
@return: The maximum allowed size in bytes, or None if no quota is set.
"""
entity_bare = entity.userhostJID()
quotas = self.settings.quotas
quota = quotas.jids.get(entity_bare)
if not quota:
is_admin = G.host.memory.is_admin_jid(entity_bare)
quota = quotas.admins if is_admin else quotas.users
return quota
async def generate_thumbnails(self, extra: dict, image_path: Path):
thumbnails = extra.setdefault(C.KEY_THUMBNAILS, [])
@ -552,7 +610,7 @@ class FileSharing:
assert C.KEY_PROGRESS_ID in file_data
filename = file_data["name"]
assert filename and not "/" in filename
quota = self.get_quota(client, peer_jid)
quota = self.get_quota(peer_jid)
if quota is not None:
used_space = await self.host.memory.file_get_used_space(client, peer_jid)
@ -676,7 +734,7 @@ class FileSharing:
if request.from_.host not in client._file_sharing_allowed_hosts:
raise error.StanzaError("forbidden")
quota = self.get_quota(client, request.from_)
quota = self.get_quota(request.from_)
if quota is not None:
used_space = await self.host.memory.file_get_used_space(client, request.from_)
@ -786,7 +844,9 @@ class Comments_handler(pubsub.PubSubService):
self.plugin_parent = plugin_parent
self.discoIdentity = {
"category": "pubsub",
"type": "virtual", # FIXME: non standard, here to avoid this service being considered as main pubsub one
"type": (
"virtual"
), # FIXME: non standard, here to avoid this service being considered as main pubsub one
"name": "files commenting service",
}

13
libervia/backend/plugins/plugin_dbg_manhole.py
View file

@ -17,6 +17,8 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from pydantic import Field
from libervia.backend import G
from libervia.backend.core.i18n import _
from libervia.backend.core.constants import Const as C
from libervia.backend.core.log import getLogger
@ -26,6 +28,8 @@ from twisted.internet import reactor, protocol, defer
from twisted.words.protocols.jabber import jid
from twisted.conch.manhole import ColoredManhole
from libervia.backend.tools.config import PluginSettings
log = getLogger(__name__)
PLUGIN_INFO = {
@ -40,11 +44,18 @@ PLUGIN_INFO = {
}
class PluginManholeSettings(PluginSettings, plugin_info=PLUGIN_INFO):
dangerous_manhole_debug_port: int = Field(
default=0, description="Debug port for manhole"
)
class Manhole(object):
def __init__(self, host):
self.host = host
port = int(host.memory.config_get(None, "manhole_debug_dangerous_port_int", 0))
settings = G.config.get_settings(PluginManholeSettings)
port = settings.dangerous_manhole_debug_port
if port:
self.start_manhole(port)

116
libervia/backend/plugins/plugin_misc_account.py
View file

@ -16,6 +16,9 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from typing import cast
from pydantic import Field
from libervia.backend import G
from libervia.backend.core.i18n import _, D_
from libervia.backend.core.log import getLogger
@ -24,11 +27,11 @@ from libervia.backend.tools import xml_tools
from libervia.backend.memory.memory import Sessions
from libervia.backend.memory.crypto import PasswordHasher
from libervia.backend.core.constants import Const as C
import configparser
from twisted.internet import defer
from twisted.python.failure import Failure
from twisted.words.protocols.jabber import jid
from libervia.backend.tools.common import email as sat_email
from libervia.backend.tools.config import EmailSettings, PluginSettings
log = getLogger(__name__)
@ -40,7 +43,7 @@ log = getLogger(__name__)
PLUGIN_INFO = {
C.PI_NAME: "Account Plugin",
C.PI_IMPORT_NAME: "MISC-ACCOUNT",
C.PI_IMPORT_NAME: "account",
C.PI_TYPE: "MISC",
C.PI_PROTOCOLS: [],
C.PI_DEPENDENCIES: ["XEP-0077"],
@ -69,11 +72,14 @@ default_conf = {
"admin_email": "",
"new_account_server": "localhost",
"new_account_domain": "", #  use xmpp_domain if not found
"reserved_list": ["libervia"], # profiles which can't be used
}
# profiles which can't be used for accound creation. Must be lowercase.
RESERVED_LIST: list[str] = ["libervia"]
WELCOME_MSG = D_(
"""Welcome to Libervia, the web interface of Salut à Toi.
"""Welcome to Libervia.
Your account on {domain} has been successfully created.
This is a demonstration version to show you the current status of the project.
@ -91,16 +97,23 @@ In the beginning, you have nobody to talk to. To find some contacts, you may use
Any feedback welcome. Thank you!
Salut à Toi association
https://www.salut-a-toi.org
The Libervia team.
https://www.libervia.org
"""
)
DEFAULT_DOMAIN = "example.net"
class PluginsAccountSettings(PluginSettings, plugin_info=PLUGIN_INFO):
# Use global email settings by default.
email: EmailSettings | None = Field(
default=None,
description="Email settings, default to settings from ``email`` section, use "
"this if you want different ones for account creation.",
)
class MiscAccount(object):
"""Account plugin: create a SàT + XMPP account, used by Libervia"""
"""Account plugin: create a Libervia + XMPP account."""
# XXX: This plugin was initialy a Q&D one used for the demo.
# TODO: cleaning, separate email handling, more configuration/tests, fixes
@ -121,7 +134,7 @@ class MiscAccount(object):
".plugin",
in_sign="",
out_sign="s",
method=self.account_domain_new_get,
method=lambda: G.config.common.new_account_domain,
async_=False,
)
host.bridge.add_method(
@ -141,7 +154,6 @@ class MiscAccount(object):
async_=True,
)
self.fix_email_admins()
self._sessions = Sessions()
self.__account_cb_id = host.register_callback(
@ -170,46 +182,16 @@ class MiscAccount(object):
self.__delete_account_cb, with_data=True
)
# FIXME: remove this after some time, when the deprecated parameter is really abandoned
def fix_email_admins(self):
"""Handle deprecated config option "admin_email" to fix the admin emails list"""
admin_email = self.config_get("admin_email")
if not admin_email:
return
log.warning(
"admin_email parameter is deprecated, please use email_admins_list instead"
)
param_name = "email_admins_list"
try:
section = ""
value = self.host.memory.config_get(section, param_name, Exception)
except (configparser.NoOptionError, configparser.NoSectionError):
section = CONFIG_SECTION
value = self.host.memory.config_get(
section, param_name, default_conf[param_name]
)
def get_email_conf(self) -> EmailSettings:
"""Return settings to send an email.
value = set(value)
value.add(admin_email)
self.host.memory.config.set(section, param_name, ",".join(value))
def config_get(self, name, section=CONFIG_SECTION):
if name.startswith("email_"):
# XXX: email_ parameters were first in [plugin account] section
# but as it make more sense to have them in common with other plugins,
# they can now be in [DEFAULT] section
try:
value = self.host.memory.config_get(None, name, Exception)
except (configparser.NoOptionError, configparser.NoSectionError):
pass
else:
return value
if section == CONFIG_SECTION:
default = default_conf[name]
Will use settings of this plugins is set, otherwise global ones.
"""
plugin_settings = G.config.get_settings(PluginsAccountSettings)
if plugin_settings.email is not None:
return plugin_settings.email
else:
default = None
return self.host.memory.config_get(section, name, default)
return G.config.email
def _register_account(self, email, password, profile):
return self.registerAccount(email, password, None, profile)
@ -244,7 +226,7 @@ class MiscAccount(object):
if not password or not profile:
raise exceptions.DataError
if profile.lower() in self.config_get("reserved_list"):
if profile.lower() in RESERVED_LIST:
return defer.fail(Failure(exceptions.ConflictError))
d = self.host.memory.create_profile(profile, password)
@ -265,7 +247,7 @@ class MiscAccount(object):
d = defer.succeed(None)
jid_ = jid.JID(jid_s)
else:
jid_s = profile + "@" + self.account_domain_new_get()
jid_s = f"{profile}@{G.config.common.new_account_domain}"
jid_ = jid.JID(jid_s)
d = self.host.plugins["XEP-0077"].register_new_account(jid_, password)
@ -299,10 +281,10 @@ class MiscAccount(object):
def send_emails(self, email, profile):
# time to send the email
domain = self.account_domain_new_get()
domain = G.config.common.new_account_domain
# email to the administrators
admins_emails = self.config_get("email_admins_list")
admins_emails = G.config.common.email_admins_list
if not admins_emails:
log.warning(
"No known admin email, we can't send email to administrator(s).\n"
@ -314,7 +296,11 @@ class MiscAccount(object):
# there is no email when an existing XMPP account is used
body = f"New account created on {domain}: {profile} [{email or '<no email>'}]"
d_admin = sat_email.send_email(
self.host.memory.config, admins_emails, subject, body
self.get_email_conf(),
self.host.memory.config,
admins_emails,
subject,
body,
)
admins_emails_txt = ", ".join(["<" + addr + ">" for addr in admins_emails])
@ -343,7 +329,7 @@ class MiscAccount(object):
# XXX: this will not fail when the email address doesn't exist
# FIXME: check email reception to validate email given by the user
# FIXME: delete the profile if the email could not been sent?
d_user = sat_email.send_email(self.host.memory.config, [email], subject, body)
d_user = sat_email.send_email(self.get_email_conf, [email], subject, body)
d_user.addCallbacks(
lambda __: log.debug(
"Account creation confirmation sent to <{}>".format(email)
@ -353,21 +339,6 @@ class MiscAccount(object):
)
return defer.DeferredList([d_user, d_admin])
def account_domain_new_get(self):
"""get the domain that will be set to new account"""
domain = self.config_get("new_account_domain") or self.config_get(
"xmpp_domain", None
)
if not domain:
log.warning(
_(
'xmpp_domain needs to be set in sat.conf. Using "{default}" meanwhile'
).format(default=DEFAULT_DOMAIN)
)
return DEFAULT_DOMAIN
return domain
def _get_account_dialog_ui(self, profile):
"""Get the main dialog to manage your account
@param menu_data
@ -540,7 +511,7 @@ class MiscAccount(object):
return d
def __delete_account(self, profile):
"""Ask for a confirmation before deleting the XMPP account and SàT profile
"""Ask for a confirmation before deleting the XMPP account and Libervia profile
@param profile
"""
form_ui = xml_tools.XMLUI(
@ -558,9 +529,8 @@ class MiscAccount(object):
)
form_ui.addText(
D_(
"All your data stored on %(server)s, including your %(target)s will be erased."
)
% {"server": self.account_domain_new_get(), "target": target}
"All your data stored on {server}, including your {target} will be erased."
).format(server=G.config.common.new_account_domain, target=target)
)
form_ui.addText(
D_(

5
libervia/backend/plugins/plugin_misc_android.py
View file

@ -1,5 +1,3 @@
#!/usr/bin/env python3
# SAT plugin for file tansfer
# Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
@ -30,6 +28,7 @@ from twisted.internet import protocol
from twisted.internet import abstract
from twisted.internet import error as int_error
from twisted.internet import _sslverify
from libervia.backend import G
from libervia.backend.core.i18n import _, D_
from libervia.backend.core.constants import Const as C
from libervia.backend.core.log import getLogger
@ -284,7 +283,7 @@ class AndroidPlugin(object):
self.am = activity.getSystemService(Context.AUDIO_SERVICE)
# sound notification
media_dir = Path(host.memory.config_get("", "media_dir"))
media_dir = G.config.common.media_dir
assert media_dir is not None
notif_path = media_dir / "sounds" / "notifications" / "music-box.mp3"
self.notif_player = MediaPlayer()

3
libervia/backend/plugins/plugin_misc_app_manager/__init__.py
View file

@ -30,6 +30,7 @@ from twisted.internet import defer
from twisted.python.procutils import which
from yaml.constructor import ConstructorError
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.i18n import _
@ -177,7 +178,7 @@ class AppManager:
"is expected"
)
value = self.host.memory.config_get(section, name, default)
value = getattr(getattr(G.config, section), name) or default
# FIXME: "public_url" is used only here and doesn't take multi-sites into account
if name == "public_url" and (not value or value.startswith("http")):
if not value:

10
libervia/backend/plugins/plugin_misc_email_invitation.py
View file

@ -22,6 +22,7 @@ from twisted.internet import defer
from twisted.words.protocols.jabber import jid
from twisted.words.protocols.jabber import error
from twisted.words.protocols.jabber import sasl
from libervia.backend import G
from libervia.backend.core.i18n import _, D_
from libervia.backend.core.constants import Const as C
from libervia.backend.core import exceptions
@ -229,13 +230,8 @@ class InvitationsPlugin(object):
jid_ = kwargs.pop("jid_", None)
if not jid_:
domain = self.host.memory.config_get(None, "xmpp_domain")
if not domain:
# TODO: fallback to profile's domain
raise ValueError(_("You need to specify xmpp_domain in sat.conf"))
jid_ = "invitation-{uuid}@{domain}".format(
uuid=shortuuid.uuid(), domain=domain
)
domain = G.config.common.xmpp_domain
jid_ = f"invitation-{shortuuid.uuid()}@{domain}"
jid_ = jid.JID(jid_)
extra[KEY_JID] = jid_.full()

108
libervia/backend/plugins/plugin_misc_static_blog.py
View file

@ -1,108 +0,0 @@
#!/usr/bin/env python3
# SAT plugin for static blogs
# Copyright (C) 2014 Adrien Cossa (souliane@mailoo.org)
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from libervia.backend.core.log import getLogger
log = getLogger(__name__)
from libervia.backend.core.i18n import _, D_
from libervia.backend.core.constants import Const as C
from libervia.backend.core import exceptions
from libervia.backend.tools import xml_tools
from twisted.internet import defer
from twisted.words.protocols.jabber import jid
PLUGIN_INFO = {
C.PI_NAME: "Static Blog Plugin",
C.PI_IMPORT_NAME: "STATIC-BLOG",
C.PI_TYPE: "MISC",
C.PI_PROTOCOLS: [],
C.PI_DEPENDENCIES: [],
C.PI_RECOMMENDATIONS: [
"MISC-ACCOUNT"
], # TODO: remove when all blogs can be retrieved
C.PI_MAIN: "StaticBlog",
C.PI_HANDLER: "no",
C.PI_DESCRIPTION: _("""Plugin for static blogs"""),
}
class StaticBlog(object):
params = """
<params>
<individual>
<category name="{category_name}" label="{category_label}">
<param name="{title_name}" label="{title_label}" value="" type="string" security="0"/>
<param name="{banner_name}" label="{banner_label}" value="" type="string" security="0"/>
<param name="{background_name}" label="{background_label}" value ="" type="string" security="0"/>
<param name="{keywords_name}" label="{keywords_label}" value="" type="string" security="0"/>
<param name="{description_name}" label="{description_label}" value="" type="string" security="0"/>
</category>
</individual>
</params>
""".format(
category_name=C.STATIC_BLOG_KEY,
category_label=D_(C.STATIC_BLOG_KEY),
title_name=C.STATIC_BLOG_PARAM_TITLE,
title_label=D_("Page title"),
banner_name=C.STATIC_BLOG_PARAM_BANNER,
banner_label=D_("Banner URL"),
background_name="Background",
background_label=D_("Background image URL"),
keywords_name=C.STATIC_BLOG_PARAM_KEYWORDS,
keywords_label=D_("Keywords"),
description_name=C.STATIC_BLOG_PARAM_DESCRIPTION,
description_label=D_("Description"),
)
def __init__(self, host):
try: # TODO: remove this attribute when all blogs can be retrieved
self.domain = host.plugins["MISC-ACCOUNT"].account_domain_new_get()
except KeyError:
self.domain = None
host.memory.update_params(self.params)
# host.import_menu((D_("User"), D_("Public blog")), self._display_public_blog, security_limit=1, help_string=D_("Display public blog page"), type_=C.MENU_JID_CONTEXT)
def _display_public_blog(self, menu_data, profile):
"""Check if the blog can be displayed and answer the frontend.
@param menu_data: %(menu_data)s
@param profile: %(doc_profile)s
@return: dict
"""
# FIXME: "public_blog" key has been removed
# TODO: replace this with a more generic widget call with URIs
try:
user_jid = jid.JID(menu_data["jid"])
except KeyError:
log.error(_("jid key is not present !"))
return defer.fail(exceptions.DataError)
# TODO: remove this check when all blogs can be retrieved
if self.domain and user_jid.host != self.domain:
info_ui = xml_tools.XMLUI("popup", title=D_("Not available"))
info_ui.addText(
D_("Retrieving a blog from an external domain is not implemented yet.")
)
return {"xmlui": info_ui.toXml()}
return {"public_blog": user_jid.userhost()}

3
libervia/backend/plugins/plugin_pubsub_cache.py
View file

@ -23,6 +23,7 @@ from twisted.words.protocols.jabber import jid, error
from twisted.words.xish import domish
from twisted.internet import defer
from wokkel import pubsub, rsm
from libervia.backend import G
from libervia.backend.core.i18n import _
from libervia.backend.core.constants import Const as C
from libervia.backend.core import exceptions
@ -65,7 +66,7 @@ class PubsubCache:
def __init__(self, host):
log.info(_("PubSub Cache initialization"))
strategy = host.memory.config_get(None, "pubsub_cache_strategy")
strategy = G.config.common.pubsub_cache_strategy
if strategy == "no_cache":
log.info(
_(

18
libervia/backend/plugins/plugin_xep_0045.py
View file

@ -22,6 +22,7 @@ import time
from typing import Optional, cast
import uuid
from pydantic import Field
import shortuuid
from twisted.internet import defer
from twisted.python import failure
@ -32,6 +33,7 @@ from wokkel import rsm
from wokkel import mam
from zope.interface import implementer
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.core_types import SatXMPPEntity
from libervia.backend.core.constants import Const as C
@ -40,6 +42,7 @@ from libervia.backend.core.log import getLogger
from libervia.backend.memory import memory
from libervia.backend.plugins import plugin_misc_jid_search
from libervia.backend.tools import xml_tools, utils
from libervia.backend.tools.config import PluginSettings
log = getLogger(__name__)
@ -71,9 +74,8 @@ HISTORY_LEGACY = "legacy"
HISTORY_MAM = "mam"
CONFIG_SECTION = "plugin muc"
default_conf = {"default_muc": "sat@chat.jabberfr.org"}
class PluginsMucSettings(PluginSettings, plugin_info=PLUGIN_INFO):
default_muc: str = Field(default="sat@chat.jabberfr.org")
class AlreadyJoined(exceptions.ConflictError):
@ -653,13 +655,9 @@ class XEP_0045:
return jid.JID(f"{room_name}@{muc_service}")
def get_default_muc(self):
"""Return the default MUC.
@return: unicode
"""
return self.host.memory.config_get(
CONFIG_SECTION, "default_muc", default_conf["default_muc"]
)
"""Return the default MUC."""
plugin_settings = G.config.get_settings(PluginsMucSettings)
return plugin_settings.default_muc
def _bridge_join_eb(self, failure_, client):
failure_.trap(AlreadyJoined)

29
libervia/backend/plugins/plugin_xep_0055.py
View file

@ -17,12 +17,16 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from pydantic import Field
from libervia.backend import G
from libervia.backend.core.i18n import _, D_
from libervia.backend.core.log import getLogger
from libervia.backend.models.types import JIDType
from libervia.backend.tools.config import PluginSettings
log = getLogger(__name__)
from twisted.words.protocols.jabber.xmlstream import IQ
from twisted.words.protocols.jabber.xmlstream import IQ, XMPPHandler
from twisted.words.protocols.jabber import jid
from twisted.internet import defer
from wokkel import data_form
@ -32,10 +36,6 @@ from libervia.backend.tools import xml_tools
from wokkel import disco, iwokkel
try:
from twisted.words.protocols.xmlstream import XMPPHandler
except ImportError:
from wokkel.subprotocols import XMPPHandler
from zope.interface import implementer
@ -44,6 +44,7 @@ NS_SEARCH = "jabber:iq:search"
PLUGIN_INFO = {
C.PI_NAME: "Jabber Search",
C.PI_IMPORT_NAME: "XEP-0055",
C.PI_CONFIG_KEY: "search",
C.PI_TYPE: "XEP",
C.PI_PROTOCOLS: ["XEP-0055"],
C.PI_DEPENDENCIES: [],
@ -65,18 +66,20 @@ FIELD_CURRENT_SERVICE = (
)
class XEP_0055(object):
class PluginSearchSettings(PluginSettings, plugin_info=PLUGIN_INFO):
services: list[JIDType] = Field(
default_factory=list,
description="Default list of services for search operations.",
)
class XEP_0055:
def __init__(self, host):
log.info(_("Jabber search plugin initialization"))
self.host = host
# default search services (config file + hard-coded lists)
self.services = [
jid.JID(entry)
for entry in host.memory.config_get(
CONFIG_SECTION, CONFIG_SERVICE_LIST, DEFAULT_SERVICE_LIST
)
]
search_settings = G.config.get_settings(PluginSearchSettings)
self.services = search_settings.services
host.bridge.add_method(
"search_fields_ui_get",

9
libervia/backend/plugins/plugin_xep_0060.py
View file

@ -33,7 +33,7 @@ from typing import (
)
import urllib.parse
from pydantic import BaseModel, Field
from pydantic import BaseModel, ConfigDict, Field
from twisted.internet import defer, reactor
from twisted.words.protocols.jabber import error, jid
from twisted.words.xish import domish
@ -44,6 +44,7 @@ from wokkel import rsm
from wokkel import mam
from zope.interface import implementer
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.core_types import SatXMPPEntity
@ -91,7 +92,7 @@ DEFAULT_PUBSUB_MIN_FEAT = {
class NodeMetadata(BaseModel):
"""Node metadata model."""
model_config = {"extra": "allow"}
model_config = ConfigDict(extra="allow")
type: str | None = None
creator: JIDType | None = None
@ -481,9 +482,7 @@ class XEP_0060:
async def profile_connected(self, client):
try:
client.pubsub_service = jid.JID(
self.host.memory.config_get("", "pubsub_service")
)
client.pubsub_service = jid.JID(G.config.common.pubsub_service)
except RuntimeError:
log.info(
_(

3
libervia/backend/plugins/plugin_xep_0077.py
View file

@ -32,6 +32,7 @@ from twisted.words.xish import domish
from wokkel import data_form, disco, iwokkel
from zope.interface import implementer
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.i18n import _
@ -488,7 +489,7 @@ class XEP_0077:
@param port: port of the server to register to
"""
if host is None:
host = self.host.memory.config_get("", "xmpp_domain", "127.0.0.1")
host = G.config.common.xmpp_domain
check_certificate = host != "127.0.0.1"
authenticator = RegisteringAuthenticator(
jid_, password, email, check_certificate=check_certificate

9
libervia/backend/plugins/plugin_xep_0198.py
View file

@ -16,6 +16,7 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from libervia.backend import G
from libervia.backend.core.i18n import _
from libervia.backend.core.constants import Const as C
from libervia.backend.core import exceptions
@ -137,13 +138,7 @@ class XEP_0198(object):
host.trigger.add("xml_init", self._xml_init_trigger)
host.trigger.add("disconnecting", self._disconnecting_trigger)
host.trigger.add("disconnected", self._disconnected_trigger)
try:
self._ack_timeout = int(
host.memory.config_get("", "ack_timeout", ACK_TIMEOUT)
)
except ValueError:
log.error(_("Invalid ack_timeout value, please check your configuration"))
self._ack_timeout = ACK_TIMEOUT
self._ack_timeout = G.config.common.ack_timeout
if not self._ack_timeout:
log.info(_("Ack timeout disabled"))
else:

9
libervia/backend/plugins/plugin_xep_0329.py
View file

@ -28,11 +28,13 @@ from twisted.words.protocols.jabber import jid
from twisted.words.protocols.jabber import error as jabber_error
from twisted.internet import defer
from wokkel import disco, iwokkel, data_form
from libervia.backend import G
from libervia.backend.core.i18n import _
from libervia.backend.core.xmpp import SatXMPPEntity
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.log import getLogger
from libervia.backend.plugins.plugin_comp_file_sharing import FileSharingSettings
from libervia.backend.tools import stream
from libervia.backend.tools import utils
from libervia.backend.tools.common import regex
@ -360,9 +362,10 @@ class XEP_0329(object):
def profile_connected(self, client):
if client.is_component:
client._file_sharing_allowed_hosts = self.host.memory.config_get(
"component file_sharing", "http_upload_allowed_hosts_list"
) or [client.host]
file_sharing_settings = G.config.get_settings(FileSharingSettings)
client._file_sharing_allowed_hosts = (
file_sharing_settings.http_upload_allowed_hosts or [client.host]
)
else:
client._XEP_0329_root_node = ShareNode(
None,

9
libervia/backend/plugins/plugin_xep_0433.py
View file

@ -23,6 +23,7 @@ from twisted.internet import defer
from twisted.words.protocols.jabber import jid
from twisted.words.xish import domish
from wokkel import data_form
from libervia.backend import G
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C
from libervia.backend.core.core_types import SatXMPPEntity
@ -269,13 +270,9 @@ class XEP_0433:
host.trigger.add(
"JID_SEARCH_perform_search", self.jid_search_perform_search_trigger
)
self.allow_external = C.bool(
host.memory.config_get(None, "allow_external_search", C.BOOL_FALSE)
)
self.allow_external = G.config.common.allow_external_search
self.group_chat_search_default_jid = jid.JID(
host.memory.config_get(
None, "group_chat_search_default_jid", "api@search.jabber.network"
)
G.config.common.group_chat_search_default_jid
)
host.bridge.add_method(

8
libervia/backend/stdui/ui_contact_list.py
View file

@ -17,6 +17,7 @@
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from libervia.backend import G
from libervia.backend.core.i18n import _, D_
from libervia.backend.core.constants import Const as C
from libervia.backend.tools import xml_tools
@ -54,11 +55,8 @@ class ContactList(object):
help_string=D_("Remove contact"),
)
# FIXME: a plugin should not be used here, and current profile's jid host would be better than installation wise host
if "MISC-ACCOUNT" in self.host.plugins:
self.default_host = self.host.plugins["MISC-ACCOUNT"].account_domain_new_get()
else:
self.default_host = "example.net"
# FIXME: Current profile's jid host would be better than installation wise host
self.default_host = G.config.common.new_account_domain
def contacts_get(self, profile):
"""Return a sorted list of the contacts for that profile

3
libervia/backend/test/test_plugin_misc_radiocol.py
View file

@ -76,7 +76,8 @@ class RadiocolTest(helpers.SatTestCase):
self.host.get_client(profile) # init self.host.profiles[profile]
self.songs = []
self.playlist = []
self.sound_dir = self.host.memory.config_get("", "media_dir") + "/test/sound/"
assert self.host.memory.config.common.media_dir is not None
self.sound_dir = str(self.host.memory.config.common.media_dir / "/test/sound/")
try:
for filename in os.listdir(self.sound_dir):
if filename.endswith(".ogg") or filename.endswith(".mp3"):

26
libervia/backend/tools/common/email.py
View file

@ -20,11 +20,9 @@
"""email sending facilities"""
from configparser import ConfigParser
from email.mime.text import MIMEText
from twisted.internet import defer
from twisted.mail import smtp
from libervia.backend.core.constants import Const as C
from libervia.backend.tools import config as tools_config
from libervia.backend.core.log import getLogger
@ -32,7 +30,7 @@ log = getLogger(__name__)
def send_email(
config: ConfigParser,
config: tools_config.Settings,
to_emails: list[str] | str,
subject: str = "",
body: str = "",
@ -50,11 +48,11 @@ def send_email(
"""
if isinstance(to_emails, str):
to_emails = to_emails.split()
email_host = tools_config.config_get(config, None, "email_server") or "localhost"
email_from = from_email or tools_config.config_get(config, None, "email_from")
email_host = config.email.server
email_from = from_email or config.email.from_
# we suppose that email domain and XMPP domain are identical
domain = tools_config.config_get(config, None, "xmpp_domain")
domain = config.common.xmpp_domain
if domain is None:
if email_from is not None:
domain = email_from.split("@", 1)[-1]
@ -63,16 +61,12 @@ def send_email(
if email_from is None:
email_from = "no_reply@" + domain
email_sender_domain = tools_config.config_get(
config, None, "email_sender_domain", domain
)
email_port = int(tools_config.config_get(config, None, "email_port", 25))
email_username = tools_config.config_get(config, None, "email_username")
email_password = tools_config.config_get(config, None, "email_password")
email_auth = C.bool(tools_config.config_get(config, None, "email_auth", C.BOOL_FALSE))
email_starttls = C.bool(
tools_config.config_get(config, None, "email_starttls", C.BOOL_FALSE)
)
email_sender_domain = config.email.sender_domain or domain
email_port = config.email.port
email_username = config.email.username
email_password = config.email.password
email_auth = config.email.auth
email_starttls = config.email.starttls
msg = MIMEText(body, "plain", "UTF-8")
msg["Subject"] = subject

26
libervia/backend/tools/common/files_utils.py
View file

@ -17,8 +17,15 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""tools to help manipulating files"""
import os.path
from pathlib import Path
from libervia.backend.core.constants import xdg_config_home
from libervia.backend.core.log import getLogger
log = getLogger(__name__)
def get_unique_name(path):
"""Generate a path with a name not conflicting with existing file
@ -32,3 +39,22 @@ def get_unique_name(path):
path = ori_path.with_name(f"{ori_path.stem}_{idx}{ori_path.suffix}")
idx += 1
return path
def get_downloads_dir() -> Path:
"""Find XDG download dir."""
dirs_file = xdg_config_home / "user-dirs.dirs"
try:
with dirs_file.open("r") as f:
for line in f:
if line.startswith("XDG_DOWNLOAD_DIR"):
path_str = line.split("=", 1)[1].strip().strip('"')
# We use "expandvars" to expand $HOME.
return Path(os.path.expandvars(path_str))
except FileNotFoundError:
pass
except Exception as e:
log.warning(f"Can't read XDG user directories: {e}")
return Path.home() / "Downloads"

6
libervia/backend/tools/common/template.py
View file

@ -377,7 +377,7 @@ class Environment(jinja2.Environment):
return super(Environment, self).get_template(name, parent, globals)
class Renderer(object):
class Renderer:
def __init__(self, host, front_url_filter=None, trusted=False, private=False):
"""
@ -399,10 +399,10 @@ class Renderer(object):
}
self.sites_themes = {}
conf = config.parse_main_conf()
public_sites = config.config_get(conf, None, "sites_path_public_dict", {})
public_sites = conf.web.sites_path_public_dict
sites_data = [public_sites]
if private:
private_sites = config.config_get(conf, None, "sites_path_private_dict", {})
private_sites = conf.web.sites_path_private_dict
sites_data.append(private_sites)
for sites in sites_data:
normalised = {}

23
libervia/backend/tools/common/tls.py
View file

@ -33,10 +33,27 @@ except ImportError:
log = getLogger(__name__)
def get_options_from_config(config, section=""):
def get_options_from_config(
extra_settings: tools_config.TLSSettings | None = None,
) -> dict[str, str | None]:
"""Retrieve TLS configuration.
By default, search in main TLS settings.
@param extra_settings: Settings to use in priority if set.
@return: TLS configuration.
"""
config = tools_config.parse_main_conf()
options = {}
for option in ("tls_certificate", "tls_private_key", "tls_chain"):
options[option] = tools_config.config_get(config, section, option)
for key in ("certificate", "private_key", "chain"):
if (
extra_settings is not None
and (value := getattr(extra_settings, key)) is not None
):
options[f"tls_{key}"] = value
else:
options[f"tls_{key}"] = getattr(config.tls, key)
return options

24
libervia/backend/tools/common/utils.py
View file

@ -114,8 +114,24 @@ class OrderedSet(collections.abc.MutableSet):
self._dict.update({i: None for i in items})
def parse_size(size):
"""Parse a file size with optional multiple symbole"""
def parse_size(size: str) -> int:
"""Parse a file size with optional unit.
The size string can contain an optional unit suffix. Units are case-insensitive.
Spaces within the size string are ignored.
Examples::
"1024" -> 1024
"1 kb" -> 1000
"1 ko" -> 1000
"2 Mb" -> 2000000
"1 MiB" -> 1048576
@param size: Size string with optional unit (e.g. "1024", "1 kb", "2 Mb", "1 MiB")
@return: Size in bytes
@raise ValueError: If the size string is invalid or contains unknown units
"""
try:
return int(size)
except ValueError:
@ -138,13 +154,13 @@ def parse_size(size):
multiplier = size_units[symbol.lower()]
except KeyError:
raise ValueError(
"unknown size multiplier symbole: {symbol!r} (size: {size!r})"
"unknown size multiplier symbol: {symbol!r} (size: {size!r})"
)
else:
return number * multiplier
return number
except Exception as e:
raise ValueError(f"invalid size: {e}")
raise ValueError(f"invalid size ({size!r}): {e}")
def get_size_multiplier(size: int | str, suffix="o"):

597
libervia/backend/tools/config.py
View file

@ -1,8 +1,7 @@
#!/usr/bin/env python3
# SAT: a jabber client
# Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org)
# Libervia XMPP Client
# Copyright (C) 2009-2025 Jérôme Poisson (goffi@goffi.org)
# Copyright (C) 2013-2016 Adrien Cossa (souliane@mailoo.org)
# This program is free software: you can redistribute it and/or modify
@ -20,151 +19,485 @@
"""Configuration related useful methods"""
import os
import csv
import json
from typing import Any
from configparser import ConfigParser, DEFAULTSECT, NoOptionError, NoSectionError
from xdg import BaseDirectory
from libervia.backend.core.log import getLogger
from libervia.backend.core.constants import Const as C
from libervia.backend.core.i18n import _
from collections import defaultdict
from pathlib import Path
from socket import getfqdn
from typing import Annotated, Literal, TypeVar, cast
from pydantic import (
AfterValidator,
BaseModel,
ConfigDict,
Field,
ValidationError,
create_model,
field_validator,
)
from pydantic.types import PathType
from pydantic_settings import (
BaseSettings,
PydanticBaseSettingsSource,
SettingsConfigDict,
TomlConfigSettingsSource,
)
from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C, xdg_data_home
from libervia.backend.core.i18n import _
from libervia.backend.core.log import getLogger
from libervia.backend.tools.common.files_utils import get_downloads_dir
log = getLogger(__name__)
expanduser_validator = AfterValidator(lambda p: p.expanduser().resolve())
def fix_config_option(section, option, value, silent=True):
"""Force a configuration option value
cached_config: "Settings|None" = None
DynamicSettingsRegister = dict[str, "type[DynamicSettingsBase]"]
DynamicSettings: dict[str, DynamicSettingsRegister | type["DynamicSettingsBase"]] = (
defaultdict(DynamicSettingsRegister)
)
the option will be written in the first found user config file, a new user
config will be created if none is found.
@param section (str): the config section
@param option (str): the config option
@param value (str): the new value
@param silent (boolean): toggle logging output (must be True when called from sat.sh)
def autocreatedir(p: Path) -> Path:
"""If directory doesn't exists, create it.
Will also create parents, permission are set for user-only access.
"""
config = ConfigParser()
target_file = None
for file_ in C.CONFIG_FILES[::-1]:
# we will eventually update the existing file with the highest priority,
# if it's a user personal file...
if not silent:
log.debug(_("Testing file %s") % file_)
if os.path.isfile(file_):
if file_.startswith(os.path.expanduser("~")):
config.read([file_])
target_file = file_
break
if not target_file:
# ... otherwise we create a new config file for that user
target_file = (
f"{BaseDirectory.save_config_path(C.APP_NAME_FILE)}/{C.APP_NAME_FILE}.conf"
)
if section and section.upper() != DEFAULTSECT and not config.has_section(section):
config.add_section(section)
config.set(section, option, value)
with open(target_file, "wb") as configfile:
config.write(configfile) # for the next time that user launches sat
if not silent:
if option in ("passphrase",): # list here the options storing a password
value = "******"
log.warning(
_(
"Config auto-update: {option} set to {value} in the file "
"{config_file}."
).format(option=option, value=value, config_file=target_file)
p.mkdir(mode=0o700, parents=True, exist_ok=True)
return p
autocreatedir_validator = AfterValidator(autocreatedir)
def _validate_executable_file(path: Path) -> Path:
"""Validate that a file is executable."""
stat_result = path.stat()
if not (stat_result.st_mode & 0o100):
raise ValueError(f"File {path} is not executable.")
return path
FilePath = Annotated[Path, expanduser_validator, PathType("file")]
ExecutableFilePath = Annotated[
Path,
expanduser_validator,
PathType("file"),
AfterValidator(_validate_executable_file),
]
DirectoryPathAuto = Annotated[
Path, expanduser_validator, autocreatedir_validator, PathType("dir")
]
DirectoryPath = Annotated[Path, expanduser_validator, PathType("dir")]
class ConfigOptions(BaseModel):
"""Configuration options."""
# We need to validate default to be sure that autocreatedir_validator is run.
model_config = ConfigDict(extra="forbid", validate_default=True)
class DynamicSettingsBase(ConfigOptions):
"""Settings which are loaded at runtime."""
_key: str = ""
_sub_key: str | None = None
@classmethod
def _init_subclass(
cls, /, key: str, plugin_info: dict | None = None
) -> None:
"""Initialize the sublass, and register it in DynamicSettings.
@param key: name of the field which will be used to store those settings.
@param info: ``PLUGIN_INFO`` data of the plugin adding settings.
Use None when the data is from a frontend and not a plugin.
"""
cls._key = key
if plugin_info is not None:
sub_key = plugin_info.get(C.PI_CONFIG_KEY) or plugin_info[C.PI_IMPORT_NAME]
assert sub_key
# FIXME: We transform key here, but in the future plugins key name must be
# normalized on plugin import.
sub_key = sub_key.replace("-", "_").lower()
cls._sub_key = sub_key
register = DynamicSettings[key]
assert isinstance(register, dict)
if sub_key in register:
raise exceptions.ConflictError(
f"There is already a plugin registered for {key!r}.{sub_key!r}."
)
register[sub_key] = cls
else:
cls._sub_key = None
DynamicSettings[key] = cls
class PluginSettings(DynamicSettingsBase):
"""Configuration options for plugins."""
def __init_subclass__(cls, /, plugin_info: dict, **kwargs) -> None:
super().__init_subclass__(**kwargs)
super()._init_subclass(key="plugins", plugin_info=plugin_info)
class ComponentSettings(DynamicSettingsBase):
"""Configuration options for components."""
def __init_subclass__(cls, /, plugin_info: dict, **kwargs) -> None:
super().__init_subclass__(**kwargs)
super()._init_subclass(key="components", plugin_info=plugin_info)
class FrontendSettings(DynamicSettingsBase):
def __init_subclass__(cls, /, key: str, **kwargs) -> None:
super().__init_subclass__(**kwargs)
super()._init_subclass(key=key)
D = TypeVar("D", bound=DynamicSettingsBase)
class PbBridgePbSettings(ConfigOptions):
type: Literal["pb"] = "pb"
connection_type: Literal["unix_socket", "socket"] = Field(
default="unix_socket", description="type of connection to use."
)
port: int = Field(
default=8789, description='TCP port to use when ``connection_type`` is "socket"'
)
socket_dir: DirectoryPathAuto = Field(
default=C.TEMP_DIR,
description=(
"Directory where the UNIX socket file will be created when using unix_socket"
" connection type."
),
)
host: str = Field(
default="localhost", description="Host to connect to when using socket connection"
)
class DBusBridgeSettings(ConfigOptions):
type: Literal["dbus"] = "dbus"
int_prefix: str = Field(
default="org.libervia.Libervia", description="Interface prefix"
)
BridgeSettings = Annotated[
DBusBridgeSettings | PbBridgePbSettings, Field(discriminator="type")
]
B = TypeVar("B", bound=BridgeSettings)
class WebSettings(ConfigOptions):
# FIXME: to be moved to web frontend.
model_config = ConfigDict(extra="allow")
sites_path_public: dict[str, str] = Field(
default_factory=dict, description="Dictionary of public site paths"
)
sites_path_private_dict: dict[str, str] = Field(
default_factory=dict, description="Dictionary of private site paths"
)
class EmailSettings(ConfigOptions):
server: str = Field(default="localhost", description="Email server address")
port: int = Field(default=25, description="Email server port")
username: str | None = Field(default=None, description="Email server username")
password: str | None = Field(default=None, description="Email server password")
starttls: bool = Field(default=False, description="Use STARTTLS for email connection")
auth: bool = Field(
default=False, description="Enable authentication for email connection"
)
from_: str | None = Field(
default="NOREPLY@example.net",
description="Sender address for emails",
alias="from",
)
sender_domain: str | None = Field(default=None, description="Email sender domain")
class TLSSettings(ConfigOptions):
certificate: FilePath | None = Field(
default=None, description="Path to TLS certificate file"
)
private_key: FilePath | None = Field(
default=None, description="Path to TLS private key file"
)
chain: FilePath | None = Field(
default=None,
description="Path to TLS certificate chain file (intermediate certificates)",
)
class CommonSettings(ConfigOptions):
# "media_dir": "/usr/share/" + Const.APP_NAME_FILE + "/media",
media_dir: DirectoryPath | None = Field(
default=None, description="Directory for media files"
)
local_dir: DirectoryPathAuto = Field(
default=xdg_data_home / C.APP_NAME_FILE,
description="Local directory for Libervia data",
)
downloads_dir: DirectoryPathAuto = Field(
default_factory=get_downloads_dir,
description="Directory where files can be downloaded by default.",
)
log_dir: DirectoryPath = Field(
default_factory=lambda data: data["local_dir"],
description="Directory where log files will be written. Default to local_dir.",
)
log_level: str | None = Field(
default=None, description="Logging level (debug, info, warning, error)"
)
log_fmt: str | None = Field(default=None, description="Log message format")
bridges: list[BridgeSettings] = Field(
default_factory=list,
description=(
"Bridge is the name of the IPC used to communicate between backend and"
" frontends. Leave empty to let Libervia determine the best one. If you"
" indicate several bridges, they will run in parallel."
),
)
pid_dir: DirectoryPathAuto = Field(
default=C.TEMP_DIR,
description=(
"Directory where PID files will be stored. Default is a subdirectory of the"
" system's temporary directory."
),
)
xmpp_domain: str = Field(
default=getfqdn(),
description=(
"XMPP domain of the main server. Local hostname will be used by default."
),
)
new_account_domain: str | None = Field(
default_factory=lambda data: data["xmpp_domain"],
description=(
"Domain to use when creating new account. Default to ``xmpp_domain``."
),
)
public_url: str | None = Field(default=None, description="Public URL for the server")
# FIXME: we don't use JIDType here because config is also parsed by frontend, and
# JIDType uses Twisted's JID class.
pubsub_service: str | None = Field(
default=None,
description=(
"Main pubsub service to use. If not set, it will be discovered automatically."
),
)
admins_list: list[str] = Field(
default_factory=list, description="List of profile names of admin users"
)
email_admins_list: list[str] = Field(
default_factory=list, description="List of admin email addresses"
)
mutli_users: bool = Field(default=False, description="Enable multi-user support")
pubsub_cache_strategy: Literal["no_cache"] | None = Field(
default=None, description="PubSub cache strategy"
)
hosts_dict: dict[str, dict] = Field(
default_factory=dict, description="Dictionary of host configurations"
)
ack_timeout: int = Field(
default=35,
description=(
"Acknowledgement timeout in seconds. If the server doesn't acknowledge a"
" stanza within this time, the connection will be aborted."
),
)
force_autodisconnect: bool = Field(
default=False, description="Force automatic disconnection"
)
# FIXME: Seems not used anymore, delete?
use_local_shared_tmp: bool = Field(
default=False,
description=(
"If set, a temporary dir in cache will be used to share files between backend"
" and frontends, useful when they are separated (e.g. when using containers)."
" If unset, a temporary dir will be automatically created in os-relevant"
" location."
),
)
allow_external_search: bool = Field(
default=False, description="Allow external search functionality"
)
image_max: tuple[int, int] = Field(
default=(1200, 720), description="Maximum dimensions for images (width, height)"
)
init_wait_for_service: str | None = Field(
default=None,
description=(
"Wait for a network service to become available before starting the backend."
" Format is 'host:port[:timeout][:service_name]' where host can be an"
" IPv4/IPv6 address or hostname, port is the service port number, timeout is"
" optional (default 60 seconds) and service_name is optional (defaults to"
" 'service'). Example: 'localhost:5222:30:XmppServer'"
),
)
init_script: FilePath | None = Field(
default=None,
description="Path to an executable script to be run during backend startup.",
)
group_chat_search_default_jid: str = Field(
default="api@search.jabber.network",
description="JID of the entity used for group chat searches.",
)
@field_validator("bridges", mode="after")
@classmethod
def set_bridges_auto(cls, bridges: list[BridgeSettings]) -> list[BridgeSettings]:
if not bridges:
# FIXME: We default to DBus for now.
bridges = [DBusBridgeSettings()]
return bridges
def get_bridge_conf(self, bridge_type: type[B]) -> B:
"""Get bridge configuration by type.
@param type: The type of bridge to find (e.g. ``pb``, ``dbus``)
@return: The configuration for the specified bridge.
@raise exceptions.InternalError: If no bridge configuration is found for the given
type.
"""
for bridge_conf in self.bridges:
if isinstance(bridge_conf, bridge_type):
return bridge_conf
# We raise internal error because there should not be any request for conf if the
# bridge is not in "bridges" and thus instanciating.
raise exceptions.InternalError(
f"No bridge configuration found for type {type!r}."
)
def parse_main_conf(log_filenames=False):
"""Look for main .ini configuration file, and parse it
class Settings(BaseSettings):
model_config = SettingsConfigDict(
extra="allow",
env_prefix="LIBERVIA_",
env_nested_delimiter="__",
toml_file=C.CONFIG_FILES,
)
@param log_filenames(bool): if True, log filenames of read config files
common: CommonSettings = Field(default_factory=CommonSettings)
email: EmailSettings = Field(default_factory=EmailSettings)
tls: TLSSettings = Field(default_factory=TLSSettings)
web: WebSettings = Field(default_factory=WebSettings)
plugins: dict = Field(default_factory=dict, description="Plugin specific settings.")
components: dict = Field(
default_factory=dict, description="Component specific settings."
)
def get_settings(self, dyn_settings_cls: type[D]) -> D:
"""Get dynamic settings instance.
@param dyn_settings_cls: The dynamtic settings class to retrieve.
@return: The instance of the specified settings.
@raise exceptions.InternalError: If no settings are found. This should not happen
as the dynamtic classes are automatically registered.
"""
try:
settings = getattr(self, dyn_settings_cls._key)
if dyn_settings_cls._sub_key is None:
return settings
else:
return getattr(settings, dyn_settings_cls._sub_key)
except AttributeError:
raise exceptions.InternalError(
f"No settings found for {dyn_settings_cls._sub_key!r}, they should"
" be imported dynamically."
)
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
env_settings,
dotenv_settings,
file_secret_settings,
TomlConfigSettingsSource(
settings_cls,
# FIXME: deep_merge is not yet available, will be for pydantic-settings
# 2.13
# deep_merge=True
),
)
def parse_main_conf(log_filenames: bool = False, ignore_cache=False) -> Settings:
"""Look for main .toml configuration files, and parse them.
@param log_filenames: if True, log filenames of read config files.
@param ignore_cache: if True, settings will be reparsed even if they are already in
cache. Notable useful when dynamic setting (e.g.; from plugins) have been added.
"""
config = ConfigParser(defaults=C.DEFAULT_CONFIG)
try:
filenames = config.read(C.CONFIG_FILES)
except Exception as e:
log.error(_("Can't read main config: {msg}").format(msg=e), exc_info=True)
else:
if log_filenames:
if filenames:
log.info(
_("Configuration was read from: {filenames}").format(
filenames=", ".join(filenames)
)
global cached_config
if log_filenames:
existing_conf_file = []
for path_str in C.CONFIG_FILES:
path = Path(path_str)
if path.is_file() and path.exists():
existing_conf_file.append(str(path))
if existing_conf_file:
log.info(
_("Configuration will be read from: {filenames}.").format(
filenames=", ".join(existing_conf_file)
)
)
else:
log.info(_("No configuration file found."))
if ignore_cache or cached_config is None:
dynamic_settings_kw = {}
for key, settings in DynamicSettings.items():
if isinstance(settings, dict):
NewSettingsModel = create_model(
f"{key.title()}Settings",
**cast(
dict,
{
sub_key: (settings_type, Field(default_factory=settings_type))
for sub_key, settings_type in settings.items()
},
),
__config__ = ConfigDict(extra="forbid")
)
dynamic_settings_kw[key] = (
NewSettingsModel,
Field(default_factory=NewSettingsModel),
)
else:
log.warning(_("No configuration file found, using default settings"))
dynamic_settings_kw[key] = (
settings,
Field(default_factory=settings),
)
return config
def config_get(config, section, name, default=None):
"""Get a configuration option
@param config (ConfigParser): the configuration instance
@param section (str): section of the config file (None or '' for DEFAULT)
@param name (str): name of the option
@param default: value to use if not found, or Exception to raise an exception
@return (unicode, list, dict): parsed value
@raise: NoOptionError if option is not present and default is Exception
NoSectionError if section doesn't exists and default is Exception
exceptions.ParsingError error while parsing value
"""
if not section:
section = DEFAULTSECT
try:
value = config.get(section, name)
except (NoOptionError, NoSectionError) as e:
if default is Exception:
raise e
return default
if name.endswith("_path") or name.endswith("_dir"):
value = os.path.expanduser(value)
# thx to Brian (http://stackoverflow.com/questions/186857/splitting-a-semicolon-separated-string-to-a-dictionary-in-python/186873#186873)
elif name.endswith("_list"):
value = next(
csv.reader([value], delimiter=",", quotechar='"', skipinitialspace=True)
LiberviaSettings = create_model(
"LiberviaSettings", __base__=Settings, **dynamic_settings_kw
)
elif name.endswith("_dict"):
try:
value = json.loads(value)
except ValueError as e:
raise exceptions.ParsingError("Error while parsing data: {}".format(e))
if not isinstance(value, dict):
raise exceptions.ParsingError(
"{name} value is not a dict: {value}".format(name=name, value=value)
)
elif name.endswith("_json"):
try:
value = json.loads(value)
except ValueError as e:
raise exceptions.ParsingError("Error while parsing data: {}".format(e))
return value
cached_config = LiberviaSettings()
except ValidationError as e:
log.error(f"Settings are invalid, please fix your configuration: {e}")
raise exceptions.ConfigError
except:
log.exception("Can't parse settings.")
raise exceptions.InternalError
def get_conf(
conf: ConfigParser, prefix: str, section: str, name: str, default: Any
) -> Any:
"""Get configuration value from environment or config file
@param str: prefix to use for the varilable name (see `name` below)
@param section: config section to use
@param name: unsuffixed name.
For environment variable, `LIBERVIA_<prefix>_` will be prefixed (and name
will be set to uppercase).
For config file, `<prefix>_` will be prefixed (and DEFAULT section will be
used).
Environment variable has priority over config values. If Environment variable
is set but empty string, config value will be used.
@param default: default value to use if varilable is set neither in environment,
nor in config
"""
# XXX: This is a temporary method until parameters are refactored
value = os.getenv(f"LIBERVIA_{prefix}_{name}".upper())
return value or config_get(conf, section, f"{prefix}_{name}", default)
return cached_config

5
libervia/backend/tools/image.py
View file

@ -23,6 +23,7 @@ import mimetypes
from PIL import Image, ImageOps
from pathlib import Path
from twisted.internet import threads
from libervia.backend import G
from libervia.backend.core.i18n import _
from libervia.backend.core import exceptions
from libervia.backend.core.log import getLogger
@ -41,7 +42,7 @@ def check(host, path, max_size=None):
report will indicate if image is too large, and the recommended new size if this is
the case
@param host: SàT instance
@param host: Libervia instance
@param path(str, pathlib.Path): image to open
@param max_size(tuple[int, int]): maximum accepted size of image
None to use value set in config
@ -52,7 +53,7 @@ def check(host, path, max_size=None):
report = {}
image = Image.open(path)
if max_size is None:
max_size = tuple(host.memory.config_get(None, "image_max", (1200, 720)))
max_size = G.config.common.image_max
if image.size > max_size:
report["too_large"] = True
if image.size[0] > max_size[0]:

50
libervia/cli/base.py
View file

@ -29,9 +29,10 @@ from pathlib import Path
import sys
import termios
import tty
from typing import Optional, Set, Union
from typing import Literal, Optional, Set, Union
import xml.etree.ElementTree as ET
from pydantic import Field
from rich import console
from twisted.internet.error import ReactorNotRunning
@ -54,11 +55,33 @@ from libervia.frontends.tools.jid import JID
log.basicConfig(level=log.WARNING, format="[%(name)s] %(message)s")
###
class CLISettings(config.FrontendSettings, key="cli"):
background: Literal["auto", "dark", "light"] = Field(
default="auto", description="Background handling strategy"
)
image_cmd: str|None = Field(
default=None,
description="Command to use to display images. Must be avaialble in path."
)
editor: str = Field(
default = os.getenv("EDITOR", "vi")
)
editor_args_opt: str|None = Field(
default=None
)
syntax_ext: dict = Field(default_factory=dict)
blog_preview_open_cmd: str|None = Field(
default=None
)
blog_preview_update_cmd: str|None = Field(
default_factory=lambda data: data["blog_preview_open_cmd"]
)
## bridge handling
# we get bridge name from conf and initialise the right class accordingly
main_config = config.parse_main_conf()
bridge_name = config.config_get(main_config, "", "bridge", "dbus")
main_config = config.parse_main_conf(ignore_cache=True)
bridge_name = next(iter(main_config.common.bridges)).type
LiberviaCLILoop = get_libervia_cli_loop(bridge_name)
@ -131,7 +154,9 @@ class LiberviaCli:
by default display a message
"""
self.console = console.Console(theme=C.THEME_DEFAULT)
self.sat_conf = main_config
self.settings = main_config
self.cli_settings = main_config.get_settings(CLISettings)
self.media_dir = main_config.common.media_dir
self.set_color_theme()
bridge_module = dynamic_import.bridge(bridge_name, "libervia.frontends.bridge")
if bridge_module is None:
@ -141,10 +166,6 @@ class LiberviaCli:
self.bridge = bridge_module.AIOBridge()
self._on_quit_callbacks = []
def get_config(self, name, section=C.CONFIG_SECTION, default=None):
"""Retrieve a setting value from sat.conf"""
return config.config_get(self.sat_conf, section, name, default=default)
def guess_background(self):
# cf. https://unix.stackexchange.com/a/245568 (thanks!)
try:
@ -210,16 +231,9 @@ class LiberviaCli:
return "dark"
def set_color_theme(self):
background = self.get_config("background", default="auto")
background = self.cli_settings.background
if background == "auto":
background = self.guess_background()
if background not in ("dark", "light"):
raise exceptions.ConfigError(
_(
'Invalid value set for "background" ({background}), please check '
"your settings in libervia.conf"
).format(background=repr(background))
)
self.background = background
if background == "light":
C.A_HEADER = A.FG_MAGENTA
@ -1272,8 +1286,8 @@ class CommandBase:
self.add_parser_options()
@property
def sat_conf(self):
return self.host.sat_conf
def cli_settings(self) -> CLISettings:
return self.host.cli_settings
@property
def args(self):

2
libervia/cli/call_gui.py
View file

@ -204,7 +204,7 @@ class AVCallUI(QMainWindow):
@classmethod
async def run(cls, parent, call_data):
"""Run PyQt loop and show the app"""
media_dir = Path(await parent.host.bridge.config_get("", "media_dir"))
media_dir = parent.host.settings.common.media_dir
icons_path = media_dir / "fonts/fontawesome/svgs/solid"
app = QApplication([])
av_call_gui = cls(parent.host, icons_path)

3
libervia/cli/cmd_avatar.py
View file

@ -48,8 +48,7 @@ class Get(base.CommandBase):
self.parser.add_argument("jid", nargs="?", default="", help=_("entity"))
async def show_image(self, path):
sat_conf = config.parse_main_conf()
cmd = config.config_get(sat_conf, C.CONFIG_SECTION, "image_cmd")
cmd = self.host.cli_settings.image_cmd
cmds = [cmd] + DISPLAY_CMD if cmd else DISPLAY_CMD
for cmd in cmds:
try:

19
libervia/cli/cmd_blog.py
View file

@ -770,7 +770,7 @@ class Edit(base.CommandBase, BlogPublishCommon, common.BaseEdit):
async def start(self):
# if there are user defined extension, we use them
SYNTAX_EXT.update(
config.config_get(self.sat_conf, C.CONFIG_SECTION, CONF_SYNTAX_EXT, {})
self.host.cli_settings.syntax_ext
)
self.current_syntax = await self.get_current_syntax()
@ -948,24 +948,17 @@ class Preview(base.CommandBase, common.BaseEdit):
else:
aionotify = None
sat_conf = self.sat_conf
SYNTAX_EXT.update(
config.config_get(sat_conf, C.CONFIG_SECTION, CONF_SYNTAX_EXT, {})
self.host.cli_settings.syntax_ext
)
try:
self.open_cb_cmd = config.config_get(
sat_conf, C.CONFIG_SECTION, "blog_preview_open_cmd", Exception
)
except (NoOptionError, NoSectionError):
self.open_cb_cmd = None
self.open_cb_cmd = self.cli_settings.blog_preview_open_cmd
if self.open_cb_cmd is None:
open_cb = self.show_preview
else:
open_cb = self.open_preview_ext
open_cp = self.open_preview_ext
self.update_cb_cmd = config.config_get(
sat_conf, C.CONFIG_SECTION, "blog_preview_update_cmd", self.open_cb_cmd
)
self.update_cb_cmd = self.cli_settings.blog_preview_update_cmd
if self.update_cb_cmd is None:
update_cb = self.show_preview
else:

33
libervia/cli/common.py
View file

@ -26,6 +26,7 @@ import asyncio
import shlex
import re
from pathlib import Path
from typing import TYPE_CHECKING
from libervia.cli.constants import Const as C
from libervia.backend.core.i18n import _
from libervia.backend.core import exceptions
@ -36,6 +37,9 @@ from libervia.backend.tools import config
from configparser import NoSectionError, NoOptionError
from collections import namedtuple
if TYPE_CHECKING:
from .base import LiberviaCli
# default arguments used for some known editors (editing with metadata)
VIM_SPLIT_ARGS = "-c 'set nospr|vsplit|wincmd w|next|wincmd w'"
EMACS_SPLIT_ARGS = '--eval "(split-window-horizontally)"'
@ -83,7 +87,7 @@ def ansi_rjust(s, width):
return " " * (width - len(cleaned)) + s
def get_tmp_dir(sat_conf, cat_dir, sub_dir=None):
def get_tmp_dir(config: config.Settings, cat_dir, sub_dir=None):
"""Return directory used to store temporary files
@param sat_conf(ConfigParser.ConfigParser): instance opened on sat configuration
@ -94,8 +98,7 @@ def get_tmp_dir(sat_conf, cat_dir, sub_dir=None):
initial str)
@return (Path): path to the dir
"""
local_dir = config.config_get(sat_conf, "", "local_dir", Exception)
path_elts = [local_dir, cat_dir]
path_elts = [config.common.local_dir, cat_dir]
if sub_dir is not None:
path_elts.append(regex.path_escape(sub_dir))
return Path(*path_elts)
@ -126,7 +129,7 @@ class BaseEdit(object):
It works with temporary files in SàT local_dir, in a "cat_dir" subdir
"""
def __init__(self, host, cat_dir, use_metadata=False):
def __init__(self, host: "LiberviaCli", cat_dir, use_metadata=False):
"""
@param sat_conf(ConfigParser.ConfigParser): instance opened on sat configuration
@param cat_dir(unicode): directory to use for drafts
@ -152,7 +155,7 @@ class BaseEdit(object):
path = Path(path).resolve()
if not path.is_file:
raise OSError("path must link to a regular file")
if path.parent != get_tmp_dir(self.sat_conf, self.cat_dir):
if path.parent != get_tmp_dir(self.host.settings, self.cat_dir):
self.disp(
f"File {path} is not in SàT temporary hierarchy, we do not remove " f"it",
2,
@ -160,7 +163,7 @@ class BaseEdit(object):
return
# we have 2 files per draft with use_metadata, so we double max
unlink_max = SECURE_UNLINK_MAX * 2 if self.use_metadata else SECURE_UNLINK_MAX
backup_dir = get_tmp_dir(self.sat_conf, self.cat_dir, SECURE_UNLINK_DIR)
backup_dir = get_tmp_dir(self.host.settings, self.cat_dir, SECURE_UNLINK_DIR)
if not os.path.exists(backup_dir):
os.makedirs(backup_dir)
filename = os.path.basename(path)
@ -210,15 +213,9 @@ class BaseEdit(object):
content_file_obj.close()
# we prepare arguments
editor = config.config_get(
self.sat_conf, C.CONFIG_SECTION, "editor"
) or os.getenv("EDITOR", "vi")
try:
# is there custom arguments in sat.conf ?
editor_args = config.config_get(
self.sat_conf, C.CONFIG_SECTION, editor_args_opt, Exception
)
except (NoOptionError, NoSectionError):
editor = self.host.cli_settings.editor
editor_args = self.host.cli_settings.editor_args_opt
if editor_args is None:
# no, we check if we know the editor and have special arguments
if self.use_metadata:
editor_args = EDITOR_ARGS_MAGIC.get(os.path.basename(editor), "")
@ -291,7 +288,7 @@ class BaseEdit(object):
if len(content) == 0:
self.disp("Content is empty, cancelling the edition")
if content_file_path.parent != get_tmp_dir(self.sat_conf, self.cat_dir):
if content_file_path.parent != get_tmp_dir(self.host.settings, self.cat_dir):
self.disp(
"File are not in SàT temporary hierarchy, we do not remove them",
2,
@ -350,7 +347,7 @@ class BaseEdit(object):
"""
suff = "." + self.get_tmp_suff()
cat_dir_str = self.cat_dir
tmp_dir = get_tmp_dir(self.sat_conf, self.cat_dir, self.profile)
tmp_dir = get_tmp_dir(self.host.settings, self.cat_dir, self.profile)
if not tmp_dir.exists():
try:
tmp_dir.mkdir(parents=True)
@ -381,7 +378,7 @@ class BaseEdit(object):
# we guess the item currently edited by choosing
# the most recent file corresponding to temp file pattern
# in tmp_dir, excluding metadata files
tmp_dir = get_tmp_dir(self.sat_conf, self.cat_dir, profile)
tmp_dir = get_tmp_dir(self.host.settings, self.cat_dir, profile)
available = [
p
for p in tmp_dir.glob(f"{self.cat_dir}_*")

1
libervia/cli/constants.py
View file

@ -28,7 +28,6 @@ class Const(constants.Const):
APP_COMPONENT = "CLI"
APP_NAME_ALT = "jp"
APP_NAME_FILE = "libervia_cli"
CONFIG_SECTION = APP_COMPONENT.lower()
PLUGIN_CMD = "commands"
PLUGIN_OUTPUT = "outputs"
#: All outputs are managed directly by the command, ``extra_outputs`` must be

2
libervia/cli/output_template.py
View file

@ -68,8 +68,6 @@ class Template(object):
command's template_data_mapping attribute will be used if it exists to convert
data to a dict usable by the template.
"""
# media_dir is needed for the template
self.host.media_dir = self.host.bridge.config_get("", "media_dir")
cmd = self.host.command
try:
template_path = cmd.TEMPLATE

6
libervia/frontends/bridge/dbus_bridge.py
View file

@ -32,9 +32,9 @@ log = getLogger(__name__)
# Interface prefix
const_INT_PREFIX = config.config_get(
config.parse_main_conf(), "", "bridge_dbus_int_prefix", "org.libervia.Libervia"
)
conf = config.parse_main_conf()
bridge_conf = conf.common.get_bridge_conf(config.DBusBridgeSettings)
const_INT_PREFIX = bridge_conf.int_prefix
const_ERROR_PREFIX = const_INT_PREFIX + ".error"
const_OBJ_PATH = "/org/libervia/Libervia/bridge"
const_CORE_SUFFIX = ".core"

11
libervia/frontends/bridge/pb.py
View file

@ -137,16 +137,13 @@ class bridge(object):
def bridge_connect(self, callback, errback):
factory = pb.PBClientFactory()
conf = config.parse_main_conf()
get_conf = partial(config.get_conf, conf, "bridge_pb", "")
conn_type = get_conf("connection_type", "unix_socket")
bridge_conf = conf.common.get_bridge_conf(config.PbBridgePbSettings)
conn_type = bridge_conf.connection_type
if conn_type == "unix_socket":
local_dir = Path(config.config_get(conf, "", "local_dir")).resolve()
socket_path = local_dir / "bridge_pb"
socket_path = bridge_conf.socket_dir / "bridge_pb"
reactor.connectUNIX(str(socket_path), factory)
elif conn_type == "socket":
host = get_conf("host", "localhost")
port = int(get_conf("port", 8789))
reactor.connectTCP(host, port, factory)
reactor.connectTCP(bridge_conf.host, bridge_conf.port, factory)
else:
raise ValueError(f"Unknown pb connection type: {conn_type!r}")
d = factory.getRootObject()

62
libervia/tui/base.py
View file

@ -17,6 +17,7 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
from pydantic import Field
from libervia.backend.core.i18n import _, D_
from libervia.tui.constants import Const as C
from libervia.backend.core import log_config
@ -25,7 +26,7 @@ log_config.libervia_configure(C.LOG_BACKEND_STANDARD, C)
from libervia.backend.core import log as logging
log = logging.getLogger(__name__)
from libervia.backend.tools import config as sat_config
from libervia.backend.tools import config as tools_config
import urwid
from urwid.util import is_wide_char
from urwid_satext import sat_widgets
@ -39,17 +40,33 @@ from libervia.tui import xmlui
from libervia.tui.progress import Progress
from libervia.tui.notify import Notify
from libervia.tui.keys import action_key_map as a_key
from libervia.tui import config
# from libervia.tui import config
from libervia.frontends.tools.misc import InputHistory
from libervia.backend.tools.common import dynamic_import
from libervia.frontends.tools import jid
import signal
import sys
class TUISettings(tools_config.FrontendSettings, key="tui"):
bracketed_paste: bool = Field(
default=False,
description="""Enable or disable bracketed paste mode.
When enabled, this allows the terminal to distinguish between typed text and
pasted text, which is particularly useful for multi-line pastes in the TUI. This
prevents pasted content from being interpreted as individual keystrokes.
Note: This feature requires terminal support for bracketed paste mode (ANSI escape
sequences).
"""
)
## bridge handling
# we get bridge name from conf and initialise the right class accordingly
main_config = sat_config.parse_main_conf()
bridge_name = sat_config.config_get(main_config, "", "bridge", "dbus")
main_config = tools_config.parse_main_conf(ignore_cache=True)
bridge_name = next(iter(main_config.common.bridges)).type
if "dbus" not in bridge_name:
print("only D-Bus bridge is currently supported")
sys.exit(3)
@ -333,6 +350,7 @@ class LiberviaTUIApp(QuickApp, InputHistory):
check_options=quick_utils.check_options,
connect_bridge=False,
)
self.tui_settings = main_config.get_settings(TUISettings)
## main loop setup ##
event_loop = (
urwid.GLibEventLoop if "dbus" in bridge_name else urwid.TwistedEventLoop
@ -370,10 +388,7 @@ class LiberviaTUIApp(QuickApp, InputHistory):
# we already manage exit with a_key['APP_QUIT'], so we don't want C-c
signal.signal(signal.SIGINT, signal.SIG_IGN)
sat_conf = sat_config.parse_main_conf()
self._bracketed_paste = C.bool(
sat_config.config_get(sat_conf, C.CONFIG_SECTION, "bracketed_paste", "false")
)
self._bracketed_paste = self.tui_settings.bracketed_paste
if self._bracketed_paste:
log.debug("setting bracketed paste mode as requested")
sys.stdout.write("\033[?2004h")
@ -431,20 +446,23 @@ class LiberviaTUIApp(QuickApp, InputHistory):
self.loop.run()
def post_init(self):
try:
config.apply_config(self)
except Exception as e:
log.error("configuration error: {}".format(e))
popup = self.alert(
_("Configuration Error"),
_(
"Something went wrong while reading the configuration, please check :messages"
),
)
if self.options.profile:
self._early_popup = popup
else:
self.show_pop_up(popup)
# FIXME: legacy code to handle config. Options there (such as the ones prefixed
# with "KEY_", or "disable_mouse") are not yet managed. This code can be deleted
# once everything is managed again.
# try:
# config.apply_config(self)
# except Exception as e:
# log.error("configuration error: {}".format(e))
# popup = self.alert(
# _("Configuration Error"),
# _(
# "Something went wrong while reading the configuration, please check :messages"
# ),
# )
# if self.options.profile:
# self._early_popup = popup
# else:
# self.show_pop_up(popup)
super(LiberviaTUIApp, self).post_init(self.main_widget)
def keys_to_text(self, keys):

1
libervia/tui/constants.py
View file

@ -25,7 +25,6 @@ class Const(constants.Const):
APP_COMPONENT = "TUI"
APP_NAME_ALT = "LiberviaTUI"
APP_NAME_FILE = "libervia_tui"
CONFIG_SECTION = APP_COMPONENT.lower()
PALETTE = [
("title", "black", "light gray", "standout,underline"),
("title_focus", "white,bold", "light gray", "standout,underline"),

4
pyproject.toml
View file

@ -40,7 +40,8 @@ dependencies = [
"omemo >= 1.0.0, < 2",
"pillow >= 6.0.0",
"progressbar2 < 3.54",
"pydantic ~= 2.4",
"pydantic ~= 2.12",
"pydantic-settings ~= 2.12",
"pygments < 3",
"pygobject ~= 3.48",
"pyopenssl ~= 24.1",
@ -48,7 +49,6 @@ dependencies = [
# currently disabled due to incompatibility, OTR feature may be removed in
# the close future
# "python-potr < 1.1",
"pyxdg < 0.30",
"pyyaml < 7.0.0",
"python-xlib ~= 0.33",
"prompt_toolkit ~= 3.0",