core: Replace legacy INI config system with Pydantic-based TOML configuration.

This is a huge configuration refactoring.

The configuration system now uses Pydantic settings, with better validation, better
support for environment variables and other goodies.

The main config files now use TOML instead of INI, this allow for better handling of
complex types, notably dict.

Plugins, components and frontends can declare their config settings in a class which will
be imported dynamically.
This commit is contained in:
Goffi 2025-11-30 12:27:35 +01:00
commit 6a7b7bc36a
59 changed files with 1392 additions and 1011 deletions

View file

@ -104,7 +104,7 @@ entities (JIDs). Permissions can be set through Ad-Hoc Commands.
Configuration Configuration
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
All options are to be set in ``[component file-sharing]`` section. All options are to be set in ``[components.file_sharing]`` section.
``http_upload_port`` ``http_upload_port``
port to use for HTTP File Upload port to use for HTTP File Upload
@ -181,148 +181,7 @@ documentation to find how this must be done.
Configuration Configuration
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
All options are to be set in ``[component ap-gateway]`` section. All options are to be set in ``[components.ap_gateway]`` section.
``public_url``
Main user-facing domain of the HTTP server, this will be used to construct all AP URLs
**default**: if not set, ``xmpp_domain`` is used. If ``xmpp_domain`` is not set either,
an error is raised.
``http_port``
port where the HTTP server should listen. Port ``80`` is not used directly as it would
require root privileges, and it is strongly recommended against launching Libervia under
a privileged account. An HTTP Proxy or a port redirection should be set to redirect the
``80`` port to the port specified here.
**default**: ``8123``
``http_connection_type``
either ``http`` or ``https``. If you have a HTTP proxy such as Apache or NGINX which
already handles HTTPS, you may want to use ``http``.
**default**: ``https``
Note that the HTTP server should always use ``https`` with end-user, the ``http`` option
is only to be used with an HTTPS proxy.
``local_only``
A boolean value indicating if the gateway is allowed to convert pubsub node from
external XMPP service or not. A JID is considered external if its domain part doesn't
end with the gateway's server. For instance, if a gateway ``ap.example.org`` is set on
the server ``example.org``, the JIDs ``pierre@example.org`` or
``some-node@pubsub.example.org`` will be considered local, but
``buenaventura@example.net`` won't (note the different domain).
Most of time, ``local_only`` should be used.
**default**: ``true``
``ap_path``
Path prefix to use for ActivityPub request. It's usually not necessary to change the
default value.
**default**: ``_ap``
``comments_max_depth``
An integer value indicating the maximum number of comment nodes that can be created. See
:ref:`ap-xmpp-threads-conversion`
``auto_mentions``
A boolean value indicating if received XMPP pubsub blog items bodies must be scanned to
find ``@user@server.tld`` type mentions. If mentions are found, they'll be added to the
resulting AP items.
**default**: ``true``
``http_sign_get``
A boolean value indicating if HTTP ``GET`` request made to AP implementations must be
signed. Some AP implementations require those requests to be signed, notably Mastodon
when "secure mode" is activated.
**default**: ``true``
``html_redirect_dict``
A dictionary used to redirect HTTP requests when it's not an ActivityPub request (i.e.
when the ``Accept`` header is not set to ``application/json``). Several ActivityPub
implementations link to original server when clicking on some links such as post author
or item ID, and this result in a request to the corresponding ActivityPub endpoint, but
made to retrieve HTML instead of ActivityPub JSON data.
By default, this request is showing the corresponding AP JSON data, but if you set HTML
redirection, you can redirect the page to your XMPP client web frontend (or anything
making sense), which results in better user experience.
The expected dictionary is a mapping URL request types to destination URL.
The URL request types is the first path element after ``ap_path``, it can be:
``actor``
Actor information. It should link to the profile page of the corresponding XMPP entity
``item``
A specific publication. It should link to the rendered item
``followers``
Entities publicly subscribed to the actor. It is usually not used for redirection.
``following``
Entities that the actor is publicly subscribed to. It is usually not used for
redirection.
In the target URL you can uses template values in curly braces (something like
``{item}``). The values that you can use are:
``jid``
Full JID corresponding to the AP actor.
``jid_user``
Only the user part of the JID (what is before the ``@``)
``node``
Name of the pubsub node corresponding to the AP actor.
``item``
ID of the XMPP pubsub item.
You can use a slash ``/`` followed by a part of a node name to filter only on specific
nodes. For instance, if you want to redirect microblog items to a different URL that
event items, you can use ``item/urn:xmpp:microblog:0`` and ``item/urn:xmpp:events:0``,
the first redirection to match will be used. Note that anything containing the filter
value will match, thus you can also use ``item/microblog`` (but it then would match an
event which has ``microblog`` in is node name).
You can also use a dictionary as value instead of the target URL. This can be useful to
filter on an other value than the node name, if you want to make specific redirection
for a particular user for instance. If you use a dictionary, you must use a ``url`` key
with the target URL (which may have template values as usual), and you may use a
``filters`` key mapping to an other dictionary where each key is a filter on a specific
template value (in other words: key is the template value, like ``jid`` or ``node``, and
value is the filter to match).
*examples*
Redirect actor page to ``/profile/<JID user part>``, blogs items to ``/blog/<full
JID>/<node name>/<item id>`` and events items to ``/event/<full JID>/<node name>/<item
id>`` ::
html_redirect_dict = {
"actor": "/profile/{jid_user}",
"item/microblog": "/blog/{jid}/{node}/{item}",
"item/events": "/event/{jid}/{node}/{item}"
}
Redirect items of user ``louise@example.org`` to ``/b/<item>``::
html_redirect_dict = {
"item": {
"url": "/b/{item}",
filters: {
"jid": "louise@example.org"
}
}
}
.. _ap-actor-from-xmpp: .. _ap-actor-from-xmpp:

View file

@ -26,6 +26,7 @@ if not wokkel.installed:
if TYPE_CHECKING: if TYPE_CHECKING:
from libervia.backend.core.main import LiberviaBackend from libervia.backend.core.main import LiberviaBackend
from libervia.backend.memory.sqla import Storage from libervia.backend.memory.sqla import Storage
from libervia.backend.tools.config import Settings
class Global: class Global:
@ -47,5 +48,9 @@ class Global:
def storage(self) -> "Storage": def storage(self) -> "Storage":
return self.host.memory.storage return self.host.memory.storage
@property
def config(self) -> "Settings":
return self.host.memory.config
G = Global() G = Global()

View file

@ -30,9 +30,9 @@ from txdbus.interface import DBusInterface, Method, Signal
log = getLogger(__name__) log = getLogger(__name__)
# Interface prefix # Interface prefix
const_INT_PREFIX = config.config_get( conf = config.parse_main_conf()
config.parse_main_conf(), "", "bridge_dbus_int_prefix", "org.libervia.Libervia" bridge_conf = conf.common.get_bridge_conf(config.DBusBridgeSettings)
) const_INT_PREFIX = bridge_conf.int_prefix
const_ERROR_PREFIX = const_INT_PREFIX + ".error" const_ERROR_PREFIX = const_INT_PREFIX + ".error"
const_OBJ_PATH = "/org/libervia/Libervia/bridge" const_OBJ_PATH = "/org/libervia/Libervia/bridge"
const_CORE_SUFFIX = ".core" const_CORE_SUFFIX = ".core"

View file

@ -32,9 +32,9 @@ log = getLogger(__name__)
# Interface prefix # Interface prefix
const_INT_PREFIX = config.config_get( conf = config.parse_main_conf()
config.parse_main_conf(), "", "bridge_dbus_int_prefix", "org.libervia.Libervia" bridge_conf = conf.common.get_bridge_conf(config.DBusBridgeSettings)
) const_INT_PREFIX = bridge_conf.int_prefix
const_ERROR_PREFIX = const_INT_PREFIX + ".error" const_ERROR_PREFIX = const_INT_PREFIX + ".error"
const_OBJ_PATH = "/org/libervia/Libervia/bridge" const_OBJ_PATH = "/org/libervia/Libervia/bridge"
const_CORE_SUFFIX = ".core" const_CORE_SUFFIX = ".core"

View file

@ -108,17 +108,16 @@ class bridge(object):
log.info("Init Perspective Broker...") log.info("Init Perspective Broker...")
self.root = PBRoot() self.root = PBRoot()
conf = config.parse_main_conf() conf = config.parse_main_conf()
get_conf = partial(config.get_conf, conf, "bridge_pb", "") bridge_conf = conf.common.get_bridge_conf(config.PbBridgePbSettings)
conn_type = get_conf("connection_type", "unix_socket") conn_type = bridge_conf.connection_type
if conn_type == "unix_socket": if conn_type == "unix_socket":
local_dir = Path(config.config_get(conf, "", "local_dir")).resolve() socket_path = bridge_conf.socket_dir / "bridge_pb"
socket_path = local_dir / "bridge_pb"
log.info(f"using UNIX Socket at {socket_path}") log.info(f"using UNIX Socket at {socket_path}")
reactor.listenUNIX( reactor.listenUNIX(
str(socket_path), pb.PBServerFactory(self.root), mode=0o600 str(socket_path), pb.PBServerFactory(self.root), mode=0o600
) )
elif conn_type == "socket": elif conn_type == "socket":
port = int(get_conf("port", 8789)) port = bridge_conf.port
log.info(f"using TCP Socket at port {port}") log.info(f"using TCP Socket at port {port}")
reactor.listenTCP(port, pb.PBServerFactory(self.root)) reactor.listenTCP(port, pb.PBServerFactory(self.root))
else: else:

View file

@ -137,16 +137,13 @@ class bridge(object):
def bridge_connect(self, callback, errback): def bridge_connect(self, callback, errback):
factory = pb.PBClientFactory() factory = pb.PBClientFactory()
conf = config.parse_main_conf() conf = config.parse_main_conf()
get_conf = partial(config.get_conf, conf, "bridge_pb", "") bridge_conf = conf.common.get_bridge_conf(config.PbBridgePbSettings)
conn_type = get_conf("connection_type", "unix_socket") conn_type = bridge_conf.connection_type
if conn_type == "unix_socket": if conn_type == "unix_socket":
local_dir = Path(config.config_get(conf, "", "local_dir")).resolve() socket_path = bridge_conf.socket_dir / "bridge_pb"
socket_path = local_dir / "bridge_pb"
reactor.connectUNIX(str(socket_path), factory) reactor.connectUNIX(str(socket_path), factory)
elif conn_type == "socket": elif conn_type == "socket":
host = get_conf("host", "localhost") reactor.connectTCP(bridge_conf.host, bridge_conf.port, factory)
port = int(get_conf("port", 8789))
reactor.connectTCP(host, port, factory)
else: else:
raise ValueError(f"Unknown pb connection type: {conn_type!r}") raise ValueError(f"Unknown pb connection type: {conn_type!r}")
d = factory.getRootObject() d = factory.getRootObject()

View file

@ -30,9 +30,9 @@ from txdbus.interface import DBusInterface, Method, Signal
log = getLogger(__name__) log = getLogger(__name__)
# Interface prefix # Interface prefix
const_INT_PREFIX = config.config_get( conf = config.parse_main_conf()
config.parse_main_conf(), "", "bridge_dbus_int_prefix", "org.libervia.Libervia" bridge_conf = conf.common.get_bridge_conf(config.DBusBridgeSettings)
) const_INT_PREFIX = bridge_conf.int_prefix
const_ERROR_PREFIX = const_INT_PREFIX + ".error" const_ERROR_PREFIX = const_INT_PREFIX + ".error"
const_OBJ_PATH = "/org/libervia/Libervia/bridge" const_OBJ_PATH = "/org/libervia/Libervia/bridge"
const_CORE_SUFFIX = ".core" const_CORE_SUFFIX = ".core"

View file

@ -108,17 +108,16 @@ class bridge(object):
log.info("Init Perspective Broker...") log.info("Init Perspective Broker...")
self.root = PBRoot() self.root = PBRoot()
conf = config.parse_main_conf() conf = config.parse_main_conf()
get_conf = partial(config.get_conf, conf, "bridge_pb", "") bridge_conf = conf.common.get_bridge_conf(config.PbBridgePbSettings)
conn_type = get_conf("connection_type", "unix_socket") conn_type = bridge_conf.connection_type
if conn_type == "unix_socket": if conn_type == "unix_socket":
local_dir = Path(config.config_get(conf, "", "local_dir")).resolve() socket_path = bridge_conf.socket_dir / "bridge_pb"
socket_path = local_dir / "bridge_pb"
log.info(f"using UNIX Socket at {socket_path}") log.info(f"using UNIX Socket at {socket_path}")
reactor.listenUNIX( reactor.listenUNIX(
str(socket_path), pb.PBServerFactory(self.root), mode=0o600 str(socket_path), pb.PBServerFactory(self.root), mode=0o600
) )
elif conn_type == "socket": elif conn_type == "socket":
port = int(get_conf("port", 8789)) port = bridge_conf.port
log.info(f"using TCP Socket at port {port}") log.info(f"using TCP Socket at port {port}")
reactor.listenTCP(port, pb.PBServerFactory(self.root)) reactor.listenTCP(port, pb.PBServerFactory(self.root))
else: else:

View file

@ -16,12 +16,8 @@
# You should have received a copy of the GNU Affero General Public License # You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
try: import os
from xdg import BaseDirectory from pathlib import Path
from os.path import expanduser, realpath
except ImportError:
BaseDirectory = None
from os.path import dirname, join
import tempfile import tempfile
from typing import Final from typing import Final
@ -29,16 +25,22 @@ from libervia import backend
try: try:
from getpass import getuser from getpass import getuser
except ImportError:
logged_user = "unkonwn"
else:
logged_user = getuser() logged_user = getuser()
except Exception as e:
logged_user = "unknown"
bool_type = bool bool_type = bool
xdg_config_home = Path(os.environ.get("XDG_CONFIG_HOME") or Path.home() / ".config")
xdg_config_dirs = [
d for d in os.environ.get("XDG_CONFIG_DIRS", "/etc/xdg").split(":") if d
]
xdg_data_home = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share")
class Const(object):
class Const:
## Application ## ## Application ##
APP_NAME = "Libervia" APP_NAME = "Libervia"
@ -50,6 +52,36 @@ class Const(object):
APP_RELEASE_NAME = "La Ruche" APP_RELEASE_NAME = "La Ruche"
APP_URL = "https://libervia.org" APP_URL = "https://libervia.org"
## Configuration ##
CONFIG_PATHS = [
# /etc/_ is used for system-related settings (e.g. when media_dir is set by the
# distribution and has not reason to change, or in a Docker image)
Path("/etc/_"),
Path("/etc/"),
Path.home(),
Path.home() / ".",
xdg_config_home / APP_NAME,
# We use backend root as last config path, this is useful for per instance
# configurations.
Path(backend.__file__).parent.parent,
]
for d in xdg_config_dirs:
CONFIG_PATHS.append(Path(d) / APP_NAME)
# on recent versions of Flatpak, FLATPAK_ID is set at run time
# it seems that this is not the case on older versions,
# but FLATPAK_SANDBOX_DIR seems set then
if os.getenv("FLATPAK_ID") or os.getenv("FLATPAK_SANDBOX_DIR"):
# for Flatpak, the conf can't be set in /etc or $HOME, so we have
# to add /app
CONFIG_PATHS.append(Path("/app/"))
CONFIG_FILES: list[Path] = []
for p in CONFIG_PATHS:
CONFIG_FILES.append(p / f"{APP_NAME_FILE}.toml")
TEMP_DIR = Path(tempfile.gettempdir(), f"{APP_NAME_FILE}_{logged_user}")
## Runtime ## ## Runtime ##
PLUGIN_EXT = "py" PLUGIN_EXT = "py"
HISTORY_SKIP = "skip" HISTORY_SKIP = "skip"
@ -222,6 +254,7 @@ class Const(object):
# to make the header more readable # to make the header more readable
PI_NAME = "name" PI_NAME = "name"
PI_IMPORT_NAME = "import_name" PI_IMPORT_NAME = "import_name"
PI_CONFIG_KEY = "config_key"
PI_MAIN = "main" PI_MAIN = "main"
PI_HANDLER = "handler" PI_HANDLER = "handler"
PI_TYPE = "type" #  FIXME: should be types, and should handle single unicode type or tuple of types (e.g. "blog" and "import") PI_TYPE = "type" #  FIXME: should be types, and should handle single unicode type or tuple of types (e.g. "blog" and "import")
@ -319,7 +352,7 @@ class Const(object):
LOG_BACKEND_CUSTOM = "custom" LOG_BACKEND_CUSTOM = "custom"
LOG_BASE_LOGGER = "root" LOG_BASE_LOGGER = "root"
LOG_TWISTED_LOGGER = "twisted" LOG_TWISTED_LOGGER = "twisted"
LOG_OPT_SECTION = "DEFAULT" # section of sat.conf where log options should be LOG_OPT_SECTION = "" # section of libervia.toml where log options should be
LOG_OPT_PREFIX = "log_" LOG_OPT_PREFIX = "log_"
# (option_name, default_value) tuples # (option_name, default_value) tuples
LOG_OPT_COLORS = ( LOG_OPT_COLORS = (
@ -435,7 +468,6 @@ class Const(object):
## Misc ## ## Misc ##
SAVEFILE_DATABASE = APP_NAME_FILE + ".db" SAVEFILE_DATABASE = APP_NAME_FILE + ".db"
IQ_SET = '/iq[@type="set"]' IQ_SET = '/iq[@type="set"]'
ENV_PREFIX = "SAT_" # Prefix used for environment variables
IGNORE = "ignore" IGNORE = "ignore"
NO_LIMIT = -1 # used in bridge when a integer value is expected NO_LIMIT = -1 # used in bridge when a integer value is expected
DEFAULT_MAX_AGE = 1209600 # default max age of cached files, in seconds DEFAULT_MAX_AGE = 1209600 # default max age of cached files, in seconds
@ -472,90 +504,38 @@ class Const(object):
## Configuration ## ## Configuration ##
if ( # if "org.libervia.cagou" in BaseDirectory.__file__:
BaseDirectory # # FIXME: hack to make config read from the right location on Android
): # skipped when xdg module is not available (should not happen in backend) # # TODO: fix it in a more proper way
if "org.libervia.cagou" in BaseDirectory.__file__:
# FIXME: hack to make config read from the right location on Android
# TODO: fix it in a more proper way
# we need to use Android API to get downloads directory # # we need to use Android API to get downloads directory
import os.path # import os.path
from jnius import autoclass # from jnius import autoclass
# we don't want the very verbose jnius log when we are in DEBUG level # # we don't want the very verbose jnius log when we are in DEBUG level
import logging # import logging
logging.getLogger("jnius").setLevel(logging.WARNING) # logging.getLogger("jnius").setLevel(logging.WARNING)
logging.getLogger("jnius.reflect").setLevel(logging.WARNING) # logging.getLogger("jnius.reflect").setLevel(logging.WARNING)
Environment = autoclass("android.os.Environment") # Environment = autoclass("android.os.Environment")
BaseDirectory = None # BaseDirectory = None
Const.DEFAULT_CONFIG = { # Const.DEFAULT_CONFIG = {
"local_dir": "/data/data/org.libervia.cagou/app", # "local_dir": "/data/data/org.libervia.cagou/app",
"media_dir": "/data/data/org.libervia.cagou/files/app/media", # "media_dir": "/data/data/org.libervia.cagou/files/app/media",
# FIXME: temporary location for downloads, need to call API properly # # FIXME: temporary location for downloads, need to call API properly
"downloads_dir": os.path.join( # "downloads_dir": os.path.join(
Environment.getExternalStoragePublicDirectory( # Environment.getExternalStoragePublicDirectory(
Environment.DIRECTORY_DOWNLOADS # Environment.DIRECTORY_DOWNLOADS
).getAbsolutePath(), # ).getAbsolutePath(),
Const.APP_NAME_FILE, # Const.APP_NAME_FILE,
), # ),
"pid_dir": join(tempfile.gettempdir(), Const.APP_NAME_FILE, logged_user), # "pid_dir": join(tempfile.gettempdir(), Const.APP_NAME_FILE, logged_user),
"log_dir": "%(local_dir)s", # "log_dir": "%(local_dir)s",
} # }
Const.CONFIG_FILES = [ # Const.CONFIG_FILES = [
"/data/data/org.libervia.cagou/files/app/android/" # "/data/data/org.libervia.cagou/files/app/android/"
+ Const.APP_NAME_FILE # + Const.APP_NAME_FILE
+ ".conf" # + ".toml"
] # ]
else:
import os
# we use parent of "sat" module dir as last config path, this is useful for
# per instance configurations (e.g. a dev instance and a main instance)
root_dir = dirname(dirname(backend.__file__)) + "/"
Const.CONFIG_PATHS = (
# /etc/_sat.conf is used for system-related settings (e.g. when media_dir
# is set by the distribution and has not reason to change, or in a Docker
# image)
["/etc/_", "/etc/", "~/", "~/."]
+ [
"{}/".format(path)
for path in list(BaseDirectory.load_config_paths(Const.APP_NAME_FILE))
]
# this is to handle legacy sat.conf
+ [
"{}/".format(path)
for path in list(BaseDirectory.load_config_paths("sat"))
]
+ [root_dir]
)
# on recent versions of Flatpak, FLATPAK_ID is set at run time
# it seems that this is not the case on older versions,
# but FLATPAK_SANDBOX_DIR seems set then
if os.getenv("FLATPAK_ID") or os.getenv("FLATPAK_SANDBOX_DIR"):
# for Flatpak, the conf can't be set in /etc or $HOME, so we have
# to add /app
Const.CONFIG_PATHS.append("/app/")
## Configuration ##
Const.DEFAULT_CONFIG = {
"media_dir": "/usr/share/" + Const.APP_NAME_FILE + "/media",
"local_dir": BaseDirectory.save_data_path(Const.APP_NAME_FILE),
"downloads_dir": "~/Downloads/" + Const.APP_NAME_FILE,
"pid_dir": join(tempfile.gettempdir(), Const.APP_NAME_FILE, logged_user),
"log_dir": "%(local_dir)s",
}
# List of the configuration filenames sorted by ascending priority
Const.CONFIG_FILES = [
realpath(expanduser(path) + Const.APP_NAME_FILE + ".conf")
for path in Const.CONFIG_PATHS
] + [
# legacy sat.conf
realpath(expanduser(path) + "sat.conf")
for path in Const.CONFIG_PATHS
]

View file

@ -1,7 +1,7 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
# Libervia: an XMPP client # Libervia: an XMPP client
# Copyright (C) 2009-2021 Jérôme Poisson (goffi@goffi.org) # Copyright (C) 2009-2025 Jérôme Poisson (goffi@goffi.org)
# This program is free software: you can redistribute it and/or modify # This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by # it under the terms of the GNU Affero General Public License as published by
@ -16,16 +16,16 @@
# You should have received a copy of the GNU Affero General Public License # You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Script launching SàT backend""" """Script launching Libervia backend"""
import sys import sys
import os import os
import argparse import argparse
from pathlib import Path from pathlib import Path
from configparser import ConfigParser
from twisted.application import app from twisted.application import app
from twisted.python import usage from twisted.python import usage
from libervia.backend.core.constants import Const as C from libervia.backend.core.constants import Const as C
from libervia.backend.tools.config import parse_main_conf
class LiberviaLogger(app.AppLogger): class LiberviaLogger(app.AppLogger):
@ -46,6 +46,13 @@ class Launcher:
def NOT_RUNNING_MSG(self): def NOT_RUNNING_MSG(self):
return f"{self.APP_NAME} is *NOT* running" return f"{self.APP_NAME} is *NOT* running"
def __init__(self):
try:
self.config = parse_main_conf()
except Exception as e:
print(rf"/!\ Can't read main config! {e}")
sys.exit(1)
def cmd_no_subparser(self, args): def cmd_no_subparser(self, args):
"""Command launched by default""" """Command launched by default"""
args.extra_args = [] args.extra_args = []
@ -64,8 +71,7 @@ class Launcher:
import signal import signal
import time import time
config = self.get_config() pid_file = self.get_pid_file()
pid_file = self.get_pid_file(config)
if not pid_file.is_file(): if not pid_file.is_file():
print(self.NOT_RUNNING_MSG) print(self.NOT_RUNNING_MSG)
sys.exit(0) sys.exit(0)
@ -110,8 +116,7 @@ class Launcher:
sys.exit(0) sys.exit(0)
def cmd_status(self, args): def cmd_status(self, args):
config = self.get_config() pid_file = self.get_pid_file()
pid_file = self.get_pid_file(config)
if pid_file.is_file(): if pid_file.is_file():
import errno import errno
@ -186,19 +191,8 @@ class Launcher:
return parser.parse_args() return parser.parse_args()
def get_config(self): def get_pid_file(self) -> Path:
config = ConfigParser(defaults=C.DEFAULT_CONFIG) return self.config.common.pid_dir / f"{self.APP_NAME_FILE}.pid"
try:
config.read(C.CONFIG_FILES)
except Exception as e:
print(rf"/!\ Can't read main config! {e}")
sys.exit(1)
return config
def get_pid_file(self, config):
pid_dir = Path(config.get("DEFAULT", "pid_dir")).expanduser()
pid_dir.mkdir(parents=True, exist_ok=True)
return pid_dir / f"{self.APP_NAME_FILE}.pid"
def wait_for_service( def wait_for_service(
self, service_host: str, service_port: int, timeout: int, service_name: str self, service_host: str, service_port: int, timeout: int, service_name: str
@ -252,12 +246,9 @@ class Launcher:
app_runner.loggerFactory = LiberviaLogger app_runner.loggerFactory = LiberviaLogger
server_options = ServerOptions() server_options = ServerOptions()
config = self.get_config()
# wait for a service (e.g. XMPP server) # wait for a service (e.g. XMPP server)
wait_for_service_value = config.get( wait_for_service_value = self.config.common.init_wait_for_service
"DEFAULT", "init_wait_for_service", fallback=None
)
if wait_for_service_value is not None: if wait_for_service_value is not None:
try: try:
@ -291,8 +282,8 @@ class Launcher:
else: else:
self.wait_for_service(host, port, timeout, service_name) self.wait_for_service(host, port, timeout, service_name)
pid_file = self.get_pid_file(config) pid_file = self.get_pid_file()
log_dir = Path(config.get("DEFAULT", "log_dir")).expanduser() log_dir = self.config.common.log_dir
log_file = log_dir / f"{self.APP_NAME_FILE}.log" log_file = log_dir / f"{self.APP_NAME_FILE}.log"
server_opts = [ server_opts = [
"--no_save", "--no_save",
@ -322,7 +313,7 @@ class Launcher:
if runner._exitSignal is not None: if runner._exitSignal is not None:
app._exitWithSignal(runner._exitSignal) app._exitWithSignal(runner._exitSignal)
try: try:
sys.exit(app._exitCode) sys.exit(app._exit_code)
except AttributeError: except AttributeError:
pass pass

View file

@ -20,8 +20,10 @@
"""High level logging functions""" """High level logging functions"""
# XXX: this module use standard logging module when possible, but as SàT can work in different cases where logging is not the best choice (twisted, pyjamas, etc), it is necessary to have a dedicated module. Additional feature like environment variables and colors are also managed. # XXX: this module use standard logging module when possible, but as SàT can work in different cases where logging is not the best choice (twisted, pyjamas, etc), it is necessary to have a dedicated module. Additional feature like environment variables and colors are also managed.
from libervia.backend import G
from libervia.backend.core.constants import Const as C from libervia.backend.core.constants import Const as C
from libervia.backend.core import log from libervia.backend.core import log
from libervia.backend.tools.config import parse_main_conf
class TwistedLogger(log.Logger): class TwistedLogger(log.Logger):
@ -391,26 +393,25 @@ def libervia_configure(backend=C.LOG_BACKEND_STANDARD, const=None, backend_data=
- C.LOG_BACKEND_BASIC: print based backend - C.LOG_BACKEND_BASIC: print based backend
- C.LOG_BACKEND_TWISTED: Twisted logging backend - C.LOG_BACKEND_TWISTED: Twisted logging backend
- C.LOG_BACKEND_STANDARD: standard logging backend - C.LOG_BACKEND_STANDARD: standard logging backend
@param const: Const class to use instead of sat.core.constants.Const (mainly used to change default values) @param const: Const class to use instead of sat.core.constants.Const (mainly used to
change default values)
""" """
if const is not None: if const is not None:
global C global C
C = const C = const
log.C = const log.C = const
from libervia.backend.tools import config
import os
log_conf = {} log_conf = {}
sat_conf = config.parse_main_conf() # FIXME: Q&D adaptation following configuration refactoring. All this code should be
# removed in close future to use standard logging directly.
for opt_name, opt_default in C.LOG_OPTIONS(): for opt_name, opt_default in C.LOG_OPTIONS():
config = parse_main_conf()
try: try:
log_conf[opt_name] = os.environ[ value = getattr(config.common, f"{C.LOG_OPT_PREFIX}{opt_name}")
"".join((C.ENV_PREFIX, C.LOG_OPT_PREFIX.upper(), opt_name.upper())) log_conf[opt_name] = value if value else opt_default
] except AttributeError:
except KeyError: log_conf[opt_name] = opt_default
log_conf[opt_name] = config.config_get( log.debug(f"Using default log option {opt_name!r}: {opt_default!r}")
sat_conf, C.LOG_OPT_SECTION, C.LOG_OPT_PREFIX + opt_name, opt_default
)
_parse_options(log_conf) _parse_options(log_conf)
configure(backend, backend_data=backend_data, **log_conf) configure(backend, backend_data=backend_data, **log_conf)

View file

@ -23,14 +23,17 @@ import hashlib
import copy import copy
from pathlib import Path from pathlib import Path
import shutil import shutil
from typing import Optional, List, Tuple, Dict, cast from typing import Optional, List, Tuple, Dict
from pydantic import ValidationError
from libervia import backend from libervia import backend
from libervia.backend.core.i18n import _, D_, language_switch from libervia.backend.core.i18n import _, D_, language_switch
from libervia.backend.core import patches from libervia.backend.core import patches
from libervia.backend.tools.config import DBusBridgeSettings
patches.apply() patches.apply()
from twisted.application import service from twisted.application import app, service
from twisted.internet import defer from twisted.internet import defer
from twisted.words.protocols.jabber import jid from twisted.words.protocols.jabber import jid
from twisted.internet import reactor from twisted.internet import reactor
@ -85,14 +88,15 @@ class LiberviaBackend(service.Service):
} }
self.memory = memory.Memory(self) self.memory = memory.Memory(self)
config = self.memory.config
# trigger are used to change Libervia behaviour # trigger are used to change Libervia behaviour
self.trigger = trigger.TriggerManager() self.trigger = trigger.TriggerManager()
bridge_name = os.getenv("LIBERVIA_BRIDGE_NAME") or self.memory.config_get( bridges = config.common.bridges
"", "bridge", "dbus" # FIXME: We only handle first bridge for now.
) bridge = next(iter(bridges))
bridge_name: str = bridge.type
bridge_module = dynamic_import.bridge(bridge_name) bridge_module = dynamic_import.bridge(bridge_name)
if bridge_module is None: if bridge_module is None:
log.error(f"Can't find bridge module of name {bridge_name}") log.error(f"Can't find bridge module of name {bridge_name}")
@ -104,13 +108,8 @@ class LiberviaBackend(service.Service):
log.exception("bridge can't be initialised, can't start Libervia Backend") log.exception("bridge can't be initialised, can't start Libervia Backend")
sys.exit(1) sys.exit(1)
# If set, a temporary dir in cache will be used to share files between backend and # FIXME: Seems not used anymore, maybe delete?
# frontends, useful when they are separated (e.g. when using containers). If if config.common.use_local_shared_tmp:
# unset, a temporary dir will be automatically created in os-relevant location.
use_local_shared_tmp: bool = C.bool(
self.memory.config_get("", "use_local_shared_tmp", C.BOOL_FALSE)
)
if use_local_shared_tmp:
self.local_shared_path = self.get_local_path( self.local_shared_path = self.get_local_path(
None, C.CACHE_DIR, C.LOCAL_SHARED_DIR None, C.CACHE_DIR, C.LOCAL_SHARED_DIR
) )
@ -280,31 +279,24 @@ class LiberviaBackend(service.Service):
self._import_plugins() self._import_plugins()
ui_contact_list.ContactList(self) ui_contact_list.ContactList(self)
ui_profile_manager.ProfileManager(self) ui_profile_manager.ProfileManager(self)
except Exception as e: except exceptions.ConfigError as e:
log.error(f"Could not initialize backend: {e}") log.error(f"Configuration error, please check and fix your configuration: {e}")
sys.exit(1) self.stop(exit_code=C.EXIT_BAD_ARG)
except Exception:
log.exception(f"Could not initialize backend")
self.stop(exit_code=C.EXIT_ERROR)
self._add_base_menus() self._add_base_menus()
self.init_pre_script.callback(None) self.init_pre_script.callback(None)
init_script = self.memory.config_get(None, "init_script_path") if (init_script := G.config.common.init_script) is not None:
if init_script is not None: log.info(f"Running init script {init_script!r}.")
init_script = cast(str, init_script) try:
script_path = Path(init_script) await async_process.run(str(init_script), verbose=True)
if not script_path.exists(): except RuntimeError as e:
log.error(f"Init script doesn't exist: {init_script}") log.error(f"Init script failed: {e}")
sys.exit(C.EXIT_BAD_ARG) self.stopService()
elif not script_path.is_file(): sys.exit(C.EXIT_ERROR)
log.error(f"Init script is not a file: {init_script}")
sys.exit(C.EXIT_BAD_ARG)
else:
log.info(f"Running init script {init_script!r}.")
try:
await async_process.run(str(init_script), verbose=True)
except RuntimeError as e:
log.error(f"Init script failed: {e}")
self.stopService()
sys.exit(C.EXIT_ERROR)
self.initialised.callback(None) self.initialised.callback(None)
log.info(_("Backend is ready")) log.info(_("Backend is ready"))
@ -444,6 +436,10 @@ class LiberviaBackend(service.Service):
) )
continue continue
plugins_to_import[import_name] = (plugin_path, mod, plugin_info) plugins_to_import[import_name] = (plugin_path, mod, plugin_info)
# We reparse configuration to handle plugin settings before initialising them.
self.memory.reparse_configuration()
while True: while True:
try: try:
self._import_plugins_from_dict(plugins_to_import) self._import_plugins_from_dict(plugins_to_import)
@ -701,8 +697,10 @@ class LiberviaBackend(service.Service):
log.debug(_("running app")) log.debug(_("running app"))
reactor.run() reactor.run()
def stop(self): def stop(self, exit_code: int|None = None):
log.debug(_("stopping app")) log.debug(_("stopping app"))
if exit_code:
app._exit_code = exit_code
reactor.stop() reactor.stop()
## Misc methods ## ## Misc methods ##
@ -751,14 +749,22 @@ class LiberviaBackend(service.Service):
raise exceptions.ProfileKeyUnknown raise exceptions.ProfileKeyUnknown
return [self.profiles[profile]] return [self.profiles[profile]]
def _get_config(self, section, name): def _get_config(self, section, name) -> str:
"""Get the main configuration option """Get the main configuration option
@param section: section of the config file (None or '' for DEFAULT) @param section: section of the config
@param name: name of the option @param name: name of the option
@return: unicode representation of the option @return: Serialised otpion value
""" """
return str(self.memory.config_get(section, name, "")) # FIXME: This is a quick adaptation of old code.
if not section or section == "DEFAULT":
section = "common"
try:
value = getattr(getattr(G.config, section), name)
except:
raise exceptions.InternalError(f"Can't retrieve value {section=} {name=}.")
else:
return data_format.serialise(value)
def log_errback(self, failure_, msg=_("Unexpected error: {failure_}")): def log_errback(self, failure_, msg=_("Unexpected error: {failure_}")):
"""Generic errback logging """Generic errback logging
@ -977,10 +983,8 @@ class LiberviaBackend(service.Service):
@param component: if True, path will be prefixed with C.COMPONENTS_DIR @param component: if True, path will be prefixed with C.COMPONENTS_DIR
@return: path @return: path
""" """
local_dir = self.memory.config_get("", "local_dir") local_dir = G.config.common.local_dir
if not local_dir: path_elts: list[Path | str] = [local_dir]
raise exceptions.InternalError("local_dir must be set")
path_elts = [local_dir]
if component: if component:
path_elts.append(C.COMPONENTS_DIR) path_elts.append(C.COMPONENTS_DIR)
path_elts.append(regex.path_escape(dir_name)) path_elts.append(regex.path_escape(dir_name))

View file

@ -24,7 +24,7 @@ import mimetypes
from pathlib import Path from pathlib import Path
import sys import sys
import time import time
from typing import Callable, Dict, Tuple, Optional, cast from typing import Callable, Dict, Optional, Tuple, cast
from urllib.parse import unquote, urlparse from urllib.parse import unquote, urlparse
import uuid import uuid
@ -42,6 +42,7 @@ from wokkel import component
from wokkel import delay from wokkel import delay
from zope.interface import implementer from zope.interface import implementer
from libervia.backend import G
from libervia.backend.core import exceptions from libervia.backend.core import exceptions
from libervia.backend.core import core_types from libervia.backend.core import core_types
from libervia.backend.core.constants import Const as C from libervia.backend.core.constants import Const as C
@ -964,7 +965,7 @@ class SatXMPPClient(SatXMPPEntity, core_types.SatXMPPClient, wokkel_client.XMPPC
# for now we consider Android devices to be always phones # for now we consider Android devices to be always phones
self.identities = [disco.DiscoIdentity("client", "phone", C.APP_NAME)] self.identities = [disco.DiscoIdentity("client", "phone", C.APP_NAME)]
hosts_map = host_app.memory.config_get(None, "hosts_dict", {}) hosts_map = G.config.common.hosts_dict
if host is None and user_jid.host in hosts_map: if host is None and user_jid.host in hosts_map:
host_data = hosts_map[user_jid.host] host_data = hosts_map[user_jid.host]
if isinstance(host_data, str): if isinstance(host_data, str):

View file

@ -24,6 +24,7 @@ from typing import Any
from pydantic import BaseModel, ValidationError from pydantic import BaseModel, ValidationError
from libervia.backend import G
from libervia.backend.core import exceptions from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C from libervia.backend.core.constants import Const as C
from libervia.backend.core.i18n import _ from libervia.backend.core.i18n import _
@ -58,7 +59,7 @@ class Cache:
if None, the cache will be common for all profiles if None, the cache will be common for all profiles
""" """
self.profile = profile self.profile = profile
path_elts = [host.memory.config_get("", "local_dir"), C.CACHE_DIR] path_elts = [G.config.common.local_dir, C.CACHE_DIR]
if profile: if profile:
path_elts.extend(["profiles", regex.path_escape(profile)]) path_elts.extend(["profiles", regex.path_escape(profile)])
else: else:

View file

@ -261,12 +261,21 @@ class Memory:
self.subscriptions = {} self.subscriptions = {}
self.auth_sessions = PasswordSessions() # remember the authenticated profiles self.auth_sessions = PasswordSessions() # remember the authenticated profiles
self.disco = Discovery(host) self.disco = Discovery(host)
self.config = tools_config.parse_main_conf(log_filenames=True) self.config: tools_config.Settings = tools_config.parse_main_conf(
self._cache_path = Path(self.config_get("", "local_dir"), C.CACHE_DIR) log_filenames=True
self.admins = self.config_get("", "admins_list", []) )
self._cache_path = self.config.common.local_dir / C.CACHE_DIR
self.admins = self.config.common.admins_list
self.admin_jids = set() self.admin_jids = set()
self._file_path_lock = defer.DeferredLock() self._file_path_lock = defer.DeferredLock()
def reparse_configuration(self) -> None:
"""Reparse configuration.
This is needed when dynamic settings has been added (after importing plugins).
"""
self.config = tools_config.parse_main_conf(ignore_cache=True)
async def initialise(self): async def initialise(self):
self.storage = Storage() self.storage = Storage()
await self.storage.initialise() await self.storage.initialise()
@ -296,18 +305,6 @@ class Memory:
## Configuration ## ## Configuration ##
def config_get(
self, section: str | None, name: str, default: Any = None
) -> str | list | dict:
"""Get the main configuration option
@param section: section of the config file (None or '' for DEFAULT)
@param name: name of the option
@param default: value to use if not found
@return: str, list or dict
"""
return tools_config.config_get(self.config, section, name, default)
def load_xml(self, filename): def load_xml(self, filename):
"""Load parameters template from xml file """Load parameters template from xml file

View file

@ -16,7 +16,6 @@
# You should have received a copy of the GNU Affero General Public License # You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
from pathlib import Path
from urllib.parse import quote from urllib.parse import quote
from libervia.backend.core.constants import Const as C from libervia.backend.core.constants import Const as C
from libervia.backend.tools import config from libervia.backend.tools import config
@ -30,8 +29,7 @@ def get_db_config() -> dict:
- path: path to the sqlite DB - path: path to the sqlite DB
""" """
main_conf = config.parse_main_conf() main_conf = config.parse_main_conf()
local_dir = Path(config.config_get(main_conf, "", "local_dir")) database_path = main_conf.common.local_dir / C.SAVEFILE_DATABASE
database_path = (local_dir / C.SAVEFILE_DATABASE).expanduser()
url = f"sqlite+aiosqlite:///{quote(str(database_path))}?timeout=30" url = f"sqlite+aiosqlite:///{quote(str(database_path))}?timeout=30"
return { return {
"type": "sqlite", "type": "sqlite",

View file

@ -54,14 +54,14 @@ override:
PEERTUBE_WEBSERVER_HOSTNAME: !libervia_conf ["application peertube", "hostname"] PEERTUBE_WEBSERVER_HOSTNAME: !libervia_conf ["application peertube", "hostname"]
PEERTUBE_TRUST_PROXY: '["127.0.0.1", "loopback", "172.18.0.0/16"]' PEERTUBE_TRUST_PROXY: '["127.0.0.1", "loopback", "172.18.0.0/16"]'
PEERTUBE_SECRET: !libervia_generate_pwd {name: secret} PEERTUBE_SECRET: !libervia_generate_pwd {name: secret}
PEERTUBE_SMTP_HOSTNAME: !libervia_conf ["", "email_server"] PEERTUBE_SMTP_HOSTNAME: !libervia_conf ["email", "server"]
PEERTUBE_SMTP_USERNAME: !libervia_conf ["", "email_username"] PEERTUBE_SMTP_USERNAME: !libervia_conf ["email", "username"]
PEERTUBE_SMTP_PASSWORD: !libervia_conf ["", "email_password"] PEERTUBE_SMTP_PASSWORD: !libervia_conf ["email", "password"]
PEERTUBE_SMTP_DISABLE_STARTTLS: !libervia_conf ["", "email_starttls", "false", "not"] PEERTUBE_SMTP_DISABLE_STARTTLS: !libervia_conf ["email", "starttls", "false", "not"]
PEERTUBE_SMTP_PORT: !libervia_conf ["", "email_port"] PEERTUBE_SMTP_PORT: !libervia_conf ["email", "port"]
PEERTUBE_SMTP_FROM: !libervia_conf ["application peertube", "email_from", "peertube@example.com"] PEERTUBE_SMTP_FROM: !libervia_conf ["application peertube", "email_from", "peertube@example.com"]
PEERTUBE_SMTP_TLS: "false" PEERTUBE_SMTP_TLS: "false"
PEERTUBE_ADMIN_EMAIL: !libervia_conf ["", "email_admins_list", "", "first"] PEERTUBE_ADMIN_EMAIL: !libervia_conf ["common", "email_admins_list", "", "first"]
PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PUBLIC: "public-read" PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PUBLIC: "public-read"
PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PRIVATE: "private" PEERTUBE_OBJECT_STORAGE_UPLOAD_ACL_PRIVATE: "private"

View file

@ -21,7 +21,7 @@ override:
WEBLATE_DEFAULT_FROM_EMAIL: !libervia_conf ["application weblate", "email_from", "weblate@example.com"] WEBLATE_DEFAULT_FROM_EMAIL: !libervia_conf ["application weblate", "email_from", "weblate@example.com"]
WEBLATE_SITE_DOMAIN: !libervia_conf ["application weblate", "public_url"] WEBLATE_SITE_DOMAIN: !libervia_conf ["application weblate", "public_url"]
WEBLATE_ADMIN_PASSWORD: !libervia_generate_pwd {name: admin} WEBLATE_ADMIN_PASSWORD: !libervia_generate_pwd {name: admin}
WEBLATE_ADMIN_EMAIL: !libervia_conf ["", "email_admins_list", "", "first"] WEBLATE_ADMIN_EMAIL: !libervia_conf ["common", "email_admins_list", "", "first"]
WEBLATE_ENABLE_HTTPS: !libervia_conf ["application weblate", "enable_https", "1"] WEBLATE_ENABLE_HTTPS: !libervia_conf ["application weblate", "enable_https", "1"]
volumes: volumes:
- ./settings-override.py:/app/data/settings-override.py:ro - ./settings-override.py:/app/data/settings-override.py:ro

View file

@ -53,6 +53,7 @@ from twisted.words.protocols.jabber import error, jid
from twisted.words.xish import domish from twisted.words.xish import domish
from wokkel import pubsub, rsm from wokkel import pubsub, rsm
from libervia.backend import G
from libervia.backend.core import exceptions from libervia.backend.core import exceptions
from libervia.backend.core.constants import Const as C from libervia.backend.core.constants import Const as C
from libervia.backend.core.core_types import SatXMPPEntity from libervia.backend.core.core_types import SatXMPPEntity
@ -66,14 +67,15 @@ from libervia.backend.tools.common import data_format, tls, uri
from libervia.backend.tools.common.async_utils import async_lru from libervia.backend.tools.common.async_utils import async_lru
from .ad_hoc import APAdHocService from .ad_hoc import APAdHocService
from .events import APEvents # We import PLUGIN_INFO as it's needed for the plugin, and we put it in .constants because
# it's necessary to create the ComponentSettings subclass.
from .constants import PLUGIN_INFO
from .constants import ( from .constants import (
ACTIVITY_OBJECT_MANDATORY, ACTIVITY_OBJECT_MANDATORY,
ACTIVITY_TARGET_MANDATORY, ACTIVITY_TARGET_MANDATORY,
ACTIVITY_TYPES, ACTIVITY_TYPES,
ACTIVITY_TYPES_LOWER, ACTIVITY_TYPES_LOWER,
COMMENTS_MAX_PARENTS, COMMENTS_MAX_PARENTS,
CONF_SECTION,
IMPORT_NAME, IMPORT_NAME,
LRU_MAX_SIZE, LRU_MAX_SIZE,
MEDIA_TYPE_AP, MEDIA_TYPE_AP,
@ -84,55 +86,23 @@ from .constants import (
TYPE_EVENT, TYPE_EVENT,
TYPE_FOLLOWERS, TYPE_FOLLOWERS,
TYPE_ITEM, TYPE_ITEM,
TYPE_JOIN,
TYPE_LEAVE,
TYPE_LIKE, TYPE_LIKE,
TYPE_MENTION, TYPE_MENTION,
TYPE_REACTION, TYPE_REACTION,
TYPE_TOMBSTONE, TYPE_TOMBSTONE,
TYPE_JOIN,
TYPE_LEAVE,
) )
from .events import APEvents
from .http_server import HTTPServer from .http_server import HTTPServer
from .pubsub_service import APPubsubService from .pubsub_service import APPubsubService
from .regex import RE_MENTION from .regex import RE_MENTION
from .settings import APGatewaySettings
log = getLogger(__name__) log = getLogger(__name__)
IMPORT_NAME = "ap-gateway" IMPORT_NAME = "ap-gateway"
PLUGIN_INFO = {
C.PI_NAME: "ActivityPub Gateway component",
C.PI_IMPORT_NAME: IMPORT_NAME,
C.PI_MODES: [C.PLUG_MODE_COMPONENT],
C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
C.PI_PROTOCOLS: [],
C.PI_DEPENDENCIES: [
"XEP-0050",
"XEP-0054",
"XEP-0060",
"XEP-0084",
"XEP-0106",
"XEP-0277",
"XEP-0292",
"XEP-0329",
"XEP-0372",
"XEP-0424",
"XEP-0465",
"XEP-0470",
"XEP-0447",
"XEP-0471",
"PUBSUB_CACHE",
"TEXT_SYNTAXES",
"IDENTITY",
],
C.PI_RECOMMENDATIONS: [],
C.PI_MAIN: "APGateway",
C.PI_HANDLER: C.BOOL_TRUE,
C.PI_DESCRIPTION: _(
"Gateway for bidirectional communication between XMPP and ActivityPub."
),
}
HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})" HEXA_ENC = r"(?P<hex>[0-9a-fA-f]{2})"
RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}") RE_PERIOD_ENC = re.compile(f"\\.{HEXA_ENC}")
RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}") RE_PERCENT_ENC = re.compile(f"%{HEXA_ENC}")
@ -181,6 +151,7 @@ class APGateway:
self.pubsub_service = APPubsubService(self) self.pubsub_service = APPubsubService(self)
self.ad_hoc = APAdHocService(self) self.ad_hoc = APAdHocService(self)
self.ap_events = APEvents(self) self.ap_events = APEvents(self)
self.html_redirect: HTMLRedirectDict = {}
host.trigger.add_with_check( host.trigger.add_with_check(
"message_received", self, self._message_received_trigger, priority=-1000 "message_received", self, self._message_received_trigger, priority=-1000
) )
@ -242,14 +213,19 @@ class APGateway:
# params # params
# URL and port # URL and port
self.public_url = self.host.memory.config_get( # Get component settings
CONF_SECTION, "public_url" ap_settings = G.config.get_settings(APGatewaySettings)
) or self.host.memory.config_get(CONF_SECTION, "xmpp_domain")
self.public_url = (
ap_settings.public_url
or ap_settings.xmpp_domain
or G.config.common.xmpp_domain
)
if self.public_url is None: if self.public_url is None:
log.error( log.error(
'"public_url" not set in configuration, this is mandatory to have' '"public_url" not set in configuration, this is mandatory to have'
"ActivityPub Gateway running. Please set this option it to public facing " "ActivityPub Gateway running. Please set this option it to public facing "
f"url in {CONF_SECTION!r} configuration section." f"url in {IMPORT_NAME!r} configuration section."
) )
return return
if parse.urlparse(self.public_url).scheme: if parse.urlparse(self.public_url).scheme:
@ -258,65 +234,34 @@ class APGateway:
'"public_url" configuration option. ActivityPub Gateway won\'t be run.' '"public_url" configuration option. ActivityPub Gateway won\'t be run.'
) )
return return
self.http_port = int(self.host.memory.config_get(CONF_SECTION, "http_port", 8123)) self.http_port = ap_settings.http_port
connection_type = self.host.memory.config_get( connection_type = ap_settings.http_connection_type
CONF_SECTION, "http_connection_type", "https"
)
if connection_type not in ("http", "https"): if connection_type not in ("http", "https"):
raise exceptions.ConfigError( raise exceptions.ConfigError(
'bad ap-gateay http_connection_type, you must use one of "http" or ' 'bad ap-gateay http_connection_type, you must use one of "http" or '
'"https"' '"https"'
) )
self.http_sign_get = C.bool( self.http_sign_get = ap_settings.http_sign_get
self.host.memory.config_get(CONF_SECTION, "http_sign_get", C.BOOL_TRUE) self.max_items = ap_settings.new_node_max_items
) self.comments_max_depth = ap_settings.comments_max_depth
self.max_items = int( self.ap_path = ap_settings.ap_path
self.host.memory.config_get(CONF_SECTION, "new_node_max_items", 50)
)
self.comments_max_depth = int(
self.host.memory.config_get(CONF_SECTION, "comments_max_depth", 0)
)
self.ap_path = self.host.memory.config_get(CONF_SECTION, "ap_path", "_ap")
self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/") self.base_ap_url = parse.urljoin(f"https://{self.public_url}", f"{self.ap_path}/")
# True (default) if we provide gateway only to entities/services from our server # True (default) if we provide gateway only to entities/services from our server
self.local_only = C.bool( self.local_only = ap_settings.local_only
self.host.memory.config_get(CONF_SECTION, "local_only", C.BOOL_TRUE)
)
# if True (default), mention will be parsed in non-private content coming from # if True (default), mention will be parsed in non-private content coming from
# XMPP. This is necessary as XEP-0372 are coming separately from item where the # XMPP. This is necessary as XEP-0372 are coming separately from item where the
# mention is done, which is hard to impossible to translate to ActivityPub (where # mention is done, which is hard to impossible to translate to ActivityPub (where
# mention specified inside the item directly). See documentation for details. # mention specified inside the item directly). See documentation for details.
self.auto_mentions = C.bool( self.auto_mentions = ap_settings.auto_mentions
self.host.memory.config_get(CONF_SECTION, "auto_mentions", C.BOOL_TRUE)
)
html_redirect: Dict[str, Union[str, dict]] = self.host.memory.config_get( self.html_redirect = ap_settings.html_redirect
CONF_SECTION, "html_redirect_dict", {}
)
self.html_redirect: Dict[str, List[dict]] = {}
for url_type, target in html_redirect.items():
if isinstance(target, str):
target = {"url": target}
elif not isinstance(target, dict):
raise exceptions.ConfigError(
f"html_redirect target must be a URL or a dict, not {target!r}"
)
filters = target.setdefault("filters", {})
if "url" not in target:
log.warning(f"invalid HTML redirection, missing target URL: {target}")
continue
# a slash in the url_type is a syntactic shortcut to have a node filter
if "/" in url_type:
url_type, node_filter = url_type.split("/", 1)
filters["node"] = node_filter
self.html_redirect.setdefault(url_type, []).append(target)
# HTTP server launch # HTTP server launch
self.server = HTTPServer(self) self.server = HTTPServer(self)
if connection_type == "http": if connection_type == "http":
reactor.listenTCP(self.http_port, self.server) reactor.listenTCP(self.http_port, self.server)
else: else:
options = tls.get_options_from_config(self.host.memory.config, CONF_SECTION) options = tls.get_options_from_config(ap_settings.tls)
tls.tls_options_check(options) tls.tls_options_check(options)
context_factory = tls.get_tls_context_factory(options) context_factory = tls.get_tls_context_factory(options)
reactor.listenSSL(self.http_port, self.server, context_factory) reactor.listenSSL(self.http_port, self.server, context_factory)
@ -1596,8 +1541,10 @@ class APGateway:
elif start_index > 5000: elif start_index > 5000:
raise error.StanzaError( raise error.StanzaError(
"feature-not-implemented", "feature-not-implemented",
text="Maximum limit for previous_index has been reached, this limit" text=(
"is set to avoid DoS", "Maximum limit for previous_index has been reached, this limit"
"is set to avoid DoS"
),
) )
else: else:
# we'll convert "start_index" to "after_id", thus we need the item just # we'll convert "start_index" to "after_id", thus we need the item just
@ -1972,7 +1919,7 @@ class APGateway:
except StopIteration: except StopIteration:
# no item corresponding to this author, we use ap_account # no item corresponding to this author, we use ap_account
log.warning( log.warning(
"Can't find a single cached item for parent item " f"{parent_item!r}" f"Can't find a single cached item for parent item {parent_item!r}"
) )
parent_ap_account = ap_account parent_ap_account = ap_account
else: else:
@ -2419,7 +2366,7 @@ class APGateway:
if parsed_anchor["type"] != "pubsub": if parsed_anchor["type"] != "pubsub":
log.warning( log.warning(
f"ignoring reference with non pubsub anchor, this is not supported: " f"ignoring reference with non pubsub anchor, this is not supported: "
"{reference_data!r}" f"{{reference_data!r}}"
) )
return False return False

View file

@ -17,8 +17,10 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>. # along with this program. If not, see <http://www.gnu.org/licenses/>.
from libervia.backend.core.constants import Const as C
from libervia.backend.core.i18n import _
IMPORT_NAME = "ap-gateway" IMPORT_NAME = "ap-gateway"
CONF_SECTION = f"component {IMPORT_NAME}"
CONTENT_TYPE_WEBFINGER = "application/jrd+json; charset=utf-8" CONTENT_TYPE_WEBFINGER = "application/jrd+json; charset=utf-8"
CONTENT_TYPE_AP = "application/activity+json; charset=utf-8" CONTENT_TYPE_AP = "application/activity+json; charset=utf-8"
TYPE_ACTOR = "actor" TYPE_ACTOR = "actor"
@ -129,3 +131,36 @@ MAX_AVATAR_SIZE = 1024 * 1024 * 5
# storage prefixes # storage prefixes
ST_AVATAR = "[avatar]" ST_AVATAR = "[avatar]"
ST_AP_CACHE = "[AP_item_cache]" ST_AP_CACHE = "[AP_item_cache]"
PLUGIN_INFO = {
C.PI_NAME: "ActivityPub Gateway component",
C.PI_IMPORT_NAME: IMPORT_NAME,
C.PI_MODES: [C.PLUG_MODE_COMPONENT],
C.PI_TYPE: C.PLUG_TYPE_ENTRY_POINT,
C.PI_PROTOCOLS: [],
C.PI_DEPENDENCIES: [
"XEP-0050",
"XEP-0054",
"XEP-0060",
"XEP-0084",
"XEP-0106",
"XEP-0277",
"XEP-0292",
"XEP-0329",
"XEP-0372",
"XEP-0424",
"XEP-0465",
"XEP-0470",
"XEP-0447",
"XEP-0471",
"PUBSUB_CACHE",
"TEXT_SYNTAXES",
"IDENTITY",
],
C.PI_RECOMMENDATIONS: [],
C.PI_MAIN: "APGateway",
C.PI_HANDLER: C.BOOL_TRUE,
C.PI_DESCRIPTION: _(
"Gateway for bidirectional communication between XMPP and ActivityPub."
),
}

Some files were not shown because too many files have changed in this diff Show more