Inheritance diagram for searx.sqlitedb.SQLiteAppl:

Collaboration diagram for searx.sqlitedb.SQLiteAppl:

Public Member Functions
	__init__ (self, str db_url)
sqlite3.Connection	connect (self)
	register_functions (self, sqlite3.Connection conn)
sqlite3.Connection	DB (self)
bool	init (self, sqlite3.Connection conn)
	create_schema (self, sqlite3.Connection conn)

Public Attributes
str	db_url = db_url
SQLiteProperties	properties = SQLiteProperties(db_url)

Static Public Attributes
dict	DDL_CREATE_TABLES = {}
int	DB_SCHEMA = 1
dict	SQLITE_THREADING_MODE
str	SQLITE_JOURNAL_MODE = "WAL"
dict	SQLITE_CONNECT_ARGS

Protected Member Functions
	_compatibility (self)
sqlite3.Connection	_connect (self)

Protected Attributes
bool	_init_done = False
sqlite3.Connection\|None	_DB = None

Detailed Description

Abstract base class for implementing convenient DB access in SQLite
applications.  In the constructor, a :py:obj:`SQLiteProperties` instance is
already aggregated under ``self.properties``.

Definition at line 91 of file sqlitedb.py.

Constructor & Destructor Documentation

◆ init()

searx.sqlitedb.SQLiteAppl.__init__	(		self,
		str	db_url )

Definition at line 154 of file sqlitedb.py.

    def __init__(self, db_url: str):
 
        self.db_url: str = db_url
        self.properties: SQLiteProperties = SQLiteProperties(db_url)
        self._init_done: bool = False
        self._compatibility()
        # atexit.register(self.tear_down)
 

Member Function Documentation

◆ _compatibility()

searx.sqlitedb.SQLiteAppl._compatibility ( self )

protected

Definition at line 170 of file sqlitedb.py.

    def _compatibility(self):
 
        if self.SQLITE_THREADING_MODE == "serialized":
            self._DB: sqlite3.Connection | None = None
        else:
            msg = (
                f"SQLite library is compiled with {self.SQLITE_THREADING_MODE} mode,"
                " read https://docs.python.org/3/library/sqlite3.html#sqlite3.threadsafety"
            )
            if threading.active_count() > 1:
                logger.error(msg)
            else:
                logger.warning(msg)
 
        if sqlite3.sqlite_version_info <= (3, 35):
            # See "Generalize UPSERT:" in https://sqlite.org/releaselog/3_35_0.html
            logger.critical(
                "SQLite runtime library version %s is not supported (require >= 3.35)", sqlite3.sqlite_version
            )
 

References SQLITE_THREADING_MODE.

◆ _connect()

sqlite3.Connection searx.sqlitedb.SQLiteAppl._connect ( self )

protected

Definition at line 190 of file sqlitedb.py.

    def _connect(self) -> sqlite3.Connection:
        conn = sqlite3.Connection(self.db_url, **self.SQLITE_CONNECT_ARGS)  # type: ignore
        conn.execute(f"PRAGMA journal_mode={self.SQLITE_JOURNAL_MODE}")
        self.register_functions(conn)
        return conn
 

References searx.cache.ExpireCacheCfg.db_url, searx.favicons.cache.FaviconCacheConfig.db_url, db_url, register_functions(), and SQLITE_CONNECT_ARGS.

Referenced by connect().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ connect()

sqlite3.Connection searx.sqlitedb.SQLiteAppl.connect ( self )

Creates a new DB connection (:py:obj:`SQLITE_CONNECT_ARGS`).  If not
already done, the DB schema is set up.  The caller must take care of
closing the resource.  Alternatively, :py:obj:`SQLiteAppl.DB` can also
be used (the resource behind `self.DB` is automatically closed when the
process or thread is terminated).

Definition at line 196 of file sqlitedb.py.

    def connect(self) -> sqlite3.Connection:
        """Creates a new DB connection (:py:obj:`SQLITE_CONNECT_ARGS`).  If not
        already done, the DB schema is set up.  The caller must take care of
        closing the resource.  Alternatively, :py:obj:`SQLiteAppl.DB` can also
        be used (the resource behind `self.DB` is automatically closed when the
        process or thread is terminated).
        """
        if sys.version_info < (3, 12):
            # Prior Python 3.12 there is no "autocommit" option
            self.SQLITE_CONNECT_ARGS.pop("autocommit", None)  # pyright: ignore[reportUnreachable]
 
        msg = (
            f"[{threading.current_thread().ident}] {self.__class__.__name__}({self.db_url})"
            f" {self.SQLITE_CONNECT_ARGS} // {self.SQLITE_JOURNAL_MODE}"
        )
        logger.debug(msg)
 
        with self._connect() as conn:
            self.init(conn)
        return conn
 

References _connect(), searx.cache.ExpireCacheSQLite.init(), searx.data.currencies.CurrenciesDB.init(), searx.data.tracker_patterns.TrackerPatternsDB.init(), searx.enginelib.Engine.init(), searx.plugins._core.Plugin.init(), searx.plugins._core.PluginStorage.init(), searx.plugins.ahmia_filter.SXNGPlugin.init(), searx.plugins.hostnames.SXNGPlugin.init(), searx.plugins.tracker_url_remover.SXNGPlugin.init(), searx.search.processors.ProcessorMap.init(), init(), and SQLITE_CONNECT_ARGS.

Referenced by searx.cache.ExpireCacheSQLite.create_table(), searx.favicons.cache.FaviconCacheSQLite.set(), and searx.cache.ExpireCacheSQLite.truncate_tables().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ create_schema()

searx.sqlitedb.SQLiteAppl.create_schema	(		self,
		sqlite3.Connection	conn )

Reimplemented in searx.sqlitedb.SQLiteProperties.

Definition at line 308 of file sqlitedb.py.

    def create_schema(self, conn: sqlite3.Connection):
 
        logger.debug("create schema ..")
        self.properties.set("DB_SCHEMA", self.DB_SCHEMA)
        self.properties.set("LAST_MAINTENANCE", "")
        with conn:
            for table_name, sql in self.DDL_CREATE_TABLES.items():
                conn.execute(sql)
                self.properties.set(f"Table {table_name} created", table_name)
 
 

References searx.cache.ExpireCacheSQLite.DB_SCHEMA, searx.favicons.cache.FaviconCacheSQLite.DB_SCHEMA, DB_SCHEMA, searx.cache.ExpireCacheSQLite.DDL_CREATE_TABLES, searx.favicons.cache.FaviconCacheSQLite.DDL_CREATE_TABLES, DDL_CREATE_TABLES, and properties.

Referenced by init(), and searx.sqlitedb.SQLiteProperties.init().

Here is the caller graph for this function:

◆ DB()

sqlite3.Connection searx.sqlitedb.SQLiteAppl.DB ( self )

Provides a DB connection.  The connection is a *singleton* and
therefore well suited for read access.  If
:py:obj:`SQLITE_THREADING_MODE` is ``serialized`` only one DB connection
is created for all threads.

.. note::

   For dedicated `transaction control`_, it is recommended to create a
   new connection (:py:obj:`SQLiteAppl.connect`).

.. _transaction control:
    https://docs.python.org/3/library/sqlite3.html#sqlite3-controlling-transactions

Definition at line 242 of file sqlitedb.py.

    def DB(self) -> sqlite3.Connection:
        """Provides a DB connection.  The connection is a *singleton* and
        therefore well suited for read access.  If
        :py:obj:`SQLITE_THREADING_MODE` is ``serialized`` only one DB connection
        is created for all threads.
 
        .. note::
 
           For dedicated `transaction control`_, it is recommended to create a
           new connection (:py:obj:`SQLiteAppl.connect`).
 
        .. _transaction control:
            https://docs.python.org/3/library/sqlite3.html#sqlite3-controlling-transactions
        """
 
        conn: sqlite3.Connection
 
        if self.SQLITE_THREADING_MODE == "serialized":
            # Theoretically it is possible to reuse the DB cursor across threads
            # as of Python 3.12, in practice the threading of the cursor seems
            # to me a little faulty that I prefer to establish one connection
            # per thread.
            #
            # may we can activate this code one day ..
            # if self._DB is None:
            #     self._DB = self.connect()
            # conn = self._DB
            conn = DBSession.get_connect(self)
        else:
            conn = DBSession.get_connect(self)
 
        # Since more than one instance of SQLiteAppl share the same DB
        # connection, we need to make sure that each SQLiteAppl instance has run
        # its init method at least once.
        self.init(conn)
 
        return conn
 

References searx.cache.ExpireCacheSQLite.init(), searx.data.currencies.CurrenciesDB.init(), searx.data.tracker_patterns.TrackerPatternsDB.init(), searx.enginelib.Engine.init(), searx.plugins._core.Plugin.init(), searx.plugins._core.PluginStorage.init(), searx.plugins.ahmia_filter.SXNGPlugin.init(), searx.plugins.hostnames.SXNGPlugin.init(), searx.plugins.tracker_url_remover.SXNGPlugin.init(), searx.search.processors.ProcessorMap.init(), init(), and SQLITE_THREADING_MODE.

Referenced by searx.favicons.cache.FaviconCacheSQLite.__call__(), and searx.favicons.cache.FaviconCacheSQLite._query_val().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ init()

bool searx.sqlitedb.SQLiteAppl.init	(		self,
		sqlite3.Connection	conn )

Initializes the DB schema and properties, is only executed once even
if called several times.

If the initialization has not yet taken place, it is carried out and a
`True` is returned to the caller at the end.  If the initialization has
already been carried out in the past, `False` is returned.

Reimplemented in searx.cache.ExpireCacheSQLite, and searx.sqlitedb.SQLiteProperties.

Definition at line 280 of file sqlitedb.py.

    def init(self, conn: sqlite3.Connection) -> bool:
        """Initializes the DB schema and properties, is only executed once even
        if called several times.
 
        If the initialization has not yet taken place, it is carried out and a
        `True` is returned to the caller at the end.  If the initialization has
        already been carried out in the past, `False` is returned.
        """
 
        if self._init_done:
            return False
        self._init_done = True
 
        logger.debug("init DB: %s", self.db_url)
        self.properties.init(conn)
 
        ver = self.properties("DB_SCHEMA")
        if ver is None:
            with conn:
                self.create_schema(conn)
        else:
            ver = int(ver)
            if ver != self.DB_SCHEMA:
                raise sqlite3.DatabaseError("Expected DB schema v%s, DB schema is v%s" % (self.DB_SCHEMA, ver))
            logger.debug("DB_SCHEMA = %s", ver)
 
        return True
 

References _init_done, create_schema(), searx.cache.ExpireCacheSQLite.DB_SCHEMA, searx.favicons.cache.FaviconCacheSQLite.DB_SCHEMA, DB_SCHEMA, searx.cache.ExpireCacheCfg.db_url, searx.favicons.cache.FaviconCacheConfig.db_url, db_url, init(), and properties.

Referenced by connect(), DB(), and init().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ register_functions()

searx.sqlitedb.SQLiteAppl.register_functions	(		self,
		sqlite3.Connection	conn )

Create user-defined_ SQL functions.

``REGEXP(<pattern>, <field>)`` : 0 | 1
   `re.search`_ returns (int) 1 for a match and 0 for none match of
   ``<pattern>`` in ``<field>``.

   .. code:: sql

      SELECT '12' AS field WHERE REGEXP('^[0-9][0-9]$', field)
      -- 12

      SELECT REGEXP('[0-9][0-9]', 'X12Y')
      -- 1
      SELECT REGEXP('[0-9][0-9]', 'X1Y')
      -- 0

.. _user-defined: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.create_function
.. _deterministic: https://sqlite.org/deterministic.html
.. _re.search: https://docs.python.org/3/library/re.html#re.search

Definition at line 217 of file sqlitedb.py.

    def register_functions(self, conn: sqlite3.Connection):
        """Create user-defined_ SQL functions.
 
        ``REGEXP(<pattern>, <field>)`` : 0 | 1
           `re.search`_ returns (int) 1 for a match and 0 for none match of
           ``<pattern>`` in ``<field>``.
 
           .. code:: sql
 
              SELECT '12' AS field WHERE REGEXP('^[0-9][0-9]$', field)
              -- 12
 
              SELECT REGEXP('[0-9][0-9]', 'X12Y')
              -- 1
              SELECT REGEXP('[0-9][0-9]', 'X1Y')
              -- 0
 
        .. _user-defined: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.create_function
        .. _deterministic: https://sqlite.org/deterministic.html
        .. _re.search: https://docs.python.org/3/library/re.html#re.search
        """
 
        conn.create_function("regexp", 2, lambda x, y: 1 if re.search(x, y) else 0, deterministic=True)  # type: ignore
 

Referenced by _connect().

Here is the caller graph for this function:

Member Data Documentation

◆ _DB

sqlite3.Connection | None searx.sqlitedb.SQLiteAppl._DB = None

protected

Definition at line 173 of file sqlitedb.py.

◆ _init_done

bool searx.sqlitedb.SQLiteAppl._init_done = False

protected

Definition at line 158 of file sqlitedb.py.

Referenced by init(), and searx.sqlitedb.SQLiteProperties.init().

◆ DB_SCHEMA

searx.sqlitedb.SQLiteAppl.DB_SCHEMA = 1

static

Definition at line 98 of file sqlitedb.py.

Referenced by create_schema(), and init().

◆ db_url

searx.sqlitedb.SQLiteAppl.db_url = db_url

Definition at line 156 of file sqlitedb.py.

Referenced by _connect(), init(), and searx.sqlitedb.SQLiteProperties.init().

◆ DDL_CREATE_TABLES

dict searx.sqlitedb.SQLiteAppl.DDL_CREATE_TABLES = {}

static

Definition at line 96 of file sqlitedb.py.

Referenced by create_schema().

◆ properties

SQLiteProperties searx.sqlitedb.SQLiteAppl.properties = SQLiteProperties(db_url)

Definition at line 157 of file sqlitedb.py.

Referenced by create_schema(), searx.cache.ExpireCacheSQLite.create_table(), searx.cache.ExpireCacheSQLite.init(), init(), searx.cache.ExpireCacheSQLite.next_maintenance_time(), and searx.favicons.cache.FaviconCacheSQLite.next_maintenance_time().

◆ SQLITE_CONNECT_ARGS

dict searx.sqlitedb.SQLiteAppl.SQLITE_CONNECT_ARGS

static

Initial value:

=  {
        # "timeout": 5.0,
        # "detect_types": 0,
        "check_same_thread": bool(SQLITE_THREADING_MODE != "serialized"),
        "cached_statements": 0,  # https://github.com/python/cpython/issues/118172
        # "uri": False,
        "isolation_level": None,
    }

Definition at line 124 of file sqlitedb.py.

Referenced by _connect(), and connect().

◆ SQLITE_JOURNAL_MODE

str searx.sqlitedb.SQLiteAppl.SQLITE_JOURNAL_MODE = "WAL"

static

Definition at line 118 of file sqlitedb.py.

◆ SQLITE_THREADING_MODE

dict searx.sqlitedb.SQLiteAppl.SQLITE_THREADING_MODE

static

Initial value:

=  {
        0: "single-thread",
        1: "multi-thread",
        3: "serialized"}[sqlite3.threadsafety]

Definition at line 103 of file sqlitedb.py.

Referenced by _compatibility(), and DB().

The documentation for this class was generated from the following file:

/home/andrew/Documents/code/public/searxng/searx/sqlitedb.py

Public Member Functions

Public Attributes

Static Public Attributes

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ _compatibility()

◆ _connect()

◆ connect()

◆ create_schema()

◆ DB()

◆ init()

◆ register_functions()

Member Data Documentation

◆ _DB

◆ _init_done

◆ DB_SCHEMA

◆ db_url

◆ DDL_CREATE_TABLES

◆ properties

◆ SQLITE_CONNECT_ARGS

◆ SQLITE_JOURNAL_MODE

◆ SQLITE_THREADING_MODE

◆ init()