Inheritance diagram for searx.sqlitedb.SQLiteAppl:

Collaboration diagram for searx.sqlitedb.SQLiteAppl:

Public Member Functions
	__init__ (self, db_url)

sqlite3.Connection	connect (self)

	register_functions (self, conn)

sqlite3.Connection	DB (self)

	init (self)

	create_schema (self, conn)

Public Attributes
	db_url = db_url

	properties = SQLiteProperties(db_url)

	thread_local = threading.local()

Static Public Attributes
dict	DDL_CREATE_TABLES = {}

int	DB_SCHEMA = 1

dict	SQLITE_THREADING_MODE

str	SQLITE_JOURNAL_MODE = "WAL"

dict	SQLITE_CONNECT_ARGS

Protected Member Functions
	_compatibility (self)

Protected Attributes
bool	_init_done = False

	_DB = None

Detailed Description

Abstract base class for implementing convenient DB access in SQLite
applications.  In the constructor, a :py:obj:`SQLiteProperties` instance is
already aggregated under ``self.properties``.

Definition at line 26 of file sqlitedb.py.

Constructor & Destructor Documentation

◆ init()

searx.sqlitedb.SQLiteAppl.__init__	(		self,
			db_url )

Reimplemented in searx.favicons.cache.FaviconCacheSQLite, and searx.sqlitedb.SQLiteProperties.

Definition at line 88 of file sqlitedb.py.

    def __init__(self, db_url):
 
        self.db_url = db_url
        self.properties = SQLiteProperties(db_url)
        self.thread_local = threading.local()
        self._init_done = False
        self._compatibility()
 

Member Function Documentation

◆ _compatibility()

searx.sqlitedb.SQLiteAppl._compatibility ( self )

protected

Definition at line 96 of file sqlitedb.py.

    def _compatibility(self):
 
        if self.SQLITE_THREADING_MODE == "serialized":
            self._DB = None
        else:
            msg = (
                f"SQLite library is compiled with {self.SQLITE_THREADING_MODE} mode,"
                " read https://docs.python.org/3/library/sqlite3.html#sqlite3.threadsafety"
            )
            if threading.active_count() > 1:
                logger.error(msg)
            else:
                logger.warning(msg)
 
        if sqlite3.sqlite_version_info <= (3, 35):
            # See "Generalize UPSERT:" in https://sqlite.org/releaselog/3_35_0.html
            logger.critical(
                "SQLite runtime library version %s is not supported (require >= 3.35)", sqlite3.sqlite_version
            )
 

References searx.sqlitedb.SQLiteAppl.SQLITE_THREADING_MODE.

◆ connect()

sqlite3.Connection searx.sqlitedb.SQLiteAppl.connect ( self )

Creates a new DB connection (:py:obj:`SQLITE_CONNECT_ARGS`).  If not
already done, the DB schema is set up

Definition at line 116 of file sqlitedb.py.

    def connect(self) -> sqlite3.Connection:
        """Creates a new DB connection (:py:obj:`SQLITE_CONNECT_ARGS`).  If not
        already done, the DB schema is set up
        """
        if sys.version_info < (3, 12):
            # Prior Python 3.12 there is no "autocommit" option
            self.SQLITE_CONNECT_ARGS.pop("autocommit", None)
 
        self.init()
        logger.debug("%s: connect to DB: %s // %s", self.__class__.__name__, self.db_url, self.SQLITE_CONNECT_ARGS)
        conn = sqlite3.Connection(self.db_url, **self.SQLITE_CONNECT_ARGS)  # type: ignore
        conn.execute(f"PRAGMA journal_mode={self.SQLITE_JOURNAL_MODE}")
        self.register_functions(conn)
        return conn
 

References searx.favicons.cache.FaviconCacheStats.__class__, searx.favicons.cache.FaviconCacheConfig.db_url, searx.sqlitedb.SQLiteAppl.db_url, searx.sqlitedb.SQLiteProperties.db_url, searx.sqlitedb.SQLiteAppl.init(), searx.sqlitedb.SQLiteProperties.init(), searx.sqlitedb.SQLiteAppl.register_functions(), searx.sqlitedb.SQLiteAppl.SQLITE_CONNECT_ARGS, and searx.sqlitedb.SQLiteProperties.SQLITE_CONNECT_ARGS.

Referenced by searx.sqlitedb.SQLiteAppl.DB(), and searx.favicons.cache.FaviconCacheSQLite.set().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ create_schema()

searx.sqlitedb.SQLiteAppl.create_schema	(		self,
			conn )

Reimplemented in searx.sqlitedb.SQLiteProperties.

Definition at line 213 of file sqlitedb.py.

    def create_schema(self, conn):
 
        logger.debug("create schema ..")
        with conn:
            for table_name, sql in self.DDL_CREATE_TABLES.items():
                conn.execute(sql)
                self.properties.set(f"Table {table_name} created", table_name)
                self.properties.set("DB_SCHEMA", self.DB_SCHEMA)
                self.properties.set("LAST_MAINTENANCE", "")
 
 

References searx.favicons.cache.FaviconCacheSQLite.DB_SCHEMA, searx.sqlitedb.SQLiteAppl.DB_SCHEMA, searx.favicons.cache.FaviconCacheSQLite.DDL_CREATE_TABLES, searx.sqlitedb.SQLiteAppl.DDL_CREATE_TABLES, and searx.sqlitedb.SQLiteAppl.properties.

Referenced by searx.sqlitedb.SQLiteAppl.init(), and searx.sqlitedb.SQLiteProperties.init().

Here is the caller graph for this function:

◆ DB()

sqlite3.Connection searx.sqlitedb.SQLiteAppl.DB ( self )

Provides a DB connection.  The connection is a *singleton* and
therefore well suited for read access.  If
:py:obj:`SQLITE_THREADING_MODE` is ``serialized`` only one DB connection
is created for all threads.

.. note::

   For dedicated `transaction control`_, it is recommended to create a
   new connection (:py:obj:`SQLiteAppl.connect`).

.. _transaction control:
    https://docs.python.org/3/library/sqlite3.html#sqlite3-controlling-transactions

Definition at line 156 of file sqlitedb.py.

    def DB(self) -> sqlite3.Connection:
        """Provides a DB connection.  The connection is a *singleton* and
        therefore well suited for read access.  If
        :py:obj:`SQLITE_THREADING_MODE` is ``serialized`` only one DB connection
        is created for all threads.
 
        .. note::
 
           For dedicated `transaction control`_, it is recommended to create a
           new connection (:py:obj:`SQLiteAppl.connect`).
 
        .. _transaction control:
            https://docs.python.org/3/library/sqlite3.html#sqlite3-controlling-transactions
        """
 
        if getattr(self.thread_local, 'DB', None) is None:
            self.thread_local.DB = self.connect()
 
        # Theoretically it is possible to reuse the DB cursor across threads as
        # of Python 3.12, in practice the threading of the cursor seems to me to
        # be so faulty that I prefer to establish one connection per thread
 
        self.thread_local.DB.commit()
        return self.thread_local.DB
 
        # In "serialized" mode, SQLite can be safely used by multiple threads
        # with no restriction.
        #
        # if self.SQLITE_THREADING_MODE != "serialized":
        #     if getattr(self.thread_local, 'DB', None) is None:
        #         self.thread_local.DB = self.connect()
        #     return self.thread_local.DB
        #
        # if self._DB is None:
        #     self._DB = self.connect()  # pylint: disable=attribute-defined-outside-init
        # return self._DB
 

References searx.sqlitedb.SQLiteAppl.connect(), searx.sqlitedb.SQLiteAppl.thread_local, and searx.sqlitedb.SQLiteProperties.thread_local.

Referenced by searx.favicons.cache.FaviconCacheSQLite.__call__(), searx.sqlitedb.SQLiteProperties.__call__(), searx.favicons.cache.FaviconCacheSQLite._query_val(), searx.sqlitedb.SQLiteProperties.init(), searx.sqlitedb.SQLiteProperties.m_time(), searx.sqlitedb.SQLiteProperties.row(), and searx.sqlitedb.SQLiteProperties.set().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ init()

searx.sqlitedb.SQLiteAppl.init ( self )

Initializes the DB schema and properties, is only executed once even
if called several times.

Reimplemented in searx.sqlitedb.SQLiteProperties.

Definition at line 193 of file sqlitedb.py.

    def init(self):
        """Initializes the DB schema and properties, is only executed once even
        if called several times."""
 
        if self._init_done:
            return
        self._init_done = True
 
        logger.debug("init DB: %s", self.db_url)
        self.properties.init()
        ver = self.properties("DB_SCHEMA")
        if ver is None:
            with self.properties.DB:
                self.create_schema(self.properties.DB)
        else:
            ver = int(ver)
            if ver != self.DB_SCHEMA:
                raise sqlite3.DatabaseError("Expected DB schema v%s, DB schema is v%s" % (self.DB_SCHEMA, ver))
            logger.debug("DB_SCHEMA = %s", ver)
 

References searx.sqlitedb.SQLiteAppl._init_done, searx.sqlitedb.SQLiteProperties._init_done, searx.sqlitedb.SQLiteAppl.create_schema(), searx.sqlitedb.SQLiteProperties.create_schema(), searx.favicons.cache.FaviconCacheSQLite.DB_SCHEMA, searx.sqlitedb.SQLiteAppl.DB_SCHEMA, searx.favicons.cache.FaviconCacheConfig.db_url, searx.sqlitedb.SQLiteAppl.db_url, searx.sqlitedb.SQLiteProperties.db_url, and searx.sqlitedb.SQLiteAppl.properties.

Referenced by searx.sqlitedb.SQLiteAppl.connect().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ register_functions()

searx.sqlitedb.SQLiteAppl.register_functions	(		self,
			conn )

Create user-defined_ SQL functions.

``REGEXP(<pattern>, <field>)`` : 0 | 1
   `re.search`_ returns (int) 1 for a match and 0 for none match of
   ``<pattern>`` in ``<field>``.

   .. code:: sql

      SELECT '12' AS field WHERE REGEXP('^[0-9][0-9]$', field)
      -- 12

      SELECT REGEXP('[0-9][0-9]', 'X12Y')
      -- 1
      SELECT REGEXP('[0-9][0-9]', 'X1Y')
      -- 0

.. _user-defined: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.create_function
.. _deterministic: https://sqlite.org/deterministic.html
.. _re.search: https://docs.python.org/3/library/re.html#re.search

Definition at line 131 of file sqlitedb.py.

    def register_functions(self, conn):
        """Create user-defined_ SQL functions.
 
        ``REGEXP(<pattern>, <field>)`` : 0 | 1
           `re.search`_ returns (int) 1 for a match and 0 for none match of
           ``<pattern>`` in ``<field>``.
 
           .. code:: sql
 
              SELECT '12' AS field WHERE REGEXP('^[0-9][0-9]$', field)
              -- 12
 
              SELECT REGEXP('[0-9][0-9]', 'X12Y')
              -- 1
              SELECT REGEXP('[0-9][0-9]', 'X1Y')
              -- 0
 
        .. _user-defined: https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.create_function
        .. _deterministic: https://sqlite.org/deterministic.html
        .. _re.search: https://docs.python.org/3/library/re.html#re.search
        """
 
        conn.create_function('regexp', 2, lambda x, y: 1 if re.search(x, y) else 0, deterministic=True)
 

Referenced by searx.sqlitedb.SQLiteAppl.connect().

Here is the caller graph for this function:

Member Data Documentation

◆ _DB

searx.sqlitedb.SQLiteAppl._DB = None

protected

Definition at line 99 of file sqlitedb.py.

◆ _init_done

bool searx.sqlitedb.SQLiteAppl._init_done = False

protected

Definition at line 93 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl.init(), and searx.sqlitedb.SQLiteProperties.init().

◆ DB_SCHEMA

searx.sqlitedb.SQLiteAppl.DB_SCHEMA = 1

static

Definition at line 33 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl.create_schema(), and searx.sqlitedb.SQLiteAppl.init().

◆ db_url

searx.sqlitedb.SQLiteAppl.db_url = db_url

Definition at line 90 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl.connect(), searx.sqlitedb.SQLiteAppl.init(), and searx.sqlitedb.SQLiteProperties.init().

◆ DDL_CREATE_TABLES

dict searx.sqlitedb.SQLiteAppl.DDL_CREATE_TABLES = {}

static

Definition at line 31 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl.create_schema().

◆ properties

searx.sqlitedb.SQLiteAppl.properties = SQLiteProperties(db_url)

Definition at line 91 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl.create_schema(), searx.sqlitedb.SQLiteAppl.init(), and searx.favicons.cache.FaviconCacheSQLite.next_maintenance_time().

◆ SQLITE_CONNECT_ARGS

searx.sqlitedb.SQLiteAppl.SQLITE_CONNECT_ARGS

static

Initial value:

=  {
        # "timeout": 5.0,
        # "detect_types": 0,
        "check_same_thread": bool(SQLITE_THREADING_MODE != "serialized"),
        "cached_statements": 0,  # https://github.com/python/cpython/issues/118172
        # "uri": False,
        "autocommit": False,
    }

Definition at line 54 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl.connect().

◆ SQLITE_JOURNAL_MODE

str searx.sqlitedb.SQLiteAppl.SQLITE_JOURNAL_MODE = "WAL"

static

Definition at line 53 of file sqlitedb.py.

◆ SQLITE_THREADING_MODE

dict searx.sqlitedb.SQLiteAppl.SQLITE_THREADING_MODE

static

Initial value:

=  {
        0: "single-thread",
        1: "multi-thread",
        3: "serialized"}[sqlite3.threadsafety]

Definition at line 38 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl._compatibility().

◆ thread_local

searx.sqlitedb.SQLiteAppl.thread_local = threading.local()

Definition at line 92 of file sqlitedb.py.

Referenced by searx.sqlitedb.SQLiteAppl.DB().

The documentation for this class was generated from the following file:

/home/andrew/Documents/code/public/searxng/searx/sqlitedb.py

Public Member Functions

Public Attributes

Static Public Attributes

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ _compatibility()

◆ connect()

◆ create_schema()

◆ DB()

◆ init()

◆ register_functions()

Member Data Documentation

◆ _DB

◆ _init_done

◆ DB_SCHEMA

◆ db_url

◆ DDL_CREATE_TABLES

◆ properties

◆ SQLITE_CONNECT_ARGS

◆ SQLITE_JOURNAL_MODE

◆ SQLITE_THREADING_MODE

◆ thread_local

◆ init()