searx.plugins.hostnames Namespace Reference

Classes
class	SXNGPlugin

Functions
bool|str	filter_url_field ("Result|LegacyResult" result, str field_name, str url_src)

Variables
dict	REPLACE = {}
set	REMOVE = set()
set	HIGH = set()
set	LOW = set()

Detailed Description

During the initialization phase, the plugin checks whether a ``hostnames:``
configuration exists. If this is not the case, the plugin is not included
in the PluginStorage (it is not available for selection).

- ``hostnames.replace``: A **mapping** of regular expressions to hostnames to be
  replaced by other hostnames.

  .. code:: yaml

     hostnames:
       replace:
         '(.*\\.)?youtube\\.com$': 'invidious.example.com'
         '(.*\\.)?youtu\\.be$': 'invidious.example.com'
         ...

- ``hostnames.remove``: A **list** of regular expressions of the hostnames whose
  results should be taken from the results list.

  .. code:: yaml

     hostnames:
       remove:
         - '(.*\\.)?facebook.com$'
         - ...

- ``hostnames.high_priority``: A **list** of regular expressions for hostnames
  whose result should be given higher priority. The results from these hosts are
  arranged higher in the results list.

  .. code:: yaml

     hostnames:
       high_priority:
         - '(.*\\.)?wikipedia.org$'
         - ...

- ``hostnames.lower_priority``: A **list** of regular expressions for hostnames
  whose result should be given lower priority. The results from these hosts are
  arranged lower in the results list.

  .. code:: yaml

     hostnames:
       low_priority:
         - '(.*\\.)?google(\\..*)?$'
         - ...

If the URL matches the pattern of ``high_priority`` AND ``low_priority``, the
higher priority wins over the lower priority.

Alternatively, you can also specify a file name for the **mappings** or
**lists** to load these from an external file:

.. code:: yaml

   hostnames:
     replace: 'rewrite-hosts.yml'
     remove:
       - '(.*\\.)?facebook.com$'
       - ...
     low_priority:
       - '(.*\\.)?google(\\..*)?$'
       - ...
     high_priority:
       - '(.*\\.)?wikipedia.org$'
       - ...

The ``rewrite-hosts.yml`` from the example above must be in the folder in which
the ``settings.yml`` file is already located (``/etc/searxng``). The file then
only contains the lists or the mapping tables without further information on the
namespaces.  In the example above, this would be a mapping table that looks
something like this:

.. code:: yaml

   '(.*\\.)?youtube\\.com$': 'invidious.example.com'
   '(.*\\.)?youtu\\.be$': 'invidious.example.com'

Function Documentation

filter_url_field()

bool | str searx.plugins.hostnames.filter_url_field	(	"Result|LegacyResult"	result,
		str	field_name,
		str	url_src )

Returns bool ``True`` to use URL unchanged (``False`` to ignore URL).
If URL should be modified, the returned string is the new URL to use.

Definition at line 184 of file hostnames.py.

def filter_url_field(result: "Result|LegacyResult", field_name: str, url_src: str) -> bool | str:
    """Returns bool ``True`` to use URL unchanged (``False`` to ignore URL).
    If URL should be modified, the returned string is the new URL to use."""
 
    if not url_src:
        log.debug("missing a URL in field %s", field_name)
        return True
 
    url_src_parsed = urlparse(url=url_src)
 
    for pattern in REMOVE:
        if pattern.search(url_src_parsed.netloc):
            return False
 
    for pattern, replacement in REPLACE.items():
        if pattern.search(url_src_parsed.netloc):
            new_url = url_src_parsed._replace(netloc=pattern.sub(replacement, url_src_parsed.netloc))
            new_url = urlunparse(new_url)
            return new_url
 
    return True

Variable Documentation

HIGH

set searx.plugins.hostnames.HIGH = set()

Definition at line 110 of file hostnames.py.

LOW

set searx.plugins.hostnames.LOW = set()

Definition at line 111 of file hostnames.py.

REMOVE

set searx.plugins.hostnames.REMOVE = set()

Definition at line 109 of file hostnames.py.

REPLACE

dict searx.plugins.hostnames.REPLACE = {}

Definition at line 108 of file hostnames.py.