Functions
bool	is_vpn_connected (html.HtmlElement dom)

dict	assign_headers (dict headers)

	request (str query, dict params)

	extract_result (list[html.HtmlElement] dom_result)

	extract_results (html.HtmlElement search_results)

	response (Response resp)

	fetch_traits (EngineTraits engine_traits)

Variables
	logger = logging.getLogger()

bool	use_cache = True

str	leta_engine = 'google'

str	search_url = "https://leta.mullvad.net"

dict	about

list	categories = ['general', 'web']

bool	paging = True

int	max_page = 50

bool	time_range_support = True

dict	time_range_dict

list	available_leta_engines

Detailed Description

This is the implementation of the Mullvad-Leta meta-search engine.

This engine **REQUIRES** that searxng operate within a Mullvad VPN

If using docker, consider using gluetun for easily connecting to the Mullvad

- https://github.com/qdm12/gluetun

Otherwise, follow instructions provided by Mullvad for enabling the VPN on Linux

- https://mullvad.net/en/help/install-mullvad-app-linux

.. hint::

   The :py:obj:`EngineTraits` is empty by default.  Maintainers have to run
   ``make data.traits`` (in the Mullvad VPN / :py:obj:`fetch_traits`) and rebase
   the modified JSON file ``searx/data/engine_traits.json`` on every single
   update of SearXNG!

Function Documentation

◆ assign_headers()

dict searx.engines.mullvad_leta.assign_headers ( dict headers )

Assigns the headers to make a request to Mullvad Leta

Definition at line 80 of file mullvad_leta.py.

def assign_headers(headers: dict) -> dict:
    """Assigns the headers to make a request to Mullvad Leta"""
    headers['Accept'] = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"
    headers['Content-Type'] = "application/x-www-form-urlencoded"
    headers['Host'] = "leta.mullvad.net"
    headers['Origin'] = "https://leta.mullvad.net"
    return headers
 
 

Referenced by fetch_traits(), and request().

Here is the caller graph for this function:

◆ extract_result()

searx.engines.mullvad_leta.extract_result ( list[html.HtmlElement] dom_result )

Definition at line 130 of file mullvad_leta.py.

def extract_result(dom_result: list[html.HtmlElement]):
    # Infoboxes sometimes appear in the beginning and will have a length of 0
    if len(dom_result) == 3:
        [a_elem, h3_elem, p_elem] = dom_result
    elif len(dom_result) == 4:
        [_, a_elem, h3_elem, p_elem] = dom_result
    else:
        return None
 
    return {
        'url': extract_text(a_elem.text),
        'title': extract_text(h3_elem),
        'content': extract_text(p_elem),
    }
 
 

Referenced by extract_results().

Here is the caller graph for this function:

◆ extract_results()

searx.engines.mullvad_leta.extract_results ( html.HtmlElement search_results )

Definition at line 146 of file mullvad_leta.py.

def extract_results(search_results: html.HtmlElement):
    for search_result in search_results:
        dom_result = eval_xpath_list(search_result, 'div/div/*')
        result = extract_result(dom_result)
        if result is not None:
            yield result
 
 

References extract_result().

Referenced by response().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ fetch_traits()

searx.engines.mullvad_leta.fetch_traits ( EngineTraits engine_traits )

Fetch languages and regions from Mullvad-Leta

.. warning::

    Fetching the engine traits also requires a Mullvad VPN connection. If
    not connected, then an error message will print and no traits will be
    updated.

Definition at line 165 of file mullvad_leta.py.

def fetch_traits(engine_traits: EngineTraits):
    """Fetch languages and regions from Mullvad-Leta
 
    .. warning::
 
        Fetching the engine traits also requires a Mullvad VPN connection. If
        not connected, then an error message will print and no traits will be
        updated.
    """
    # pylint: disable=import-outside-toplevel
    # see https://github.com/searxng/searxng/issues/762
    from searx.network import post as http_post
 
    # pylint: enable=import-outside-toplevel
    resp = http_post(search_url, headers=assign_headers({}))
    if not isinstance(resp, Response):
        print("ERROR: failed to get response from mullvad-leta. Are you connected to the VPN?")
        return
    if not resp.ok:
        print("ERROR: response from mullvad-leta is not OK. Are you connected to the VPN?")
        return
    dom = html.fromstring(resp.text)
    if not is_vpn_connected(dom):
        print('ERROR: Not connected to Mullvad VPN')
        return
    # supported region codes
    options = eval_xpath_list(dom.body, '//main/div/form/div[2]/div/select[1]/option')
    if options is None or len(options) <= 0:
        print('ERROR: could not find any results. Are you connected to the VPN?')
    for x in options:
        eng_country = x.get("value")
 
        sxng_locales = get_official_locales(eng_country, engine_traits.languages.keys(), regional=True)
 
        if not sxng_locales:
            print(
                "ERROR: can't map from Mullvad-Leta country %s (%s) to a babel region."
                % (x.get('data-name'), eng_country)
            )
            continue
 
        for sxng_locale in sxng_locales:
            engine_traits.regions[region_tag(sxng_locale)] = eng_country

References assign_headers(), and is_vpn_connected().

Here is the call graph for this function:

◆ is_vpn_connected()

bool searx.engines.mullvad_leta.is_vpn_connected ( html.HtmlElement dom )

Returns true if the VPN is connected, False otherwise

Definition at line 74 of file mullvad_leta.py.

def is_vpn_connected(dom: html.HtmlElement) -> bool:
    """Returns true if the VPN is connected, False otherwise"""
    connected_text = extract_text(eval_xpath(dom, '//main/div/p[1]'))
    return connected_text != 'You are not connected to Mullvad VPN.'
 
 

Referenced by fetch_traits(), and response().

Here is the caller graph for this function:

◆ request()

searx.engines.mullvad_leta.request	(	str	query,
		dict	params )

Definition at line 89 of file mullvad_leta.py.

def request(query: str, params: dict):
    country = traits.get_region(params.get('searxng_locale', 'all'), traits.all_locale)  # type: ignore
 
    result_engine = leta_engine
    if leta_engine not in available_leta_engines:
        result_engine = available_leta_engines[0]
        logger.warning(
            'Configured engine "%s" not one of the available engines %s, defaulting to "%s"',
            leta_engine,
            available_leta_engines,
            result_engine,
        )
 
    params['url'] = search_url
    params['method'] = 'POST'
    params['data'] = {
        "q": query,
        "gl": country if country is str else '',
        'engine': result_engine,
    }
    # pylint: disable=undefined-variable
    if use_cache:
        params['data']['oc'] = "on"
    # pylint: enable=undefined-variable
 
    if params['time_range'] in time_range_dict:
        params['dateRestrict'] = time_range_dict[params['time_range']]
    else:
        params['dateRestrict'] = ''
 
    if params['pageno'] > 1:
        #  Page 1 is n/a, Page 2 is 11, page 3 is 21, ...
        params['data']['start'] = ''.join([str(params['pageno'] - 1), "1"])
 
    if params['headers'] is None:
        params['headers'] = {}
 
    assign_headers(params['headers'])
    return params
 
 

References assign_headers().

Here is the call graph for this function:

◆ response()

searx.engines.mullvad_leta.response ( Response resp )

Checks if connected to Mullvad VPN, then extracts the search results from
the DOM resp: requests response object

Definition at line 154 of file mullvad_leta.py.

def response(resp: Response):
    """Checks if connected to Mullvad VPN, then extracts the search results from
    the DOM resp: requests response object"""
 
    dom = html.fromstring(resp.text)
    if not is_vpn_connected(dom):
        raise SearxEngineResponseException('Not connected to Mullvad VPN')
    search_results = eval_xpath(dom.body, '//main/div[2]/div')
    return list(extract_results(search_results))
 
 

References extract_results(), and is_vpn_connected().

Here is the call graph for this function:

Variable Documentation

◆ about

dict searx.engines.mullvad_leta.about

Initial value:

=  {
    "website": search_url,
    "wikidata_id": 'Q47008412',  # the Mullvad id - not leta, but related
    "official_api_documentation": 'https://leta.mullvad.net/faq',
    "use_official_api": False,
    "require_api_key": False,
    "results": 'HTML',
}

Definition at line 47 of file mullvad_leta.py.

◆ available_leta_engines

list searx.engines.mullvad_leta.available_leta_engines

Initial value:

=  [
    'google',  # first will be default if provided engine is invalid
    'brave',
]

Definition at line 68 of file mullvad_leta.py.

◆ categories

list searx.engines.mullvad_leta.categories = ['general', 'web']

Definition at line 57 of file mullvad_leta.py.

◆ leta_engine

str searx.engines.mullvad_leta.leta_engine = 'google'

Definition at line 42 of file mullvad_leta.py.

◆ logger

searx.engines.mullvad_leta.logger = logging.getLogger()

Definition at line 36 of file mullvad_leta.py.

◆ max_page

int searx.engines.mullvad_leta.max_page = 50

Definition at line 59 of file mullvad_leta.py.

◆ paging

bool searx.engines.mullvad_leta.paging = True

Definition at line 58 of file mullvad_leta.py.

◆ search_url

str searx.engines.mullvad_leta.search_url = "https://leta.mullvad.net"

Definition at line 44 of file mullvad_leta.py.

◆ time_range_dict

dict searx.engines.mullvad_leta.time_range_dict

Initial value:

=  {
    "day": "d1",
    "week": "w1",
    "month": "m1",
    "year": "y1",
}

Definition at line 61 of file mullvad_leta.py.

◆ time_range_support

bool searx.engines.mullvad_leta.time_range_support = True

Definition at line 60 of file mullvad_leta.py.

◆ use_cache

bool searx.engines.mullvad_leta.use_cache = True

Definition at line 40 of file mullvad_leta.py.

Functions

Variables

Detailed Description

Function Documentation

◆ assign_headers()

◆ extract_result()

◆ extract_results()

◆ fetch_traits()

◆ is_vpn_connected()

◆ request()

◆ response()

Variable Documentation

◆ about

◆ available_leta_engines

◆ categories

◆ leta_engine

◆ logger

◆ max_page

◆ paging

◆ search_url

◆ time_range_dict

◆ time_range_support

◆ use_cache