.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.engines Namespace Reference

::1337x More...

Namespaces

namespace  1337x
namespace  360search
namespace  360search_videos
namespace  9gag
namespace  acfun
namespace  adobe_stock
namespace  ahmia
namespace  alpinelinux
namespace  annas_archive
namespace  ansa
namespace  apkmirror
namespace  apple_app_store
namespace  apple_maps
namespace  archlinux
namespace  artic
namespace  arxiv
namespace  ask
namespace  astrophysics_data_system
namespace  baidu
namespace  bandcamp
namespace  base
namespace  bilibili
namespace  bing
namespace  bing_images
namespace  bing_news
namespace  bing_videos
namespace  bitchute
namespace  bpb
namespace  brave
namespace  bt4g
namespace  btdigg
namespace  ccc_media
namespace  chefkoch
namespace  chinaso
namespace  cloudflareai
namespace  command
namespace  core
namespace  cppreference
namespace  crates
namespace  crossref
namespace  currency_convert
namespace  dailymotion
namespace  deepl
namespace  deezer
namespace  demo_offline
namespace  demo_online
namespace  destatis
namespace  deviantart
namespace  dictzone
namespace  digbt
namespace  discourse
namespace  docker_hub
namespace  doku
namespace  duckduckgo
namespace  duckduckgo_definitions
namespace  duckduckgo_extra
namespace  duckduckgo_weather
namespace  duden
namespace  dummy
 -offline
namespace  dummy-offline
namespace  ebay
namespace  elasticsearch
namespace  emojipedia
namespace  fdroid
namespace  findthatmeme
namespace  flickr
namespace  flickr_noapi
namespace  freesound
namespace  frinkiac
namespace  fyyd
namespace  geizhals
namespace  genius
namespace  gitea
namespace  github
namespace  github_code
namespace  gitlab
namespace  goodreads
namespace  google
namespace  google_images
namespace  google_news
namespace  google_play
namespace  google_scholar
namespace  google_videos
namespace  hackernews
namespace  hex
namespace  huggingface
namespace  il_post
namespace  imdb
namespace  imgur
namespace  ina
namespace  invidious
namespace  ipernity
namespace  iqiyi
namespace  jisho
namespace  json_engine
namespace  kickass
namespace  lemmy
namespace  lib_rs
namespace  libretranslate
namespace  lingva
namespace  livespace
namespace  loc
namespace  marginalia
namespace  mariadb_server
namespace  mastodon
namespace  material_icons
namespace  mediathekviewweb
namespace  mediawiki
namespace  meilisearch
namespace  metacpan
namespace  microsoft_learn
namespace  mixcloud
namespace  mojeek
namespace  mongodb
namespace  moviepilot
namespace  mozhi
namespace  mrs
namespace  mullvad_leta
namespace  mwmbl
namespace  mysql_server
namespace  naver
namespace  niconico
namespace  npm
namespace  nyaa
namespace  odysee
namespace  ollama
namespace  open_meteo
namespace  openalex
namespace  openclipart
namespace  openlibrary
namespace  opensemantic
namespace  openstreetmap
namespace  openverse
namespace  pdbe
namespace  peertube
namespace  photon
namespace  pinterest
namespace  piped
namespace  piratebay
namespace  pixabay
namespace  pixiv
namespace  pkg_go_dev
namespace  podcastindex
namespace  postgresql
namespace  presearch
namespace  public_domain_image_archive
namespace  pubmed
namespace  pypi
namespace  quark
namespace  qwant
namespace  radio_browser
namespace  recoll
namespace  reddit
namespace  repology
namespace  reuters
namespace  rottentomatoes
namespace  rumble
namespace  scanr_structures
namespace  searchcode_code
namespace  searx_engine
namespace  seekr
namespace  selfhst
namespace  semantic_scholar
namespace  senscritique
namespace  sepiasearch
namespace  seznam
namespace  sogou
namespace  sogou_images
namespace  sogou_videos
namespace  sogou_wechat
namespace  solidtorrents
namespace  solr
namespace  soundcloud
namespace  spotify
namespace  springer
namespace  sqlite
namespace  stackexchange
namespace  startpage
namespace  steam
namespace  stract
namespace  svgrepo
namespace  tagesschau
namespace  tineye
namespace  tokyotoshokan
namespace  tootfinder
namespace  torznab
namespace  translated
namespace  tubearchivist
namespace  unsplash
namespace  uxwing
namespace  valkey_server
namespace  vimeo
namespace  voidlinux
namespace  wallhaven
namespace  wikicommons
namespace  wikidata
namespace  wikipedia
namespace  wolframalpha_api
namespace  wolframalpha_noapi
namespace  wordnik
namespace  wttr
namespace  www1x
namespace  xpath
namespace  yacy
namespace  yahoo
namespace  yahoo_news
namespace  yandex
namespace  yandex_music
namespace  yep
namespace  youtube_api
namespace  youtube_noapi
namespace  zlibrary

Functions

 check_engine_module (types.ModuleType module)
"Engine | types.ModuleType | None" load_engine (dict[str, t.Any] engine_data)
 set_loggers ("Engine|types.ModuleType" engine, str engine_name)
 update_engine_attributes ("Engine | types.ModuleType" engine, dict[str, t.Any] engine_data)
 update_attributes_for_tor ("Engine | types.ModuleType" engine)
 is_missing_required_attributes ("Engine | types.ModuleType" engine)
 using_tor_proxy ("Engine | types.ModuleType" engine)
 is_engine_active ("Engine | types.ModuleType" engine)
 register_engine ("Engine | types.ModuleType" engine)
 load_engines (list[dict[str, t.Any]] engine_list)

Variables

 logger = logger.getChild('engines')
 ENGINE_DIR = dirname(realpath(__file__))
dict ENGINE_DEFAULT_ARGS
str DEFAULT_CATEGORY = 'other'
dict categories = {'general': []}
dict engines = {}
dict engine_shortcuts = {}

Detailed Description

::1337x

::9gag

::360search_videos

::360search

1337x
360Search search engine for searxng
360Search-Videos: A search engine for retrieving videos from 360Search.
9GAG (social media)
Load and initialize the ``engines``, see :py:func:`load_engines` and register
:py:obj:`engine_shortcuts`.

usage::

    load_engines( settings['engines'] )

Function Documentation

◆ check_engine_module()

searx.engines.check_engine_module ( types.ModuleType module)

Definition at line 66 of file __init__.py.

66def check_engine_module(module: types.ModuleType):
67 # probe unintentional name collisions / for example name collisions caused
68 # by import statements in the engine module ..
69
70 # network: https://github.com/searxng/searxng/issues/762#issuecomment-1605323861
71 obj = getattr(module, 'network', None)
72 if obj and inspect.ismodule(obj):
73 msg = f'type of {module.__name__}.network is a module ({obj.__name__}), expected a string'
74 # logger.error(msg)
75 raise TypeError(msg)
76
77

Referenced by load_engine().

Here is the caller graph for this function:

◆ is_engine_active()

searx.engines.is_engine_active ( "Engine | types.ModuleType" engine)

Definition at line 214 of file __init__.py.

214def is_engine_active(engine: "Engine | types.ModuleType"):
215 # check if engine is inactive
216 if engine.inactive is True:
217 return False
218
219 # exclude onion engines if not using tor
220 if 'onions' in engine.categories and not using_tor_proxy(engine):
221 return False
222
223 return True
224
225

References using_tor_proxy().

Referenced by load_engine().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ is_missing_required_attributes()

searx.engines.is_missing_required_attributes ( "Engine | types.ModuleType" engine)
An attribute is required when its name doesn't start with ``_`` (underline).
Required attributes must not be ``None``.

Definition at line 196 of file __init__.py.

196def is_missing_required_attributes(engine: "Engine | types.ModuleType"):
197 """An attribute is required when its name doesn't start with ``_`` (underline).
198 Required attributes must not be ``None``.
199
200 """
201 missing = False
202 for engine_attr in dir(engine):
203 if not engine_attr.startswith('_') and getattr(engine, engine_attr) is None:
204 logger.error('Missing engine config attribute: "{0}.{1}"'.format(engine.name, engine_attr))
205 missing = True
206 return missing
207
208

Referenced by load_engine().

Here is the caller graph for this function:

◆ load_engine()

"Engine | types.ModuleType | None" searx.engines.load_engine ( dict[str, t.Any] engine_data)
Load engine from ``engine_data``.

:param dict engine_data:  Attributes from YAML ``settings:engines/<engine>``
:return: initialized namespace of the ``<engine>``.

1. create a namespace and load module of the ``<engine>``
2. update namespace with the defaults from :py:obj:`ENGINE_DEFAULT_ARGS`
3. update namespace with values from ``engine_data``

If engine *is active*, return namespace of the engine, otherwise return
``None``.

This function also returns ``None`` if initialization of the namespace fails
for one of the following reasons:

- engine name contains underscore
- engine name is not lowercase
- required attribute is not set :py:func:`is_missing_required_attributes`

Definition at line 78 of file __init__.py.

78def load_engine(engine_data: dict[str, t.Any]) -> "Engine | types.ModuleType | None":
79 """Load engine from ``engine_data``.
80
81 :param dict engine_data: Attributes from YAML ``settings:engines/<engine>``
82 :return: initialized namespace of the ``<engine>``.
83
84 1. create a namespace and load module of the ``<engine>``
85 2. update namespace with the defaults from :py:obj:`ENGINE_DEFAULT_ARGS`
86 3. update namespace with values from ``engine_data``
87
88 If engine *is active*, return namespace of the engine, otherwise return
89 ``None``.
90
91 This function also returns ``None`` if initialization of the namespace fails
92 for one of the following reasons:
93
94 - engine name contains underscore
95 - engine name is not lowercase
96 - required attribute is not set :py:func:`is_missing_required_attributes`
97
98 """
99 # pylint: disable=too-many-return-statements
100
101 engine_name = engine_data.get('name')
102 if engine_name is None:
103 logger.error('An engine does not have a "name" field')
104 return None
105 if '_' in engine_name:
106 logger.error('Engine name contains underscore: "{}"'.format(engine_name))
107 return None
108
109 if engine_name.lower() != engine_name:
110 logger.warning('Engine name is not lowercase: "{}", converting to lowercase'.format(engine_name))
111 engine_name = engine_name.lower()
112 engine_data['name'] = engine_name
113
114 # load_module
115 module_name = engine_data.get('engine')
116 if module_name is None:
117 logger.error('The "engine" field is missing for the engine named "{}"'.format(engine_name))
118 return None
119 try:
120 engine = load_module(module_name + '.py', ENGINE_DIR)
121 except (SyntaxError, KeyboardInterrupt, SystemExit, SystemError, ImportError, RuntimeError):
122 logger.exception('Fatal exception in engine "{}"'.format(module_name))
123 sys.exit(1)
124 except BaseException:
125 logger.exception('Cannot load engine "{}"'.format(module_name))
126 return None
127
128 check_engine_module(engine)
129 update_engine_attributes(engine, engine_data)
130 update_attributes_for_tor(engine)
131
132 # avoid cyclic imports
133 # pylint: disable=import-outside-toplevel
134 from searx.enginelib.traits import EngineTraitsMap
135
136 trait_map = EngineTraitsMap.from_data()
137 trait_map.set_traits(engine)
138
139 if not is_engine_active(engine):
140 return None
141
142 if is_missing_required_attributes(engine):
143 return None
144
145 set_loggers(engine, engine_name)
146
147 if not any(cat in settings['categories_as_tabs'] for cat in engine.categories):
148 engine.categories.append(DEFAULT_CATEGORY)
149
150 return engine
151
152

References check_engine_module(), is_engine_active(), is_missing_required_attributes(), set_loggers(), update_attributes_for_tor(), and update_engine_attributes().

Referenced by load_engines().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ load_engines()

searx.engines.load_engines ( list[dict[str, t.Any]] engine_list)
usage: ``engine_list = settings['engines']``

Definition at line 241 of file __init__.py.

241def load_engines(engine_list: list[dict[str, t.Any]]):
242 """usage: ``engine_list = settings['engines']``"""
243 engines.clear()
244 engine_shortcuts.clear()
245 categories.clear()
246 categories['general'] = []
247 for engine_data in engine_list:
248 engine = load_engine(engine_data)
249 if engine:
250 register_engine(engine)
251 return engines

References load_engine(), and register_engine().

Here is the call graph for this function:

◆ register_engine()

searx.engines.register_engine ( "Engine | types.ModuleType" engine)

Definition at line 226 of file __init__.py.

226def register_engine(engine: "Engine | types.ModuleType"):
227 if engine.name in engines:
228 logger.error('Engine config error: ambiguous name: {0}'.format(engine.name))
229 sys.exit(1)
230 engines[engine.name] = engine
231
232 if engine.shortcut in engine_shortcuts:
233 logger.error('Engine config error: ambiguous shortcut: {0}'.format(engine.shortcut))
234 sys.exit(1)
235 engine_shortcuts[engine.shortcut] = engine.name
236
237 for category_name in engine.categories:
238 categories.setdefault(category_name, []).append(engine)
239
240

Referenced by load_engines().

Here is the caller graph for this function:

◆ set_loggers()

searx.engines.set_loggers ( "Engine|types.ModuleType" engine,
str engine_name )

Definition at line 153 of file __init__.py.

153def set_loggers(engine: "Engine|types.ModuleType", engine_name: str):
154 # set the logger for engine
155 engine.logger = logger.getChild(engine_name)
156 # the engine may have load some other engines
157 # may sure the logger is initialized
158 # use sys.modules.copy() to avoid "RuntimeError: dictionary changed size during iteration"
159 # see https://github.com/python/cpython/issues/89516
160 # and https://docs.python.org/3.10/library/sys.html#sys.modules
161 modules = sys.modules.copy()
162 for module_name, module in modules.items():
163 if (
164 module_name.startswith("searx.engines")
165 and module_name != "searx.engines.__init__"
166 and not hasattr(module, "logger")
167 ):
168 module_engine_name = module_name.split(".")[-1]
169 module.logger = logger.getChild(module_engine_name) # type: ignore
170
171

Referenced by load_engine().

Here is the caller graph for this function:

◆ update_attributes_for_tor()

searx.engines.update_attributes_for_tor ( "Engine | types.ModuleType" engine)

Definition at line 190 of file __init__.py.

190def update_attributes_for_tor(engine: "Engine | types.ModuleType"):
191 if using_tor_proxy(engine) and hasattr(engine, 'onion_url'):
192 engine.search_url = engine.onion_url + getattr(engine, 'search_path', '') # type: ignore
193 engine.timeout += settings['outgoing'].get('extra_proxy_timeout', 0) # type: ignore
194
195

References using_tor_proxy().

Referenced by load_engine().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ update_engine_attributes()

searx.engines.update_engine_attributes ( "Engine | types.ModuleType" engine,
dict[str, t.Any] engine_data )

Definition at line 172 of file __init__.py.

172def update_engine_attributes(engine: "Engine | types.ModuleType", engine_data: dict[str, t.Any]):
173 # set engine attributes from engine_data
174 for param_name, param_value in engine_data.items():
175 if param_name == 'categories':
176 if isinstance(param_value, str):
177 param_value = list(map(str.strip, param_value.split(',')))
178 engine.categories = param_value # type: ignore
179 elif hasattr(engine, 'about') and param_name == 'about':
180 engine.about = {**engine.about, **engine_data['about']} # type: ignore
181 else:
182 setattr(engine, param_name, param_value)
183
184 # set default attributes
185 for arg_name, arg_value in ENGINE_DEFAULT_ARGS.items():
186 if not hasattr(engine, arg_name):
187 setattr(engine, arg_name, copy.deepcopy(arg_value))
188
189

Referenced by load_engine().

Here is the caller graph for this function:

◆ using_tor_proxy()

searx.engines.using_tor_proxy ( "Engine | types.ModuleType" engine)
Return True if the engine configuration declares to use Tor.

Definition at line 209 of file __init__.py.

209def using_tor_proxy(engine: "Engine | types.ModuleType"):
210 """Return True if the engine configuration declares to use Tor."""
211 return settings['outgoing'].get('using_tor_proxy') or getattr(engine, 'using_tor_proxy', False)
212
213

Referenced by is_engine_active(), and update_attributes_for_tor().

Here is the caller graph for this function:

Variable Documentation

◆ categories

dict searx.engines.categories = {'general': []}

Definition at line 53 of file __init__.py.

◆ DEFAULT_CATEGORY

str searx.engines.DEFAULT_CATEGORY = 'other'

Definition at line 51 of file __init__.py.

◆ ENGINE_DEFAULT_ARGS

dict searx.engines.ENGINE_DEFAULT_ARGS
Initial value:
1= {
2 # Common options in the engine module
3 "engine_type": "online",
4 "paging": False,
5 "time_range_support": False,
6 "safesearch": False,
7 # settings.yml
8 "categories": ["general"],
9 "enable_http": False,
10 "shortcut": "-",
11 "timeout": settings["outgoing"]["request_timeout"],
12 "display_error_messages": True,
13 "disabled": False,
14 "inactive": False,
15 "about": {},
16 "using_tor_proxy": False,
17 "send_accept_language_header": False,
18 "tokens": [],
19 "max_page": 0,
20}

Definition at line 30 of file __init__.py.

◆ ENGINE_DIR

searx.engines.ENGINE_DIR = dirname(realpath(__file__))

Definition at line 27 of file __init__.py.

◆ engine_shortcuts

dict searx.engines.engine_shortcuts = {}

Definition at line 55 of file __init__.py.

◆ engines

dict searx.engines.engines = {}

Definition at line 54 of file __init__.py.

◆ logger

searx.engines.logger = logger.getChild('engines')

Definition at line 26 of file __init__.py.