.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.engines Namespace Reference

::1337x More...

Namespaces

namespace  1337x
namespace  360search
namespace  360search_videos
namespace  9gag
namespace  acfun
namespace  adobe_stock
namespace  ahmia
namespace  alpinelinux
namespace  annas_archive
namespace  ansa
namespace  apkmirror
namespace  apple_app_store
namespace  apple_maps
namespace  archlinux
namespace  artic
namespace  arxiv
namespace  ask
namespace  astrophysics_data_system
namespace  baidu
namespace  bandcamp
namespace  base
namespace  bilibili
namespace  bing
namespace  bing_images
namespace  bing_news
namespace  bing_videos
namespace  bitchute
namespace  bpb
namespace  brave
namespace  bt4g
namespace  btdigg
namespace  ccc_media
namespace  chefkoch
namespace  chinaso
namespace  cloudflareai
namespace  command
namespace  core
namespace  cppreference
namespace  crates
namespace  crossref
namespace  currency_convert
namespace  dailymotion
namespace  deepl
namespace  deezer
namespace  demo_offline
namespace  demo_online
namespace  destatis
namespace  deviantart
namespace  dictzone
namespace  digbt
namespace  discourse
namespace  docker_hub
namespace  doku
namespace  duckduckgo
namespace  duckduckgo_definitions
namespace  duckduckgo_extra
namespace  duckduckgo_weather
namespace  duden
namespace  dummy
 -offline
namespace  dummy-offline
namespace  ebay
namespace  elasticsearch
namespace  emojipedia
namespace  fdroid
namespace  findthatmeme
namespace  flickr
namespace  flickr_noapi
namespace  freesound
namespace  frinkiac
namespace  fyyd
namespace  geizhals
namespace  genius
namespace  gitea
namespace  github
namespace  github_code
namespace  gitlab
namespace  goodreads
namespace  google
namespace  google_images
namespace  google_news
namespace  google_play
namespace  google_scholar
namespace  google_videos
namespace  hackernews
namespace  hex
namespace  huggingface
namespace  il_post
namespace  imdb
namespace  imgur
namespace  ina
namespace  invidious
namespace  ipernity
namespace  iqiyi
namespace  jisho
namespace  json_engine
namespace  kickass
namespace  lemmy
namespace  lib_rs
namespace  libretranslate
namespace  lingva
namespace  livespace
namespace  loc
namespace  marginalia
namespace  mariadb_server
namespace  mastodon
namespace  material_icons
namespace  mediathekviewweb
namespace  mediawiki
namespace  meilisearch
namespace  metacpan
namespace  microsoft_learn
namespace  mixcloud
namespace  mojeek
namespace  mongodb
namespace  moviepilot
namespace  mozhi
namespace  mrs
namespace  mullvad_leta
namespace  mwmbl
namespace  mysql_server
namespace  naver
namespace  niconico
namespace  npm
namespace  nyaa
namespace  odysee
namespace  ollama
namespace  open_meteo
namespace  openalex
namespace  openclipart
namespace  openlibrary
namespace  opensemantic
namespace  openstreetmap
namespace  openverse
namespace  pdbe
namespace  peertube
namespace  photon
namespace  pinterest
namespace  piped
namespace  piratebay
namespace  pixabay
namespace  pixiv
namespace  pkg_go_dev
namespace  podcastindex
namespace  postgresql
namespace  presearch
namespace  public_domain_image_archive
namespace  pubmed
namespace  pypi
namespace  quark
namespace  qwant
namespace  radio_browser
namespace  recoll
namespace  reddit
namespace  repology
namespace  reuters
namespace  rottentomatoes
namespace  rumble
namespace  scanr_structures
namespace  searchcode_code
namespace  searx_engine
namespace  seekr
namespace  selfhst
namespace  semantic_scholar
namespace  senscritique
namespace  sepiasearch
namespace  seznam
namespace  sogou
namespace  sogou_images
namespace  sogou_videos
namespace  sogou_wechat
namespace  solidtorrents
namespace  solr
namespace  soundcloud
namespace  spotify
namespace  springer
namespace  sqlite
namespace  stackexchange
namespace  startpage
namespace  steam
namespace  stract
namespace  svgrepo
namespace  tagesschau
namespace  tineye
namespace  tokyotoshokan
namespace  tootfinder
namespace  torznab
namespace  translated
namespace  tubearchivist
namespace  unsplash
namespace  uxwing
namespace  valkey_server
namespace  vimeo
namespace  voidlinux
namespace  wallhaven
namespace  wikicommons
namespace  wikidata
namespace  wikipedia
namespace  wolframalpha_api
namespace  wolframalpha_noapi
namespace  wordnik
namespace  wttr
namespace  www1x
namespace  xpath
namespace  yacy
namespace  yahoo
namespace  yahoo_news
namespace  yandex
namespace  yandex_music
namespace  yep
namespace  youtube_api
namespace  youtube_noapi
namespace  zlibrary

Functions

 check_engine_module (types.ModuleType module)
"Engine | types.ModuleType | None" load_engine (dict[str, t.Any] engine_data)
 set_loggers ("Engine|types.ModuleType" engine, str engine_name)
 update_engine_attributes ("Engine | types.ModuleType" engine, dict[str, t.Any] engine_data)
 update_attributes_for_tor ("Engine | types.ModuleType" engine)
 is_missing_required_attributes ("Engine | types.ModuleType" engine)
 using_tor_proxy ("Engine | types.ModuleType" engine)
 is_engine_active ("Engine | types.ModuleType" engine)
bool call_engine_setup ("Engine | types.ModuleType" engine, dict[str, t.Any] engine_data)
 register_engine ("Engine | types.ModuleType" engine)
 load_engines (list[dict[str, t.Any]] engine_list)

Variables

 logger = logger.getChild('engines')
 ENGINE_DIR = dirname(realpath(__file__))
dict ENGINE_DEFAULT_ARGS
str DEFAULT_CATEGORY = 'other'
dict categories = {'general': []}
dict engines = {}
dict engine_shortcuts = {}

Detailed Description

::1337x

::9gag

::360search_videos

::360search

1337x
360Search search engine for searxng
360Search-Videos: A search engine for retrieving videos from 360Search.
9GAG (social media)
Load and initialize the ``engines``, see :py:func:`load_engines` and register
:py:obj:`engine_shortcuts`.

usage::

    load_engines( settings['engines'] )

Function Documentation

◆ call_engine_setup()

bool searx.engines.call_engine_setup ( "Engine | types.ModuleType" engine,
dict[str, t.Any] engine_data )

Definition at line 232 of file __init__.py.

232def call_engine_setup(engine: "Engine | types.ModuleType", engine_data: dict[str, t.Any]) -> bool:
233 setup_ok = False
234 setup_func = getattr(engine, "setup", None)
235
236 if setup_func is None:
237 setup_ok = True
238 elif not callable(setup_func):
239 logger.error("engine's setup method isn't a callable (is of type: %s)", type(setup_func))
240 else:
241 try:
242 setup_ok = engine.setup(engine_data)
243 except Exception as e: # pylint: disable=broad-except
244 logger.exception('exception : {0}'.format(e))
245
246 if not setup_ok:
247 logger.error("%s: Engine setup was not successful, engine is set to inactive.", engine.name)
248 return setup_ok
249
250

Referenced by load_engine().

Here is the caller graph for this function:

◆ check_engine_module()

searx.engines.check_engine_module ( types.ModuleType module)

Definition at line 69 of file __init__.py.

69def check_engine_module(module: types.ModuleType):
70 # probe unintentional name collisions / for example name collisions caused
71 # by import statements in the engine module ..
72
73 # network: https://github.com/searxng/searxng/issues/762#issuecomment-1605323861
74 obj = getattr(module, 'network', None)
75 if obj and inspect.ismodule(obj):
76 msg = f'type of {module.__name__}.network is a module ({obj.__name__}), expected a string'
77 # logger.error(msg)
78 raise TypeError(msg)
79
80

Referenced by load_engine().

Here is the caller graph for this function:

◆ is_engine_active()

searx.engines.is_engine_active ( "Engine | types.ModuleType" engine)

Definition at line 220 of file __init__.py.

220def is_engine_active(engine: "Engine | types.ModuleType"):
221 # check if engine is inactive
222 if engine.inactive is True:
223 return False
224
225 # exclude onion engines if not using tor
226 if 'onions' in engine.categories and not using_tor_proxy(engine):
227 return False
228
229 return True
230
231

References using_tor_proxy().

Referenced by load_engine().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ is_missing_required_attributes()

searx.engines.is_missing_required_attributes ( "Engine | types.ModuleType" engine)
An attribute is required when its name doesn't start with ``_`` (underline).
Required attributes must not be ``None``.

Definition at line 202 of file __init__.py.

202def is_missing_required_attributes(engine: "Engine | types.ModuleType"):
203 """An attribute is required when its name doesn't start with ``_`` (underline).
204 Required attributes must not be ``None``.
205
206 """
207 missing = False
208 for engine_attr in dir(engine):
209 if not engine_attr.startswith('_') and getattr(engine, engine_attr) is None:
210 logger.error('Missing engine config attribute: "{0}.{1}"'.format(engine.name, engine_attr))
211 missing = True
212 return missing
213
214

Referenced by load_engine().

Here is the caller graph for this function:

◆ load_engine()

"Engine | types.ModuleType | None" searx.engines.load_engine ( dict[str, t.Any] engine_data)
Load engine from ``engine_data``.

:param dict engine_data:  Attributes from YAML ``settings:engines/<engine>``
:return: initialized namespace of the ``<engine>``.

1. create a namespace and load module of the ``<engine>``
2. update namespace with the defaults from :py:obj:`ENGINE_DEFAULT_ARGS`
3. update namespace with values from ``engine_data``

If engine *is active*, return namespace of the engine, otherwise return
``None``.

This function also returns ``None`` if initialization of the namespace fails
for one of the following reasons:

- engine name contains underscore
- engine name is not lowercase
- required attribute is not set :py:func:`is_missing_required_attributes`

Definition at line 81 of file __init__.py.

81def load_engine(engine_data: dict[str, t.Any]) -> "Engine | types.ModuleType | None":
82 """Load engine from ``engine_data``.
83
84 :param dict engine_data: Attributes from YAML ``settings:engines/<engine>``
85 :return: initialized namespace of the ``<engine>``.
86
87 1. create a namespace and load module of the ``<engine>``
88 2. update namespace with the defaults from :py:obj:`ENGINE_DEFAULT_ARGS`
89 3. update namespace with values from ``engine_data``
90
91 If engine *is active*, return namespace of the engine, otherwise return
92 ``None``.
93
94 This function also returns ``None`` if initialization of the namespace fails
95 for one of the following reasons:
96
97 - engine name contains underscore
98 - engine name is not lowercase
99 - required attribute is not set :py:func:`is_missing_required_attributes`
100
101 """
102 # pylint: disable=too-many-return-statements
103
104 engine_name = engine_data.get('name')
105 if engine_name is None:
106 logger.error('An engine does not have a "name" field')
107 return None
108 if '_' in engine_name:
109 logger.error('Engine name contains underscore: "{}"'.format(engine_name))
110 return None
111
112 if engine_name.lower() != engine_name:
113 logger.warning('Engine name is not lowercase: "{}", converting to lowercase'.format(engine_name))
114 engine_name = engine_name.lower()
115 engine_data['name'] = engine_name
116
117 # load_module
118 module_name = engine_data.get('engine')
119 if module_name is None:
120 logger.error('The "engine" field is missing for the engine named "{}"'.format(engine_name))
121 return None
122 try:
123 engine = load_module(module_name + '.py', ENGINE_DIR)
124 except (SyntaxError, KeyboardInterrupt, SystemExit, SystemError, ImportError, RuntimeError):
125 logger.exception('Fatal exception in engine "{}"'.format(module_name))
126 sys.exit(1)
127 except BaseException:
128 logger.exception('Cannot load engine "{}"'.format(module_name))
129 return None
130
131 check_engine_module(engine)
132 update_engine_attributes(engine, engine_data)
133 update_attributes_for_tor(engine)
134
135 # avoid cyclic imports
136 # pylint: disable=import-outside-toplevel
137 from searx.enginelib.traits import EngineTraitsMap
138
139 trait_map = EngineTraitsMap.from_data()
140 trait_map.set_traits(engine)
141
142 if not is_engine_active(engine):
143 return None
144
145 if is_missing_required_attributes(engine):
146 return None
147
148 set_loggers(engine, engine_name)
149
150 if not call_engine_setup(engine, engine_data):
151 return None
152
153 if not any(cat in settings['categories_as_tabs'] for cat in engine.categories):
154 engine.categories.append(DEFAULT_CATEGORY)
155
156 return engine
157
158

References call_engine_setup(), check_engine_module(), is_engine_active(), is_missing_required_attributes(), set_loggers(), update_attributes_for_tor(), and update_engine_attributes().

Referenced by load_engines().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ load_engines()

searx.engines.load_engines ( list[dict[str, t.Any]] engine_list)
usage: ``engine_list = settings['engines']``

Definition at line 266 of file __init__.py.

266def load_engines(engine_list: list[dict[str, t.Any]]):
267 """usage: ``engine_list = settings['engines']``"""
268 engines.clear()
269 engine_shortcuts.clear()
270 categories.clear()
271 categories['general'] = []
272 for engine_data in engine_list:
273 engine = load_engine(engine_data)
274 if engine:
275 register_engine(engine)
276 return engines

References load_engine(), and register_engine().

Here is the call graph for this function:

◆ register_engine()

searx.engines.register_engine ( "Engine | types.ModuleType" engine)

Definition at line 251 of file __init__.py.

251def register_engine(engine: "Engine | types.ModuleType"):
252 if engine.name in engines:
253 logger.error('Engine config error: ambiguous name: {0}'.format(engine.name))
254 sys.exit(1)
255 engines[engine.name] = engine
256
257 if engine.shortcut in engine_shortcuts:
258 logger.error('Engine config error: ambiguous shortcut: {0}'.format(engine.shortcut))
259 sys.exit(1)
260 engine_shortcuts[engine.shortcut] = engine.name
261
262 for category_name in engine.categories:
263 categories.setdefault(category_name, []).append(engine)
264
265

Referenced by load_engines().

Here is the caller graph for this function:

◆ set_loggers()

searx.engines.set_loggers ( "Engine|types.ModuleType" engine,
str engine_name )

Definition at line 159 of file __init__.py.

159def set_loggers(engine: "Engine|types.ModuleType", engine_name: str):
160 # set the logger for engine
161 engine.logger = logger.getChild(engine_name)
162 # the engine may have load some other engines
163 # may sure the logger is initialized
164 # use sys.modules.copy() to avoid "RuntimeError: dictionary changed size during iteration"
165 # see https://github.com/python/cpython/issues/89516
166 # and https://docs.python.org/3.10/library/sys.html#sys.modules
167 modules = sys.modules.copy()
168 for module_name, module in modules.items():
169 if (
170 module_name.startswith("searx.engines")
171 and module_name != "searx.engines.__init__"
172 and not hasattr(module, "logger")
173 ):
174 module_engine_name = module_name.split(".")[-1]
175 module.logger = logger.getChild(module_engine_name) # type: ignore
176
177

Referenced by load_engine().

Here is the caller graph for this function:

◆ update_attributes_for_tor()

searx.engines.update_attributes_for_tor ( "Engine | types.ModuleType" engine)

Definition at line 196 of file __init__.py.

196def update_attributes_for_tor(engine: "Engine | types.ModuleType"):
197 if using_tor_proxy(engine) and hasattr(engine, 'onion_url'):
198 engine.search_url = engine.onion_url + getattr(engine, 'search_path', '') # type: ignore
199 engine.timeout += settings['outgoing'].get('extra_proxy_timeout', 0) # type: ignore
200
201

References using_tor_proxy().

Referenced by load_engine().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ update_engine_attributes()

searx.engines.update_engine_attributes ( "Engine | types.ModuleType" engine,
dict[str, t.Any] engine_data )

Definition at line 178 of file __init__.py.

178def update_engine_attributes(engine: "Engine | types.ModuleType", engine_data: dict[str, t.Any]):
179 # set engine attributes from engine_data
180 for param_name, param_value in engine_data.items():
181 if param_name == 'categories':
182 if isinstance(param_value, str):
183 param_value = list(map(str.strip, param_value.split(',')))
184 engine.categories = param_value # type: ignore
185 elif hasattr(engine, 'about') and param_name == 'about':
186 engine.about = {**engine.about, **engine_data['about']} # type: ignore
187 else:
188 setattr(engine, param_name, param_value)
189
190 # set default attributes
191 for arg_name, arg_value in ENGINE_DEFAULT_ARGS.items():
192 if not hasattr(engine, arg_name):
193 setattr(engine, arg_name, copy.deepcopy(arg_value))
194
195

Referenced by load_engine().

Here is the caller graph for this function:

◆ using_tor_proxy()

searx.engines.using_tor_proxy ( "Engine | types.ModuleType" engine)
Return True if the engine configuration declares to use Tor.

Definition at line 215 of file __init__.py.

215def using_tor_proxy(engine: "Engine | types.ModuleType"):
216 """Return True if the engine configuration declares to use Tor."""
217 return settings['outgoing'].get('using_tor_proxy') or getattr(engine, 'using_tor_proxy', False)
218
219

Referenced by is_engine_active(), and update_attributes_for_tor().

Here is the caller graph for this function:

Variable Documentation

◆ categories

dict searx.engines.categories = {'general': []}

Definition at line 53 of file __init__.py.

◆ DEFAULT_CATEGORY

str searx.engines.DEFAULT_CATEGORY = 'other'

Definition at line 51 of file __init__.py.

◆ ENGINE_DEFAULT_ARGS

dict searx.engines.ENGINE_DEFAULT_ARGS
Initial value:
1= {
2 # Common options in the engine module
3 "engine_type": "online",
4 "paging": False,
5 "time_range_support": False,
6 "safesearch": False,
7 # settings.yml
8 "categories": ["general"],
9 "enable_http": False,
10 "shortcut": "-",
11 "timeout": settings["outgoing"]["request_timeout"],
12 "display_error_messages": True,
13 "disabled": False,
14 "inactive": False,
15 "about": {},
16 "using_tor_proxy": False,
17 "send_accept_language_header": False,
18 "tokens": [],
19 "max_page": 0,
20}

Definition at line 30 of file __init__.py.

◆ ENGINE_DIR

searx.engines.ENGINE_DIR = dirname(realpath(__file__))

Definition at line 27 of file __init__.py.

◆ engine_shortcuts

dict searx.engines.engine_shortcuts = {}

Definition at line 58 of file __init__.py.

◆ engines

dict searx.engines.engines = {}

Definition at line 55 of file __init__.py.

◆ logger

searx.engines.logger = logger.getChild('engines')

Definition at line 26 of file __init__.py.