.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.engines Namespace Reference

::1337x More...

Namespaces

namespace  1337x
namespace  360search
namespace  360search_videos
namespace  9gag
namespace  acfun
namespace  adobe_stock
namespace  ahmia
namespace  alpinelinux
namespace  annas_archive
namespace  ansa
namespace  apkmirror
namespace  apple_app_store
namespace  apple_maps
namespace  archlinux
namespace  artic
namespace  arxiv
namespace  ask
namespace  astrophysics_data_system
namespace  baidu
namespace  bandcamp
namespace  base
namespace  bilibili
namespace  bing
namespace  bing_images
namespace  bing_news
namespace  bing_videos
namespace  bitchute
namespace  bpb
namespace  brave
namespace  bt4g
namespace  btdigg
namespace  ccc_media
namespace  chefkoch
namespace  chinaso
namespace  cloudflareai
namespace  command
namespace  core
namespace  cppreference
namespace  crates
namespace  crossref
namespace  currency_convert
namespace  dailymotion
namespace  deepl
namespace  deezer
namespace  demo_offline
namespace  demo_online
namespace  destatis
namespace  deviantart
namespace  dictzone
namespace  digbt
namespace  discourse
namespace  docker_hub
namespace  doku
namespace  duckduckgo
namespace  duckduckgo_definitions
namespace  duckduckgo_extra
namespace  duckduckgo_weather
namespace  duden
namespace  dummy
 -offline
namespace  dummy-offline
namespace  ebay
namespace  elasticsearch
namespace  emojipedia
namespace  fdroid
namespace  findthatmeme
namespace  flickr
namespace  flickr_noapi
namespace  freesound
namespace  frinkiac
namespace  fyyd
namespace  geizhals
namespace  genius
namespace  gitea
namespace  github
namespace  gitlab
namespace  goodreads
namespace  google
namespace  google_images
namespace  google_news
namespace  google_play
namespace  google_scholar
namespace  google_videos
namespace  hackernews
namespace  hex
namespace  huggingface
namespace  il_post
namespace  imdb
namespace  imgur
namespace  ina
namespace  invidious
namespace  ipernity
namespace  iqiyi
namespace  jisho
namespace  json_engine
namespace  kickass
namespace  lemmy
namespace  lib_rs
namespace  libretranslate
namespace  lingva
namespace  livespace
namespace  loc
namespace  mariadb_server
namespace  mastodon
namespace  material_icons
namespace  mediathekviewweb
namespace  mediawiki
namespace  meilisearch
namespace  metacpan
namespace  microsoft_learn
namespace  mixcloud
namespace  mojeek
namespace  mongodb
namespace  moviepilot
namespace  mozhi
namespace  mrs
namespace  mullvad_leta
namespace  mwmbl
namespace  mysql_server
namespace  naver
namespace  niconico
namespace  npm
namespace  nyaa
namespace  odysee
namespace  ollama
namespace  open_meteo
namespace  openclipart
namespace  openlibrary
namespace  opensemantic
namespace  openstreetmap
namespace  openverse
namespace  pdbe
namespace  peertube
namespace  photon
namespace  pinterest
namespace  piped
namespace  piratebay
namespace  pixiv
namespace  pkg_go_dev
namespace  podcastindex
namespace  postgresql
namespace  presearch
namespace  public_domain_image_archive
namespace  pubmed
namespace  pypi
namespace  quark
namespace  qwant
namespace  radio_browser
namespace  recoll
namespace  reddit
namespace  redis_server
namespace  reuters
namespace  rottentomatoes
namespace  rumble
namespace  scanr_structures
namespace  searchcode_code
namespace  searx_engine
namespace  seekr
namespace  selfhst
namespace  semantic_scholar
namespace  senscritique
namespace  sepiasearch
namespace  seznam
namespace  sogou
namespace  sogou_images
namespace  sogou_videos
namespace  sogou_wechat
namespace  solidtorrents
namespace  solr
namespace  soundcloud
namespace  spotify
namespace  springer
namespace  sqlite
namespace  stackexchange
namespace  startpage
namespace  steam
namespace  stract
namespace  svgrepo
namespace  tagesschau
namespace  tineye
namespace  tokyotoshokan
namespace  tootfinder
namespace  torznab
namespace  translated
namespace  tubearchivist
namespace  unsplash
namespace  uxwing
namespace  vimeo
namespace  voidlinux
namespace  wallhaven
namespace  wikicommons
namespace  wikidata
namespace  wikipedia
namespace  wolframalpha_api
namespace  wolframalpha_noapi
namespace  wordnik
namespace  wttr
namespace  www1x
namespace  xpath
namespace  yacy
namespace  yahoo
namespace  yahoo_news
namespace  yandex
namespace  yandex_music
namespace  yep
namespace  youtube_api
namespace  youtube_noapi
namespace  yummly
namespace  zlibrary

Functions

 check_engine_module (types.ModuleType module)
Engine|types.ModuleType|None load_engine (dict engine_data)
 set_loggers (engine, engine_name)
 update_engine_attributes (Engine|types.ModuleType engine, engine_data)
 update_attributes_for_tor (Engine|types.ModuleType engine)
 is_missing_required_attributes (engine)
 using_tor_proxy (Engine|types.ModuleType engine)
 is_engine_active (Engine|types.ModuleType engine)
 register_engine (Engine|types.ModuleType engine)
 load_engines (engine_list)

Variables

 logger = logger.getChild('engines')
 ENGINE_DIR = dirname(realpath(__file__))
dict ENGINE_DEFAULT_ARGS
str DEFAULT_CATEGORY = 'other'
dict categories = {'general': []}
dict engines = {}
dict engine_shortcuts = {}

Detailed Description

::1337x

::9gag

::360search_videos

::360search

1337x
360Search search engine for searxng
360Search-Videos: A search engine for retrieving videos from 360Search.
9GAG (social media)
Load and initialize the ``engines``, see :py:func:`load_engines` and register
:py:obj:`engine_shortcuts`.

usage::

    load_engines( settings['engines'] )

Function Documentation

◆ check_engine_module()

searx.engines.check_engine_module ( types.ModuleType module)

Definition at line 68 of file __init__.py.

68def check_engine_module(module: types.ModuleType):
69 # probe unintentional name collisions / for example name collisions caused
70 # by import statements in the engine module ..
71
72 # network: https://github.com/searxng/searxng/issues/762#issuecomment-1605323861
73 obj = getattr(module, 'network', None)
74 if obj and inspect.ismodule(obj):
75 msg = f'type of {module.__name__}.network is a module ({obj.__name__}), expected a string'
76 # logger.error(msg)
77 raise TypeError(msg)
78
79

Referenced by load_engine().

Here is the caller graph for this function:

◆ is_engine_active()

searx.engines.is_engine_active ( Engine | types.ModuleType engine)

Definition at line 216 of file __init__.py.

216def is_engine_active(engine: Engine | types.ModuleType):
217 # check if engine is inactive
218 if engine.inactive is True:
219 return False
220
221 # exclude onion engines if not using tor
222 if 'onions' in engine.categories and not using_tor_proxy(engine):
223 return False
224
225 return True
226
227

References using_tor_proxy().

Referenced by load_engine().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ is_missing_required_attributes()

searx.engines.is_missing_required_attributes ( engine)
An attribute is required when its name doesn't start with ``_`` (underline).
Required attributes must not be ``None``.

Definition at line 198 of file __init__.py.

198def is_missing_required_attributes(engine):
199 """An attribute is required when its name doesn't start with ``_`` (underline).
200 Required attributes must not be ``None``.
201
202 """
203 missing = False
204 for engine_attr in dir(engine):
205 if not engine_attr.startswith('_') and getattr(engine, engine_attr) is None:
206 logger.error('Missing engine config attribute: "{0}.{1}"'.format(engine.name, engine_attr))
207 missing = True
208 return missing
209
210

Referenced by load_engine().

Here is the caller graph for this function:

◆ load_engine()

Engine | types.ModuleType | None searx.engines.load_engine ( dict engine_data)
Load engine from ``engine_data``.

:param dict engine_data:  Attributes from YAML ``settings:engines/<engine>``
:return: initialized namespace of the ``<engine>``.

1. create a namespace and load module of the ``<engine>``
2. update namespace with the defaults from :py:obj:`ENGINE_DEFAULT_ARGS`
3. update namespace with values from ``engine_data``

If engine *is active*, return namespace of the engine, otherwise return
``None``.

This function also returns ``None`` if initialization of the namespace fails
for one of the following reasons:

- engine name contains underscore
- engine name is not lowercase
- required attribute is not set :py:func:`is_missing_required_attributes`

Definition at line 80 of file __init__.py.

80def load_engine(engine_data: dict) -> Engine | types.ModuleType | None:
81 """Load engine from ``engine_data``.
82
83 :param dict engine_data: Attributes from YAML ``settings:engines/<engine>``
84 :return: initialized namespace of the ``<engine>``.
85
86 1. create a namespace and load module of the ``<engine>``
87 2. update namespace with the defaults from :py:obj:`ENGINE_DEFAULT_ARGS`
88 3. update namespace with values from ``engine_data``
89
90 If engine *is active*, return namespace of the engine, otherwise return
91 ``None``.
92
93 This function also returns ``None`` if initialization of the namespace fails
94 for one of the following reasons:
95
96 - engine name contains underscore
97 - engine name is not lowercase
98 - required attribute is not set :py:func:`is_missing_required_attributes`
99
100 """
101 # pylint: disable=too-many-return-statements
102
103 engine_name = engine_data.get('name')
104 if engine_name is None:
105 logger.error('An engine does not have a "name" field')
106 return None
107 if '_' in engine_name:
108 logger.error('Engine name contains underscore: "{}"'.format(engine_name))
109 return None
110
111 if engine_name.lower() != engine_name:
112 logger.warning('Engine name is not lowercase: "{}", converting to lowercase'.format(engine_name))
113 engine_name = engine_name.lower()
114 engine_data['name'] = engine_name
115
116 # load_module
117 module_name = engine_data.get('engine')
118 if module_name is None:
119 logger.error('The "engine" field is missing for the engine named "{}"'.format(engine_name))
120 return None
121 try:
122 engine = load_module(module_name + '.py', ENGINE_DIR)
123 except (SyntaxError, KeyboardInterrupt, SystemExit, SystemError, ImportError, RuntimeError):
124 logger.exception('Fatal exception in engine "{}"'.format(module_name))
125 sys.exit(1)
126 except BaseException:
127 logger.exception('Cannot load engine "{}"'.format(module_name))
128 return None
129
130 check_engine_module(engine)
131 update_engine_attributes(engine, engine_data)
132 update_attributes_for_tor(engine)
133
134 # avoid cyclic imports
135 # pylint: disable=import-outside-toplevel
136 from searx.enginelib.traits import EngineTraitsMap
137
138 trait_map = EngineTraitsMap.from_data()
139 trait_map.set_traits(engine)
140
141 if not is_engine_active(engine):
142 return None
143
144 if is_missing_required_attributes(engine):
145 return None
146
147 set_loggers(engine, engine_name)
148
149 if not any(cat in settings['categories_as_tabs'] for cat in engine.categories):
150 engine.categories.append(DEFAULT_CATEGORY)
151
152 return engine
153
154

References check_engine_module(), is_engine_active(), is_missing_required_attributes(), set_loggers(), update_attributes_for_tor(), and update_engine_attributes().

Referenced by load_engines().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ load_engines()

searx.engines.load_engines ( engine_list)
usage: ``engine_list = settings['engines']``

Definition at line 243 of file __init__.py.

243def load_engines(engine_list):
244 """usage: ``engine_list = settings['engines']``"""
245 engines.clear()
246 engine_shortcuts.clear()
247 categories.clear()
248 categories['general'] = []
249 for engine_data in engine_list:
250 engine = load_engine(engine_data)
251 if engine:
252 register_engine(engine)
253 return engines

References load_engine(), and register_engine().

Here is the call graph for this function:

◆ register_engine()

searx.engines.register_engine ( Engine | types.ModuleType engine)

Definition at line 228 of file __init__.py.

228def register_engine(engine: Engine | types.ModuleType):
229 if engine.name in engines:
230 logger.error('Engine config error: ambiguous name: {0}'.format(engine.name))
231 sys.exit(1)
232 engines[engine.name] = engine
233
234 if engine.shortcut in engine_shortcuts:
235 logger.error('Engine config error: ambiguous shortcut: {0}'.format(engine.shortcut))
236 sys.exit(1)
237 engine_shortcuts[engine.shortcut] = engine.name
238
239 for category_name in engine.categories:
240 categories.setdefault(category_name, []).append(engine)
241
242

Referenced by load_engines().

Here is the caller graph for this function:

◆ set_loggers()

searx.engines.set_loggers ( engine,
engine_name )

Definition at line 155 of file __init__.py.

155def set_loggers(engine, engine_name):
156 # set the logger for engine
157 engine.logger = logger.getChild(engine_name)
158 # the engine may have load some other engines
159 # may sure the logger is initialized
160 # use sys.modules.copy() to avoid "RuntimeError: dictionary changed size during iteration"
161 # see https://github.com/python/cpython/issues/89516
162 # and https://docs.python.org/3.10/library/sys.html#sys.modules
163 modules = sys.modules.copy()
164 for module_name, module in modules.items():
165 if (
166 module_name.startswith("searx.engines")
167 and module_name != "searx.engines.__init__"
168 and not hasattr(module, "logger")
169 ):
170 module_engine_name = module_name.split(".")[-1]
171 module.logger = logger.getChild(module_engine_name) # type: ignore
172
173

Referenced by load_engine().

Here is the caller graph for this function:

◆ update_attributes_for_tor()

searx.engines.update_attributes_for_tor ( Engine | types.ModuleType engine)

Definition at line 192 of file __init__.py.

192def update_attributes_for_tor(engine: Engine | types.ModuleType):
193 if using_tor_proxy(engine) and hasattr(engine, 'onion_url'):
194 engine.search_url = engine.onion_url + getattr(engine, 'search_path', '') # type: ignore
195 engine.timeout += settings['outgoing'].get('extra_proxy_timeout', 0) # type: ignore
196
197

References using_tor_proxy().

Referenced by load_engine().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ update_engine_attributes()

searx.engines.update_engine_attributes ( Engine | types.ModuleType engine,
engine_data )

Definition at line 174 of file __init__.py.

174def update_engine_attributes(engine: Engine | types.ModuleType, engine_data):
175 # set engine attributes from engine_data
176 for param_name, param_value in engine_data.items():
177 if param_name == 'categories':
178 if isinstance(param_value, str):
179 param_value = list(map(str.strip, param_value.split(',')))
180 engine.categories = param_value # type: ignore
181 elif hasattr(engine, 'about') and param_name == 'about':
182 engine.about = {**engine.about, **engine_data['about']} # type: ignore
183 else:
184 setattr(engine, param_name, param_value)
185
186 # set default attributes
187 for arg_name, arg_value in ENGINE_DEFAULT_ARGS.items():
188 if not hasattr(engine, arg_name):
189 setattr(engine, arg_name, copy.deepcopy(arg_value))
190
191

Referenced by load_engine().

Here is the caller graph for this function:

◆ using_tor_proxy()

searx.engines.using_tor_proxy ( Engine | types.ModuleType engine)
Return True if the engine configuration declares to use Tor.

Definition at line 211 of file __init__.py.

211def using_tor_proxy(engine: Engine | types.ModuleType):
212 """Return True if the engine configuration declares to use Tor."""
213 return settings['outgoing'].get('using_tor_proxy') or getattr(engine, 'using_tor_proxy', False)
214
215

Referenced by is_engine_active(), and update_attributes_for_tor().

Here is the caller graph for this function:

Variable Documentation

◆ categories

dict searx.engines.categories = {'general': []}

Definition at line 55 of file __init__.py.

◆ DEFAULT_CATEGORY

str searx.engines.DEFAULT_CATEGORY = 'other'

Definition at line 50 of file __init__.py.

◆ ENGINE_DEFAULT_ARGS

dict searx.engines.ENGINE_DEFAULT_ARGS
Initial value:
1= {
2 # Common options in the engine module
3 "engine_type": "online",
4 "paging": False,
5 "time_range_support": False,
6 "safesearch": False,
7 # settings.yml
8 "categories": ["general"],
9 "enable_http": False,
10 "shortcut": "-",
11 "timeout": settings["outgoing"]["request_timeout"],
12 "display_error_messages": True,
13 "disabled": False,
14 "inactive": False,
15 "about": {},
16 "using_tor_proxy": False,
17 "send_accept_language_header": False,
18 "tokens": [],
19 "max_page": 0,
20}

Definition at line 29 of file __init__.py.

◆ ENGINE_DIR

searx.engines.ENGINE_DIR = dirname(realpath(__file__))

Definition at line 28 of file __init__.py.

◆ engine_shortcuts

dict searx.engines.engine_shortcuts = {}

Definition at line 57 of file __init__.py.

◆ engines

dict searx.engines.engines = {}

Definition at line 56 of file __init__.py.

◆ logger

searx.engines.logger = logger.getChild('engines')

Definition at line 27 of file __init__.py.