.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.results.ResultContainer Class Reference
Collaboration diagram for searx.results.ResultContainer:

Public Member Functions

 __init__ (self)
 extend (self, str|None engine_name, list[Result|LegacyResult] results)
 close (self)
list[MainResult|LegacyResultget_ordered_results (self)
int number_of_results (self)
 add_unresponsive_engine (self, str engine_name, str error_type, bool suspended=False)
 add_timing (self, str engine_name, float engine_time, float page_load_time)
list[Timingget_timings (self)

Public Attributes

 answers = AnswerSet()
dict[str, dict[str, str]] engine_data = defaultdict(dict)
bool paging = False
set[UnresponsiveEngineunresponsive_engines = set()
list timings = []
str|None redirect_url = None
t.Callable[[Result|LegacyResult], bool] on_result = lambda _: True

Static Public Attributes

dict main_results_map [int, MainResult | LegacyResult]
list infoboxes [LegacyResult]
set suggestions [str]
set corrections [str]

Protected Member Functions

 _merge_infobox (self, LegacyResult new_infobox)
 _merge_main_result (self, MainResult|LegacyResult result, int position)

Protected Attributes

list _number_of_results = []
bool _closed = False
RLock _lock = RLock()
list[MainResult|LegacyResult_main_results_sorted = None

Detailed Description

In the result container, the results are collected, sorted and duplicates
will be merged.

Definition at line 53 of file results.py.

Constructor & Destructor Documentation

◆ __init__()

searx.results.ResultContainer.__init__ ( self)

Definition at line 65 of file results.py.

65 def __init__(self):
66 self.main_results_map = {}
67 self.infoboxes = []
68 self.suggestions = set()
69 self.answers = AnswerSet()
70 self.corrections = set()
71
72 self._number_of_results: list[int] = []
73 self.engine_data: dict[str, dict[str, str]] = defaultdict(dict)
74 self._closed: bool = False
75 self.paging: bool = False
76 self.unresponsive_engines: set[UnresponsiveEngine] = set()
77 self.timings: list[Timing] = []
78 self.redirect_url: str | None = None
79 self.on_result: t.Callable[[Result | LegacyResult], bool] = lambda _: True
80 self._lock: RLock = RLock()
81 self._main_results_sorted: list[MainResult | LegacyResult] = None # type: ignore
82

References infoboxes, main_results_map, and suggestions.

Member Function Documentation

◆ _merge_infobox()

searx.results.ResultContainer._merge_infobox ( self,
LegacyResult new_infobox )
protected

Definition at line 160 of file results.py.

160 def _merge_infobox(self, new_infobox: LegacyResult):
161 add_infobox = True
162
163 new_id = getattr(new_infobox, "id", None)
164 if new_id is not None:
165 with self._lock:
166 for existing_infobox in self.infoboxes:
167 if new_id == getattr(existing_infobox, "id", None):
168 merge_two_infoboxes(existing_infobox, new_infobox)
169 add_infobox = False
170 if add_infobox:
171 self.infoboxes.append(new_infobox)
172

References searx.metrics.models.Histogram._lock, _lock, infoboxes, and searx.results.merge_two_infoboxes().

Referenced by extend().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ _merge_main_result()

searx.results.ResultContainer._merge_main_result ( self,
MainResult | LegacyResult result,
int position )
protected

Definition at line 173 of file results.py.

173 def _merge_main_result(self, result: MainResult | LegacyResult, position: int):
174 result_hash = hash(result)
175
176 with self._lock:
177
178 merged = self.main_results_map.get(result_hash)
179 if not merged:
180 # if there is no duplicate in the merged results, append result
181 result.positions = [position]
182 self.main_results_map[result_hash] = result
183 return
184
185 merge_two_main_results(merged, result)
186 # add the new position
187 merged.positions.append(position)
188

References searx.metrics.models.Histogram._lock, _lock, main_results_map, and searx.results.merge_two_main_results().

Referenced by extend().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ add_timing()

searx.results.ResultContainer.add_timing ( self,
str engine_name,
float engine_time,
float page_load_time )

Definition at line 282 of file results.py.

282 def add_timing(self, engine_name: str, engine_time: float, page_load_time: float):
283 with self._lock:
284 if self._closed:
285 log.error("call to ResultContainer.add_timing after ResultContainer.close")
286 return
287 self.timings.append(Timing(engine_name, total=engine_time, load=page_load_time))
288

References _closed, searx.metrics.models.Histogram._lock, _lock, searx.extended_types.SXNG_Request.timings, and timings.

◆ add_unresponsive_engine()

searx.results.ResultContainer.add_unresponsive_engine ( self,
str engine_name,
str error_type,
bool suspended = False )

Definition at line 274 of file results.py.

274 def add_unresponsive_engine(self, engine_name: str, error_type: str, suspended: bool = False):
275 with self._lock:
276 if self._closed:
277 log.error("call to ResultContainer.add_unresponsive_engine after ResultContainer.close")
278 return
279 if searx.engines.engines[engine_name].display_error_messages:
280 self.unresponsive_engines.add(UnresponsiveEngine(engine_name, error_type, suspended))
281

References _closed, searx.metrics.models.Histogram._lock, _lock, and unresponsive_engines.

◆ close()

searx.results.ResultContainer.close ( self)

Definition at line 189 of file results.py.

189 def close(self):
190 self._closed = True
191
192 for result in self.main_results_map.values():
193 result.score = calculate_score(result, result.priority)
194 for eng_name in result.engines:
195 counter_add(result.score, 'engine', eng_name, 'score')
196

References _closed, searx.results.calculate_score(), and main_results_map.

Referenced by get_ordered_results().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ extend()

searx.results.ResultContainer.extend ( self,
str | None engine_name,
list[Result | LegacyResult] results )

Definition at line 83 of file results.py.

85 ): # pylint: disable=too-many-branches
86 if self._closed:
87 log.debug("container is closed, ignoring results: %s", results)
88 return
89 main_count = 0
90
91 for result in list(results):
92
93 if isinstance(result, Result):
94 result.engine = result.engine or engine_name
95 result.normalize_result_fields()
96 if not self.on_result(result):
97 continue
98
99 if isinstance(result, BaseAnswer):
100 self.answers.add(result)
101 elif isinstance(result, MainResult):
102 main_count += 1
103 self._merge_main_result(result, main_count)
104 else:
105 # more types need to be implemented in the future ..
106 raise NotImplementedError(f"no handler implemented to process the result of type {result}")
107
108 else:
109 result["engine"] = result.get("engine") or engine_name or ""
110 result = LegacyResult(result) # for backward compatibility, will be romeved one day
111 result.normalize_result_fields()
112
113 if "suggestion" in result:
114 if self.on_result(result):
115 self.suggestions.add(result["suggestion"])
116 continue
117
118 if "answer" in result:
119 if self.on_result(result):
120 warnings.warn(
121 f"answer results from engine {result.engine}"
122 " are without typification / migrate to Answer class.",
123 DeprecationWarning,
124 )
125 self.answers.add(result) # type: ignore
126 continue
127
128 if "correction" in result:
129 if self.on_result(result):
130 self.corrections.add(result["correction"])
131 continue
132
133 if "infobox" in result:
134 if self.on_result(result):
135 self._merge_infobox(result)
136 continue
137
138 if "number_of_results" in result:
139 if self.on_result(result):
140 self._number_of_results.append(result["number_of_results"])
141 continue
142
143 if "engine_data" in result:
144 if self.on_result(result):
145 if result.engine:
146 self.engine_data[result.engine][result["key"]] = result["engine_data"]
147 continue
148
149 if self.on_result(result):
150 main_count += 1
151 self._merge_main_result(result, main_count)
152 continue
153
154 if engine_name in searx.engines.engines:
155 eng = searx.engines.engines[engine_name]
156 histogram_observe(main_count, "engine", eng.name, "result", "count")
157 if not self.paging and eng.paging:
158 self.paging = True
159

References _closed, _merge_infobox(), _merge_main_result(), _number_of_results, answers, corrections, engine_data, on_result, paging, and suggestions.

Here is the call graph for this function:

◆ get_ordered_results()

list[MainResult | LegacyResult] searx.results.ResultContainer.get_ordered_results ( self)
Returns a sorted list of results to be displayed in the main result
area (:ref:`result types`).

Definition at line 197 of file results.py.

197 def get_ordered_results(self) -> list[MainResult | LegacyResult]:
198 """Returns a sorted list of results to be displayed in the main result
199 area (:ref:`result types`)."""
200
201 if not self._closed:
202 self.close()
203
204 if self._main_results_sorted:
205 return self._main_results_sorted
206
207 # first pass, sort results by "score" (descanding)
208 results = sorted(self.main_results_map.values(), key=lambda x: x.score, reverse=True)
209
210 # pass 2 : group results by category and template
211 gresults: list[MainResult | LegacyResult] = []
212 categoryPositions: dict[str, t.Any] = {}
213 max_count = 8
214 max_distance = 20
215
216 for res in results:
217 # do we need to handle more than one category per engine?
218 engine = searx.engines.engines.get(res.engine or "")
219 if engine:
220 res.category = engine.categories[0] if len(engine.categories) > 0 else ""
221
222 # do we need to handle more than one category per engine?
223 category = f"{res.category}:{res.template}:{'img_src' if (res.thumbnail or res.img_src) else ''}"
224 grp = categoryPositions.get(category)
225
226 # group with previous results using the same category, if the group
227 # can accept more result and is not too far from the current
228 # position
229
230 if (grp is not None) and (grp["count"] > 0) and (len(gresults) - grp["index"] < max_distance):
231 # group with the previous results using the same category with
232 # this one
233 index = grp["index"]
234 gresults.insert(index, res)
235
236 # update every index after the current one (including the
237 # current one)
238 for item in categoryPositions.values():
239 v = item["index"]
240 if v >= index:
241 item["index"] = v + 1
242
243 # update this category
244 grp["count"] -= 1
245
246 else:
247 gresults.append(res)
248 # update categoryIndex
249 categoryPositions[category] = {"index": len(gresults), "count": max_count}
250 continue
251
252 self._main_results_sorted = gresults
253 return self._main_results_sorted
254

References _closed, _main_results_sorted, close(), and main_results_map.

Referenced by number_of_results().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ get_timings()

list[Timing] searx.results.ResultContainer.get_timings ( self)

Definition at line 289 of file results.py.

289 def get_timings(self) -> list[Timing]:
290 with self._lock:
291 if not self._closed:
292 log.error("call to ResultContainer.get_timings before ResultContainer.close")
293 return []
294 return self.timings
295
296

References _closed, searx.metrics.models.Histogram._lock, _lock, searx.extended_types.SXNG_Request.timings, and timings.

◆ number_of_results()

int searx.results.ResultContainer.number_of_results ( self)
Returns the average of results number, returns zero if the average
result number is smaller than the actual result count.

Definition at line 256 of file results.py.

256 def number_of_results(self) -> int:
257 """Returns the average of results number, returns zero if the average
258 result number is smaller than the actual result count."""
259
260 if not self._closed:
261 log.error("call to ResultContainer.number_of_results before ResultContainer.close")
262 return 0
263
264 with self._lock:
265 resultnum_sum = sum(self._number_of_results)
266 if not resultnum_sum or not self._number_of_results:
267 return 0
268
269 average = int(resultnum_sum / len(self._number_of_results))
270 if average < len(self.get_ordered_results()):
271 average = 0
272 return average
273

References _closed, searx.metrics.models.Histogram._lock, _lock, _number_of_results, and get_ordered_results().

Here is the call graph for this function:

Member Data Documentation

◆ _closed

bool searx.results.ResultContainer._closed = False
protected

◆ _lock

RLock searx.results.ResultContainer._lock = RLock()
protected

◆ _main_results_sorted

list[MainResult | LegacyResult] searx.results.ResultContainer._main_results_sorted = None
protected

Definition at line 81 of file results.py.

Referenced by get_ordered_results().

◆ _number_of_results

list searx.results.ResultContainer._number_of_results = []
protected

Definition at line 72 of file results.py.

Referenced by extend(), and number_of_results().

◆ answers

searx.results.ResultContainer.answers = AnswerSet()

Definition at line 69 of file results.py.

Referenced by extend().

◆ corrections

set searx.results.ResultContainer.corrections [str]
static

Definition at line 63 of file results.py.

Referenced by extend().

◆ engine_data

dict[str, dict[str, str]] searx.results.ResultContainer.engine_data = defaultdict(dict)

Definition at line 73 of file results.py.

Referenced by searx.search.models.SearchQuery.__copy__(), and extend().

◆ infoboxes

list searx.results.ResultContainer.infoboxes [LegacyResult]
static

Definition at line 60 of file results.py.

Referenced by __init__(), and _merge_infobox().

◆ main_results_map

dict searx.results.ResultContainer.main_results_map [int, MainResult | LegacyResult]
static

Definition at line 59 of file results.py.

Referenced by __init__(), _merge_main_result(), close(), and get_ordered_results().

◆ on_result

t.Callable[[Result | LegacyResult], bool] searx.results.ResultContainer.on_result = lambda _: True

Definition at line 79 of file results.py.

Referenced by extend().

◆ paging

bool searx.results.ResultContainer.paging = False

Definition at line 75 of file results.py.

Referenced by extend().

◆ redirect_url

str | None searx.results.ResultContainer.redirect_url = None

Definition at line 78 of file results.py.

◆ suggestions

set searx.results.ResultContainer.suggestions [str]
static

Definition at line 61 of file results.py.

Referenced by __init__(), and extend().

◆ timings

list searx.results.ResultContainer.timings = []

Definition at line 77 of file results.py.

Referenced by add_timing(), and get_timings().

◆ unresponsive_engines

set[UnresponsiveEngine] searx.results.ResultContainer.unresponsive_engines = set()

Definition at line 76 of file results.py.

Referenced by add_unresponsive_engine().


The documentation for this class was generated from the following file:
  • /home/andrew/Documents/code/public/searxng/searx/results.py