.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.results.ResultContainer Class Reference
Collaboration diagram for searx.results.ResultContainer:

Public Member Functions

 __init__ (self)
 extend (self, str|None engine_name, results)
 close (self)
list[MainResult|LegacyResult] get_ordered_results (self)
int number_of_results (self)
 add_unresponsive_engine (self, str engine_name, str error_type, bool suspended=False)
 add_timing (self, str engine_name, float engine_time, float page_load_time)
 get_timings (self)

Public Attributes

 answers = AnswerSet()
dict[str, dict[str, str]] engine_data = defaultdict(dict)
bool paging = False
Set[UnresponsiveEngine] unresponsive_engines = set()
list timings = []
str|None redirect_url = None
bool on_result = lambda _: True

Static Public Attributes

dict main_results_map [int, MainResult | LegacyResult]
list infoboxes [LegacyResult]
set suggestions [str]
set corrections [str]

Protected Member Functions

 _merge_infobox (self, LegacyResult new_infobox)
 _merge_main_result (self, MainResult|LegacyResult result, position)

Protected Attributes

list _number_of_results = []
bool _closed = False
 _lock = RLock()
list[MainResult|LegacyResult] _main_results_sorted = None

Detailed Description

In the result container, the results are collected, sorted and duplicates
will be merged.

Definition at line 50 of file results.py.

Constructor & Destructor Documentation

◆ __init__()

searx.results.ResultContainer.__init__ ( self)

Definition at line 62 of file results.py.

62 def __init__(self):
63 self.main_results_map = {}
64 self.infoboxes = []
65 self.suggestions = set()
66 self.answers = AnswerSet()
67 self.corrections = set()
68
69 self._number_of_results: list[int] = []
70 self.engine_data: dict[str, dict[str, str]] = defaultdict(dict)
71 self._closed: bool = False
72 self.paging: bool = False
73 self.unresponsive_engines: Set[UnresponsiveEngine] = set()
74 self.timings: List[Timing] = []
75 self.redirect_url: str | None = None
76 self.on_result = lambda _: True
77 self._lock = RLock()
78 self._main_results_sorted: list[MainResult | LegacyResult] = None # type: ignore
79

References infoboxes, main_results_map, and suggestions.

Member Function Documentation

◆ _merge_infobox()

searx.results.ResultContainer._merge_infobox ( self,
LegacyResult new_infobox )
protected

Definition at line 155 of file results.py.

155 def _merge_infobox(self, new_infobox: LegacyResult):
156 add_infobox = True
157
158 new_id = getattr(new_infobox, "id", None)
159 if new_id is not None:
160 with self._lock:
161 for existing_infobox in self.infoboxes:
162 if new_id == getattr(existing_infobox, "id", None):
163 merge_two_infoboxes(existing_infobox, new_infobox)
164 add_infobox = False
165 if add_infobox:
166 self.infoboxes.append(new_infobox)
167

References searx.metrics.models.Histogram._lock, _lock, infoboxes, and searx.results.merge_two_infoboxes().

Referenced by extend().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ _merge_main_result()

searx.results.ResultContainer._merge_main_result ( self,
MainResult | LegacyResult result,
position )
protected

Definition at line 168 of file results.py.

168 def _merge_main_result(self, result: MainResult | LegacyResult, position):
169 result_hash = hash(result)
170
171 with self._lock:
172
173 merged = self.main_results_map.get(result_hash)
174 if not merged:
175 # if there is no duplicate in the merged results, append result
176 result.positions = [position]
177 self.main_results_map[result_hash] = result
178 return
179
180 merge_two_main_results(merged, result)
181 # add the new position
182 merged.positions.append(position)
183

References searx.metrics.models.Histogram._lock, _lock, main_results_map, and searx.results.merge_two_main_results().

Referenced by extend().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ add_timing()

searx.results.ResultContainer.add_timing ( self,
str engine_name,
float engine_time,
float page_load_time )

Definition at line 277 of file results.py.

277 def add_timing(self, engine_name: str, engine_time: float, page_load_time: float):
278 with self._lock:
279 if self._closed:
280 log.error("call to ResultContainer.add_timing after ResultContainer.close")
281 return
282 self.timings.append(Timing(engine_name, total=engine_time, load=page_load_time))
283

References _closed, searx.metrics.models.Histogram._lock, _lock, searx.extended_types.SXNG_Request.timings, and timings.

◆ add_unresponsive_engine()

searx.results.ResultContainer.add_unresponsive_engine ( self,
str engine_name,
str error_type,
bool suspended = False )

Definition at line 269 of file results.py.

269 def add_unresponsive_engine(self, engine_name: str, error_type: str, suspended: bool = False):
270 with self._lock:
271 if self._closed:
272 log.error("call to ResultContainer.add_unresponsive_engine after ResultContainer.close")
273 return
274 if searx.engines.engines[engine_name].display_error_messages:
275 self.unresponsive_engines.add(UnresponsiveEngine(engine_name, error_type, suspended))
276

References _closed, searx.metrics.models.Histogram._lock, _lock, and unresponsive_engines.

◆ close()

searx.results.ResultContainer.close ( self)

Definition at line 184 of file results.py.

184 def close(self):
185 self._closed = True
186
187 for result in self.main_results_map.values():
188 result.score = calculate_score(result, result.priority)
189 for eng_name in result.engines:
190 counter_add(result.score, 'engine', eng_name, 'score')
191

References _closed, searx.results.calculate_score(), and main_results_map.

Referenced by get_ordered_results().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ extend()

searx.results.ResultContainer.extend ( self,
str | None engine_name,
results )

Definition at line 80 of file results.py.

80 def extend(self, engine_name: str | None, results): # pylint: disable=too-many-branches
81 if self._closed:
82 log.debug("container is closed, ignoring results: %s", results)
83 return
84 main_count = 0
85
86 for result in list(results):
87
88 if isinstance(result, Result):
89 result.engine = result.engine or engine_name
90 result.normalize_result_fields()
91 if not self.on_result(result):
92 continue
93
94 if isinstance(result, BaseAnswer):
95 self.answers.add(result)
96 elif isinstance(result, MainResult):
97 main_count += 1
98 self._merge_main_result(result, main_count)
99 else:
100 # more types need to be implemented in the future ..
101 raise NotImplementedError(f"no handler implemented to process the result of type {result}")
102
103 else:
104 result["engine"] = result.get("engine") or engine_name or ""
105 result = LegacyResult(result) # for backward compatibility, will be romeved one day
106 result.normalize_result_fields()
107
108 if "suggestion" in result:
109 if self.on_result(result):
110 self.suggestions.add(result["suggestion"])
111 continue
112
113 if "answer" in result:
114 if self.on_result(result):
115 warnings.warn(
116 f"answer results from engine {result.engine}"
117 " are without typification / migrate to Answer class.",
118 DeprecationWarning,
119 )
120 self.answers.add(result) # type: ignore
121 continue
122
123 if "correction" in result:
124 if self.on_result(result):
125 self.corrections.add(result["correction"])
126 continue
127
128 if "infobox" in result:
129 if self.on_result(result):
130 self._merge_infobox(result)
131 continue
132
133 if "number_of_results" in result:
134 if self.on_result(result):
135 self._number_of_results.append(result["number_of_results"])
136 continue
137
138 if "engine_data" in result:
139 if self.on_result(result):
140 if result.engine:
141 self.engine_data[result.engine][result["key"]] = result["engine_data"]
142 continue
143
144 if self.on_result(result):
145 main_count += 1
146 self._merge_main_result(result, main_count)
147 continue
148
149 if engine_name in searx.engines.engines:
150 eng = searx.engines.engines[engine_name]
151 histogram_observe(main_count, "engine", eng.name, "result", "count")
152 if not self.paging and eng.paging:
153 self.paging = True
154

References _closed, _merge_infobox(), _merge_main_result(), _number_of_results, answers, corrections, engine_data, on_result, paging, and suggestions.

Here is the call graph for this function:

◆ get_ordered_results()

list[MainResult | LegacyResult] searx.results.ResultContainer.get_ordered_results ( self) 
Returns a sorted list of results to be displayed in the main result
area (:ref:`result types`).

Definition at line 192 of file results.py.

192 def get_ordered_results(self) -> list[MainResult | LegacyResult]:
193 """Returns a sorted list of results to be displayed in the main result
194 area (:ref:`result types`)."""
195
196 if not self._closed:
197 self.close()
198
199 if self._main_results_sorted:
200 return self._main_results_sorted
201
202 # first pass, sort results by "score" (descanding)
203 results = sorted(self.main_results_map.values(), key=lambda x: x.score, reverse=True)
204
205 # pass 2 : group results by category and template
206 gresults = []
207 categoryPositions = {}
208 max_count = 8
209 max_distance = 20
210
211 for res in results:
212 # do we need to handle more than one category per engine?
213 engine = searx.engines.engines.get(res.engine or "")
214 if engine:
215 res.category = engine.categories[0] if len(engine.categories) > 0 else ""
216
217 # do we need to handle more than one category per engine?
218 category = f"{res.category}:{res.template}:{'img_src' if (res.thumbnail or res.img_src) else ''}"
219 grp = categoryPositions.get(category)
220
221 # group with previous results using the same category, if the group
222 # can accept more result and is not too far from the current
223 # position
224
225 if (grp is not None) and (grp["count"] > 0) and (len(gresults) - grp["index"] < max_distance):
226 # group with the previous results using the same category with
227 # this one
228 index = grp["index"]
229 gresults.insert(index, res)
230
231 # update every index after the current one (including the
232 # current one)
233 for item in categoryPositions.values():
234 v = item["index"]
235 if v >= index:
236 item["index"] = v + 1
237
238 # update this category
239 grp["count"] -= 1
240
241 else:
242 gresults.append(res)
243 # update categoryIndex
244 categoryPositions[category] = {"index": len(gresults), "count": max_count}
245 continue
246
247 self._main_results_sorted = gresults
248 return self._main_results_sorted
249

References _closed, _main_results_sorted, close(), and main_results_map.

Referenced by number_of_results().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ get_timings()

searx.results.ResultContainer.get_timings ( self)

Definition at line 284 of file results.py.

284 def get_timings(self):
285 with self._lock:
286 if not self._closed:
287 log.error("call to ResultContainer.get_timings before ResultContainer.close")
288 return []
289 return self.timings
290
291

References _closed, searx.metrics.models.Histogram._lock, _lock, searx.extended_types.SXNG_Request.timings, and timings.

◆ number_of_results()

int searx.results.ResultContainer.number_of_results ( self) 
Returns the average of results number, returns zero if the average
result number is smaller than the actual result count.

Definition at line 251 of file results.py.

251 def number_of_results(self) -> int:
252 """Returns the average of results number, returns zero if the average
253 result number is smaller than the actual result count."""
254
255 if not self._closed:
256 log.error("call to ResultContainer.number_of_results before ResultContainer.close")
257 return 0
258
259 with self._lock:
260 resultnum_sum = sum(self._number_of_results)
261 if not resultnum_sum or not self._number_of_results:
262 return 0
263
264 average = int(resultnum_sum / len(self._number_of_results))
265 if average < len(self.get_ordered_results()):
266 average = 0
267 return average
268

References _closed, searx.metrics.models.Histogram._lock, _lock, _number_of_results, and get_ordered_results().

Here is the call graph for this function:

Member Data Documentation

◆ _closed

bool searx.results.ResultContainer._closed = False
protected

Definition at line 71 of file results.py.

Referenced by add_timing(), add_unresponsive_engine(), close(), extend(), get_ordered_results(), get_timings(), and number_of_results().

◆ _lock

searx.results.ResultContainer._lock = RLock()
protected

Definition at line 77 of file results.py.

Referenced by _merge_infobox(), _merge_main_result(), add_timing(), add_unresponsive_engine(), get_timings(), and number_of_results().

◆ _main_results_sorted

list[MainResult | LegacyResult] searx.results.ResultContainer._main_results_sorted = None
protected

Definition at line 78 of file results.py.

Referenced by get_ordered_results().

◆ _number_of_results

list searx.results.ResultContainer._number_of_results = []
protected

Definition at line 69 of file results.py.

Referenced by extend(), and number_of_results().

◆ answers

searx.results.ResultContainer.answers = AnswerSet()

Definition at line 66 of file results.py.

Referenced by extend().

◆ corrections

set searx.results.ResultContainer.corrections [str]
static

Definition at line 60 of file results.py.

Referenced by extend().

◆ engine_data

dict[str, dict[str, str]] searx.results.ResultContainer.engine_data = defaultdict(dict)

Definition at line 70 of file results.py.

Referenced by searx.search.models.SearchQuery.__copy__(), and extend().

◆ infoboxes

list searx.results.ResultContainer.infoboxes [LegacyResult]
static

Definition at line 57 of file results.py.

Referenced by __init__(), and _merge_infobox().

◆ main_results_map

dict searx.results.ResultContainer.main_results_map [int, MainResult | LegacyResult]
static

Definition at line 56 of file results.py.

Referenced by __init__(), _merge_main_result(), close(), and get_ordered_results().

◆ on_result

bool searx.results.ResultContainer.on_result = lambda _: True

Definition at line 76 of file results.py.

Referenced by extend().

◆ paging

bool searx.results.ResultContainer.paging = False

Definition at line 72 of file results.py.

Referenced by extend().

◆ redirect_url

str | None searx.results.ResultContainer.redirect_url = None

Definition at line 75 of file results.py.

◆ suggestions

set searx.results.ResultContainer.suggestions [str]
static

Definition at line 58 of file results.py.

Referenced by __init__(), and extend().

◆ timings

list searx.results.ResultContainer.timings = []

Definition at line 74 of file results.py.

Referenced by add_timing(), and get_timings().

◆ unresponsive_engines

Set[UnresponsiveEngine] searx.results.ResultContainer.unresponsive_engines = set()

Definition at line 73 of file results.py.

Referenced by add_unresponsive_engine().

The documentation for this class was generated from the following file:
  • /home/andrew/Documents/code/public/searxng/searx/results.py