.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.engines.yahoo Namespace Reference

Functions

 request (query, params)
 
 parse_url (url_string)
 
 response (resp)
 

Variables

dict about
 
list categories = ['general', 'web']
 
bool paging = True
 
bool time_range_support = True
 
dict time_range_dict
 
dict region2domain
 
dict lang2domain
 
dict yahoo_languages
 

Detailed Description

Yahoo Search (Web)

Languages are supported by mapping the language to a domain.  If domain is not
found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.

Function Documentation

◆ parse_url()

searx.engines.yahoo.parse_url ( url_string)
remove yahoo-specific tracking-url

Definition at line 160 of file yahoo.py.

160def parse_url(url_string):
161 """remove yahoo-specific tracking-url"""
162
163 endings = ['/RS', '/RK']
164 endpositions = []
165 start = url_string.find('http', url_string.find('/RU=') + 1)
166
167 for ending in endings:
168 endpos = url_string.rfind(ending)
169 if endpos > -1:
170 endpositions.append(endpos)
171
172 if start == 0 or len(endpositions) == 0:
173 return url_string
174
175 end = min(endpositions)
176 return unquote(url_string[start:end])
177
178

Referenced by response().

+ Here is the caller graph for this function:

◆ request()

searx.engines.yahoo.request ( query,
params )
build request

Definition at line 130 of file yahoo.py.

130def request(query, params):
131 """build request"""
132
133 lang, region = (params["language"].split("-") + [None])[:2]
134 lang = yahoo_languages.get(lang, "any")
135
136 offset = (params['pageno'] - 1) * 7 + 1
137 age, btf = time_range_dict.get(params['time_range'], ('', ''))
138
139 args = urlencode(
140 {
141 'p': query,
142 'ei': 'UTF-8',
143 'fl': 1,
144 'vl': 'lang_' + lang,
145 'btf': btf,
146 'fr2': 'time',
147 'age': age,
148 'b': offset,
149 'xargs': 0,
150 }
151 )
152
153 domain = region2domain.get(region)
154 if not domain:
155 domain = lang2domain.get(lang, '%s.search.yahoo.com' % lang)
156 params['url'] = 'https://%s/search?%s' % (domain, args)
157 params['domain'] = domain
158
159

◆ response()

searx.engines.yahoo.response ( resp)
parse response

Definition at line 179 of file yahoo.py.

179def response(resp):
180 """parse response"""
181
182 results = []
183 dom = html.fromstring(resp.text)
184
185 url_xpath = './/div[contains(@class,"compTitle")]/h3/a/@href'
186 title_xpath = './/h3//a/@aria-label'
187
188 domain = resp.search_params['domain']
189 if domain == "search.yahoo.com":
190 url_xpath = './/div[contains(@class,"compTitle")]/a/@href'
191 title_xpath = './/div[contains(@class,"compTitle")]/a/h3/span'
192
193 # parse results
194 for result in eval_xpath_list(dom, '//div[contains(@class,"algo-sr")]'):
195 url = eval_xpath_getindex(result, url_xpath, 0, default=None)
196 if url is None:
197 continue
198 url = parse_url(url)
199
200 title = eval_xpath_getindex(result, title_xpath, 0, default='')
201 title: str = extract_text(title)
202 content = eval_xpath_getindex(result, './/div[contains(@class, "compText")]', 0, default='')
203 content: str = extract_text(content, allow_none=True)
204
205 # append result
206 results.append(
207 {
208 'url': url,
209 # title sometimes contains HTML tags / see
210 # https://github.com/searxng/searxng/issues/3790
211 'title': " ".join(html_to_text(title).strip().split()),
212 'content': " ".join(html_to_text(content).strip().split()),
213 }
214 )
215
216 for suggestion in eval_xpath_list(dom, '//div[contains(@class, "AlsoTry")]//table//a'):
217 # append suggestion
218 results.append({'suggestion': extract_text(suggestion)})
219
220 return results

References parse_url().

+ Here is the call graph for this function:

Variable Documentation

◆ about

dict searx.engines.yahoo.about
Initial value:
1= {
2 "website": 'https://search.yahoo.com/',
3 "wikidata_id": None,
4 "official_api_documentation": 'https://developer.yahoo.com/api/',
5 "use_official_api": False,
6 "require_api_key": False,
7 "results": 'HTML',
8}

Definition at line 26 of file yahoo.py.

◆ categories

list searx.engines.yahoo.categories = ['general', 'web']

Definition at line 36 of file yahoo.py.

◆ lang2domain

dict searx.engines.yahoo.lang2domain
Initial value:
1= {
2 'zh_chs': 'hk.search.yahoo.com',
3 'zh_cht': 'tw.search.yahoo.com',
4 'any': 'search.yahoo.com',
5 'en': 'search.yahoo.com',
6 'bg': 'search.yahoo.com',
7 'cs': 'search.yahoo.com',
8 'da': 'search.yahoo.com',
9 'el': 'search.yahoo.com',
10 'et': 'search.yahoo.com',
11 'he': 'search.yahoo.com',
12 'hr': 'search.yahoo.com',
13 'ja': 'search.yahoo.com',
14 'ko': 'search.yahoo.com',
15 'sk': 'search.yahoo.com',
16 'sl': 'search.yahoo.com',
17}

Definition at line 70 of file yahoo.py.

◆ paging

bool searx.engines.yahoo.paging = True

Definition at line 37 of file yahoo.py.

◆ region2domain

dict searx.engines.yahoo.region2domain
Initial value:
1= {
2 "CO": "co.search.yahoo.com", # Colombia
3 "TH": "th.search.yahoo.com", # Thailand
4 "VE": "ve.search.yahoo.com", # Venezuela
5 "CL": "cl.search.yahoo.com", # Chile
6 "HK": "hk.search.yahoo.com", # Hong Kong
7 "PE": "pe.search.yahoo.com", # Peru
8 "CA": "ca.search.yahoo.com", # Canada
9 "DE": "de.search.yahoo.com", # Germany
10 "FR": "fr.search.yahoo.com", # France
11 "TW": "tw.search.yahoo.com", # Taiwan
12 "GB": "uk.search.yahoo.com", # United Kingdom
13 "UK": "uk.search.yahoo.com",
14 "BR": "br.search.yahoo.com", # Brazil
15 "IN": "in.search.yahoo.com", # India
16 "ES": "espanol.search.yahoo.com", # Espanol
17 "PH": "ph.search.yahoo.com", # Philippines
18 "AR": "ar.search.yahoo.com", # Argentina
19 "MX": "mx.search.yahoo.com", # Mexico
20 "SG": "sg.search.yahoo.com", # Singapore
21}

Definition at line 47 of file yahoo.py.

◆ time_range_dict

dict searx.engines.yahoo.time_range_dict
Initial value:
1= {
2 'day': ('1d', 'd'),
3 'week': ('1w', 'w'),
4 'month': ('1m', 'm'),
5}

Definition at line 41 of file yahoo.py.

◆ time_range_support

bool searx.engines.yahoo.time_range_support = True

Definition at line 38 of file yahoo.py.

◆ yahoo_languages

dict searx.engines.yahoo.yahoo_languages

Definition at line 89 of file yahoo.py.