.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.engines.youtube_noapi Namespace Reference

Functions

 request (query, params)
 response (resp)
 parse_next_page_response (response_text)
 parse_first_page_response (response_text)
 get_text_from_json (element)

Variables

dict about
list categories = ['videos', 'music']
bool paging = True
bool language_support = False
bool time_range_support = True
str base_url = 'https://www.youtube.com/results'
str search_url = base_url + '?search_query={query}&page={page}'
str time_range_url = '&sp=EgII{time_range}%253D%253D'
str next_page_url = 'https://www.youtube.com/youtubei/v1/search?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8'
dict time_range_dict = {'day': 'Ag', 'week': 'Aw', 'month': 'BA', 'year': 'BQ'}
str base_youtube_url = 'https://www.youtube.com/watch?v='

Detailed Description

Youtube (Videos)

Function Documentation

◆ get_text_from_json()

searx.engines.youtube_noapi.get_text_from_json ( element)

Definition at line 169 of file youtube_noapi.py.

169def get_text_from_json(element):
170 if 'runs' in element:
171 return reduce(lambda a, b: a + b.get('text', ''), element.get('runs'), '')
172 return element.get('simpleText', '')

Referenced by parse_first_page_response().

Here is the caller graph for this function:

◆ parse_first_page_response()

searx.engines.youtube_noapi.parse_first_page_response ( response_text)

Definition at line 112 of file youtube_noapi.py.

112def parse_first_page_response(response_text):
113 results = []
114 results_data = extr(response_text, 'ytInitialData = ', ';</script>')
115
116 results_json = loads(results_data) if results_data else {}
117 sections = (
118 results_json.get('contents', {})
119 .get('twoColumnSearchResultsRenderer', {})
120 .get('primaryContents', {})
121 .get('sectionListRenderer', {})
122 .get('contents', [])
123 )
124
125 for section in sections:
126 if "continuationItemRenderer" in section:
127 next_page_token = (
128 section["continuationItemRenderer"]
129 .get("continuationEndpoint", {})
130 .get("continuationCommand", {})
131 .get("token", "")
132 )
133 if next_page_token:
134 results.append(
135 {
136 "engine_data": next_page_token,
137 "key": "next_page_token",
138 }
139 )
140 for video_container in section.get('itemSectionRenderer', {}).get('contents', []):
141 video = video_container.get('videoRenderer', {})
142 videoid = video.get('videoId')
143 if videoid is not None:
144 url = base_youtube_url + videoid
145 thumbnail = 'https://i.ytimg.com/vi/' + videoid + '/hqdefault.jpg'
146 title = get_text_from_json(video.get('title', {}))
147 content = get_text_from_json(video.get('descriptionSnippet', {}))
148 author = get_text_from_json(video.get('ownerText', {}))
149 length = get_text_from_json(video.get('lengthText', {}))
150
151 # append result
152 results.append(
153 {
154 'url': url,
155 'title': title,
156 'content': content,
157 'author': author,
158 'length': length,
159 'template': 'videos.html',
160 'iframe_src': 'https://www.youtube-nocookie.com/embed/' + videoid,
161 'thumbnail': thumbnail,
162 }
163 )
164
165 # return results
166 return results
167
168

References get_text_from_json().

Referenced by response().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ parse_next_page_response()

searx.engines.youtube_noapi.parse_next_page_response ( response_text)

Definition at line 67 of file youtube_noapi.py.

67def parse_next_page_response(response_text):
68 results = []
69 result_json = loads(response_text)
70 for section in (
71 result_json['onResponseReceivedCommands'][0]
72 .get('appendContinuationItemsAction')['continuationItems'][0]
73 .get('itemSectionRenderer')['contents']
74 ):
75 if 'videoRenderer' not in section:
76 continue
77 section = section['videoRenderer']
78 content = "-"
79 if 'descriptionSnippet' in section:
80 content = ' '.join(x['text'] for x in section['descriptionSnippet']['runs'])
81 results.append(
82 {
83 'url': base_youtube_url + section['videoId'],
84 'title': ' '.join(x['text'] for x in section['title']['runs']),
85 'content': content,
86 'author': section['ownerText']['runs'][0]['text'],
87 'length': section['lengthText']['simpleText'],
88 'template': 'videos.html',
89 'iframe_src': 'https://www.youtube-nocookie.com/embed/' + section['videoId'],
90 'thumbnail': section['thumbnail']['thumbnails'][-1]['url'],
91 }
92 )
93 try:
94 token = (
95 result_json['onResponseReceivedCommands'][0]
96 .get('appendContinuationItemsAction')['continuationItems'][1]
97 .get('continuationItemRenderer')['continuationEndpoint']
98 .get('continuationCommand')['token']
99 )
100 results.append(
101 {
102 "engine_data": token,
103 "key": "next_page_token",
104 }
105 )
106 except: # pylint: disable=bare-except
107 pass
108
109 return results
110
111

Referenced by response().

Here is the caller graph for this function:

◆ request()

searx.engines.youtube_noapi.request ( query,
params )

Definition at line 40 of file youtube_noapi.py.

40def request(query, params):
41 params['cookies']['CONSENT'] = "YES+"
42 if not params['engine_data'].get('next_page_token'):
43 params['url'] = search_url.format(query=quote_plus(query), page=params['pageno'])
44 if params['time_range'] in time_range_dict:
45 params['url'] += time_range_url.format(time_range=time_range_dict[params['time_range']])
46 else:
47 params['url'] = next_page_url
48 params['method'] = 'POST'
49 params['data'] = dumps(
50 {
51 'context': {"client": {"clientName": "WEB", "clientVersion": "2.20210310.12.01"}},
52 'continuation': params['engine_data']['next_page_token'],
53 }
54 )
55 params['headers']['Content-Type'] = 'application/json'
56
57 return params
58
59
60# get response from search-request

◆ response()

searx.engines.youtube_noapi.response ( resp)

Definition at line 61 of file youtube_noapi.py.

61def response(resp):
62 if resp.search_params.get('engine_data'):
63 return parse_next_page_response(resp.text)
64 return parse_first_page_response(resp.text)
65
66

References parse_first_page_response(), and parse_next_page_response().

Here is the call graph for this function:

Variable Documentation

◆ about

dict searx.engines.youtube_noapi.about
Initial value:
1= {
2 "website": 'https://www.youtube.com/',
3 "wikidata_id": 'Q866',
4 "official_api_documentation": 'https://developers.google.com/youtube/v3/docs/search/list?apix=true',
5 "use_official_api": False,
6 "require_api_key": False,
7 "results": 'HTML',
8}

Definition at line 13 of file youtube_noapi.py.

◆ base_url

str searx.engines.youtube_noapi.base_url = 'https://www.youtube.com/results'

Definition at line 29 of file youtube_noapi.py.

◆ base_youtube_url

str searx.engines.youtube_noapi.base_youtube_url = 'https://www.youtube.com/watch?v='

Definition at line 36 of file youtube_noapi.py.

◆ categories

list searx.engines.youtube_noapi.categories = ['videos', 'music']

Definition at line 23 of file youtube_noapi.py.

◆ language_support

bool searx.engines.youtube_noapi.language_support = False

Definition at line 25 of file youtube_noapi.py.

◆ next_page_url

str searx.engines.youtube_noapi.next_page_url = 'https://www.youtube.com/youtubei/v1/search?key=AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8'

Definition at line 33 of file youtube_noapi.py.

◆ paging

bool searx.engines.youtube_noapi.paging = True

Definition at line 24 of file youtube_noapi.py.

◆ search_url

str searx.engines.youtube_noapi.search_url = base_url + '?search_query={query}&page={page}'

Definition at line 30 of file youtube_noapi.py.

◆ time_range_dict

dict searx.engines.youtube_noapi.time_range_dict = {'day': 'Ag', 'week': 'Aw', 'month': 'BA', 'year': 'BQ'}

Definition at line 34 of file youtube_noapi.py.

◆ time_range_support

bool searx.engines.youtube_noapi.time_range_support = True

Definition at line 26 of file youtube_noapi.py.

◆ time_range_url

str searx.engines.youtube_noapi.time_range_url = '&sp=EgII{time_range}%253D%253D'

Definition at line 31 of file youtube_noapi.py.