.oO SearXNG Developer Documentation Oo.
Loading...
Searching...
No Matches
searx.engines.tokyotoshokan Namespace Reference

Functions

 request (query, params)
 
 response (resp)
 

Variables

dict about
 
list categories = ['files']
 
bool paging = True
 
str base_url = 'https://www.tokyotosho.info/'
 
str search_url = base_url + 'search.php?{query}'
 

Detailed Description

Tokyo Toshokan (A BitTorrent Library for Japanese Media)

Function Documentation

◆ request()

searx.engines.tokyotoshokan.request ( query,
params )

Definition at line 33 of file tokyotoshokan.py.

33def request(query, params):
34 query = urlencode({'page': params['pageno'], 'terms': query})
35 params['url'] = search_url.format(query=query)
36 return params
37
38
39# get response from search-request

◆ response()

searx.engines.tokyotoshokan.response ( resp)

Definition at line 40 of file tokyotoshokan.py.

40def response(resp):
41 results = []
42
43 dom = html.fromstring(resp.text)
44 rows = dom.xpath('//table[@class="listing"]//tr[contains(@class, "category_0")]')
45
46 # check if there are no results or page layout was changed so we cannot parse it
47 # currently there are two rows for each result, so total count must be even
48 if len(rows) == 0 or len(rows) % 2 != 0:
49 return []
50
51 # regular expression for parsing torrent size strings
52 size_re = re.compile(r'[\d.]+(T|G|M)?B', re.IGNORECASE)
53
54 # processing the results, two rows at a time
55 for i in range(0, len(rows), 2):
56 # parse the first row
57 name_row = rows[i]
58
59 links = name_row.xpath('./td[@class="desc-top"]/a')
60 params = {'template': 'torrent.html', 'url': links[-1].attrib.get('href'), 'title': extract_text(links[-1])}
61 # I have not yet seen any torrents without magnet links, but
62 # it's better to be prepared to stumble upon one some day
63 if len(links) == 2:
64 magnet = links[0].attrib.get('href')
65 if magnet.startswith('magnet'):
66 # okay, we have a valid magnet link, let's add it to the result
67 params['magnetlink'] = magnet
68
69 # no more info in the first row, start parsing the second one
70 info_row = rows[i + 1]
71 desc = extract_text(info_row.xpath('./td[@class="desc-bot"]')[0])
72 for item in desc.split('|'):
73 item = item.strip()
74 if item.startswith('Size:'):
75 try:
76 params['filesize'] = size_re.search(item).group()
77 except: # pylint: disable=bare-except
78 pass
79 elif item.startswith('Date:'):
80 try:
81 # Date: 2016-02-21 21:44 UTC
82 date = datetime.strptime(item, 'Date: %Y-%m-%d %H:%M UTC')
83 params['publishedDate'] = date
84 except: # pylint: disable=bare-except
85 pass
86 elif item.startswith('Comment:'):
87 params['content'] = item
88 stats = info_row.xpath('./td[@class="stats"]/span')
89 # has the layout not changed yet?
90 if len(stats) == 3:
91 params['seed'] = int_or_zero(extract_text(stats[0]))
92 params['leech'] = int_or_zero(extract_text(stats[1]))
93
94 results.append(params)
95
96 return results

Variable Documentation

◆ about

dict searx.engines.tokyotoshokan.about
Initial value:
1= {
2 "website": 'https://www.tokyotosho.info/',
3 "wikidata_id": None,
4 "official_api_documentation": None,
5 "use_official_api": False,
6 "require_api_key": False,
7 "results": 'HTML',
8}

Definition at line 14 of file tokyotoshokan.py.

◆ base_url

str searx.engines.tokyotoshokan.base_url = 'https://www.tokyotosho.info/'

Definition at line 28 of file tokyotoshokan.py.

◆ categories

list searx.engines.tokyotoshokan.categories = ['files']

Definition at line 24 of file tokyotoshokan.py.

◆ paging

bool searx.engines.tokyotoshokan.paging = True

Definition at line 25 of file tokyotoshokan.py.

◆ search_url

str searx.engines.tokyotoshokan.search_url = base_url + 'search.php?{query}'

Definition at line 29 of file tokyotoshokan.py.