searx.utils._HTMLTextExtractor Class Reference

Inheritance diagram for searx.utils._HTMLTextExtractor:

Collaboration diagram for searx.utils._HTMLTextExtractor:

Public Member Functions
	__init__ (self)
	handle_starttag (self, tag, attrs)
	handle_endtag (self, tag)
	is_valid_tag (self)
	handle_data (self, data)
	handle_charref (self, name)
	handle_entityref (self, name)
	get_text (self)
	error (self, message)

Public Attributes
list	result = []
list	tags = []

Detailed Description

Internal class to extract text from HTML

Definition at line 81 of file utils.py.

Constructor & Destructor Documentation

__init__()

searx.utils._HTMLTextExtractor.__init__

(

self

)

Definition at line 84 of file utils.py.

    def __init__(self):
        HTMLParser.__init__(self)
        self.result = []
        self.tags = []
 

Member Function Documentation

error()

searx.utils._HTMLTextExtractor.error	(		self,
			message )

Definition at line 130 of file utils.py.

    def error(self, message):
        # error handle is needed in <py3.10
        # https://github.com/python/cpython/pull/8562/files
        raise AssertionError(message)
 
 

get_text()

searx.utils._HTMLTextExtractor.get_text

(

self

)

Definition at line 127 of file utils.py.

    def get_text(self):
        return ''.join(self.result).strip()
 

References result.

handle_charref()

searx.utils._HTMLTextExtractor.handle_charref	(		self,
			name )

Definition at line 111 of file utils.py.

    def handle_charref(self, name):
        if not self.is_valid_tag():
            return
        if name[0] in ('x', 'X'):
            codepoint = int(name[1:], 16)
        else:
            codepoint = int(name)
        self.result.append(chr(codepoint))
 

References is_valid_tag(), and result.

Here is the call graph for this function:

handle_data()

searx.utils._HTMLTextExtractor.handle_data	(		self,
			data )

Definition at line 106 of file utils.py.

    def handle_data(self, data):
        if not self.is_valid_tag():
            return
        self.result.append(data)
 

References is_valid_tag(), and result.

Here is the call graph for this function:

handle_endtag()

searx.utils._HTMLTextExtractor.handle_endtag	(		self,
			tag )

Definition at line 94 of file utils.py.

    def handle_endtag(self, tag):
        if not self.tags:
            return
 
        if tag != self.tags[-1]:
            raise _HTMLTextExtractorException()
 
        self.tags.pop()
 

References tags.

handle_entityref()

searx.utils._HTMLTextExtractor.handle_entityref	(		self,
			name )

Definition at line 120 of file utils.py.

    def handle_entityref(self, name):
        if not self.is_valid_tag():
            return
        # codepoint = htmlentitydefs.name2codepoint[name]
        # self.result.append(chr(codepoint))
        self.result.append(name)
 

References is_valid_tag(), and result.

Here is the call graph for this function:

handle_starttag()

searx.utils._HTMLTextExtractor.handle_starttag	(		self,
			tag,
			attrs )

Definition at line 89 of file utils.py.

    def handle_starttag(self, tag, attrs):
        self.tags.append(tag)
        if tag == 'br':
            self.result.append(' ')
 

References result, and tags.

is_valid_tag()

searx.utils._HTMLTextExtractor.is_valid_tag

(

self

)

Definition at line 103 of file utils.py.

    def is_valid_tag(self):
        return not self.tags or self.tags[-1] not in _BLOCKED_TAGS
 

References tags.

Referenced by handle_charref(), handle_data(), and handle_entityref().

Here is the caller graph for this function:

Member Data Documentation

result

searx.utils._HTMLTextExtractor.result = []

Definition at line 86 of file utils.py.

Referenced by get_text(), handle_charref(), handle_data(), handle_entityref(), and handle_starttag().

tags

list searx.utils._HTMLTextExtractor.tags = []

Definition at line 87 of file utils.py.

Referenced by handle_endtag(), handle_starttag(), and is_valid_tag().

---

The documentation for this class was generated from the following file:

• /home/andrew/Documents/code/public/searxng/searx/utils.py