Loading...
Searching...
No Matches
Functions | |
| dump_request (SXNG_Request request) | |
| werkzeug.Response|None | too_many_requests (IPv4Network|IPv6Network network, str log_msg) |
| IPv4Network|IPv6Network | get_network (IPv4Address|IPv6Address real_ip, config.Config cfg) |
| _log_error_only_once (err_msg) | |
| str | get_real_ip (SXNG_Request request) |
Variables | |
| logger = logger.getChild('botdetection') | |
| list | _logged_errors = [] |
Function Documentation
◆ _log_error_only_once()
|
protected |
Definition at line 69 of file _helpers.py.
69def _log_error_only_once(err_msg):
70 if err_msg not in _logged_errors:
71 logger.error(err_msg)
72 _logged_errors.append(err_msg)
73
74
Referenced by get_real_ip().
Here is the caller graph for this function:◆ dump_request()
| searx.botdetection._helpers.dump_request | ( | SXNG_Request | request | ) |
Definition at line 24 of file _helpers.py.
24def dump_request(request: SXNG_Request):
25 return (
26 request.path
27 + " || X-Forwarded-For: %s" % request.headers.get('X-Forwarded-For')
28 + " || X-Real-IP: %s" % request.headers.get('X-Real-IP')
29 + " || form: %s" % request.form
30 + " || Accept: %s" % request.headers.get('Accept')
31 + " || Accept-Language: %s" % request.headers.get('Accept-Language')
32 + " || Accept-Encoding: %s" % request.headers.get('Accept-Encoding')
33 + " || Content-Type: %s" % request.headers.get('Content-Type')
34 + " || Content-Length: %s" % request.headers.get('Content-Length')
35 + " || Connection: %s" % request.headers.get('Connection')
36 + " || User-Agent: %s" % request.headers.get('User-Agent')
37 + " || Sec-Fetch-Site: %s" % request.headers.get('Sec-Fetch-Site')
38 + " || Sec-Fetch-Mode: %s" % request.headers.get('Sec-Fetch-Mode')
39 + " || Sec-Fetch-Dest: %s" % request.headers.get('Sec-Fetch-Dest')
40 )
41
42
◆ get_network()
| IPv4Network | IPv6Network searx.botdetection._helpers.get_network | ( | IPv4Address | IPv6Address | real_ip, |
| config.Config | cfg ) |
Returns the (client) network of whether the real_ip is part of.
Definition at line 54 of file _helpers.py.
54def get_network(real_ip: IPv4Address | IPv6Address, cfg: config.Config) -> IPv4Network | IPv6Network:
55 """Returns the (client) network of whether the real_ip is part of."""
56
57 if real_ip.version == 6:
58 prefix = cfg['real_ip.ipv6_prefix']
59 else:
60 prefix = cfg['real_ip.ipv4_prefix']
61 network = ip_network(f"{real_ip}/{prefix}", strict=False)
62 # logger.debug("get_network(): %s", network.compressed)
63 return network
64
65
◆ get_real_ip()
| str searx.botdetection._helpers.get_real_ip | ( | SXNG_Request | request | ) |
Returns real IP of the request. Since not all proxies set all the HTTP headers and incoming headers can be faked it may happen that the IP cannot be determined correctly. .. sidebar:: :py:obj:`flask.Request.remote_addr` SearXNG uses Werkzeug's ProxyFix_ (with it default ``x_for=1``). This function tries to get the remote IP in the order listed below, additional some tests are done and if inconsistencies or errors are detected, they are logged. The remote IP of the request is taken from (first match): - X-Forwarded-For_ header - `X-real-IP header <https://github.com/searxng/searxng/issues/1237#issuecomment-1147564516>`__ - :py:obj:`flask.Request.remote_addr` .. _ProxyFix: https://werkzeug.palletsprojects.com/middleware/proxy_fix/ .. _X-Forwarded-For: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For
Definition at line 75 of file _helpers.py.
75def get_real_ip(request: SXNG_Request) -> str:
76 """Returns real IP of the request. Since not all proxies set all the HTTP
77 headers and incoming headers can be faked it may happen that the IP cannot
78 be determined correctly.
79
80 .. sidebar:: :py:obj:`flask.Request.remote_addr`
81
82 SearXNG uses Werkzeug's ProxyFix_ (with it default ``x_for=1``).
83
84 This function tries to get the remote IP in the order listed below,
85 additional some tests are done and if inconsistencies or errors are
86 detected, they are logged.
87
88 The remote IP of the request is taken from (first match):
89
90 - X-Forwarded-For_ header
91 - `X-real-IP header <https://github.com/searxng/searxng/issues/1237#issuecomment-1147564516>`__
92 - :py:obj:`flask.Request.remote_addr`
93
94 .. _ProxyFix:
95 https://werkzeug.palletsprojects.com/middleware/proxy_fix/
96
97 .. _X-Forwarded-For:
98 https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For
99
100 """
101
102 forwarded_for = request.headers.get("X-Forwarded-For")
103 real_ip = request.headers.get('X-Real-IP')
104 remote_addr = request.remote_addr
105 # logger.debug(
106 # "X-Forwarded-For: %s || X-Real-IP: %s || request.remote_addr: %s", forwarded_for, real_ip, remote_addr
107 # )
108
109 if not forwarded_for:
110 _log_error_only_once("X-Forwarded-For header is not set!")
111 else:
112 from . import cfg # pylint: disable=import-outside-toplevel, cyclic-import
113
114 forwarded_for = [x.strip() for x in forwarded_for.split(',')]
115 x_for: int = cfg['real_ip.x_for'] # type: ignore
116 forwarded_for = forwarded_for[-min(len(forwarded_for), x_for)]
117
118 if not real_ip:
119 _log_error_only_once("X-Real-IP header is not set!")
120
121 if forwarded_for and real_ip and forwarded_for != real_ip:
122 logger.warning("IP from X-Real-IP (%s) is not equal to IP from X-Forwarded-For (%s)", real_ip, forwarded_for)
123
124 if forwarded_for and remote_addr and forwarded_for != remote_addr:
125 logger.warning(
126 "IP from WSGI environment (%s) is not equal to IP from X-Forwarded-For (%s)", remote_addr, forwarded_for
127 )
128
129 if real_ip and remote_addr and real_ip != remote_addr:
130 logger.warning("IP from WSGI environment (%s) is not equal to IP from X-Real-IP (%s)", remote_addr, real_ip)
131
132 request_ip = ip_address(forwarded_for or real_ip or remote_addr or '0.0.0.0')
133 if request_ip.version == 6 and request_ip.ipv4_mapped:
134 request_ip = request_ip.ipv4_mapped
135
136 # logger.debug("get_real_ip() -> %s", request_ip)
137 return str(request_ip)
References _log_error_only_once().
Here is the call graph for this function:◆ too_many_requests()
| werkzeug.Response | None searx.botdetection._helpers.too_many_requests | ( | IPv4Network | IPv6Network | network, |
| str | log_msg ) |
Returns a HTTP 429 response object and writes a ERROR message to the 'botdetection' logger. This function is used in part by the filter methods to return the default ``Too Many Requests`` response.
Definition at line 43 of file _helpers.py.
43def too_many_requests(network: IPv4Network | IPv6Network, log_msg: str) -> werkzeug.Response | None:
44 """Returns a HTTP 429 response object and writes a ERROR message to the
45 'botdetection' logger. This function is used in part by the filter methods
46 to return the default ``Too Many Requests`` response.
47
48 """
49
50 logger.debug("BLOCK %s: %s", network.compressed, log_msg)
51 return flask.make_response(('Too Many Requests', 429))
52
53
Variable Documentation
◆ _logged_errors
|
protected |
Definition at line 66 of file _helpers.py.
◆ logger
| searx.botdetection._helpers.logger = logger.getChild('botdetection') |
Definition at line 21 of file _helpers.py.
