Classes
class	AsyncHTTPTransportNoHttp
class	AsyncProxyTransportFixed

Functions
	shuffle_ciphers (SSLContext ssl_context)
SSLContext	get_sslcontexts (str\|None proxy_url=None, CertTypes\|None cert=None, bool verify=True, bool trust_env=True)
	get_transport_for_socks_proxy (bool verify, bool http2, str local_address, str proxy_url, httpx.Limits limit, int retries)
	get_transport (bool verify, bool http2, str local_address, str\|None proxy_url, httpx.Limits limit, int retries)
httpx.AsyncClient	new_client (bool enable_http, bool verify, bool enable_http2, int max_connections, int max_keepalive_connections, float keepalive_expiry, dict[str, str] proxies, str local_address, int retries, int max_redirects, t.Callable[..., t.Any]\|None hook_log_response)
asyncio.AbstractEventLoop	get_loop ()
	init ()

Variables
	CertTypes = str \| tuple[str, str] \| tuple[str, str, str]
	SslContextKeyType = tuple[str \| None, CertTypes \| None, bool, bool]
	logger = logger.getChild('searx.network.client')
asyncio	LOOP = None
dict	SSLCONTEXTS = {}

Function Documentation

◆ get_loop()

asyncio.AbstractEventLoop searx.network.client.get_loop ( )

Definition at line 209 of file client.py.

def get_loop() -> asyncio.AbstractEventLoop:
    return LOOP

Referenced by get_transport_for_socks_proxy().

Here is the caller graph for this function:

◆ get_sslcontexts()

SSLContext searx.network.client.get_sslcontexts	(	str \| None	proxy_url = None,
		CertTypes \| None	cert = None,
		bool	verify = True,
		bool	trust_env = True )

Definition at line 51 of file client.py.

) -> SSLContext:
    key: SslContextKeyType = (proxy_url, cert, verify, trust_env)
    if key not in SSLCONTEXTS:
        SSLCONTEXTS[key] = httpx.create_ssl_context(verify, cert, trust_env)
    shuffle_ciphers(SSLCONTEXTS[key])
    return SSLCONTEXTS[key]
 
 

References shuffle_ciphers().

Referenced by get_transport(), and get_transport_for_socks_proxy().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_transport()

searx.network.client.get_transport	(	bool	verify,
		bool	http2,
		str	local_address,
		str \| None	proxy_url,
		httpx.Limits	limit,
		int	retries )

Definition at line 145 of file client.py.

):
    _verify = get_sslcontexts(None, None, verify, True) if verify is True else verify
    return httpx.AsyncHTTPTransport(
        # pylint: disable=protected-access
        verify=_verify,
        http2=http2,
        limits=limit,
        proxy=httpx._config.Proxy(proxy_url) if proxy_url else None,  # pyright: ignore[reportPrivateUsage]
        local_address=local_address,
        retries=retries,
    )
 
 

References get_sslcontexts().

Referenced by new_client().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_transport_for_socks_proxy()

searx.network.client.get_transport_for_socks_proxy	(	bool	verify,
		bool	http2,
		str	local_address,
		str	proxy_url,
		httpx.Limits	limit,
		int	retries )

Definition at line 114 of file client.py.

):
    # support socks5h (requests compatibility):
    # https://requests.readthedocs.io/en/master/user/advanced/#socks
    # socks5://   hostname is resolved on client side
    # socks5h://  hostname is resolved on proxy side
    rdns = False
    socks5h = 'socks5h://'
    if proxy_url.startswith(socks5h):
        proxy_url = 'socks5://' + proxy_url[len(socks5h) :]
        rdns = True
 
    proxy_type, proxy_host, proxy_port, proxy_username, proxy_password = parse_proxy_url(proxy_url)
    _verify = get_sslcontexts(proxy_url, None, verify, True) if verify is True else verify
    return AsyncProxyTransportFixed(
        proxy_type=proxy_type,
        proxy_host=proxy_host,
        proxy_port=proxy_port,
        username=proxy_username,
        password=proxy_password,
        rdns=rdns,
        loop=get_loop(),
        verify=_verify,  # pyright: ignore[reportArgumentType]
        http2=http2,
        local_address=local_address,
        limits=limit,
        retries=retries,
    )
 
 

References get_loop(), and get_sslcontexts().

Referenced by new_client().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ init()

searx.network.client.init ( )

Definition at line 213 of file client.py.

def init():
    # log
    for logger_name in (
        'httpx',
        'httpcore.proxy',
        'httpcore.connection',
        'httpcore.http11',
        'httpcore.http2',
        'hpack.hpack',
        'hpack.table',
    ):
        logging.getLogger(logger_name).setLevel(logging.WARNING)
 
    # loop
    def loop_thread():
        global LOOP
        LOOP = asyncio.new_event_loop()
        LOOP.run_forever()
 
    thread = threading.Thread(
        target=loop_thread,
        name='asyncio_loop',
        daemon=True,
    )
    thread.start()
 
 

References init().

Referenced by init().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ new_client()

httpx.AsyncClient searx.network.client.new_client	(	bool	enable_http,
		bool	verify,
		bool	enable_http2,
		int	max_connections,
		int	max_keepalive_connections,
		float	keepalive_expiry,
		dict[str, str]	proxies,
		str	local_address,
		int	retries,
		int	max_redirects,
		t.Callable[..., t.Any] \| None	hook_log_response )

Definition at line 160 of file client.py.

) -> httpx.AsyncClient:
    limit = httpx.Limits(
        max_connections=max_connections,
        max_keepalive_connections=max_keepalive_connections,
        keepalive_expiry=keepalive_expiry,
    )
    # See https://www.python-httpx.org/advanced/#routing
    mounts = {}
    mounts: None | (dict[str, t.Any | None]) = {}
    for pattern, proxy_url in proxies.items():
        if not enable_http and pattern.startswith('http://'):
            continue
        if proxy_url.startswith('socks4://') or proxy_url.startswith('socks5://') or proxy_url.startswith('socks5h://'):
            mounts[pattern] = get_transport_for_socks_proxy(
                verify, enable_http2, local_address, proxy_url, limit, retries
            )
        else:
            mounts[pattern] = get_transport(verify, enable_http2, local_address, proxy_url, limit, retries)
 
    if not enable_http:
        mounts['http://'] = AsyncHTTPTransportNoHttp()
 
    transport = get_transport(verify, enable_http2, local_address, None, limit, retries)
 
    event_hooks = None
    if hook_log_response:
        event_hooks = {'response': [hook_log_response]}
 
    return httpx.AsyncClient(
        transport=transport,
        mounts=mounts,
        max_redirects=max_redirects,
        event_hooks=event_hooks,
    )
 
 

References get_transport(), and get_transport_for_socks_proxy().

Here is the call graph for this function:

◆ shuffle_ciphers()

searx.network.client.shuffle_ciphers ( SSLContext ssl_context )

Shuffle httpx's default ciphers of a SSL context randomly.

From `What Is TLS Fingerprint and How to Bypass It`_

> When implementing TLS fingerprinting, servers can't operate based on a
> locked-in whitelist database of fingerprints.  New fingerprints appear
> when web clients or TLS libraries release new versions. So, they have to
> live off a blocklist database instead.
> ...
> It's safe to leave the first three as is but shuffle the remaining ciphers
> and you can bypass the TLS fingerprint check.

.. _What Is TLS Fingerprint and How to Bypass It:
   https://www.zenrows.com/blog/what-is-tls-fingerprint#how-to-bypass-tls-fingerprinting

Definition at line 28 of file client.py.

def shuffle_ciphers(ssl_context: SSLContext):
    """Shuffle httpx's default ciphers of a SSL context randomly.
 
    From `What Is TLS Fingerprint and How to Bypass It`_
 
    > When implementing TLS fingerprinting, servers can't operate based on a
    > locked-in whitelist database of fingerprints.  New fingerprints appear
    > when web clients or TLS libraries release new versions. So, they have to
    > live off a blocklist database instead.
    > ...
    > It's safe to leave the first three as is but shuffle the remaining ciphers
    > and you can bypass the TLS fingerprint check.
 
    .. _What Is TLS Fingerprint and How to Bypass It:
       https://www.zenrows.com/blog/what-is-tls-fingerprint#how-to-bypass-tls-fingerprinting
 
    """
    c_list = [cipher["name"] for cipher in ssl_context.get_ciphers()]
    sc_list, c_list = c_list[:3], c_list[3:]
    random.shuffle(c_list)
    ssl_context.set_ciphers(":".join(sc_list + c_list))
 
 

Referenced by get_sslcontexts().

Here is the caller graph for this function:

Variable Documentation

◆ CertTypes

searx.network.client.CertTypes = str | tuple[str, str] | tuple[str, str, str]

Definition at line 19 of file client.py.

◆ logger

searx.network.client.logger = logger.getChild('searx.network.client')

Definition at line 22 of file client.py.

◆ LOOP

asyncio searx.network.client.LOOP = None

Definition at line 23 of file client.py.

◆ SslContextKeyType

searx.network.client.SslContextKeyType = tuple[str | None, CertTypes | None, bool, bool]

Definition at line 20 of file client.py.

◆ SSLCONTEXTS

dict searx.network.client.SSLCONTEXTS = {}

Definition at line 25 of file client.py.

Classes

Functions

Variables

Function Documentation

◆ get_loop()

◆ get_sslcontexts()

◆ get_transport()

◆ get_transport_for_socks_proxy()

◆ init()

◆ new_client()

◆ shuffle_ciphers()

Variable Documentation

◆ CertTypes

◆ logger

◆ LOOP

◆ SslContextKeyType

◆ SSLCONTEXTS