Functions
	value_to_https_link (value)
	value_to_website_link (value)
	value_wikipedia_link (value)
	value_with_prefix (prefix, value)
	request (query, params)
EngineResults	response (resp)
	get_wikipedia_image (raw_value)
	fetch_wikidata (nominatim_json, user_language)
	get_title_address (result)
	get_url_osm_geojson (result)
	get_img_src (result)
	get_links (result, user_language)
	get_data (result, user_language, ignore_keys)
	get_key_rank (k)
	get_label (labels, lang)
	get_tag_label (tag_category, tag_name, lang)
	get_key_label (key_name, lang)

Variables
dict	about
list	categories = ['map']
bool	paging = False
bool	language_support = True
bool	send_accept_language_header = True
str	base_url = 'https://nominatim.openstreetmap.org/'
str	search_string = 'search?{query}&polygon_geojson=1&format=jsonv2&addressdetails=1&extratags=1&dedupe=1'
str	result_id_url = 'https://openstreetmap.org/{osm_type}/{osm_id}'
str	result_lat_lon_url = 'https://www.openstreetmap.org/?mlat={lat}&mlon={lon}&zoom={zoom}&layers=M'
str	route_url = 'https://graphhopper.com/maps'
str	wikidata_image_sparql
dict	VALUE_TO_LINK
list	KEY_ORDER
dict	KEY_RANKS = {k: i for i, k in enumerate(KEY_ORDER)}

Detailed Description

OpenStreetMap (Map)

Function Documentation

◆ fetch_wikidata()

searx.engines.openstreetmap.fetch_wikidata	(		nominatim_json,
			user_language )

Update nominatim_json using the result of an unique to wikidata

For result in nominatim_json:
    If result['extratags']['wikidata'] or r['extratags']['wikidata link']:
        Set result['wikidata'] to { 'image': ..., 'image_sign':..., 'image_symbal':... }
        Set result['extratags']['wikipedia'] if not defined
        Set result['extratags']['contact:website'] if not defined

Definition at line 214 of file openstreetmap.py.

def fetch_wikidata(nominatim_json, user_language):
    """Update nominatim_json using the result of an unique to wikidata
 
    For result in nominatim_json:
        If result['extratags']['wikidata'] or r['extratags']['wikidata link']:
            Set result['wikidata'] to { 'image': ..., 'image_sign':..., 'image_symbal':... }
            Set result['extratags']['wikipedia'] if not defined
            Set result['extratags']['contact:website'] if not defined
    """
    wikidata_ids = []
    wd_to_results = {}
    for result in nominatim_json:
        extratags = result['extratags']
        # ignore brand:wikidata
        wd_id = extratags.get('wikidata', extratags.get('wikidata link'))
        if wd_id and wd_id not in wikidata_ids:
            wikidata_ids.append('wd:' + wd_id)
            wd_to_results.setdefault(wd_id, []).append(result)
 
    if wikidata_ids:
        user_language = 'en' if user_language == 'all' else user_language.split('-')[0]
        wikidata_ids_str = " ".join(wikidata_ids)
        query = wikidata_image_sparql.replace('%WIKIDATA_IDS%', sparql_string_escape(wikidata_ids_str)).replace(
            '%LANGUAGE%', sparql_string_escape(user_language)
        )
        wikidata_json = send_wikidata_query(query)
        for wd_result in wikidata_json.get('results', {}).get('bindings', {}):
            wd_id = wd_result['item']['value'].replace('http://www.wikidata.org/entity/', '')
            for result in wd_to_results.get(wd_id, []):
                result['wikidata'] = {
                    'itemLabel': wd_result['itemLabel']['value'],
                    'image': get_wikipedia_image(wd_result.get('image', {}).get('value')),
                    'image_sign': get_wikipedia_image(wd_result.get('sign', {}).get('value')),
                    'image_symbol': get_wikipedia_image(wd_result.get('symbol', {}).get('value')),
                }
                # overwrite wikipedia link
                wikipedia_name = wd_result.get('wikipediaName', {}).get('value')
                if wikipedia_name:
                    result['extratags']['wikipedia'] = user_language + ':' + wikipedia_name
                # get website if not already defined
                website = wd_result.get('website', {}).get('value')
                if (
                    website
                    and not result['extratags'].get('contact:website')
                    and not result['extratags'].get('website')
                ):
                    result['extratags']['contact:website'] = website
 
 

References get_wikipedia_image().

Referenced by response().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_data()

searx.engines.openstreetmap.get_data	(	result,
		user_language,
		ignore_keys )

Return key, value of result['extratags']

Must be call after get_links

Note: the values are not translated

Definition at line 378 of file openstreetmap.py.

def get_data(result, user_language, ignore_keys):
    """Return key, value of result['extratags']
 
    Must be call after get_links
 
    Note: the values are not translated
    """
    data = []
    for k, v in result['extratags'].items():
        if k in ignore_keys:
            continue
        if get_key_rank(k) is None:
            continue
        k_label = get_key_label(k, user_language)
        if k_label:
            data.append(
                {
                    'label': k_label,
                    'key': k,
                    'value': v,
                }
            )
    data.sort(key=lambda entry: (get_key_rank(entry['key']), entry['label']))
    return data
 
 

References get_key_label(), and get_key_rank().

Referenced by response().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_img_src()

searx.engines.openstreetmap.get_img_src ( result )

Get image URL from either wikidata or r['extratags']

Definition at line 329 of file openstreetmap.py.

def get_img_src(result):
    """Get image URL from either wikidata or r['extratags']"""
    # wikidata
    img_src = None
    if 'wikidata' in result:
        img_src = result['wikidata']['image']
        if not img_src:
            img_src = result['wikidata']['image_symbol']
        if not img_src:
            img_src = result['wikidata']['image_sign']
 
    # img_src
    extratags = result['extratags']
    if not img_src and extratags.get('image'):
        img_src = extratags['image']
        del extratags['image']
    if not img_src and extratags.get('wikimedia_commons'):
        img_src = get_external_url('wikimedia_image', extratags['wikimedia_commons'])
        del extratags['wikimedia_commons']
 
    return img_src
 
 

Referenced by response().

Here is the caller graph for this function:

◆ get_key_label()

searx.engines.openstreetmap.get_key_label	(		key_name,
			lang )

Get key label from OSM_KEYS_TAGS

Definition at line 441 of file openstreetmap.py.

def get_key_label(key_name, lang):
    """Get key label from OSM_KEYS_TAGS"""
    if key_name.startswith('currency:'):
        # currency:EUR --> get the name from the CURRENCIES variable
        # see https://wiki.openstreetmap.org/wiki/Key%3Acurrency
        # and for example https://taginfo.openstreetmap.org/keys/currency:EUR#values
        # but there is also currency=EUR (currently not handled)
        # https://taginfo.openstreetmap.org/keys/currency#values
        currency = key_name.split(':')
        if len(currency) > 1:
            o = CURRENCIES['iso4217'].get(currency[1])
            if o:
                return get_label(o, lang).lower()
            return currency[1]
 
    labels = OSM_KEYS_TAGS['keys']
    for k in key_name.split(':') + ['*']:
        labels = labels.get(k)
        if labels is None:
            return None
    return get_label(labels, lang)

References get_label().

Referenced by get_data(), get_links(), and response().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_key_rank()

searx.engines.openstreetmap.get_key_rank ( k )

Get OSM key rank

The rank defines in which order the key are displayed in the HTML result

Definition at line 404 of file openstreetmap.py.

def get_key_rank(k):
    """Get OSM key rank
 
    The rank defines in which order the key are displayed in the HTML result
    """
    key_rank = KEY_RANKS.get(k)
    if key_rank is None:
        # "payment:*" in KEY_ORDER matches "payment:cash", "payment:debit card", etc...
        key_rank = KEY_RANKS.get(k.split(':')[0] + ':*')
    return key_rank
 
 

Referenced by get_data().

Here is the caller graph for this function:

◆ get_label()

searx.engines.openstreetmap.get_label	(		labels,
			lang )

Get label from labels in OSM_KEYS_TAGS

in OSM_KEYS_TAGS, labels have key == '*'

Definition at line 416 of file openstreetmap.py.

def get_label(labels, lang):
    """Get label from labels in OSM_KEYS_TAGS
 
    in OSM_KEYS_TAGS, labels have key == '*'
    """
    tag_label = labels.get(lang.lower())
    if tag_label is None:
        # example: if 'zh-hk' is not found, check 'zh'
        tag_label = labels.get(lang.split('-')[0])
    if tag_label is None and lang != 'en':
        # example: if 'zh' is not found, check 'en'
        tag_label = labels.get('en')
    if tag_label is None and len(labels.values()) > 0:
        # example: if still not found, use the first entry
        tag_label = labels.values()[0]
    return tag_label
 
 

Referenced by get_key_label(), and get_tag_label().

Here is the caller graph for this function:

◆ get_links()

searx.engines.openstreetmap.get_links	(		result,
			user_language )

Return links from result['extratags']

Definition at line 352 of file openstreetmap.py.

def get_links(result, user_language):
    """Return links from result['extratags']"""
    links = []
    link_keys = set()
    extratags = result['extratags']
    if not extratags:
        # minor optimization : no need to check VALUE_TO_LINK if extratags is empty
        return links, link_keys
    for k, mapping_function in VALUE_TO_LINK.items():
        raw_value = extratags.get(k)
        if not raw_value:
            continue
        url, url_label = mapping_function(raw_value)
        if url.startswith('https://wikidata.org'):
            url_label = result.get('wikidata', {}).get('itemLabel') or url_label
        links.append(
            {
                'label': get_key_label(k, user_language),
                'url': url,
                'url_label': url_label,
            }
        )
        link_keys.add(k)
    return links, link_keys
 
 

References get_key_label().

Referenced by response().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_tag_label()

searx.engines.openstreetmap.get_tag_label	(	tag_category,
		tag_name,
		lang )

Get tag label from OSM_KEYS_TAGS

Definition at line 434 of file openstreetmap.py.

def get_tag_label(tag_category, tag_name, lang):
    """Get tag label from OSM_KEYS_TAGS"""
    tag_name = '' if tag_name is None else tag_name
    tag_labels = OSM_KEYS_TAGS['tags'].get(tag_category, {}).get(tag_name, {})
    return get_label(tag_labels, lang)
 
 

References get_label().

Referenced by response().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ get_title_address()

searx.engines.openstreetmap.get_title_address ( result )

Return title and address

title may be None

Definition at line 263 of file openstreetmap.py.

def get_title_address(result):
    """Return title and address
 
    title may be None
    """
    address_raw = result.get('address')
    address_name = None
    address = {}
 
    # get name
    if (
        result['category'] == 'amenity'
        or result['category'] == 'shop'
        or result['category'] == 'tourism'
        or result['category'] == 'leisure'
    ):
        if address_raw.get('address29'):
            # https://github.com/osm-search/Nominatim/issues/1662
            address_name = address_raw.get('address29')
        else:
            address_name = address_raw.get(result['category'])
    elif result['type'] in address_raw:
        address_name = address_raw.get(result['type'])
 
    # add rest of adressdata, if something is already found
    if address_name:
        title = address_name
        address.update(
            {
                'name': address_name,
                'house_number': address_raw.get('house_number'),
                'road': address_raw.get('road'),
                'locality': address_raw.get(
                    'city', address_raw.get('town', address_raw.get('village'))  # noqa
                ),  # noqa
                'postcode': address_raw.get('postcode'),
                'country': address_raw.get('country'),
                'country_code': address_raw.get('country_code'),
            }
        )
    else:
        title = result.get('display_name')
 
    return title, address
 
 

Referenced by response().

Here is the caller graph for this function:

◆ get_url_osm_geojson()

searx.engines.openstreetmap.get_url_osm_geojson ( result )

Get url, osm and geojson

Definition at line 309 of file openstreetmap.py.

def get_url_osm_geojson(result):
    """Get url, osm and geojson"""
    osm_type = result.get('osm_type', result.get('type'))
    if 'osm_id' not in result:
        # see https://github.com/osm-search/Nominatim/issues/1521
        # query example: "EC1M 5RF London"
        url = result_lat_lon_url.format(lat=result['lat'], lon=result['lon'], zoom=12)
        osm = {}
    else:
        url = result_id_url.format(osm_type=osm_type, osm_id=result['osm_id'])
        osm = {'type': osm_type, 'id': result['osm_id']}
 
    geojson = result.get('geojson')
    # if no geojson is found and osm_type is a node, add geojson Point
    if not geojson and osm_type == 'node':
        geojson = {'type': 'Point', 'coordinates': [result['lon'], result['lat']]}
 
    return url, osm, geojson
 
 

Referenced by response().

Here is the caller graph for this function:

◆ get_wikipedia_image()

searx.engines.openstreetmap.get_wikipedia_image ( raw_value )

Definition at line 208 of file openstreetmap.py.

def get_wikipedia_image(raw_value):
    if not raw_value:
        return None
    return get_external_url('wikimedia_image', raw_value)
 
 

Referenced by fetch_wikidata().

Here is the caller graph for this function:

◆ request()

searx.engines.openstreetmap.request	(		query,
			params )

Definition at line 139 of file openstreetmap.py.

def request(query, params):
    params['url'] = base_url + search_string.format(query=urllib.parse.urlencode({'q': query}))
    return params
 
 

◆ response()

EngineResults searx.engines.openstreetmap.response ( resp )

Definition at line 144 of file openstreetmap.py.

def response(resp) -> EngineResults:
    results = EngineResults()
 
    nominatim_json = resp.json()
    user_language = resp.search_params['language']
 
    l = re.findall(r"from\s+(.*)\s+to\s+(.+)", resp.search_params["query"])
    if not l:
        l = re.findall(r"\s*(.*)\s+to\s+(.+)", resp.search_params["query"])
    if l:
        point1, point2 = [urllib.parse.quote_plus(p) for p in l[0]]
 
        results.add(
            results.types.Answer(
                answer=gettext('Show route in map ..'),
                url=f"{route_url}/?point={point1}&point={point2}",
            )
        )
 
    # simplify the code below: make sure extratags is a dictionary
    for result in nominatim_json:
        if not isinstance(result.get('extratags'), dict):
            result["extratags"] = {}
 
    # fetch data from wikidata
    fetch_wikidata(nominatim_json, user_language)
 
    # create results
    for result in nominatim_json:
        title, address = get_title_address(result)
 
        # ignore result without title
        if not title:
            continue
 
        url, osm, geojson = get_url_osm_geojson(result)
        thumbnail = get_thumbnail(get_img_src(result))
        links, link_keys = get_links(result, user_language)
        data = get_data(result, user_language, link_keys)
 
        results.append(
            {
                'template': 'map.html',
                'title': title,
                'address': address,
                'address_label': get_key_label('addr', user_language),
                'url': url,
                'osm': osm,
                'geojson': geojson,
                'thumbnail': thumbnail,
                'links': links,
                'data': data,
                'type': get_tag_label(result.get('category'), result.get('type', ''), user_language),
                'type_icon': result.get('icon'),
                'content': '',
                'longitude': result['lon'],
                'latitude': result['lat'],
                'boundingbox': result['boundingbox'],
            }
        )
 
    return results
 
 

References fetch_wikidata(), get_data(), get_img_src(), get_key_label(), get_links(), get_tag_label(), get_title_address(), and get_url_osm_geojson().

Here is the call graph for this function:

◆ value_to_https_link()

searx.engines.openstreetmap.value_to_https_link ( value )

Definition at line 70 of file openstreetmap.py.

def value_to_https_link(value):
    http = 'http://'
    if value.startswith(http):
        value = 'https://' + value[len(http) :]
    return (value, value)
 
 

◆ value_to_website_link()

searx.engines.openstreetmap.value_to_website_link ( value )

Definition at line 77 of file openstreetmap.py.

def value_to_website_link(value):
    value = value.split(';')[0]
    return (value, value)
 
 

◆ value_wikipedia_link()

searx.engines.openstreetmap.value_wikipedia_link ( value )

Definition at line 82 of file openstreetmap.py.

def value_wikipedia_link(value):
    value = value.split(':', 1)
    return ('https://{0}.wikipedia.org/wiki/{1}'.format(*value), '{1} ({0})'.format(*value))
 
 

◆ value_with_prefix()

searx.engines.openstreetmap.value_with_prefix	(		prefix,
			value )

Definition at line 87 of file openstreetmap.py.

def value_with_prefix(prefix, value):
    return (prefix + value, value)

Variable Documentation

◆ about

dict searx.engines.openstreetmap.about

Initial value:

=  {
    "website": 'https://www.openstreetmap.org/',
    "wikidata_id": 'Q936',
    "official_api_documentation": 'http://wiki.openstreetmap.org/wiki/Nominatim',
    "use_official_api": True,
    "require_api_key": False,
    "results": 'JSON',
}

Definition at line 19 of file openstreetmap.py.

◆ base_url

str searx.engines.openstreetmap.base_url = 'https://nominatim.openstreetmap.org/'

Definition at line 35 of file openstreetmap.py.

◆ categories

list searx.engines.openstreetmap.categories = ['map']

Definition at line 29 of file openstreetmap.py.

◆ KEY_ORDER

list searx.engines.openstreetmap.KEY_ORDER

Initial value:

=  [
    'cuisine',
    'organic',
    'delivery',
    'delivery:covid19',
    'opening_hours',
    'opening_hours:covid19',
    'fee',
    'payment:*',
    'currency:*',
    'outdoor_seating',
    'bench',
    'wheelchair',
    'level',
    'building:levels',
    'bin',
    'public_transport',
    'internet_access:ssid',
]

Definition at line 117 of file openstreetmap.py.

◆ KEY_RANKS

dict searx.engines.openstreetmap.KEY_RANKS = {k: i for i, k in enumerate(KEY_ORDER)}

Definition at line 136 of file openstreetmap.py.

◆ language_support

bool searx.engines.openstreetmap.language_support = True

Definition at line 31 of file openstreetmap.py.

◆ paging

bool searx.engines.openstreetmap.paging = False

Definition at line 30 of file openstreetmap.py.

◆ result_id_url

str searx.engines.openstreetmap.result_id_url = 'https://openstreetmap.org/{osm_type}/{osm_id}'

Definition at line 37 of file openstreetmap.py.

◆ result_lat_lon_url

str searx.engines.openstreetmap.result_lat_lon_url = 'https://www.openstreetmap.org/?mlat={lat}&mlon={lon}&zoom={zoom}&layers=M'

Definition at line 38 of file openstreetmap.py.

◆ route_url

str searx.engines.openstreetmap.route_url = 'https://graphhopper.com/maps'

Definition at line 40 of file openstreetmap.py.

◆ search_string

str searx.engines.openstreetmap.search_string = 'search?{query}&polygon_geojson=1&format=jsonv2&addressdetails=1&extratags=1&dedupe=1'

Definition at line 36 of file openstreetmap.py.

◆ send_accept_language_header

bool searx.engines.openstreetmap.send_accept_language_header = True

Definition at line 32 of file openstreetmap.py.

◆ VALUE_TO_LINK

dict searx.engines.openstreetmap.VALUE_TO_LINK

Initial value:

=  {
    'website': value_to_website_link,
    'contact:website': value_to_website_link,
    'email': partial(value_with_prefix, 'mailto:'),
    'contact:email': partial(value_with_prefix, 'mailto:'),
    'contact:phone': partial(value_with_prefix, 'tel:'),
    'phone': partial(value_with_prefix, 'tel:'),
    'fax': partial(value_with_prefix, 'fax:'),
    'contact:fax': partial(value_with_prefix, 'fax:'),
    'contact:mastodon': value_to_https_link,
    'facebook': value_to_https_link,
    'contact:facebook': value_to_https_link,
    'contact:foursquare': value_to_https_link,
    'contact:instagram': value_to_https_link,
    'contact:linkedin': value_to_https_link,
    'contact:pinterest': value_to_https_link,
    'contact:telegram': value_to_https_link,
    'contact:tripadvisor': value_to_https_link,
    'contact:twitter': value_to_https_link,
    'contact:yelp': value_to_https_link,
    'contact:youtube': value_to_https_link,
    'contact:webcam': value_to_website_link,
    'wikipedia': value_wikipedia_link,
    'wikidata': partial(value_with_prefix, 'https://wikidata.org/wiki/'),
    'brand:wikidata': partial(value_with_prefix, 'https://wikidata.org/wiki/'),
}

Definition at line 91 of file openstreetmap.py.

◆ wikidata_image_sparql

str searx.engines.openstreetmap.wikidata_image_sparql

Initial value:

=  """
select ?item ?itemLabel ?image ?sign ?symbol ?website ?wikipediaName
where {
  hint:Query hint:optimizer "None".
  values ?item { %WIKIDATA_IDS% }
  OPTIONAL { ?item wdt:P18|wdt:P8517|wdt:P4291|wdt:P5252|wdt:P3451|wdt:P4640|wdt:P5775|wdt:P2716|wdt:P1801|wdt:P4896 ?image }
  OPTIONAL { ?item wdt:P1766|wdt:P8505|wdt:P8667 ?sign }
  OPTIONAL { ?item wdt:P41|wdt:P94|wdt:P154|wdt:P158|wdt:P2910|wdt:P4004|wdt:P5962|wdt:P8972 ?symbol }
  OPTIONAL { ?item wdt:P856 ?website }
  SERVICE wikibase:label {
    bd:serviceParam wikibase:language "%LANGUAGE%,en".
    ?item rdfs:label ?itemLabel .
  }
  OPTIONAL {
    ?wikipediaUrl schema:about ?item;
                  schema:isPartOf/wikibase:wikiGroup "wikipedia";
                  schema:name ?wikipediaName;
                  schema:inLanguage "%LANGUAGE%" .
  }
}
ORDER by ?item
"""

Definition at line 42 of file openstreetmap.py.

Functions

Variables

Detailed Description

Function Documentation

◆ fetch_wikidata()

◆ get_data()

◆ get_img_src()

◆ get_key_label()

◆ get_key_rank()

◆ get_label()

◆ get_links()

◆ get_tag_label()

◆ get_title_address()

◆ get_url_osm_geojson()

◆ get_wikipedia_image()

◆ request()

◆ response()

◆ value_to_https_link()

◆ value_to_website_link()

◆ value_wikipedia_link()

◆ value_with_prefix()

Variable Documentation

◆ about

◆ base_url

◆ categories

◆ KEY_ORDER

◆ KEY_RANKS

◆ language_support

◆ paging

◆ result_id_url

◆ result_lat_lon_url

◆ route_url

◆ search_string

◆ send_accept_language_header

◆ VALUE_TO_LINK

◆ wikidata_image_sparql