This Patent Uses A Image Content Analysis With A Geo-Semantic Index To Build Search Recommendations
It’s interesting seeing Google combining Google Maps with a better understanding of the Semantic Composition of areas within that map, analyzing images from sources such as streetviews. This patent reminded me of another one recently granted to Google, which I wrote about in August in the post, Locally Prominent Semantic Features.
Another patent I wrote about recently that worked to analyze images from places such as streetview was one that looked for information about different brands in different locations. So along with this patent, Google seems to be paying more attention to what it is finding using information about different locations. I wrote about that patent in the post
Detecting Brand Penetration Over Geographic Locations.
Searchers often look for locations to accommodate specific human activities. Typical search queries usually fail to yield valid results when searching for nearby places with certain facilities and physical attributes.
For example, a searcher may use a mapping application to view a map of an area and attempt to determine locations for a particular activity or search for labeled elements that may state the availability of the physical activity.
Those search techniques may be cumbersome and may yield inconsistent or incorrect results. For example, facilities and attributes may become unmapped and too many or prone to change. Thus, improved systems for map searching to identify locations associated with activities are needed (the purpose behind this patent).
Advantages of This Image Content Analysis Patent Include
- Obtaining a vocabulary of image features associated with searcher activities
- Receiving image content analysis on the collection of imagery based on the vocabulary of image feature types
- Generating at least one activity score for location cells in a geo-semantic index based on the vocabulary of image feature types
- Populating the geo-semantic index of location cells including data indicative of at least one activity score for each location cell
- Providing the geo-semantic index of location cells for use in generating location recommendations in response to a query
The Image Content Analysis Patent is at:
Map Search Recommendation System Based on Image Content Analysis Driven Geo-Semantic Index
Inventors Yan Mayster, Brian Daniel Shucker
Application Number 17251399
File Date: July 1, 2019
Publication Number 20210248180
Publication Date August 12, 2021
Applicants Google LLC
Abstract
The patent provides systems and methods that enable map search recommendations based on a geo-semantic index developed using image content analysis.
In one example, a computer-implemented method can include obtaining, by computing devices, a vocabulary of image feature types associated with searcher activities.
The process can include receiving a collection of imagery.
The method can include performing image content analysis on the collection of imagery based on the vocabulary of image feature types.
The process can include generating at least one activity score for each of a plurality of location cells in a geo-semantic index based at least in part on the vocabulary of image feature types.
The method can include populating the geo-semantic index of location cells, the geo-semantic index of location cells, including data indicative of at least one activity score for each location cell.
The method can include providing the geo-semantic index of location cells to generate location recommendations in response to a query.
Systems And Methods To Enable Map Search Recommendations Based On A Geo-Semantic Index Developed Using Image Content Analysis
The patent can help create a geo-semantic index of locations that identify features associated with searcher activities. Such a geo-semantic index provides map search recommendations, including geographic areas that accommodate particular searcher activities.
A searcher may want to search for nearby locations that can accommodate a particular activity. Such activities can include:
- Nearby places that are kid-friendly
- Playgrounds with shady areas
- Places where one can sit and read
A geo-semantic index of locations, including rich semantic contextual information for each area, can be constructed based on image content analysis. This can be done by looking at street-level imagery and identifying features associated with activities (a reason for Streetviews Cars.)
The geo-semantic index can then identify location cells within some geographic areas, such as a fixed radius of the searcher location. The desired activity is based on location cell activity scores. The identified sites can be provided to the searcher as query result recommendations, such as mapped locations and a list of places. The location recommendations can include descriptions of the sites based on the activity and features associated with the areas.
Answering Searcher Queries For Locations With Requested Features
Thus, the systems and methods of the patent can provide for answering searcher queries for locations with requested features and points that can accommodate specific activities more fully and van guide the searcher to such places.
A computing system can ease the construction and periodic updating of a geo-semantic index of locations, such as on a grid of location cells. It may have rich semantic contextual information for each site generated using an image content analysis engine. Image content analysis systems can recognize a wide array of semantic elements, such as on the order of tens of thousands of parts. Many of those parts refer to features and attributes of the physical and urban landscape.
Such image content analysis systems can process substantial image corpora periodically. The geo-semantic index can provide a semantic context map of the world using actual knowledge of physical places deduced from collected imagery and ease search recommendations for businesses that can support desired activities and suggestions within a particular area.
A computing system can provide for cataloging a vocabulary of known feature and attribute types (e.g., from an image content analysis lexicon, etc.) along with the searcher activities that they can accommodate (e.g., search Searchers and applications are likely to use to get interested in).
For example, such feature/attribute vocabulary could include “bench” for reading; “alley” for walking; “loading zone” or “parking meter” for a passenger pick-up or parking; “statue,” “fountain,” or “boutique” for meet-ups; and the like. The feature/attribute vocabulary can have a many-to-many relationship between activities and vocabulary terms. The feature/attribute vocabulary can include verb attribute terms besides physical features, for example, identifying people reading and the like in imagery.
Imagery Collections that May Be Used in Image Content Analysis
The computing system can use imagery collections (e.g., street-level imagery, aerial imagery, etc.), for example, from an updated imagery corpus, publicly available imagery, searcher-generated imagery, and the like. The computing system can perform image content analysis on the obtained imagery to identify feature/attribute type terms from the generated vocabulary. The computing system can populate and update a geo-semantic index of location cells associated with the imagery (e.g., on a grid of location cells mapped to a geographic area).
The geo-semantic index may also include data about collection times for the imagery having the identified features. The index can get updated for matching features to activities, for example, via manual updates from human operators, based on searcher feedback, and the like. The image content analysis may include filtering out specific image data, such as features identified in private and restricted areas and the like.
For every activity supported by the vocabulary, the computing system can generate a weighted activity score for each location cell based at least in part on the amount, diversity, and recency of the semantic terms corresponding to the activities that get associated with the location cell. The activity scores can get generated based on metric definitions provided by a client application associated with that activity. For example, many searcher applications get focused on specific human activities, which can get supported by search recommendations using such a geo-semantic index as described herein.
Some of the activities in the geo-semantic index and vocabulary may become defined based on the needs of a specific searcher application that describes the action and the metric for the activity score. The activity scoring can be different based on the type of activity, the type of application, and the like, for example, the elements that may be essential to the action. The generation of such activity scores, along with the repeated refresh of information provided through the image content analysis from a large imagery corpus (e.g., street-level imagery corpus, etc.), can enable the functionality of the geo-semantic index in providing more comprehensive recommendations to Searchers based on desired activities.
A Query From A Searcher About Nearby Locations That Accommodate A Searcher Activity
A computing system can get a query from a searcher about nearby locations that can accommodate a particular searcher activity. The computing system can analyze the geo-semantic index to identify nearby high-scoring location cells (e.g., within a fixed radius of the search area indicated by the searcher, around the searcher’s current location, etc.) and provide those locations that match the desired activity as search recommendations. The computing system can generate a textual description for each site based on the features and actions associated with the location. Such reports can be generated based on an application of natural language processing, curated by human operators, and the like. These location descriptions can be provided to the searcher as part of the location recommendations to assist the searcher in selecting and finding the desired location.
For example, a searcher query may get submitted to the computing system, such as “Find a place where I can walk and enjoy some shade.” The computing system can screen the query submission for any known activity words associated with the vocabulary. For example, the system could identify “walk” and “enjoy shade” as activity terms in the language. The computing system can analyze the index to identify location cells that match these activities based on specific vocabulary terms in the cells. Thus, the specified cell locations may have among their defining features phrases such as “alley,” “trees,” “sidewalk,” “gravel path,” and other similar words. The computing system can return the identified locations as recommendations to the searcher, for example, based on the activity scores associated with the determined location cells.
A geo-semantic index can get used to providing recommendations of activities within a particular geographic area as opposed to requests for locations to accommodate a specific action. For example, rather than answering a query of “Where can I do
In such cases, the same geo-semantic index can use the searcher’s current location and the semantic elements stored in the index (e.g., developed based on the image content analysis) to provide recommendations for certain activities supported by these elements within the specified geographic area.
The searcher may not issue such activity queries via desktop search or voice assistant in some cases. Instead, the potential Searchers may also interact with a recommendation system through their use of various client applications (e.g., via RPC calls). Many applications get focused on specific human activities that the geo-semantic index can support. Some good examples can include applications for meet-ups, ride-hailing, sports and outdoor exercise, virtual reality gaming, dog walking, bird watching, and the like. Each application may need to have its activity terms defined and specific weights and metric functions with additional algorithmic filtering criteria (e.g., avoiding roads or residential buildings with a particular margin of distance, etc.).
Knowledge Of Public Space To Power Its Activity-Based Feature Set To Searchers
The patent can provide an application that needs knowledge of public space to power its activity-based feature set to Searchers. In the case of passenger pick-ups, the computer may return locations with “loading zone” signs, parking meters, visible landmarks, and other terms that make a place easy to find for both the driver and the passenger and legally stop a vehicle at and pick up a person. In the example of meet-ups, the activity score of a location cell may factor in terms that make a place “walkable,” for instance, having elements such as “walk path” and various pedestrian facilities, “fountain,” “sculpture,” and the like.
A computing system can get a vocabulary of image feature types associated with searcher activities, and the wording may include data that identifies the image feature types. A plurality of activity types can get associated with a respective subset of image feature types from the vocabulary of image feature types. The computing system can get a collection of imagery, such as images. The computing system can perform image content analysis on the collection of imagery based on the vocabulary of image feature types. For example, the computing system may determine whether corresponding imagery for each of a plurality of location cells of a geo-semantic index includes feature types from the vocabulary of image feature types.
Generating an Activity Score for A Location Cell
The computing system can generate at least one activity score for each of the location cells in the geo-semantic index based on the vocabulary of image feature types. For each location cell, the computing system can determine an activity score for each activity type based at least in part on whether the location cell includes some respective subset of image feature types for such activity type. The computing system can populate the geo-semantic index of location cells with data indicating at least one activity score for each location cell. The computing system can provide the geo-semantic index of location cells to generate location recommendations in response to a query.
The systems described herein provide many technical effects and benefits. For instance, the systems and methods of the patent can provide improvements to search recommendation systems. The systems and methods can, for example, enable querying for nearby locations based on the presence of certain facilities and attributes that are likely to be yet unmapped and too many or prone to change. The latter, in particular, can make it infeasible to rely on fixed permanent records of such entities (e.g., searcher-reported) without the ability to refresh them. Thus, the systems and methods of the patent can provide for answering searcher queries for locations with requested features and attributes that can accommodate specific activities more comprehensively and guide the searcher to such places.
An improved search recommendation system and related searcher interface can get provided by associating locations with activities based on image content analysis. Image content analysis of imagery related to a place can get used to identifying the presence of features that correspond to a vocabulary of image features. The image features in the terminology can get associated with activities or activity types. As such, a geo-semantic index as described herein can include information related to activities at a particular location, where those activities get determined based on image content analysis. Such techniques may represent improvements over manual or other processes.
Facilitating Search Recommendations Based On A Geo-Semantic Index Developed Using Image Content Analysis
The system gets provided as one example only. Other computing systems that include different components can get used besides the design. The system can consist of searcher computing devices, such as a searcher computing device, and remote computing devices, such as server computing systems that get coupled over networks, such as communication networks. Although only one searcher computing device is illustrated, many can connect to the server computing system over the web.
The searcher computing device can be any computing device, such as, for example, a personal computing device (e.g., laptop or desktop), a mobile computing device (e.g., smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.
The searcher computing device includes processors and memories. The processors can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.). They can be one processor or a plurality of processors that get connected. The memories can include non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memories can store data and instructions executed by the processor to cause the searcher computing device to perform operations, such as described herein. More particularly, as described herein, a searcher computing device can generate query requests and receive response data from the server computing system over the communication network.
The searcher computing device can also include input/output interfacesInputut/output interfaces can include, for example, devices for receiving information from or providing information to a searcher, such as through a display device, touch screen, touchpad, mouse, data entry keys, an audio output device such as speakers, a microphone, camera, haptic feedback device, etc. The searcher computing device can also include communication/network interfaces used to communicate with systems or devices, including designs or devices that get located from the searcher computing device.
The server computing system can include or be otherwise implemented by server computing devices. In instances where the server computing system includes many devices, such servers can operate according to sequential computing architectures, parallel computing architectures, or combinations.
The Geographic Information System
The server computing system can host a geographic information system, such as a geographic information system associated with a mapping service. The geographic information system (GIS) can pue a mapping application, a virtual globe application, or any other suitable GIS.
The GIS can provide archiving, retrieving, and manipulating geospatial data indexed and stored according to geographic coordinates. These coordinates include latitude, longitude, and altitude associated with the geospatial data. The GIS combines satellite imagery, photographs, maps, models, and other geographic data, and Internet search capability to enable a searcher to view the planet, such as map data associated with a plurality of geographic areas, and related geographic information, such as locales like islands and cities, points of interest such as local restaurants, hospitals, parks, hotels, and schools. The GIS can further allow a searcher to conduct local searches, get travel directions to a location or between two locations, or otherwise retrieve map data for a selected region.
Results can become displayed in a two-dimensional or three-dimensional representation of the area of interest. The searcher can pan, tilt, and rotate the view to see three-dimensional terrain and buildings.
The GIS can provide search results, recommendations, and the like, in response to a searcher query. These results can be based on a geo-semantic index as described herein. For example, GIS can include or otherwise get provided access to a geo-semantic index, such as geo-semantic index, which consists of semantic context associated with a plurality of location cells, such as features, attributes, activities, etc. associated with each location cell. The geo-semantic index can get populated and updated based on image content analysis, manual entry, and the like.
According to an aspect of the patent, the server computing system can include an image content analysis engine that can ease the implementation of embodiments of the patent. For example, the server computing system can get data comprising a collection of imagery, such as street-level imagery collected periodically.
The Image Content Analysis Engine
The image content analysis engine can get tasked to process obtained imagery periodically to recognize semantic elements which refer to features and attributes of the physical landscape, for example, associated with human activities uncataloged in a vocabulary of feature/attribute types as described herein.
For instance, feature/attribute types may get associated with or indicative of physical objects and the physical landscape within the imagery, such as structures, infrastructure, buildings, objects, and the like. For example, features may include “playground,” “park,” “alley,” “path,” “boutique,” “loading zone,” “parking meter,” “bench,” “fountain,” “statute,” “artwork,” “tree,” “bush,” and the like. Data indicative of features within imagery may include feature vectors indicative of aspects of the identified quality.
The Geo-Semantic Index
According to an aspect of the patent, the server computing system can include a geo-semantic index. The geo-semantic index can comprise an index of geographic locations, including rich semantic contextual information for each area constructed and maintained based on image content analysis provided by an image content analysis engine.
As an example, the geo-semantic index may include, for each location cell, data indicative of features identified in imagery associated with the location cell (e.g., bench, path, fountain, mural, parking meter, playground, etc.), data about collection time(s) (e.g., timestamps, etc.) of analyzed images having the identified features, activity scores for searcher activities associated with and accommodated by the specified elements in a location cell, textual descriptions based on the features and actions related to the location cell, and the like.
Obtained imagery associated with location cells in the geo-semantic index may get processed by an image content analysis system using an image content analysis engine. That engine can identify features in the image included within a feature vocabulary associated with the geo-semantic index, such as the playground, bench, fountain, parking meter, tree, path. The features identified within the imagery are related to a location cell that can be matched to searcher activities and accommodated by the elements, using the feature vocabulary associated with the geo-semantic index). Activity scores can get generated for location cells, for example, based on the type and numbers of features identified, the diversity of features identified, the recency of the element identified, such as time since the imagery, including the quality, got collected. And the like corresponding to activities associated with the location cell.
Identifying Locations Within a Geographic Area
The geo-semantic index can be used to identify locations within some geographic area that can accommodate the desired searcher activity (e.g., based on data obtained from a searcher computing devices associated with a search query, etc.). The server computing device can provide the identified location data, for example, to the searcher computing device, as part of query result recommendations (e.g., mapped locations, list of sites, etc.).
The server computing system can also include communication/network interfaces used to communicate with systems or devices, including designs or devices that get located from the server computing system, such as searcher computing device, for example. The server computing system can also include input/output interfaces.
The network can be any communications network, such as a local area network (e.g., intranet), vast area network (e.g., Internet), or some combination. It can include any number of wired or wireless links. Communication over the web can get carried via any wired and wireless connection, using a wide variety of communication protocols (e.g., TCP/IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and protection schemes (e.g., VPN, secure HTTP, SSL).
A street-level image can get obtained as part of an imagery corpus and processed by an image content analysis system (e.g., image content analysis engine of FIG. 1) to identify features and attributes within the image. In particular, an image content analysis system may identify various parts within a vision to ease creating and updating a geo-semantic index as described herein.
The image may get associated with a location cell within the geo-semantic index. The featured identified may get used to updating data related to the location cell within the geo-semantic index.
An Image Content Analysis System Identifies Features With Confidence Indicative of the Feature Within The Image
These features can include windows, doors, trees, plants, etc. (e.g., “window 1.00”, “door 0.93”, “tree 0.97”, etc.). The identified features can be compared to a vocabulary of features/characteristics associated with the geo-semantic index (e.g., components/attribute types associated with searcher activity types maintained within the geo-semantic index) and used to facilitate updating of the geo-semantic index (e.g., generating/updating data related to a location cell, generating/updating activity scores for a location cell associated with the image, etc.).
For example, obtained images can get processed by the image content analysis system to identify features located within the imagery associated with the vocabulary related to features and activities accommodated by the features of the geo-semantic index. The components identified within the imagery can get matched to searcher activities unadapted by the elements, such as the vocabulary related to the geo-semantic index.
Activity scores can get generated for each location cell in the geo-semantic index, for example, based on the:
- Type of features identified
- Number of features identified
- Diversity of features identified
- Recency of the features identified, such as the time since the imagery including when the feature got collected
- Like corresponding to the activities that get associated with the location cell
Street Level Imagery Used In The Imagery Corpus
A street-level image can get obtained as part of an imagery corpus and processed by an image content analysis system, such as the image content analysis engine of FIG. 1, to identify features and attributes. In particular, an image content analysis system may identify elements within an image, such as specifying that the image is associated with characteristics of a public space such as a park. The image contains a water feature, a fountain, plants, and the like. The identified components may be associated with a geo-semantic index vocabulary, such as associating elements with activities accommodated by the features).
The image may be associated with a location cell within the geo-semantic index and the features identified may be used to generate and update data related to the location cell within the geo-semantic index, such as developing/updating feature data related to a location cell, generating/updating activity scores for a location cell associated with the image.
A street-level image can be part of an imagery corpus and processed by an image content analysis system, such as an image content analysis engine. The image content analysis engine can identify features and attributes. In particular, an image content analysis system may identify various elements within an image, such as remembering that the image characteristics associated with characteristics of public space and playground, that the image contains features may be related to a location cell within the geo-semantic index and the components identified may generate and update data associated with the location cell within the geo-semantic index. The data may create an activity score for a location cell associated with the image.
Example Methods
The patent methods are not limited to the particularly illustrated order or arrangement. The various stages of the procedure can get omitted, rearranged, combined, and adapted in multiple ways without deviating from the scope of the patent. The method can get implemented by computing devices.
The method can include obtaining, for example, a computing device such as a server computing system, a vocabulary of feature types (e.g., a terminology that associates features identified in imagery with activity types that can get accommodated by the elements). In particular, the computing device can ease cataloging a vocabulary of known quality and attribute types, for example, from an image content analysis lexicon, etc., along with searcher activity types that the element and attribute types can accommodate.
A Vocabulary of Feature/Attribute Types
For example, a vocabulary of feature/attribute types could include “bench” for reading; “alley” for walking; “loading zone” or “parking meter” for a passenger pick-up or parking; “statue,” “fountain,” or “boutique” for meet-ups; and the like. The feature/attribute vocabulary can have a many-to-many relationship between activities and feature/attribute terms.
The method can include obtaining a corpus of imagery, such as street-level imagery, aerial imagery, and the like, collected periodically by the computing device. For example, the computing device can get imagery collections from an updated imagery corpus, available imagery, searcher-generated imagery, and the like.
The method can include performing image content analysis on the imagery collected by the computing device based at least in part on the vocabulary of feature types. For instance, the computing device can perform image content analysis on the obtained imagery to identify feature/attribute type terms from the generated vocabulary (e.g., identify semantic elements that refer to qualities and characteristics of the physical landscape associated with searcher activities). Image content analysis may include using machine-learned models to identify features within images contained in the imagery collection.
Processing Images Using the Image Content Analysis System
As an example, obtained imagery can get processed by the image content analysis system to identify features located within the imagery that gets associated with the vocabulary (e.g., language associating elements with activities accommodated by the features) of the geo-semantic index. The components identified within the imagery can be matched to searcher activities related to and adapted by the elements, such as the vocabulary related to the geo-semantic index.
The method can include populating and updating a geo-semantic index of location cells by the computing device based on the image content analysis of the imagery collection. In particular, images within the imagery collection can identify individual location cells mapped to a geographic area. Based on the features identified via the image content analysis for imagery associated with a location cell, the computing device can populate and update a geo-semantic index of the location cells, based in part on the vocabulary of feature types.
For example, the geo-semantic index may get populated based on data associated with feature/attribute types associated with each location cell, activity types related to the elements in each location cell, and the like. The geo-semantic index may also include data about collection times for the imagery having the identified features. The geo-semantic index’s image content analysis and population may filter out specific image data, such as features/details specified in private and restricted areas and the like.
Activity Scores For Each Location Cell
The method can include generating activity scores for each location cell, such as associated with each activity that may get accommodated by features identified in the location cell. In particular, the computer can generate weighted activity scores for each location cell based at least in part on the amount, diversity, and recency of the feature types corresponding to the activities that get associated with the location cell.
The activity scores can get generated based on metric definitions provided by a client application associated with that activity. Some activities in the geo-semantic index and vocabulary may be defined based on a specific searcher’s needs that define the training and the metric for the activity score. The scoring can be different based on the type of activity, the type of application, and the like, for example, the elements essential to the action.
The geo-semantic index may include, for each location cell, data indicative of features identified in imagery associated with the location cell (e.g., bench, path, fountain, mural, parking meter, playground, etc.), data about collection time(s) (e.g., timestamps, etc.) of analyzed images having the identified features, activity scores for searcher activities associated with and accommodated by the specified elements in a location cell, textual descriptions based on the features and actions related to the location cell, and the like.
The method can include providing, by the computing device, the geo-semantic index for use in generating map recommendations, for example, in response to searcher queries received from a searcher computing device about desired activities, etc.
For example, the computing device may get a query from a searcher about nearby locations that can accommodate a particular searcher activity. The computing device can analyze the geo-semantic index to identify high scoring location cells that are nearby (e.g., within a fixed radius of the search area indicated by the searcher, around the searcher current location, etc.) and provide data associated with those locations that match the desired activity as search recommendations.
According to aspects of the patent, the image content analysis and updating of a geo-semantic index can get performed based on a systematic collection of new imagery.
Providing Search Recommendations Based On A Geo-Semantic Index
.
The method can include generating a vocabulary of feature types by a server computing device. For example, the server computing device can ease constructing a language of known feature and attribute types, for instance, from an image content analysis lexicon, etc., along with searcher activity types that the element and attribute types can accommodate. For example, a vocabulary of feature types could include “bench” for reading; “alley” for walking; “loading zone” or “parking meter” for a passenger pick-up or parking; “statue,” “fountain,” or “boutique” for meet-ups; and the like. The feature vocabulary can have a many-to-many relationship between activities and feature terms.
The method can include obtaining a corpus of imagery by the server computing device, such as street-level imagery, aerial imagery, and the like. For example, the computing device can get imagery collections from an updated imagery corpus, publicly available imagery, searcher-generated imagery, and the like.
The method can include performing, by the server computing device, image content analysis on the imagery corpus-based at least in part on the vocabulary of feature types. For instance, the server computing device can perform image content analysis on the obtained imagery to identify feature/attribute type terms from the generated vocabulary (e.g., identify semantic elements which refer to qualities and characteristics of the physical landscape associated with searcher activities). For example, the server computing device may determine whether corresponding imagery for each plurality of location cells of a geo-semantic index includes feature/attribute types from the vocabulary.
The method can include generating and updating a geo-semantic index of location cells by the server computing device based on the image content analysis of images within the imagery collection. In particular, the imagery collection’s embodiments can get identified with special location cells mapped to a geographic area. Based on the features identified via the image content analysis for each image associated with a location cell, the server computing device can populate and update a geo-semantic index of the location cells based on the vocabulary of feature types. The server computing device can also generate/update activity scores for each location cell of the geo-semantic index (e.g., associated with each activity that may get accommodated by features identified in the location cell).
Obtaining Data Associated with a Query
The method can include the searcher computing device obtaining data associated with a query (e.g., from a searcher, application, etc.) about nearby locations that can accommodate a particular searcher activity. For example, a searcher may submit a search query at the searcher computing device such as “Find a place where I can walk and enjoy some shade.”The method can include the searcher computing device providing data associated with the query to the server computing device.
The method may further include wherein generating at least one activity score for each location cell comprises generating at least one activity score for each location cell based at least in part on corresponding image feature types identified by the image content analysis. The method may further include associating each activity type of many activity types with a respective subset of image feature types from the vocabulary of image feature types; wherein performing the image content analysis on the collection of imagery includes determining whether corresponding imagery for each location cell includes feature types from the vocabulary of image feature types; and wherein generating, at least one activity score for each of the location cells includes determining an activity score for each activity type based at least in part on whether such location cell consists of the respective subset of image feature types for such activity type.
The method may include a first location cell consisting of a first activity score associated with a first activity type and a second activity score for a second activity type. The first activity score gets based in part on identifying the first subset of feature types associated with the first location cell and the second activity score gets based in part on identifying the second subset of feature types related to the first location cell.
Obtaining A Query For a Location Recommendation for a Desired Activity
The method may further include obtaining a query for location recommendations for the desired activity; receiving, the geo-semantic index of location cells; determining, location cells within a geographic area associated with the question based on an activity score associated with the desired activity from the geo-semantic index of location cells; and providing, by the computing devices, a recommendation including selected locations based on the determined location cells.
The method may further include filtering out feature types identified by the image content analysis in restricted geographic locations not populated in the geo-semantic index. The process may further include creating an activity score for each of the location cells in the geo-semantic index based on supported searcher activities consists of a weighted activity score for each location cell based at least in part on an amount, diversity, and recency of identified image feature types about activities associated with the location cell. The method may further include obtaining a vocabulary of image feature types associated with searcher activities includes cataloging a language of known feature types from an image content analysis lexicon along with searcher activities that the feature types can accommodate. The method may further include the collection of imagery consisting of a rare collection of street-level imagery wherein.
The Image Content Analysis Operations
- Performing image content analysis on the collection of imagery based on the vocabulary of image feature type
- Generating an activity score for each of a plurality of location cells in a geo-semantic index based at least in part on the vocabulary of image feature types
- Populating the geo-semantic index of location cells, the geo-semantic index of location cells includes data indicative of at least one activity score for each location cell
- Providing the geo-semantic index of location cells for use in generating location recommendations in response to a query
Comments