Search Results Evaluation Quiz

Maps Search Evaluation Guidelines

• Guidelines for evaluating and rating maps search results
• The rating tool is web-based
• Evaluation assesses query and result relevance, and data accuracy (name, address, pin location)
• Search 2.0 is focused on examining data accuracy
• Examples in the document reflect real-world locations at the time of publication.
• Rating workflow involves evaluating and rating each drop-down independently
• Result relevance is rated considering user intent, distance, and prominence disregarding data inaccuracies
• Evaluating data accuracy includes name, address, and pin accuracy assessments.
• Comments are mandatory for any rating except for Excellent, to explain any demotions.

Understanding User Intent

• User intent is determined using query, user location, viewport, and viewport age.
• Types of queries include:
  • Address queries (containing street number, name, state, country, etc.)
  • Point of Interest (POI) queries(e.g., famous locations)
  • Business queries (e.g., specific business names)
  • Category queries (e.g., seeking a type of store or service)
  • Product and service queries (e.g., seeking products or services offered)

Rating Result-Level Issues

• A rating for Name and Category Accuracy is crucial.
• Result Name/Title in unexpected Language or Script should be rated incorrectly when the name is not in the expected language or script.
• Business/POI is closed or does not exist should be checked if research shows the business is closed.
• Address Accuracy should be checked according to address components

Result Relevance Rating

• Relevance rating considers user intent, distance, and prominence.
• Data accuracy is assessed separately from relevance
• Relevance is rated for whether results match user intent,
• Demoting results may occur due to lack of relevance and/or significant distance to user

Rating Interface

• Navigational result questions should be answered about query whether there are any real world results that satisfy user intent
• The rating interface for individual results evaluates problems, including result name/title, language/script, and category
• Checkboxes for result issues, including unexpected language or script, and if business/POI is closed or does not exist need to be evaluated
• Relevance rating has Excellent, Good, Acceptable, Bad, and N/A ratings
• Comments should explain any demotions; include user intent, guidelines' sections used, and sources of information
• Data accuracy (name, address, and pin) is rated when applying Search 2.0 rating methodology

Pin Accuracy

• Pin accuracy is rated on a scale of Perfect, Approximate, Next Door, Wrong, Can't Verify, based on location of the pin relative to the feature
• Consider vector maps, street imagery, and other resources, giving preference to images confirming location, when rating
• Pin accuracy depends on the feature's location as it's shown on the map, not on the appearance of the feature

Address Accuracy: Components of Issues Rating

• Address accuracy should consider components, including street number, unit, street name, sub-locality, locality, region, postal code, and country.
• Address should be checked if inaccurate Language/Script, missing components such as Street Name or Sub-Locality or if the address does not exist
• When a Language/Script Issue is raised, address components for accuracy must be checked
• Address Accuracy should be Incorrect – Street Number, Incorrect – Street Name, or Address Does Not Exist, for issues in the address component ratings, according to specific guidelines

Address Accuracy: Result Expectations

• Business/POI results should have a street number, street name, locality, region, postal code, and country
• Address is correct if it matches existing official data from other resources
• Unexpected results are generally rated as Bad

Top Rating Tips

• Address queries with only a street name should be rated as Navigational
• When a user asks for a business in an area, the query is considered a location modifier. In these instances, the most relevant result is rated as Excellent

Query-Level Navigational Result Questions

• The query should be evaluated against the real world for if there are any results that completely satisfy the user intent
• If no such result exists in the real world then the answer to the query is "no"
• If such a result exists, then the answer is "yes"

Result Relevance Demotion Checkboxes

• Check if result only partially fulfill the query intent
• Check if result demoted for distance based on user/viewport location
• Check if result is less prominent based on the query intent

Test your understanding of how to evaluate search results based on specific visual elements and information display. This quiz covers essential topics such as high-level questions, color correspondence, and result evaluation techniques. Perfect for anyone looking to improve their search skills!