Performance Evaluation of Information Retrieval Systems Quiz

Retrieval models, algorithms, and systems are not comparable to each other.

Relevancy is always a binary judgment, either relevant or not relevant.

Determining relevancy of retrieved items is an objective process that does not depend on the user's judgment.

The ranking function, term selection, and term weighting components of an IR system are unimportant for evaluating its performance.

The number of relevant documents a user needs to find is irrelevant for evaluating an IR system's performance.

Dynamic is a term that refers to changes that occur over time.

The Cranfield paradigm is a type of experimental science used to evaluate information retrieval systems.

Precision is the ratio of the number of relevant documents retrieved to the total number of documents retrieved.

Recall is the ratio of the number of relevant documents retrieved to the total number of relevant documents in the collection.

The most effective information retrieval system is one that maximizes both precision and recall.

RankPower is defined as the average rank of the returned relevant documents.

The number of relevant documents, $CN$, is always less than or equal to the total number of returned documents, $N$.

In the F-Measure example, the total number of relevant documents is 8.

The RankPower formula can be expressed as $\frac{\sum_{i=1}^{CN} L_i}{CN^2}$, where $L_i$ is the rank of the $i$-th relevant document.

For R=3/6 and P=3/4, the F-Measure calculated is 0.5

In the second example, the RankPower is higher than in the first example.

The RankPower metric does not take into account the number of relevant documents returned.

In the parameterized F-Measure formula, when β > 1, precision is weighted more.

Precision @ 5 and Precision @ K are two different metrics used in Information Retrieval.

Mean Average Precision (MAP) is a measure used in information search systems like web search.

Normalized DCG (NDCG) stands for Direct Cumulative Gain.