xapian-core
1.5.0
|
Xapian::Weight subclass implementing the tf-idf weighting scheme. More...
Public Types | |
enum | wdf_norm : unsigned char { wdf_norm::NONE = 1, wdf_norm::BOOLEAN = 2, wdf_norm::SQUARE = 3, wdf_norm::LOG = 4, wdf_norm::PIVOTED = 5, wdf_norm::LOG_AVERAGE = 6, wdf_norm::AUG_LOG = 7, wdf_norm::SQRT = 8, wdf_norm::AUG_AVERAGE = 9, wdf_norm::MAX = 10, wdf_norm::AUG = 11 } |
Wdf normalizations. More... | |
enum | idf_norm : unsigned char { idf_norm::NONE = 1, idf_norm::TFIDF = 2, idf_norm::SQUARE = 3, idf_norm::FREQ = 4, idf_norm::PROB = 5, idf_norm::PIVOTED = 6, idf_norm::GLOBAL_FREQ = 7, idf_norm::LOG_GLOBAL_FREQ = 8, idf_norm::INCREMENTED_GLOBAL_FREQ = 9, idf_norm::SQRT_GLOBAL_FREQ = 10 } |
Idf normalizations. More... | |
enum | wt_norm : unsigned char { wt_norm::NONE = 1 } |
Weight normalizations. More... | |
![]() | |
enum | type_smoothing { TWO_STAGE_SMOOTHING = 1, DIRICHLET_SMOOTHING = 2, ABSOLUTE_DISCOUNT_SMOOTHING = 3, JELINEK_MERCER_SMOOTHING = 4, DIRICHLET_PLUS_SMOOTHING = 5 } |
Type of smoothing to use with the Language Model Weighting scheme. More... | |
Public Member Functions | |
TfIdfWeight (const std::string &normalizations) | |
Construct a TfIdfWeight. More... | |
TfIdfWeight (const std::string &normalizations, double slope, double delta) | |
Construct a TfIdfWeight. More... | |
TfIdfWeight (wdf_norm wdf_normalization, idf_norm idf_normalization, wt_norm wt_normalization) | |
Construct a TfIdfWeight. More... | |
TfIdfWeight (wdf_norm wdf_norm_, idf_norm idf_norm_, wt_norm wt_norm_, double slope, double delta) | |
Construct a TfIdfWeight. More... | |
TfIdfWeight () | |
Construct a TfIdfWeight using the default normalizations ("ntn"). | |
std::string | name () const |
Return the name of this weighting scheme. More... | |
std::string | short_name () const |
Return the short name of the weighting scheme. More... | |
std::string | serialise () const |
Return this object's parameters serialised as a single string. More... | |
TfIdfWeight * | unserialise (const std::string &serialised) const |
Unserialise parameters. More... | |
double | get_sumpart (Xapian::termcount wdf, Xapian::termcount doclen, Xapian::termcount uniqterm, Xapian::termcount wdfdocmax) const |
Calculate the weight contribution for this object's term to a document. More... | |
double | get_maxpart () const |
Return an upper bound on what get_sumpart() can return for any document. More... | |
double | get_sumextra (Xapian::termcount doclen, Xapian::termcount uniqterms, Xapian::termcount wdfdocmax) const |
Calculate the term-independent weight component for a document. More... | |
double | get_maxextra () const |
Return an upper bound on what get_sumextra() can return for any document. More... | |
TfIdfWeight * | create_from_parameters (const char *params) const |
Return the parameterised weighting scheme object. More... | |
![]() | |
Weight () | |
Default constructor, needed by subclass constructors. | |
virtual | ~Weight () |
Virtual destructor, because we have virtual methods. | |
Additional Inherited Members | |
![]() | |
static const Weight * | create (const std::string &scheme, const Registry ®=Registry()) |
Return the appropriate weighting scheme object. More... | |
![]() | |
enum | stat_flags { COLLECTION_SIZE = 1, RSET_SIZE = 2, AVERAGE_LENGTH = 4, TERMFREQ = 8, RELTERMFREQ = 16, QUERY_LENGTH = 32, WQF = 64, WDF = 128, DOC_LENGTH = 256, DOC_LENGTH_MIN = 512, DOC_LENGTH_MAX = 1024, WDF_MAX = 2048, COLLECTION_FREQ = 4096, UNIQUE_TERMS = 8192, TOTAL_LENGTH = 16384, WDF_DOC_MAX = 32768 } |
Stats which the weighting scheme can use (see need_stat()). More... | |
![]() | |
void | need_stat (stat_flags flag) |
Tell Xapian that your subclass will want a particular statistic. More... | |
Weight (const Weight &) | |
Don't allow copying. More... | |
Xapian::doccount | get_collection_size () const |
The number of documents in the collection. | |
Xapian::doccount | get_rset_size () const |
The number of documents marked as relevant. | |
Xapian::doclength | get_average_length () const |
The average length of a document in the collection. | |
Xapian::doccount | get_termfreq () const |
The number of documents which this term indexes. | |
Xapian::doccount | get_reltermfreq () const |
The number of relevant documents which this term indexes. | |
Xapian::termcount | get_collection_freq () const |
The collection frequency of the term. | |
Xapian::termcount | get_query_length () const |
The length of the query. | |
Xapian::termcount | get_wqf () const |
The within-query-frequency of this term. | |
Xapian::termcount | get_doclength_upper_bound () const |
An upper bound on the maximum length of any document in the database. More... | |
Xapian::termcount | get_doclength_lower_bound () const |
A lower bound on the minimum length of any document in the database. More... | |
Xapian::termcount | get_wdf_upper_bound () const |
An upper bound on the wdf of this term. More... | |
Xapian::totallength | get_total_length () const |
Total length of all documents in the collection. | |
Xapian::Weight subclass implementing the tf-idf weighting scheme.
|
strong |
Idf normalizations.
|
strong |
Wdf normalizations.
|
strong |
Weight normalizations.
Enumerator | |
---|---|
NONE | None. wtn=tfn*idfn |
|
inlineexplicit |
Construct a TfIdfWeight.
normalizations | A three character string indicating the normalizations to be used for the tf(wdf), idf and document weight. (default: "ntn") |
The normalizations string works like so:
Implementing support for more normalizations of each type would require extending the backend to track more statistics.
Xapian::TfIdfWeight::TfIdfWeight | ( | const std::string & | normalizations, |
double | slope, | ||
double | delta | ||
) |
Construct a TfIdfWeight.
normalizations | A three character string indicating the normalizations to be used for the tf(wdf), idf and document weight. (default: "ntn") |
slope | Extra parameter for "Pivoted" tf normalization. (default: 0.2) |
delta | Extra parameter for "Pivoted" tf normalization. (default: 1.0) |
The normalizations string works like so:
Implementing support for more normalizations of each type would require extending the backend to track more statistics.
|
inline |
Construct a TfIdfWeight.
wdf_norm_ | The normalization for the wdf. |
idf_norm_ | The normalization for the idf. |
wt_norm_ | The normalization for the document weight. |
Implementing support for more normalizations of each type would require extending the backend to track more statistics.
Xapian::TfIdfWeight::TfIdfWeight | ( | wdf_norm | wdf_norm_, |
idf_norm | idf_norm_, | ||
wt_norm | wt_norm_, | ||
double | slope, | ||
double | delta | ||
) |
Construct a TfIdfWeight.
wdf_norm_ | The normalization for the wdf. |
idf_norm_ | The normalization for the idf. |
wt_norm_ | The normalization for the document weight. |
slope | Extra parameter for "Pivoted" tf normalization. (default: 0.2) |
delta | Extra parameter for "Pivoted" tf normalization. (default: 1.0) |
Implementing support for more normalizations of each type would require extending the backend to track more statistics.
|
virtual |
Return the parameterised weighting scheme object.
params | the pointer to the string containing parameter values for a weighting scheme |
Reimplemented from Xapian::Weight.
|
virtual |
Return an upper bound on what get_sumextra() can return for any document.
This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.
Implements Xapian::Weight.
|
virtual |
Return an upper bound on what get_sumpart() can return for any document.
This information is used by the matcher to perform various optimisations, so strive to make the bound as tight as possible.
Implements Xapian::Weight.
|
virtual |
Calculate the term-independent weight component for a document.
The parameter gives information about the document which may be used in the calculations:
doclen | The document's length (unnormalised). |
uniqterms | The number of unique terms in the document. |
Implements Xapian::Weight.
|
virtual |
Calculate the weight contribution for this object's term to a document.
The parameters give information about the document which may be used in the calculations:
wdf | The within document frequency of the term in the document. |
doclen | The document's length (unnormalised). |
uniqterms | Number of unique terms in the document (used for absolute smoothing). |
wdfdocmax | Maximum wdf value in the document. |
Implements Xapian::Weight.
|
virtual |
Return the name of this weighting scheme.
This name is used by the remote backend. It is passed along with the serialised parameters to the remote server so that it knows which class to create.
Return the full namespace-qualified name of your class here - if your class is called FooWeight, return "FooWeight" from this method (Xapian::BM25Weight returns "Xapian::BM25Weight" here).
If you don't want to support the remote backend, you can use the default implementation which simply returns an empty string.
Reimplemented from Xapian::Weight.
|
virtual |
Return this object's parameters serialised as a single string.
If you don't want to support the remote backend, you can use the default implementation which simply throws Xapian::UnimplementedError.
Reimplemented from Xapian::Weight.
|
virtual |
|
virtual |
Unserialise parameters.
This method unserialises parameters serialised by the serialise() method and allocates and returns a new object initialised with them.
If you don't want to support the remote backend, you can use the default implementation which simply throws Xapian::UnimplementedError.
Note that the returned object will be deallocated by Xapian after use with "delete". If you want to handle the deletion in a special way (for example when wrapping the Xapian API for use from another language) then you can define a static operator delete
method in your subclass as shown here: https://trac.xapian.org/ticket/554#comment:1
serialised | A string containing the serialised parameters. |
Reimplemented from Xapian::Weight.