For every detection task TRACER computes a number of files, including, as previously seen, the text reuse results in a
.score file. This page describes all of these.
The results of every TRACER run are automatically stored in a newly generated folder called
TRACER_DATA under TRACER's
data > corpora > Bible:
If you’re working with your own texts –not TRACER’s default Bible data– the
TRACER_DATA folder would be created in your specified directory (e.g.
data > corpora > MyTexts).
TRACER organises the results in a deep folder structure with long folder names that describe the parameters used for a particular analysis. For every new analysis, TRACER creates a separate folder.
The folder name in the Figure above reads as follows:
Let's break it down:
Abbreviation in folder name
Meaning of abbreviation
Preprocessing step (01)-word-level (02)-Word Level Processing
Synonym replacement ENABLED
Replace String Similar words DISABLED
Reduced words option DISABLED-to the most significant 5 letter ngram-significance is measured by the Log-Likelihood Ratio ENABLED
Everything to lower case DISABLED
Remove diacritics DISABLED
Replace word by word length DISABLED-For words of 5 letters or more
This folder represents the first step in your detection task, Preprocessing. The other five steps of the detection process are all nested within this folder. So, if you open the Preprocessing folder, you’ll find the second folder in the sequence, the Featuring/Training folder. Within the Featuring/Training folder, you’ll find the Selection folder, and so on until you reach the final Scoring folder. In each and every folder you’ll find the relevant files resulting from the settings you specified in the
inv stands for inverted list. This file works like a word index and is the heart of any retrieval system. It shows you the position of a given word in a given textual segment/unit. For example, the row
114 4003870 2 in an
.inv TRACER file means that word
114 can be found in segment
4003870 in position
wnc stands for words number complete. This file provides a list of all the types in the corpus, including rank, word length and frequency. The list and the IDs are frequency-sorted.
RANK WORD LENGTH FREQUENCY
This file provides an overview (the "metadata" as it were) of the corpus segmentation:
WORD_TYPES: number of unique words as dictionary entries.
WORD_TOKENS: occurrences of a word; every word in a text, no matter how many times it occurs.
SOURCES: for example, a book.
SSIM_THRESHOLD: degree of similarity required to consider two words as similarly written.
SSIM_EDGES: number of links or word pairs satisfying the similarity requirements stated in
BOW_WORD_TOKENS: tokens or unique words that appear in a line.
This file is a tokenized version of the source text. This means that all punctuation has been removed or separated from the word (depending on the settings). The default setting in TRACER is
This file displays the distribution of the characters across the corpus. This file is useful to clean up overlooked dirt in the text(s) under analysis. If the file contains an unexpected character, thanks to the
.char information the user can more easily identify and remove it from the text(s).
This file displays the distribution of letter bigrams across the corpus.
Everything comes together in this file. The first and second columns represent the two words; the third column is the overlap of letter bigrams; the fourth column is the weighted overlap
by Broder’s Resemblance measure.
WORD-ID WORD FREQUENCY
This file contains three columns of data:
FEATURE-ID REUSE-ID POS
FEATURE-ID = see below.
REUSE-ID = ID from the input file, first column.
POS = position of the feature in the reuse under
REUSE-ID. In the case of n-grams as features the position is always that of the first word in the n-gram.
This two-column file maps the feature ID from the
.train file to the word(s) that make up that feature. Specifically, if you are using words as features, the numbers will be identical. If, instead, you are using n-grams, the second column contains the IDs of each word in the n-gram).
.sel file is the same as the
.train file only without the removed words.
.score file contains all computed reuse pairs. The first two columns list the IDs of the aligned reuse units, the third column displays the number of shared features (absolute overlap) and the fourth column the degree of similarity (weighted overlap). A similarity of
0.1 is 10%, of
0.2 is 20% and so on until
1 = 100%. In the example below, the two sentences have two features in common for a total similarity of 50%:
1101991 1300887 2.0 0.5
.score file lists results bidirectionally and thus redundantly. The two results below, for example, represent the same reuse alignment but the order of the IDs is inverted:
1102581 1300887 2.0 0.5
1300887 1102581 2.0 0.5