Text Variability Measures in Corpus Design for Setswana Lexicography

This book is about the design of a Setswana corpus for lexicography. While various corpora have been compiled and a variety of corpora-based research has been attempted in African languages, no effort has been made towards corpus design. Additionally, although extensive analysis of the Setswana language has been done by missionaries, grammarians and linguists since the 1800s, none of this research is in corpus design. Most research has been largely on the grammatical study of the language. The recent corpora research in African languages in general has been on the use of corpora for the compilation of dictionaries and little of it is in corpus design. Pioneers of this kind of corpora research in African languages are Prinsloo and De Schryver (1999), De Schryver and Prisloo (2000 and 2001) and Gouws and Prisloo (2005). Because of a lack of research in corpora design particularly in African languages, this book attempts to fill that gap, especially for Setswana. It is hoped that the finding of this study will inspire similar designs in other languages comparable to Setswana. We explore corpus design by focusing on measuring a variety of text types for lexical richness at comparable token points. The study explores the question of whether a corpus compiled for lexicography must comprise a variety of texts drawn from different text types or whether the quality of retrieved information for lexicographic purposes from a corpus comprising diverse text varieties could be equally extracted from a corpus with a single text type. This study therefore determines whether linguistic variability is crucial in corpus design for lexicography.