Lee, Soon Tat
(2024)
Measuring language proficiency in Malay first and second language speakers.
PhD thesis, University of Nottingham.
Abstract
Measuring language proficiency is essential in linguistics and psycholinguistics research that investigate bilingual language processing (e.g., Fromont et al., 2020; Sarrett et al., 2022; Singh et al., 2022; Tosun & Filipović, 2022) and cognitive control (e.g., Luque & Morgan-Short, 2021; Ning, 2021). Despite often being construed as a moderating variable in bilingual research, there is a great variability in how language proficiency is operationalised and measured (Hulstijn, 2015; Puig-Mayenco et al., 2023; Surrain & Luk, 2019; Treffers-Daller, 2019; Tremblay, 2011). A systematic review on second language acquisition research (Park et al., 2022) revealed that about 58% of the studies estimated second language proficiency using variables that were closely related to language proficiency (e.g., years of instruction, self-rated proficiency). In contrast, only 42% of them assessed language proficiency with a test (e.g., validated vocabulary tests). Despite being shown as a more consistent measure, and importantly, correlated well with other language performances (e.g., Diependaele et al., 2013; Lemhöfer & Broersma, 2012; Wen & van Heuven, 2017b), implementation of language proficiency test is not always feasible because existing language proficiency tests might not be available in understudied languages or could be impractical (e.g., too time-consuming) for experimental settings. For instance, there is currently no freely available quick Malay proficiency test, although there are 377 million Malay speakers in the world.
This thesis aimed to improve the methodological rigour of bilingual language testing through the development of psycholinguistic tools for Malay, an understudied language. As a starting point, this thesis aimed to create a large database of Malay and English bidirectional translation norms to facilitate Malay-English cross-linguistic stimulus selection. Malay-English bidirectional translation equivalents were collected from highly proficient bilinguals. The second aim of this thesis was to develop and validate the Lexical Test for Malay Speakers (LexMAL), a vocabulary test that can distinguish Malay learners of various proficiency levels while assessing their discrete vocabulary knowledge based on a selected list of target words presented in isolation (i.e., out of context). External criterion measures of language proficiency were used to validate the test, including translation tasks, the cloze test, and self-rated proficiency. Subsequently, this thesis sought to justify the utility of the widely used yes/no vocabulary test scores to gauge bilinguals' lexical proficiency. Using LexMAL and four newly created form-meaning vocabulary tests, the relationships between form-meaning vocabulary knowledge and yes/no vocabulary test scores were examined in depth.
Chapter 2 of this thesis describes the development of the Malay-English and English-Malay translation norms through forward and backward translation tasks. Information from these translation norms guided the selection of appropriate translation stimuli used in Chapter 3 to assess language proficiency. The Malay and English translation norms presented are among the first collected from highly proficient Malay-English bilinguals. The study also investigated Malay-English translation ambiguity in bidirectional translation tasks and how it was affected by other lexical characteristics (e.g., word class or frequency of occurrence). The study gathered English translations of 1,004 Malay words and Malay translations of 845 English words. The study revealed high prevalence of translation ambiguity between the Malay and English languages. The findings corroborate that translation ambiguity could emerge due to the conceptual and morphological mapping differences between the target and source languages, as well as language specific properties of the language pairs under investigation (Schwieter & Prior, 2020). Together with lexical and semantic information of the source and target words, these norms could be useful references to aid stimuli selection for future experimental studies (e.g., Jouravlev & Jared, 2020) and computer simulations (e.g., Dijkstra et al., 2019). Serving as the first freely available translation norms database for researchers conducting language research with Malay-English bilinguals, this study is published in Behavior Research Methods (i.e., S. T. Lee et al., 2022).
Chapter 3 of this thesis presents the construction process of LexMAL, which builds on the Malay-English translation norms. The development of LexMAL began with the stimuli writing and selection, pilot testing of the LexMAL prototype, item assessment and ended with validation of the final LexMAL. To estimate language proficiency for Malay first language (L1) and second language (L2) speakers, an initial 180-items LexMAL prototype was evaluated on 60 Malay L1 and 60 L2 speakers. Sixty words and thirty nonwords with the highest discriminative power that span across the full difficulty range were selected for the final LexMAL. The validity of LexMAL was established by demonstrating reliable discrimination between L1 and L2 speakers, and significant correlations with other Malay language tasks. Importantly, LexMAL outperformed self-rated proficiency in the correlations with cloze task performance, indicating that objective measures like LexMAL are better estimates of language proficiency than subjective self-ratings (Khare et al., 2013; Lemhöfer & Broersma, 2012; Tomoschuk et al., 2019; Wen & van Heuven, 2017a). As far as we are aware, LexMAL is the first validated Malay lexical test that can reliably measure the proficiency of L1 and L2 speakers. LexMAL is useful for researchers in, for example, linguistics, psychology, and education that require a quick (less than 5 minutes), practical and objective proficiency measure. LexMAL can be taken online at https://www.lexmal.org/, or a paper and pencil version of LexMAL can be downloaded from https://osf.io/8y4ft/. The paper is published in Behavior Research Methods (i.e., S. T. Lee et al., 2023).
Chapter 4 of this thesis investigates the prediction of bilinguals' form-meaning vocabulary knowledge to their item accuracy on LexMAL. Four vocabulary tests were developed to assess bilinguals’ knowledge of meaning recognition, form recognition, meaning recall, and form recall. The study found that language dominance affected the form-meaning knowledge of bilinguals, with L1 speakers performing better than L2 speakers. Additionally, the accuracy of Meaning Recognition, Form Recognition, and Meaning Recall tests explained 59% of the variance in LexMAL scores. Importantly, LexMAL and recognition tests were found to be more effective than recall tests in distinguishing between L1 and L2 speakers' form-meaning vocabulary knowledge. With meaning recognition, form recognition, and meaning recall serving as predictors of LexMAL score, and form recognition being the positive predictor of item accuracy in LexMAL, this study provides evidence to support the use of yes/no vocabulary tests as quick and reliable lexical proficiency measures to estimate bilinguals’ receptive language proficiency. The paper has been submitted to Bilingualism: Language and Cognition journal and is currently under review.
Chapter 5 summarises and discusses the main findings of the thesis. The theoretical contributions of these findings are discussed in relation to the current understanding of test development and validation, bilingual language processing, and vocabulary testing. Additionally, this discussion critically evaluates the limitations of the studies while proposing potential directions for future research. Overall, the three empirical studies in this thesis advance current knowledge in bilingual language testing by assessing the vocabulary knowledge of the L1 and L2 speakers on the same scale. The psycholinguistic tools presented in this thesis enable researchers to make informed decisions when selecting lexical stimuli and lexical test formats, as well as interpret research findings based on accurate measures of participants' language proficiency.
| Item Type: |
Thesis (University of Nottingham only)
(PhD)
|
| Supervisors: |
Leong, Christine Xiang Ru
Price, Jessica
van Heuven, Walter |
| Keywords: |
language proficiency, bilingual language processing, cognitive control, second language acquisition, self-rated proficiency, vocabulary tests, Lexical test for Malay speakers (LexMAL), Malay-English bilinguals, translation tasks |
| Subjects: |
Q Science > QP Physiology |
| Faculties/Schools: |
University of Nottingham, Malaysia > Faculty of Science and Engineering — Science > School of Psychology |
| Item ID: |
77761 |
| Depositing User: |
Lee, Soon
|
| Date Deposited: |
27 Jul 2024 04:40 |
| Last Modified: |
27 Jul 2024 04:40 |
| URI: |
https://eprints.nottingham.ac.uk/id/eprint/77761 |
Actions (Archive Staff Only)
 |
Edit View |
Tools
Tools
