This retrospective study examined whether practice in computer-based testing (CBT) in nurse anesthesia students would improve scores on the National Certification Examination (NCE). A group of students with extensive practice in CBT were compared to a group of students whose tests were administered primarily on pencil and paper (PnP). Groups were matched on age, grade point average (GPA), and gender (n = 73 each). Higher GPA was associated with higher NCE scores overall. Adjusted for GPA (with ANCOVA), the mean NCE proficiency score (phi) in the CBT group was 2.68 (95% CI = 2.54-2.82), which was higher than the mean score in the PnP group, 2.36 (95% CI = 2.22-2.50), with an effect size of 0.52. When subgroups were examined, CBT practice improved NCE scores only in those students with graduate GPA less than 3.50, with an effect size of 1.1. It was concluded that, controlling for GPA, nurse anesthesia students at one university with greater exposure to CBT had higher scores on the NCE than a comparison group with less practice in CBT. This difference was significant only in students with GPA less than 3.50, consistent with a beneficial effect of practice in CBT.