The purpose of this research was to explore students' mathematics anxiety levels at a selected tertiary institution in South Africa. Mathematics achievement is a top priority in the world over. South Africa lags behind in mathematics education and must do everything to develop effective strategies for the instruction of mathematics. The research examined the differences in mathematics anxiety levels according to gender, age and home language. This study examined responses to a questionnaire administered to students, utilizing a quantitative research. Data on perceived notions of students' mathematical experiences was collected. The study involved 120 respondents (84 male and 36 female) randomly selected. The instrument used to measure mathematics anxiety levels was adopted from Jerran's Maths Centre (2012). The data was analysed using Statistical Package for the Social Sciences (SPSS), version 20.0. Hypotheses were formulated to test the information that was generated. A t-test for the difference between means of the anxiety levels of males and females, Chi-square tests for association and ANOVA for testing differences among language backgrounds and anxiety levels were used as statistical analyses to measure responses. The findings of the study indicated that there are high mathematics anxiety levels among the respondents. The results also show high levels of mathematics anxiety among female students. The t-test showed that the mean difference between mathematics anxiety and gender is significant. Based on the findings of this study, it is worth noting that mathematics anxiety is one psychological factor that affects students' achievement and their general practices. Therefore, facilitators/teachers should strive to understand mathematics anxiety and implement teaching and learning strategies and study habits that can help them overcome anxiety.