My favorite quick answer is…
“Arithmetic is to mathematics as spelling is to writing.”
The dictionary definitions of these two bodies of learning are:
a·rith·me·tic
(1) the branch of mathematics that deals with addition, subtraction, multiplication, and division,
(2) the use of numbers in calculations
math·e·mat·ics
(1) the study of the relationships among numbers, shapes, and quantities,
(2) it uses signs, symbols, and proofs and includes arithmetic, algebra, calculus, geometry, and trigonometry.
The most obvious difference is that arithmetic is all about numbers and mathematics is all about theory.