Noise on the power supply or on the board. Have you tried to look at the VDD signal with an oscilloscope?
Also, you can try to debug the problem by connecting the external voltage to a pin on the P0_X pins of the ADC and try to convert this voltage instead. This might give you a hint to whether it is the reference or the VDD voltage which is the problem. This can be done by removing a jumper on P18 in SmartRF05EB (For example EM_UART_RX) and connecting the battery voltage here. You can find a proper pin by looking at the schematics of the SmartRF05EB[1] and CC2530EM reference design[2]. Remember to change the input in the ADCCON2 register and set the pin as an analog input with the APCFG register.