Police cordoned off a street in the Phillipstown area of Christchurch with a bomb disposal team, ambulance and fire and emergency crews.
A 33-year-old man was arrested and a no-fly zone has been established.
Canterbury District Commander Superintendent John Price said: “Police have located a package containing a suspected explosive device and ammunition at a vacant address on Newcastle Street in Christchurch.”
Residents were earlier evacuated from Newcastle, Glasgow and Harrow streets as a precaution, local media reports.
Specialist officers were seen carrying cases of equipment close to the property.
Last month, 50 people were killed in two terrorists attacks on mosques in Christchurch.
Australian Brenton Tarrant, 28, a self-described white supremacist, live-streamed the attack on Facebook.