A row of sarsen stones that line the west side of the churchyard, and a few others scattered through the fields beyond, are believed to be all the remains of a stone circle that once surrounded the village of Ashbury near Swindon.

Some have speculated that the church and village were built on the site of a more ancient sanctuary, similar to that at Avebury in Wiltshire.

According to Christine Bloxham in her book Folklore of Oxfordshire, the sarsen stones found scattered around the surrounding fields are said to have been sheep that were turned to stone by Merlin, King Arthur's magician, although others refer to them as Druid Stones.