There are the remains of a iron age hillfort on Castle Hill, the lower of the two hills, and legend tells of a treasure hoard hidden underground at the hill by one of its former occupants.

The hollow on the side of the hill where the treasure is supposed to be buried is known locally as the Money Pit, perhaps due to the time and expense wasted on fruitlessly searching for treasure on this spot. While the area around Wittenham Clumps has provided a number of exciting finds for archaeologists, including one of the best preserved Iron Age swords ever found in the country, nothing resembling a treasure hoard has ever been found at the Money Pit site.

According to a local legend, the Money Pit is said to be guarded by a raven. The stories goes that a man once went hunting for the treasure and actually managed to dig up a large iron-bound chest. However, just as he was about to break the chest open, the raven appeared nearby and croaked 'he is not yet born!' The startled man wisely took this as advice that the person the treasure was intended was not him, and promptly buried the treasure again!

The idea of treasure being guarded by a raven is not unique to Wittenham Clumps. There is a legend about Stokesay Castle in Shropshire that states a treasure chest is concealed in a hidden vault below the castle, also guarded by a raven. This immortal bird awaits the arrival of someone bearing the true key to the chest, and preventing any attempt to open the chest by force in the meantime! Another treasure guarded by a raven can be found near Challacombe in Devon, where an abandoned mine is said to contain a fortune in natural gold. However, in this legend the raven is the very raven from Noah's ark, and it's mission is to warn would-be treasure hunters away, lest they fall prey to a more dangerous supernatural guardian said to lurk in the mine!

There is a clump of trees on another part of the hill known as the Cuckoo Pen, the name being inspired be the old belief that the cuckoo is the herald of spring and if a cuckoo can be captured then an eternal summer will be the result. This was never a seriously-held belief, but the notion appears in folklore throughout the country, usually in the context of mocking people by describing them as being silly enough to believe that capturing a cuckoo could have such an effect on the seasons!

Similar 'cuckoo pens' can be found on maps throughout the country. Other local examples include Cuckoopen Plantation near Coleshill and Cuckoo Pen Farm near Burford.

Another charming piece of local history was embodied in The Poem Tree, a large tree on Castle Hill onto which a local man named Joseph Tubb carved a long poem in the 1840s.

The poem remained visible for over a hundred years, eventually becoming too distorted by the growth of the tree to read. The tree was over 300 years old by the time it died in the 1990s, and it eventually fell over during strong winds in 2012.

To mark the poem's 150th anniversary in 1994, a stone was erected that features a brass plaque with a rubbing of the poem taken in 1965.

The two hills that make up Wittenham Clumps have gone by various names over the years. The most common is Sinodun Hills, coming from the Celtic Seno-Dunum meaning 'old fort'. Another theory is that the name is a pun on the Latin word Sinus, meaning breasts, which makes a lot of sense if you look at the hills from a certain angle!

The hills have had a number of amusing local nicknames as well. Before the county boundaries changed, they were often referred to as the Berkshire Bubs, yet another reference to their boob-like shape! Another name was Mother Dunch's Buttocks, a reference to the owner of Long Wittenham manor during the 17th century.