Bought for £1, the mysterious tower that inspired Tolkien

Charity needs £1m to turn Perrott's Folly, said to have inspired author, into centre for Birmingham community

Perrott's Folly in Edgbaston offers views over where JRR Tolkien lived and went to school. Photograph: David Sillitoe for the Guardian

It wasn't the most promising of pitches: when Ben Bradley suggested that a homeless charity buy a derelict, windblown Georgian tower in a poor district of Birmingham he expected, and got, some blank looks.

The building is spectacular but perilous. It sways slightly in strong wind and its seven rooms – one on each storey – are the size of a hearth rug. But, said Bradley: "As it turned out, my CEO is a Tolkien fanatic, and so the deal was done."

The Trident Reach the People charity paid £1, and became proud owners of one of the oldest and most eccentric structures in Birmingham, a building better known in Japan than it is on the other side of the city.

The eyeball-shaped windows at the top of Perrott's Folly look down in one direction on where JRR Tolkien lived as a child, and in the opposite direction on the Oratory, where he went to school. It also gives a spectacular view of the other tower he passed twice a day, the gothic ornamented chimney of the Edgbaston waterworks, which in the writer's day would have belched smoke from the steam engines. To Tolkien true believers, there is no point looking further for the origins of the two sinister towers that loom over the world of his Lord of the Rings.

The folly stood at the heart of a magnificent park when it was built by a local eccentric, John Perrott, in 1758. The pragmatic explanation is that it was a hunting lodge and status symbol, but legends insist he built it to look yearningly at his wife's grave 15 miles away, or that when she was alive it allowed him spy on her trysts with her gamekeeper lover. Conspiracy theorists point to the Masonic symbols in the ornate plasterwork of the top room, and there are tales of secret passages and underground chambers.

As the city ate up the park, the tower became a weather observation station for meteorologist and glass-maker Abraham Follett Osler, and then became part of Birmingham University. By 1979, when the university finally locked the arched door, the building was already in a poor state. Repairs by a local trust saved it from collapse, and it opened on a few occasions for special events, including the centenary of Tolkien's birth, but Trident Reach is now launching a £1m fundraising drive to complete the restoration and open it permanently to the public.

The Grade-II listed tower is taking over Bradley's life. He already has a more than full-time job – the charity runs accommodation and support services across the region, including a 97-room hostel that has more than doubled in size in the past few years to meet demand – but when he gets emails from Japan or Canada pleading for a visit to the tower, he gives in helplessly. If the Tolkien pilgrims can come on his Saturdays off, he responds, it will only take him about an hour to get there from home.

He has climbed the 139 steep, narrow, winding steps so many times he dismisses out of hand the suggestion that the tower could earn money as a honeymoon venue: "When I do marry, I can tell you I certainly wouldn't want to carry my bride up those steps."

He is also determined that the tower will not just become another stop on a tourist heritage trail. He urged the charity to buy it because it was such a source of pride and wonder in a district with pockets of the worst deprivation not just in the city but in the country. Already artist Lizzie Jordan is working with local groups on projects inspired by the tower: one man whose life was in chaos was so transformed by picking up a paintbrush for the first time since he was 12 that he now has his own flat, and has left the hostel a gallery of paintings which now cover the walls.

Bradley dreams of groups from toddlers to pensioners painting in the garden, of a cafe selling 50p cups of tea to the mothers from the Sure Start centre across the road, of film nights with 20p tickets for the teenagers who are so gruffly proud of their extraordinary neighbour.

"We're working in estates where the history is of agencies coming in, doing projects and pulling out again – essentially these places have been abandoned. We don't want the people here to think aliens have got out of a spacecraft and taken over a building which is, quite rightfully, theirs."

"If all we ended up with here is four-wheel-drives pulling up and Mumsy, Mimsy and Wimpy hopping out for a quick look, and then driving away again 10 minutes later, as far as I'm concerned we'd have failed."