Trustees of the Elizabeth Barrett Browning Institute in Ledbury have started to prepare for when the building ceases to be the town library.
Spokesman Bill Masefield said agents had been instructed to look for new tenants and the historic building was likely to be used as offices, rather than being converted to residential use.
Herefordshire Council plans to move the library to the Master's House in St Katherine's car park, which can offer disabled access.
Mr Masefield said: "I'm sure we'll find someone. It's ideally situated in the centre of town."
He said the lack of amenities and the need for extensive conversion for residential purposes would not make it suitable for private accommodation.
Mr Masefield said that as Herefordshire Council signed a three-year lease on the Barrett Browning Institute last autumn, the trustees would expect the rent to be honoured for the period of the lease, even if the library goes elsewhere.
"We expect the council to pay because they have signed it," he said.
The council is currently preparing a £1.5m lottery bid for St Katherine's and, if successful, would provide match-funding.
This would be used to create a new library, a council information office, a new tourist information centre and possibly even a museum and archive centre dedicated to Mr Masefield's uncle, the former Poet Laureate, John Masefield.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article