Workers for banking giant Lloyds TSB fear for their jobs despite assurances all 170 posts will be safe when part of its business is sold.
The bank is selling one of its operations, called Activa, to a subsidiary of the Bank of Ireland.
Employees at the Activa offices in Brooklands House, Lancing, are involved in processing work.
Bank bosses have said staff would be transferred to offices in Worthing by the end of the year and no jobs would be lost.
The bank says no decision had been made on whether to retain the building after it sells the operation to Active Resource.
Steve Tatlow, of the Lloyds TSB Group workers' union, said: "It is the bank's intention to redeploy everyone within its own offices in the Worthing area but we have our doubts about this.
"The union remains to be convinced that the bank can realistically absorb this number of staff."
The independent union looks after 39,000 Lloyds TSB staff throughout the country.
A spokeswoman for the bank said: "The staff will be transferred to Worthing but that does not take place until the end of the year."
The union said it would be meeting bosses frequently to discuss job security.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article