Band leader Stephen Neiman has become the target of an "obscene" hate campaign.
The chief executive of the Hanover Band, based at the Old Market in Hove, has had dog dirt and a swastika posted to his home.
Yesterday the Old Market had to be evacuated after a bomb scare.
Mr Neiman, who is of Jewish origin, said: "Somebody out there obviously wants to have a go but if you major on them too much, you drive yourself mad. I take the view that it is just part of life's rich tapestry."
Last year the Old Market was at the centre of controversy after seven permanent staff working for Old Market Enterprises, the centre's marketing arm, were made redundant four days before Christmas.
Liquidators were called in to investigate the firm's accounts and recover what it could for creditors.
Mr Neiman said: "I am not saying this is connected with former disgruntled employees. I don't know if the things are connected. It would be wrong to point the finger without any proof."
Mr Neiman, who has two young daughters, said the dog dirt was posted to his home in a match box about three months ago.
The swastika was waiting for him when he returned from holiday last weekend.
He said: "Of course it is obscene and nasty but you cannot take it too seriously.
"What I do know is that the Old Market is a thriving venue. It is almost fully booked and the people who use it are extremely pleased. It is delivering what it set out to do.
"What has been achieved is fabulous, not only in arts terms, but as a community venue. The restoration of what was a poor and rundown derelict building has brought about the regeneration of the Brunswick area.
"A lot has come out of it and any efforts to derail what we have achieved will be completely ignored."
Mr Neiman evacuated the centre for 90 minutes while police made routine checks. Business returned to normal before noon.
Hove MP Ivor Caplin, a trustee of the Old Market, said: "Whoever has perpetrated this hoax needs to be tracked down by Sussex Police and made an example of."
A police spokeswoman said racist crime was unacceptable.
She said: "We want to talk to Mr Neiman to find out what has been happening and then investigate."
The 176-year-old centre was restored at a cost of £5 million in 1998 but ran into financial difficulties soon after opening. By the end of last year it had run up debts of £1.4 million.
South East Arts and Brighton and Hove City Council withdrew their funding in December.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article