Cricket clubs have refused to play in Brighton and Hove because conditions are so poor.
Club organisers told councillors urgent improvements were needed to pitches and pavilions.
Cricketing councillor Geoffrey Theobald said some top clubs had already abandoned the city.
A special scrutiny panel has been set up by Brighton and Hove City Council to look at whether a new sports pavilion should be built at Patcham Place to replace one destroyed by fire.
The council had decided not to replace it because of the cost. Compensation money will be spread around other grounds.
Councillors on the panel will also look at whether changes should be made to the council's draft sports strategy.
Simon Wood, secretary of Brighton, Hove and District Cricket Association, told the panel it was disgraceful the pavilion had not been rebuilt.
He said: "Stanmer Park is going to hell in a handcart. Some pitches are overrun by travellers two or three times a season.
"You go deaf after playing for several hours at Braypool (because of traffic noise).
"It is a waste of money spending money at Waterhall because no one wants to play there."
He said it was time money was spent on pavilions on several grounds to stop them falling apart.
Mr Wood said: "Whenever you approach the council, you never know who you are going to deal with. Some clubs are too embarrassed to invite villages like Lindfield, Firle and Mayfield down."
Coun Theobald said: "I feel strongly that the council has a responsibility for leisure, sport and recreation. We are good on culture but are falling back on recreation."
Coun Theobald said two clubs had left the city leaving only Brighton and Hove Cricket Club.
Village sides were refusing to come to the city and there was a danger the game would disappear.
The panel will meet at least twice more before putting a series of recommendations to the council.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article