Pat Harding gave Horsham a 12th-second lead at Wealdstone but his side were unable to hold on to their advantage in a 2-1 defeat.
Harding put the Hornets ahead from the kick-off and they looked good for a while as the hosts made a sluggish start.
After that though, Wealdstone demonstrated why Hornets boss John Maggs rates them as one of the best sides in the Ryman League.
Goals either side of the break from Ryan Ashe sealed the points for the Stones and condemned Horsham to a second successive defeat.
Harding gave the visitors the advantage straight from the kick off, Horsham slipped the ball wide to Alex Haddow, and his cross was turned in from close range.
Horsham looked to dominate down the slope and Lee Carney, the subject of interest from County League Three Bridges, was pulling all the strings.
But they were unable to create further clear cut chances, and were punished three minutes before the break when Greg Ngoyi squared for Ashe to guide a neat finish beyond Craig Bradshaw.
Wealdstone then dominated the second period, a goalline clearance and a good save denied Callum Martin, before Ashe scored the winner on 65 minutes when he turned in Martin's cross following good work from on loan Watford winger Billy Gibson.
Bognor notched their first away league win of the season with a goal in each half at Cray Wanderers.
James Crane’s long-range shot gave them the lead before the break and they made sure of the points when Charlie Oatway got the second a few minutes from time.
It’s a big relief for the Rocks after five weeks without a win and a dismal run of four home games which had brought them only two points.
Despite that recent run, there was no shortage of confidence as Darin Killpartrick’s men set about their task with relish from the off at Cray.
They had to soak up a fair bit of pressure from the home side but gradually found their feet and took the lead with a terrific goal on 25 minutes.
Full-back James Crane was the hero when he went on a surging run and unleashed a shot from 25 yards which gave the keeper no chance.
That knocked the stuffing out of Cray for a while and Bognor were worth their half-time lead.
Again, the Rocks had plenty of defending to do after the break, but they soaked up the pressure in a way which belied a side who have struggled so badly on the road this season.
It remained 1-0 until the 87th minute when Oatway made sure of the points, coolly beating the keeper after being put in the clear by a through ball from midfield.
Another plus point was a solid debut in midfield from Richard Hudson, back at Nyewood Lane after a spell at Horsham.
The win still leaves them in bottom place but they are now level on points with Billericay and within three points of two other sides. The Rocks visit Hendon on Saturday.
Hastings’ tie at home to Croydon Athletic in the third round of the Isthmian League Cup was called off.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here