Adam Flanagan and Landry Zahana-Oni were both sent off in the dying stages for violent conduct as Hastings went out in disgrace.
The pair, who had both been booked earlier, were shown straight red cards for separate incidents which left Michael Blackwood on the deck clutching his face.
It was an embarrassing end to United's first ever match in the first round proper of the FA Cup and marred a spirited display from George Wakeling's Dr Martens premier division side.
Flanagan was unfortunate to be dismissed for his challenge on Blackwood when the pair went shoulder to shoulder for a high ball five minutes from time but Zahana-Oni had no complaints after blatantly catching the Stevenage defender in the face in stoppage time.
It was the first time Flanagan has been sent off for Hastings and he and his French team-mate now face three-match bans.
Flanagan said: "I went to see the referee afterwards because I was hoping it was only a second yellow but he said it was a red for use of the elbow.
"I certainly didn't mean to catch the Stevenage player. I felt we were going for a high ball together and I used my arms to lever myself up. I don't think I made contact."
Zahana-Oni, who has already been sent off this season, admitted his guilt.
He said: "It was a sending off. It was a bad thing. I got really frustrated in the last 15 minutes and just lost it."
These two unfortunate clashes epitomised a frustrating afternoon for the East Sussex side at Broadhall Way.
The visitors, cheered on by 600 vociferous fans, fell behind after just five minutes.
To make matters worse, Wakeling had specifically briefed his players on Jude Stirling's long throw which led to Richard Howell's header.
"The early goal killed us," admitted Zahana-Oni. "We tried hard but it wasn't quite enough."
After that setback, Hastings enjoyed the better of the half but despite all their possession the closest they came to equalising was Duncan McArthur's long-range left foot effort which fizzed just wide.
Hastings continued to search for the equaliser after the interval but their Conference hosts were a big, strong team who looked fitter, which is to be expected from a club with several full-time players.
Stevenage shaded the match over 90 minutes and could have doubled their lead when Blackwood's shot came back off the inside of the post.
Hastings never really troubled Mark Westhead in the home goal and Wakeling's team clearly missed Ellis Remy, who was out with a hamstring injury.
His replacement Chris Honey, who has yet to score this term, tried hard but looked desperately short on confidence and ability against an experienced Boro rearguard.
Flanagan felt Hastings more than matched Stevenage and, combined with a memorable 5-0 win over Conference outfit Kettering in the fourth qualifying round, the defender feels the outlook is bright.
Hastings are in the top ten in their league and Flanagan argues promotion to the Conference this term is still an outside possibility.
He added: "It was desperately frustrating for us and the fans. We wanted to win for them because their support was magnificent throughout.
"I was disappointed to be sent off but what hurt the most was the result. I don't like any result other than a win.
"We were told all about their long throws so that was very disappointing. Now we have to focus again on the league.
"We are well-placed in the table and if we keep the team spirit and do well at Christmas then we could still go up.
"I certainly haven't given up on it and neither have any of the other lads."
Wakeling said his players gave everything and felt the dismissals did not alter the course of the game.
He said: "I thought the first one was harsh but Zahana deserved to go. We worked hard but they are a big club and we are a small club."
Stevenage: Westhead, Travis (Wormull 45), Fraser, Trott, Jackson, Howell, Stirling, Midson (C. Macdonald 70), Blackwood, Scott (Campbell 65), G. Macdonald. Subs not used: Sigere, Wilson.
Hastings: King, Yates, Hegley, Osborne, Burt, Flanagan, Playford, McArthur, Zahana-Oni, Simmonds (Ruddy 80), Honey (Webb 80). Subs not used: Body, Brown, Cornwall.
Attendance: 1,821.
Referee: T. Kettle (Maidenhead).
Man-of-the-match: Duncan McArthur - deserved better with at least one of his speculative efforts.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article