Lewes Crown Court heard 39-year-old Stephen Fisher held the family against their will just two months after he was released following a five-year prison sentence for kidnap and robbery.
During the three-day ordeal, Fisher did not allow Sara and Kevin Ayling to eat but allowed them to feed the baby.
The terrifying ordeal started for the Aylings just two days after they arrived from the Midlands to make a fresh start in the Hastings area.
Fisher befriended the family after meeting them at the Wallace Project, in Chapel Park Road, Hastings, where temporary accommodation was given to the homeless.
When the couple moved into their new flat, Fisher insisted on helping them before taking them hostage.
A jury returned guilty verdicts on five counts of false imprisonment but found Fisher, of Chapel Park Road, not guilty on a charge of making threats to kill.
The court heard Fisher was sent to prison as one of three men who kidnapped a 64-year-old man, held him at knifepoint and threatened to kill him unless he co-operated with their demands.
The trio forced their captive to hand over credit cards and stole £600.
The court heard Fisher was epileptic and had a personality disorder and a history of alcohol abuse.
During the trial the jury heard evidence that Fisher had extreme mood swings throughout the ordeal and would change from being very friendly to violence.
Mr Justice Penry-Davey ordered Fisher to return to prison to serve 12 months of his last sentence before starting three years for the latest offences.
The judge said he accepted that Fisher had used no physical violence against the family.
But he said: "This was a very frightening experience for both of them and particularly as they had with them their tiny child."
After the case Det Con Mike Ashcroft said: "Mr and Mrs Ayling were new to the town of Hastings when they suffered this three day ordeal which left them exhausted and scared."
Converted for the new archive on 30 June 2000. Some images and formatting may have been lost in the conversion.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article