Consumers are paying higher prices in shops because of the "unjustifiably high fee" charged by payment processor MasterCard.
The Office of Fair Trading said the fees were too high and infringed the Competition Act.
Every time a transaction is made using a card the retailer's bank has to pay an interchange fee to the card issuer.
The bank then passes this fee on to the retailer, who passes it on to the consumer through higher prices.
The OFT report said: "These fees act like a tax on retail transactions paid by all consumers in shops that accept credit cards."
The interchange fee is set by members of MasterCard, which includes most of the major banks.
Individual card issuers are free to negotiate their own interchange fee with MasterCard but few do as they have no incentive as the charge is not paid by them but charged to the retailer's bank.
Vince Cable, Liberal Democrat spokesman for trade and industry, said: "The report suggests Mastercard has exploited its dominant market position - forcing up retailers' costs, and ultimately hitting all consumers in the pocket.
"If true, this is yet another example of rip-off Britain and the OFT must act to stop the abuse."
But MasterCard claimed the OFT's proposals could lead to reduced choice and poorer value for credit card customers.
A spokesman said: "If interchange fees are reduced, banks will be forced to recover costs elsewhere and some issuers may withdraw from the market."
MasterCard would not disclose what the current fee it charged was but according to the Cruickshank report, which was published in 2000, it averaged 1.1 per cent of the transaction value.
Given that about £30 billion was spent on MasterCard branded cards during 2001, the sums at stake are very significant.
The OFT is giving MasterCard an opportunity to either justify its existing agreement or to change it so it qualifies for an exemption under the Competitions Act.
If MasterCard fails to do this by the spring the OFT said it would force it to take action."
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article