Finishing fourth in Group D with a win and two defeats to their name, Serbia could nevertheless have qualified had their last match against Australia ended 2-2 instead of 2-1 to the Socceroos.
Had referee Jorge Larrionda penalised Australia's Tim Cahill for handball in the closing minutes, instead of deeming it accidental, Serbia could very well be preparing for the last 16, instead of packing their bags to leave South Africa.
"I am disappointed with some decisions," said the Manchester United defender.
"We expected to go into the second round and we are going home disappointed."
Serbia were arguably punished for their profligacy in front of goal against Australia, and such was the superiority enjoyed by coach Radomir Antic's team they could easily have been four goals to the good before half time.
"We had a great first half but I was scared after those chances were missed ," added Vidic.
"That is football, small details can determine the winner."