Two goals from Robert Vittek and a clincher from substitute Kamil Kopunek meant Slovakia went through as runners-up to Paraguay in Group F - and the 2006 winners finished bottom below New Zealand.
It was a sad end to Marcello Lippi's spell in charge as Italy joined France, the team they beat in the final four years ago, as high-profile early casualties.
The veteran, who had already decided to step down from his post, appeared not to shake the hand of Slovakia counterpart Weiss after the final whistle.
That will not have tarnished the experience for the Slovakia players though and they thoroughly deserved to come out on top in an exciting encounter.
Vittek gave them the lead in the 25th minute with a well-placed shot after Daniele De Rossi had carelessly squandered possession in his own half.
Italy had been poor but were pepped up with a double substitution at half-time that saw Fabio Quagliarella come on.
The Napoli forward thought he had equalised after his shot beat Slovakia goalkeeper Jan Mucha but was blocked on the line by Martin Skrtel, the Liverpool defender.
It needed a decision from a linesman to confirm the ball hadn't crossed the line and it was not long before Vittek took advantage of some poor defending at the other end to make it 2-0 with a smart near-post finish.
Italy were determined to go down fighting and pulled one back with nine minutes to go through Antonio Di Natale, who slotted into an empty net after Vincenzo Iaquinta had seen a shot parried.
The excellent Quagliarella put the ball in the Slovakia net soon afterwards and once more it was the linesman's call as it was ruled out for a very marginal offside.
Italy knew an equaliser would most likely keep them in the tournament but were caught out again when substitute Kopunek marked his entrance with what turned out to be the clincher, lofting the ball cooly over goalkeeper Federico Marchetti.
Still the Azzurri refused to lie down and Quagliarella made it 3-2 two minutes into a four-minute stoppage time period. Slovakia held on for the remainder however and were left to celebrate a famous scalp.
Slovakia (4-4-2): Mucha; Pekarik, Skrtel, Durica, Zabavnik; Stoch, Strba (Kopunek 86), Kucka, Jendrisek (Petras 90); Vittek, Hamsik.
Subs (not used): Pernis, Kuciak, Cech, Salata, Weiss, Kozak, Sapara, Sestak, Holosko, Jakubko.
Italy: (4-3-2-1): Marchetti; Zambrotta, Chiellini, Cannavaro, Criscito (Maggio 46); Gattuso (Quagliarella 46), De Rossi, Montolivo (Pirlo 56); Pepe, Di Natale; Iaquinta.
Subs (not used): Buffon, De Sanctis, Bocchetti, Bonucci, Marchisio, Camoranesi, Palombo, Gilardino, Pazzini.