The smallest nuclear weapon known to the public was the W54, a 10.6"x15.7" (27.3 x 40 cm) cylinder that only weighed 51 lbs (23 kg). The W54 was used in both the Davy Crockett recoilless rifle (a nuclear mortar for ground troops) and the Mk-54 SADM (Special Atomic Demolition Munition), a hand-delivered nuclear time-bomb for attacking enemy ports. The prototype for the W54, tested during Operation Hardtack in 1954, was even smaller, at just 10.6"x11.8" (27 x 30 cm), close to what many nuclear scientists think is the theoretically smallest nuclear weapon. The Davy Crockett had a 10-20 ton yield — intentionally kept low to be safe to those firing it — while the SADM had a variable yield between 10 tons and 1 kiloton.
To create a nuclear weapon requires a critical mass of a fissile material, and a chassis for a gun-type trigger or explosive lenses. A critical mass of plutonium is about 10.5 kg (23 lb), 10.1 cm (4 in) across. This is not enough to start a multiplicative chain reaction, but does produce enough radiation to be deadly if you were holding it.
To produce a chain reaction requires upping the plutonium, just a bit — just 10% over critical mass is sufficient to create a nuclear weapon with a yield of 10-20 tons, already in the range of the Davy Crockett warhead. 20% over critical mass gives a yield of 100 tons, while 35% over critical mass can reach 250 tons. The smallest nuclear weapons would have a yield somewhere in this range.
The public can't know for sure what the smallest nuclear weapon is, because it is probably classified. The Soviet Union worked on a variety of nuclear weapons that remain completely secret, and the US has as well, although there is more transparency in the latter case. One former Soviet general, Alexander Lebed, claimed the existence of "suitcase nukes" on a news interview in September 1997, setting off a chain of speculation about whether the smallest nuclear weapon might fit in a 60 x 40 x 20 cm suitcase. The general consensus is that this would be enough room to create a nuclear weapon, especially for a technologically sophisticated country. However, there is little concrete evidence for it.