assuming you're talking about treasureclassex.txt here:nodrop is added to the rest of the probs from all the other columns, then the game randomly (or not if picks is a -ve number) chooses which item to actually pick, based on prob/sum_prob, nodrop doesn't relate to any other column.
if you want to stop monsters dropping gold, remove the gold and associated prob columns and shift everything else left 2 columns. i don't think it likes it if you have empty columns that don't finish.
i'm planning an extensive overhaul to the drop system in my mod (so things like flayers/fallen can't drop big heavy armours/weapons, and non-ranged attackers will never drop bows, etc.), one thing that's stopping me is in the act X equip/melee trees, there are additional columns near the end that seem to give odds for getting items, and i don't know if they're actually calculated/used, or if they're just there for information.
any assistance on that would be appreciated 