Upping the ante comes from betting games.
The ante is the amount that all players must commit to the "pot" before the game begins.
The pot is what is won in the game, and will be the total of the ante plus all bets during the game.
Because a player must at minimum "ante up," or pay the ante, in order to be allowed to play the game, upping the ante can quickly demonstrate how serious, committed, and solvent a player is.
The significance of "upping the ante" is that it increases the overall cost of the game, hence the idiomatic meaning of "increasing the cost or risk of something."