If most people won't take your dollars/gold/... because they fear getting caught, then what use is it to you? You won't be able to spend it. There were actually people like that in early Soviet days, guys who managed to preserve a stash of their former wealth, but without the ability to use it meaningfully.

The best mitigation you have for that kind of government is having a ticket out.