I believe you would have to decrypt them a second time. For example if you wanted to be real secure you could have the USB device, an encrypted folder that holds important documents and files you want to back up, and inside of that could be a password database that requires a Yubikey or similar device.
I believe what you are talking about is kind of like using a combination of cascading algorithms like AES->Twofish–>Serpent.
I could be wrong though. If I am I hope someone can correct me.