I currently store my backup keys in Bitwarden, but like you said, that's protected by my OTP solution.
I see two options here:
store backup keys in a completely separate way (paper in a safe, encrypted off-prem storage with security key storing that password, etc)
have the OTP running in lots of places - currently just my phone, but could run it on my desktop and laptop as well
The first is probably better, but I'm planning on the second because I'm less worried about a physical breach of my home by a technical attacker than me losing/breaking my phone at a really bad time and needing my wife to read off an OTP token over the phone in a pinch.
I do have a Yubikey that I don't use though, so maybe I'll try out the more secure option.