I‘m still working on my setup so your considerations are most helpful. What stands out to me is the option to use an airgapped old crappy laptop to provision the keys. Ideally one with manually disabled modems. That way nobody without physical access should be able to compromise it.