I have built mine this way, reusing a very old 'normal' tower case:
Take a CPU with limited TDP, like 35W or 65W. Use a passive CPU cooler. Use a semi-passive power supply that is too strong for the actual need, so it's going to stay in it's passive state all the time - they are cheaper than the real passive ones.
Then use 1 really large fan (25 or 30 cm) that goes at the side of the tower case. It turns very slow because of it's size, so it is nearly inaudible. No other fans.