Bits are probably more useful when talking about specialized storage. Byte usually means 8 bits, but doesn't always need to, and not all data is stored in byte chunks.
A bit is the smallest piece of usable datum, so that makes sense when discussing this technology.