I'd fill the entire cavity. Not packed very tightly, but just enough to fill the gap. If you get mineral wool or fiberglass 6 inches thick, that should be perfect for a 5-inch gap.

--Ethan