Yeah, it seems like it would just weigh down on your head uncomfortably. I think the only thing I'd want to stash in a hat would a hair tie in case I wanted to pull it back in a ponytail later.

[And even then, she'd rather just put it into her pocket...]