Indeed. The US Presidential Seal had gone through several changes over the years, including a change in the direction the eagle faced around the time of one of the world wars (which is how this notion of it switching back and forth during war time supposedly got started). But yeah, towards the olive branch, unless you're doing a historical piece, then you'll have to look it up.