I'm Jewish from Israel (Middle East), does that make me by definition (specifically as defined by the USA) a person of color?
Is your skin white? No? Then yes. Yes? Then no.
If I have a white skin and not from the USA, am I still defined as a person of color?
You're a blanco, yes.
until I heard the term "Person of Color" as a pre-defined term used to "segregate" anyone who isn't white
Talking opposites, White can refer to a combination of all colors while black the absence of any specter of light, therefore the lack of color. So when we're talking about white people we may be talking about humans with all the colors on them, rather than segregate they seem to include. Fascinating ain't it?