Alas, it seems the only real cure is to keep trying until you don't get so nervous. It can be really tough. I can still fall apart -- much more often than I'd like! -- when I start tunes in sessions, even when they're tunes I thought I knew cold. The fingers forget, the brain forgets, whatever .... This from someone who performed classical solos for auditions, juries, recital pieces, in big ensembles, etc. for years, including an entire solo piece with an orchestra.
So in those situations, I've found that the lifeguard system helps: i.e., find a buddy who can rescue you -- if you start playing and begin to flounder, your friend can jump in and play along and thus the tune is salvaged and everyone wins. (then one day you can rescue your buddy, too)
** Oooh, here's an idea --> why not work up a duet or two or three with a friendly fiddle or whistle player or such, and then ease into the solo work from there?
But mostly, I think you just have to keep falling off the horse until you learn how to stay on.
Meanwhile, a few quick tips: don't press the flute against your lip too hard (or too lightly!), make sure it's in about the right spot on your lip before you start, think of the tune before you start,
remember to breathe frequently, even more than you do at home, try for the best tone you can but be satisfied with any tone at first, and just keep doing it, even if it means forcing yourself to play alone in front of people
(I found that a bit of quiet noodling during those "breaks" in a session (i.e., when a bunch of people head for the bar at once) can also be helpful for easing into public play -- just be very, very sensitive to the fact that someone else may be wanting to start an actual tune!).
Once again, good luck. Mostly, we just have to "play as if no one's listening", to butcher a popular aphorism.
Hang in there. We're with ya, sister!
Deja Fu: The sense that somewhere, somehow, you've been kicked in the head exactly like this before.
