Here are the rules at the DC jam I go to, they seem pretty standard as jams go:
"We all stand in a big circle around a pool table and take turns. Some stand in the front and some in the back. As we go around the circle, each person has a chance to choose a song (or instrumental). If a beginner wants to play an instrumental really slowly, then that's what we do. If you're not comfortable with your playing, stand in the back; and if you don't want a turn just say, "I pass." No need to feel pressure."
Couple tips: we tend to go clockwise around the room, so knowing that you can position yourself if you want to get a tune or three under your belt before you have to call the tune. (I assume they go counter clockwise in the Southern Hemisphere, but I will let someone from Australia chime in)
When going to a jam I am new to, I try and ID a guitar or mandolin regular and stand opposite them in
the circle. (This works if you play gtr as well as mandolin, if only mandolin then just find another mandolin player). Reason being you want to see their hands. Since you are new there will be a number of songs you don't know and sometimes get lost in the tune. I find it helpful if I can see what chord we are on as we'll as obviously trusting your ear.
Finally, with mandolin if lost in new tune don't be afraid to mute the strings and give a no chord chop in time. It will be appreciated over a wrong chord out of time. As you know the mandolin s job is the snare drum. This is not to say don't learn the tunes, but early on it is a good safety net.
Oh yeah have fun.