Let's look at an example.

Here I am proudly displaying the latest Amoeba Crunch poster, designed by Chip Kidd and available in the store for $13. Look at that reflection. It sure looks like the text (and therefore the whole mirror image) has been flipped horizontally.
However, that's only compared to how we expect it to look:
We think the text has been "flipped" around an imaginary line going from the ceiling to the floor, like this:
The problem is, how does the mirror "know" which way is up? Why didn't it just as easily do this?



(Of course, that's not quite right. The whole image would be flipped so I'd be on my head, but bear with me.)
The faulty assumption is that forward text factors into things at all. You'll notice that easily readable, forward text does not appear in the original photograph. In fact, it's backward both in front of the mirror (albeit viewed through the paper) and in the mirror image. This is easier to see if I take another photo with the paper turned around.
Now it ought to be clear that the text reads forward both in the original and the reflection, though in the reflection you're looking through the paper to text written on the other side. So in a sense the mirror hasn't flipped anything at all -- it's the same on both sides.
Another example: Consider my left hand, which has a wedding ring. To be overly pedantic, from my perspective the ring is on the hand that's to the left of my body. When I look in the mirror, as in the photo above, it also appears to the left (from my own perspective) of my body's mirror image. There is no left to right flip here. It's left and left on both sides of the glass. Likewise, the mirror hasn't changed the text. It's readable on both sides.
I can hear some of you crying foul. After all, from my mirror image's perspective he's wearing the ring on his right hand -- that's the flip. If you're thinking that, what you've done is redefine "left" and "right" in terms of his perspective. You're considering him as if here were someone in the real world. But he isn't!
Let's think about this. If in the real world a friend and I stand toe to toe facing each other, it's no stretch for us to agree her right hand is, from my perspective, to the left of her body. Without thinking, I assume the same is true for anyone facing me, including my mirror image. The trouble is, my mirror image is not in the real world. There is no guy over there! It's just me over here. The world in the mirror image is not an extension of the room I'm in. The fact is, the definitions of left and right don't change across a mirror and neither do the definitions of up and down. The left hand has the hand with the wedding ring on both sides. This can be hard to get around. We're hard wired to deal with people as real beings, and it sure looks like the guy in the mirror is a real guy standing there facing me with his right hand where I'd expect it to be. But the key is that my mirror image is just that, an image, a trick of light. The usual assumptions about people don't apply.
So if the mirror isn't swapping left to right or up to down, what is it doing? Is it showing you what you'd see if you were standing in place of the mirror? No, because if that were true the ring would be on the opposite hand (from your perspective, on the hand to the right of the body).
What a mirror does is flip things in to out. Specifically, along the imaginary line that is normal to (that is, perpendicular to) the mirror's surface. If you're facing it as I am here your mirror image is not merely you turned around, as if you stood facing the opposite way, but is you flipped through your belly to your back. (That would make a great animation. If someone has the time and expertise to put it together I'll include it.) 

Notice that the unmarked side of the paper is toward the mirror in both the real world and the mirror image. Likewise, the paper is always between the mirror and me. There is not so much an axis of symmetry as a plane of symmetry, the plane of the mirror. So the reflection has nothing to do with the mirror "knowing" left from right or which way is up. The only direction it "knows" about is toward or away from its surface.
In mathematical terms, we can define a mirror's behavior with a Cartesian coordinate system having its origin on the mirror's surface and your eyeball on the positive z-axis, like so:

The negative z-axis extends into the mirror image world. For any point (x, y, z) in the real world you can find the corresponding point (x', y', z') on the far side using these equations, suitable even for the mathophobic:
x' = x
y' = y
z' = -z
This page explains mirror images in a slightly different way that you might find more clear.