Thanks all for the info. From reading it looks like the mirror-less is about the same as my Nikon D5000, just had the mirror engineered out hence the small body. The reviews mentioned a few down sides but they seem minor to me.
What is your experience?
Mirror-less typically has a smaller sensor, so higher ISO's will have more noise. That's the main drawback. Newer camera are much better though and some even have a larger sensor. I wouldn't let that affect your decision, unless you need to take professional low light photos.
The biggest advantages are the size and weight. This is usually the biggest reason to go mirror less. The best camera is the one you have with you. I'll carry my E-M5 almost anywhere, but would not be taking my SLR with comparable lenses. I take my Olympus with me kayaking (it's weather resistant). No way I could could take a SLR with huge lens to get the same reach.
Sports photography is more of a challenge with my camera, but I've been able to capture some nice shots that were much better than the people with the huge SLR's. I've shot football (day and night), lacrosse, and basketball (indoors) for my son's teams and get asked for the photos all the time. SLR's definitely are more suited for sports/action shots though. The exception is the new OM-D E-M1 Mark II. That camera is mirror-less and one of the fastest cameras in the world. You'll pay pro prices for it though.
For an every day carry around camera, it's hard to beat the quality you get with a mirror-less camera. Just make sure the camera you buy has the lens choices you want. One thing to watch for is the 35mm equivalence. For example, a micro four-thirds 45-150mm lens is actually equivalent to a 80-300mm 35mm lens. So a much better reach in a smaller lens.
This site can give you a good comparison of size.