It is normal for fluid to leak out of both holes in the case you're describing. That is sometimes a way to gravity bleed the system when it's installed in the car too. Open the bleeders & let it flow. Also a troubleshooting test to see if the hoses are collapsed, etc.
Conversely, lets say you're replacing a front caliper & don't want to put plugs in the line or have it gravity bleed out on you. You can prop the brake pedal down just a little bit & it will stop the flow of fluid (use a prop rod, or snowbrush works good too, lol). Keep in mind, the brake lights will be on, so if it's a long term thing either pull the fuse or the bulbs, but for a short term solution it works great. I've done it for many years in the shop with no issues.