"Also when leaving does going up or down matter? Should I take those actions? Or does it not matter when you have all burdens, pick whatevers easiest?"

Well, this phase is over now, but I might as well answer this for posterity. Going up or down changed which of the four areas you were in, which, aside from finding the burdens (one in each area), just determined which main attribute was used for the basic traversal option (not spending memories of light or using glasswork). So, while on your way out with burdens, you could move up or down to try to get to the area that used whichever attribute you could boost the most to slightly decrease the chance of wasted actions.