How did congress attempt to maintain a balance of power between northern and southern states in the mid-1800s?

They made sure that the number of free states and slave states were equal.  This was bests seen in the Missouri Compromise of 1820.  Under this compromise it stated that under the Louisiana Purchase, the territory north of latitude 36° 30' that makes up Missouri's southern boundary would be established as free states and territory south of that line would be organized as slave states. Main was also admitted as a free state as part of the compromise to maintain the balance of Missouri's admission as a slave state.  

The answer is an appellate court.