If you want this effect then you need to utilize the GridBagLayout.
http://download.oracle.com/javase/tutorial/uiswing/layout/gridbag.html
Have fun with that one =P
EDIT:
You can work around the problem by employing a mixture of FlowLayout and GridLayout to get a similar effect. However, this solution will become extremely tedious and messy as your layout complexities become bigger.