Thanks for the link.
Okay, I at least figured out why my container was breaking. I had a fixed size of 900px width and when I added a border around <img src> in the second column it collapsed the whole structure. So I've made my buttons narrower to accomodate the fixed width structure.
I've still have yet to grasp how to vertically align the buttons properly. I've been looking through various tutorials and I'm still not getting it. Do the stack of 5 images as a group need to vertically align to another <img> instead of aligning themselves within the column? If that's the case, then it seems a bit crude to create another <img> element that is as long as the column to align them properly.