The brigantine began as a sail and oar driven warship used in the Mediterranean in the 16th century. It was lateen rigged (large triangular sails on long booms) on two masts and had between eight and twelve oars on each side. Its speed, manoeuvrability and ease of handling made it a favourite of the Mediterranean pirates - “brigands” - hence its name.
The Atlantic maritime nations adapted it during the 17th century by replacing the lateen sails with square-rig on the foremast and by the gaff-rigged spanker with square-rig above it on the mainmast. (In the Mediterranean the lateen rig on the mainmast tended to be kept.)
Nomenclature has changed over the years, but it is fairly safe to say that, as far as the main (rear, stern, aft) mast is concerned:
- A brig is fully square rigged plus a gaff-rigged spanker.
- A hermaphrodite brig has fully fore and aft rig (no square rig at all).
- A brigantine has a gaff-rigged spanker, with square-rig above it
Brigantines were still being built as sail-training vessels well into the 20th century.
Reference: "The Oxford Companion to Ships and the Sea."