Thomas Ivory was born in Ireland in 1731 or 1732, possibly in Cork. An article in 'Anthologia Hibernica' of May 1793 states that he served an apprenticeship as a carpenter in Dublin and that he subsequently became an architectural draughtsman. After a period working with the gunsmith Thomas Truelock, he is said to have decided to become an architect and took drawing lessons from a 'Mr Bell Myers', who was probably the surveyor and topographical draughtsman Jonas Blaymire.
In 1764 Ivory was appointed the first master of a school for the teaching of architectural draughtsmanship to boys from "modest background" established by the Dublin Society. It was a post that he held for the rest of his life. Meanwhile he practised as an architect in Dublin.
His reputation as an architect was established when, in 1773, he won the competition to design the Blue Coat School in Dublin. Other architectural work by him included 89 and 90 Harcourt Street, Dublin (1776 and 1780); Newcomen's Bank in Dublin (c.1781); Kilcarty in co. Meath for Dr, George Cleghorn (1770s); and a bridge at Lismore, co. Waterford (1773-79). In 1775 he was appointed architect to the revenue commissioners for whom he designed the Custom House at Colraine, co. Londonderry (1783). Ivory died at his house, Mount Pleasant, in Ranelagh, Dublin on 27 December 1786.