Lieutenant William Caswell chose Tenilba, now spelt Tanilba, with its beautiful harbourside location offering commanding views across the waters of Port Stephens, as the main homestead block. It was so named by Governor Ralph Darling on the land grant as the indigenous place name signifying ‘tenil’ meaning ‘white flowers’ and ‘bah’ meaning water.

Lt Caswell was an adventurer, and very determined to succeed, he had sunk most of his money into coming out to the new colony, and failure was not a word to which he was accustomed. He had high hopes and aspirations and set high standards and values.

After he arrived in Tanilba he acquired a boat for use as his means of transport across the waterways and then supervised the clearing of trees and scrub, the building of huts and outhouses, a detached kitchen, a well for fresh water, a milking shed, and a stable.

The men were all assigned tasks, usually in pairs, at Tanilba, no one was ever idle because the removal of stumps and building of stone walls and fences was a constant chore. These are some of the convict-built walls. More walls exist near Tanilba House.

As trees were felled, stumps were left, fields were ploughed, the first crops were planted out and the assigned convicts were allowed to grow their own vegetable plots near their bark huts.