When the Dutch first settled in the Cape in 1652, Governor van Riebeeck's brief included the construction of a fort and the setting out of gardens where vegetables and fruit trees could be grown. The site for these gardens was chosen close to the so-called Sweet River, a little south-west of the present Parade. By 1660, its area occupied some 18 hectares of land and it extended from Tuyn Street, now Church Street, towards Table Mountain, and coincided more or less with the lower part of the present gardens. A trip to the Mother City would perhaps not be complete without a visit to the Company's Garden in Cape Town city centre. Home to historic buildings, museums, places of worship and an art gallery, it is also simply known as the Cape Town Gardens.
The whole area was enclosed by a thick hedge of ash trees and brambles. As the new village began to take form, a number of incursions began to be made upon the area of the gardens, with spaces being carved out for the slave lodge and a new church and graveyard. In 1679 the arrival of Governor Simon van der Stel brought about a number of changes in the nature of the garden. Other properties in the vicinity of the town were now growing so much produce that its function as an orchard and vegetable garden was no longer essential to the Company, and it was slowly converted into a botanical and ornamental garden. According to accounts of that time, it was divided into rectangular blocks by hedges, which also served as windbreaks. Some of these blocks were retained for the planting of vegetables, while others were dedicated to herbs, fruit trees, indigenous plants and exotic trees. The plan was divided by equally spaced avenues and at the top of the Gardens there was a water mill fed by a mountain stream. The northern side was enclosed by a high wall with a fine entrance gateway and important visitors to the Cape were accommodated in a small summer house erected by Simon van der Stel. During the 18th century the garden was gradually extended in the direction of Table Mountain including the establishment of a zoo at its top end, and it probably reached its present boundaries in about 1790. Apart from this expansion, a fine ornamental gateway designed by Thibault was added and the summer house, with its own private garden, and had been given over to the Governor as his residence. Following the British annexation of the Cape in 1806, the talents of Thibault, Schutte and Anreith were again combined to build the southern entrance and the so-called "Lion Gateway" to the zoo. During the 19th century added areas of the gardens were excised, when Government House was enlarged, and a portion of land was granted to the Anglican Church for the erection of St George's Cathedral. In 1832 Thibault's entrance from the city was demolished, and in the 1880s, large areas were taken over by Parliament House and the Public Library. More recently, the Museum, Art Gallery, the South African College and a number of monuments and other structures further reduced its space. Government Avenue was declared a National Monument under old NMC legislation on13 August 1937, while the entrance, walls and gateways on the southern side of Government Avenue, in Orange Street, were proclaimed on 6 January 1961.