On the shoreline of Hong Kong Harbour where the City Hall stands today was once the home of Hong Kong's oldest club. Victoria Recreation Club was established here in 1849 after having been in operation in Canton since 1832. The primary functions of the club were water sports, with rowing in the main. Early members also enjoyed social gatherings in a congenial colonial setting in the spacious clubhouse, with mean facilities and a bar, and fitted with all the necessary amenities.

In the early twentieth century, the new premises of the club included a regular swimming pool for the time as well as ample storing facilities for boats, a badminton court and gymnasium. Rowing boats, outrigger fours, eights and sculls, were used extensively in the harbour. VRC Regattas were held regularly in Kowloon Bay. The South China Morning Post carried a short article of a VRC Regatta on Friday, 6 November 1903, which is undeniable proof of the VRC's contribution to the rowing history of Hong Kong. The large swimming pool was practice ground for some of Hong Kong's finest swimmers and divers over the years.

Members of the VRC were successful in their sports and participated in local events and represented Hong Kong in international events. In the present clubhouse in Deep Water Bay there are pictures showing the winners of many swimming and water-polo competitions, which illustrates the club's successes.

The proximity of the harbour made participation in the annual cross-harbour race, a popular sports feature in the colonial days, a must for VRC member at first; then, it was made open to the garrison and local residents. Before the race was forced to stop in 1973 because of the condition of the harbour, it attracted many hundred competitors ranging from young children to hardy old swimmers who competed annually. The numbers exceeded a thousand at times.

The club's charter shows that one of the major objectives of the club is ‘To promote swimming, rowing and other sports and pastimes'. This objective inspired the formation of separate associations to promote the practice of specific sports here. Notable achievements include the Hong Kong Amateur Swimming Association; it enabled Hong Kong competitors to take part in every Olympic Games since Helsinki, 1952, and the association is going on from strength to strength.

The crowing achievement was the formation of the controlling sports body for the colony soon after World War II: the Amateur Sports Federation & Olympic Committee of Hong Kong, as it was called subsequently. ASF&OC held initial meetings in the VRC and sponsored the club with a strong group of sports leaders. Indeed, it was founded under VRC's aegis; the first meetings to work out the original charter of the ASF&OC were held in the VRC clubhouse in Central.