Opened in 1866 as a specialist hospital for children at Gough House on Queens Road West (Royal Hospital Road; Tite Street). In-patient beds were provided from 1867, and new wards opened in 1874 following an expansion.

A convalescent home opened in Margate in 1876 for the hospital, and a new street, named Tite Street built next to the hospital. New out-patients building was built in the 1880s. In 1890 the hospital absorbed the St Gabriel Home for Infants, and a new convalescent home was opened in 1892 in Broadstairs, Kent (named the Victoria and Zachery Merton Convalescent Home); the Margate convalescent home continued to be used as a long-term children's hospital. The hospital was expanded in 1903, and renamed Victoria Hospital for Children in 1905.

During the First World War two wards were used by the 2nd London General Hospital, but restored as children's wards in 1916. The hospital purchased a neighbouring house in 1921, and a new physiotherapy department was opened in 1922. Further expansions were carried out in the 1920s.

At the outbreak of the Second World War, the out-patients department was taken over by the Chelsea Borough Council to use as a First Aid Post and Decontamination Centre, and a paediatric casualty service was established. Due to damage caused by bombs, in-patient were sent to hospitals outside London.

There were plans to amalgamate the hospital with the Belgrave Hospital for Children, but the hospital instead became part of the St George's Hospital Group. In 1964, the Ministry of Health closed the hospital and the services were transferred to St George's Hospital in Tooting, where a children's ward was named the Princess Louise Ward.