Charlton House was built between 1607 and 1612 for Prince Henry, Sir Adam Newton, Dean of Durham. Newton was tutor to Prince Henry, eldest son of James I. Evelyn, who was well acquainted with Newton's son, stated that the House was built for Prince Henry; Newton, however, ceased to be the Prince's tutor in 1610; the Prince Henry died in 1612 at the age of eighteen and so never saw the house completed, so that it is unlikely that the house was at any time a royal residence.
Newton continued in the employment of the King and to live in the house until his death in 1629. He is buried in Charlton Church where there is a large monument, commemorating both Adam Newton and his wife Katherine Puckering, in black and white marble. The estate passed to his son Sir Henry Puckering Newton, who as a royalist, had to leave Charlton during the Civil War although his family continued to reside at Charlton House.
In 1658 the estate was purchased by Sir William Duice, who made additions and improvements to the house. He lived at Charlton House in great style.
Sir William Langhorne was a wealthy East India merchant who purchased the estate from William Ducie in 1680.
When William Duice died without children in 1715 it passed to his nephew Sir John Conyers. It continued through Sir Baldwin Conyers (died 1731), William Langhorne Garnes, Margaret Maryon, Rev John Maryon and Margaret Maria Weller (died 1777). On Margaret Wellers death the property passed to her daughter Jane wife of Sir Thomas Spencer Wilson. One of their daughters, Jane married Spencer Perceval.
Spencer Perceval went on to become Prime Minister in 1809. In 1812 he was assassinated in the House of Commons, our only Prime Minister to suffer this fate. He is buried in Charlton Church. When Jane Wilson died the House went to her son Sir Thomas Maryon-Wilson and thereafter descended in a direct line to Sir Spencer Maryon-Wilson.
It was Sir Spencer Maryon-Wilson who sold the estate to the Council in 1925. Part of the estate was converted to a public park, part was taken over by the London County Council to form a playing field and athletic track and this is how it is today. The House was used for a while as a museum and, following the restoration work carried out on war damage to the north wing, it became a community centre. The extension that was designed in 1877 by Norman Shaw was used as a public library until a few years ago when it was moved into the main buildings
In May 2014 Charlton House officially became part of Greenwich Heritage Trust as it was handed over from Greenwich Council. Along with the council, RGHT also manages a number of other buildings and memorials within the Greenwich area.