Victorian times refers to United Kingdom (England), when Queen Victoria was the reigning queen. The rich people lived all over the country there, including the countryside and the cities (just like poor people). The rich lived in mansions, huge estates, surrounded by many servants and luxury. They owned most of the land that farmers worked on.
The rich Victorians had a large country house and one in town too.
Same as today. Wealthy in the country, poor in town.