Hi Ngutor Vanger,
The way I managed to change the theme for the overview map. One thing to note is that when you make changes on the main map, the changes also reflects on the overview map. So to be able to change te overview map, I had to switch off all the other layers except Kaura_wards_overview and OpenStreetMaps layers on the main map. Then I had to ensure I labelled the wards on the Kaura_wards_overview layer on the main map. Then on the Kaura Wards Atlas Template, click to select the Overview Map.
You can use Interactively Edit Map Extent to change the zoom of the map to cover the area of interest. You can also change the map scale manualy to fit the area of interest. Once you are happy with the overview map, under layers, you tick lock layers and lock styles for layers so that the overview does not change even after you change the main map. See snapshot below.
Then you can now switch on the other layers on the main map and style them the way you want, the overview map should remain the same because you have locked the layers and styles on the overview map. Hope this will help.