Simmer the fruits in 300ml/½ pint water mixed with 115g/4oz caster sugar for about 15 minutes covered. Turn the fruit from time to time. Drain the fruit and retain the liquid.
Place the cooking liquid in a saucepan and make it up to 300ml/½ pint with added water. Add the remaining sugar. Boil the mixture until it begins to turn golden and forms a syrup. Brush the sides of the pan just above the syrup level with water to prevent the sugar crystallising.
Boil until the syrup becomes deep golden but not too dark. Pour the syrup into the bottom of a four pint gratin dish and spread it out evenly.
Meanwhile, for the pudding, whisk the egg yolks with the sugar until pale and frothy. Carefully whisk in the milk and cream, then fold in the citrus zests, vanilla, orange water (if using) and the nutmeg.
Mix together the brioche bread and the poached fruit and scatter the mix over the caramel in the gratin dish. Pour over the egg and cream mix and allow to stand for at least one hour or preferably overnight. This helps the pudding to rise when it is cooked.
Preheat the oven to 180C/360F/Gas 4. Place the gratin dish in a water bath or bain-marie (or place in a large deep-sided roasting tin filled with enough water to come halfway up the side of the gratin dish) and cook until nicely browned (about one and a quarter hours). Half-way through cooking, dot with teaspoons of apricot jam and a sprinkling of caster sugar.