Heat the stockpot over medium heat and add the butter. Add the saffron threads and cook gently until fragrant, then add the shallots and continue to sauté for another minute. Add the garlic, fennel, and leeks and stir. Cover and sweat the vegetables for 8-10 minutes, keeping the heat low enough so the veggies don’t brown.
Add the sherry and the lemon and orange zest and cook for about 2 minutes. Then add the potatoes, tomatoes, bay leaf, and lots of freshly-ground pepper, and finally, the broth. Add salt to taste. Bring it all to a boil and simmer for about 15 minutes until the potatoes are cooked.
While the potatoes cook, prepare the fish. Remove any bones and skin and cut into large chunks.
Once the potatoes are soft, remove the orange zest, the lemon zest, and the bay leaves. Then remove about 3 cups of the stew to a blender, blend until smooth, and return to the pot, bringing the mixture back to a slow boil.
Add the fish and the clams if you are using. Cook for five minutes, then add the shrimp. Taste and adjust seasoning as needed. (Salt, pepper, sherry, lemon juice). Cook for two minutes and remove from the heat.
Serve with crusty bread with butter or goat cheese or rouille.