Preheat the oven to 350°F.
Place 1/2 cup of the bread crumbs in a skillet and drizzle with oil. Mix well, and place over medium heat, stirring until golden.
Remove to a plate and let cool. Set aside.
Mix remaining breadcrumbs with the tomatoes, olives, capers, pine nuts, parsley, red pepper flakes, and orange zest. Drizzle with oil and mix well. You want enough oil so that, when the mixture is squeezed, it holds together in a clump. Set aside.
Set out the swordfish slices and have toothpicks ready. Take a small fistful of the filling - a couple of tablespoons - and squeeze it into a log shape. Place it at the narrow end of a swordfish slice, and then roll it up, and secure with a toothpick. Repeat with the other 15 slices of swordfish.
Line a roasting pan with parchment. If there is any leftover filling, spread it over the parchment as a base for the involtini.
Place the swordfish rolls in the pan and drizzle lightly with oil. Sprinkle the reserved golden breadcrumbs over the top.
Bake for 13-15 minutes. Serve hot with an arugula salad, and some of the leftover stuffing, if desired.