Whisk together the flours, baking powder, cinnamon, ginger, and salt.
In a separate bowl, whisk together the sweet potato*, eggs, and milk.
Fold the two mixtures together with the pecans, if using.
Preheat the oven to the lowest setting 150-200 degrees. Place an oven safe serving dish or casserole pan in there.
Heat 1 or 2 skillets over medium-low heat. Add a dab of butter and swirl around the pan(s). Dollop in about 1-2 ounces of batter per pancake. Cover with a lid. This is an unusual step for pancakes but we found that it helps the thick batter cook through enough to make flipping them easier.
When the top of the pancakes appear dry and the bottom is golden, flip them over. Repeat with the remaining batter. Place in the warming oven as you finish them.
Serve with butter, fruit, yogurt or more nuts.
*If you need cooked and mashed sweet potatoes, simply preheat the oven to 400. Pierce a large sweet potato all over. Place in the oven for 30-60 minutes until it is soft. The microwave also works. Cook the pierced sweet potato for 3-4 minutes at a time, rotate and continue cooking until it is soft to the touch. The peel will be easy to pull off at this point.