This should normally not affect it since items have different modifiers for each action, basically all this does is increase the gather proficiency of the tool, for it to affect the speed of breaking walls it would need to modify the demolish proficiency.