Pretty simple idea, but something that has been bugging me for a while, and that I constantly notice myself getting annoyed at.
Basically, the durability of a tool will only decrease if that tool is the appropriate tool for the current task (or in other words, if the tool is faster at mining that particular block compared to default speed, or has some sort of reason to be used on that block).
So swords will only lose durability if used to attack (and break webs etc.), pickaxes only for rock metal etc.
I don't think this mod would feel too unnatural, I think it'd be a nice thing to have. Would just make inventory/tool management less of a hassle (I can run around with my sword and break dirt blocks without having to worry about losing durability)
Not sure how difficult it might be to implement though!