Does Quick Fix Work: Unveiling the Truth Behind Shortcuts