Well, glad it isn't just me then! I found that when one of the hideout modules is built that modifies XP gain, the memory and mag drills skill spams a lot. When it isn't, the overall spam was lesser.
Out of curiosity, how do you modify your XP for specific skills as such? Because I agree, the way some skills progress vs others is not ideal.
You need an All In One (AIO) mod where you can change the skill XP modifiers, such as multipliers, limit before debuff, etc.