Invariant mass is mass of a particle (or of any object) that is a mathematical idea rather than a real physical feature conneted with inertia of an object. The feature connected with inertia is called relativistic mass. The difference between the two masses is such as between a photograph of a man (corresponding to invariant mass that never changes) and the man himself (corresponding to relativistic mass depending e.g. on his velocity).
The name invariant came from relativity theory according to which many ideas in physics are relative, which in physics means "depending on the fame of reference". The simplst example of such "dependence on the frame of reference" is velocity (that should be distinguished from speed, a magnitude of velocity of an object). The velocity is always of something in relation to some reference frame (E.g. to a train, or to the Earth, or to the Solar system). E.g. we may be at rest in relation to a train (zero velocity in relation to the train) (even soundly sleeping in it) and moving with speed of 100km/h with respect to the Earth. Everybody who was ever riding a train understands this. That the velocity is relative feature of an object.
Not everybody knows though that even if an object may have a relative velocity its mass (or rather energy) grows with this velocity (which means that energy has inertia (relativistic mass) and at the speed of light it would be infinitely big (and that's why it can never reach speed of light. Many folks consider it a strange thing (almost impossible). When they do they go to any physicist and ask her: "Is it really so strange? How come?"
Then the poor physicists has to say: "Yes, it is true. When the volocity grows, the kinetic energy of the object grows as well and since the energy has mass (or rather inertia) the inertia grows too with velocity and when speed is close to the speed of light the inertia is so big that with each push that object gets the inertia (the relativistic mass) increases more and more and so it absorbs almost the whole energy of that push and so the velocity increases only a smal bit. It can never exceed the speed of light in any reference frame that this velocity is observed. That's the relativity of velocity.
So the physicists, not to get crazy, invented a mathematical idea of Invariant mass which never changes and so it can be used in math of the phenomenon not to make physicists to go crazy. This invariant mass is just the relativistic mass divided by the γ factor that depends on velocity at which we observe the moving object. So if we see that something has inertia M its invariant mass will be m = M / γ.
If you are a really curious and want to know what γ is then I can tell you that γ2 = 1 / (1 − v2 / c2), and all physicists have it embedded in their genes so it is easy for them to convert any relativistic mass M that normal folks use to measure ther goods with, to invariant mass m, said the physicists and wiped swet from her forehead, promptly walking away from the curious non physicist, not to look dumb, since the non physicist wanted also to know how do we know that γ factor is just as she desribed it, since it would need some theoretical knowledge and she was just a regular experimentalist not knowing much about ancient features of relativity. Especially its theory.