After all, when you invert one you get the other.
Huh? Please explain.
Csus4 = C F G
Fsus2 = F G C
I would make my decision on the bass note, I don't think these chords are inverted too often - and when they are it usually seems to imply another chord.
Plebious is correct, you need to listen to the bass and the interval that the suspended note creates with the bass. These chords aren't typically inverted because doing that ruins the effect that suspensions create - that is, the tense suspended note resolving, a 2nd falling to a tonic or a 4th falling to a 3rd of a triad. The bass must be prominent in order to hear the tension followed by the resolution. Of course, not every suspension resolves, but most do and certainly almost all of them in classical music. Jazz is a different thing altogether.
I suspect the first hurdle is that you haven't done enough ear-training to be able to pick out intervals quickly. You need to practise this a lot because you won't understand these tension/resolution effects until you do. You need to be able to relate any tone you focus on to the bass note. Get a friend to help you by playing random pairs of notes, and you have to figure out the interval. You can do that on your own two if you close your eyes and play a note with each hand. Eventually you will learn to hear intervals very clearly and accurately. You need to do the same thing with triads, then 7th chords. Then you move on to suspended chords.
It will take time and patience but it's an incredibly important skill for any musician. Keep at it.