My teacher learned me to not play the wrong notes from the beginning.
I have read that some advocate it was best to stop and NOT play the key if I wasn't sure it was the right note; that way I wouldn't practice or play the wrong note to start out with. That is a hard "rule" not to break and I'm not 100% diligent on this, especially if I'm on a roll and things are going rather well.
Well, I have to clarify this a bit. When playing a note, when dropping the finger onto the key, the hand moves to next note at the same time. So you must know which key to play otherwise you can't make that move correctly. Then for the duration of the note, the finger on the key relaxes and the next finger hangs straight above the next note. Then when you play the next note, you only have to drop your finger. There is no way you can miss that note, because if you let your finger fall you hit the right key.
If you rhythm is correct, but you misses notes, then your technique isn't good enough. You'll have to slow down until you don't miss notes. Work on your technique, so after this you can speed up. This is when learning a piece.
If you have learned a piece and you play it and have fun with it, so you're not so concentrated when learning. Well an error sometimes isn't that bad if you don't stop playing. But if you played a piece many times, and the piece get worse. Sometimes it helps when you take a few measures and start learning the notes slowly again.