Piano World Home Page Forums Our Most Popular Forums Piano Teachers Forum The future of piano playing?

Why use your hands when you can go direct from brain to keyboard electronically?

I think technology is fascinating and the progress made, especially when it will be of help to people that need it, is fantastic. The connection the player can have to their instrument I don't believe will be replicated by technology, ever. It is like our fingerprint or a snowflake, Each one is different and unique.

rada

I think it's great for those with disabilities, but I doubt it would ever replace actually playing an instrument. First of all, the technology would really have to become much more sensitive to not only determine which finger/note to press, but also the velocity to the point where it can be musical. Personally, I think that would be harder, as I'm a kinesthetic learner, so I wouldn't really be doing anything I could feel. Still, I think it's a great idea for those with injuries and inability to actually play.

I wouldn't dismiss it quite so quickly. Ten years ago, I saw some amazing video developed in Germany, University of Dusseldorf I believe, which allowed us to see brain waves of people imagining playing the piano. Imagining, not actually playing. Although complex, I don't see too much more required which is necessary to hook up to a Yamaha piano player, for example.

The pleasure of playing is precisely that of using your body, senses, and mind in interaction with an instrument. So it's like proposing a way of ingesting the nutrients of ice cream without having to taste it or savouring its coldness on a hot day.

I think one day this would be great for composing, just imagine the music in your mind and publish the score on Amazon, all without getting up from your bed

This brings a whole new meaning to the phrase "mental practice"

Aha, this is the same argument we "acoustic" piano lovers use when salesmen hawk electronic keyboards as "just as good as a piano" argument.

So, the wired performer is in the recital hall, totally rapt, mentating the best performance of their life, in a pppassage, when someone in the audience opens a candy...

And it's true (the acoustic piano lovers argument, that is, not the salesman).

That may not be universally true.

For some it may be the act of creation, and some mechanical aids may allow them to produce a more complex work of art.

The piano is limiting. You only have ten fingers, 88 keys, and they can only go so fast. There is lots of music that can be made within those confines - but it isn't necessarily wrong to want to go beyond those either.

You're familiar with "conductor" programs?

The "EEG cap" creeps me out.

Fascinating! Let's wait and see how fine grained the signals that can be picked up get over time. Indeed this has the potential to change the way we interface to many tools, including musical instruments.

The aim of the system as presented in the program was to do things like allowing a paraplegic to point his wheelchair to the right and make it go forward or back. Nothing like music was ever mentioned, not even getting specific notes to sound, let alone the subtle touch of a musician. The system works by having the person imagine crude large motions such as moving a hand (arm?) to the left or right or up and down. Sensors pick up the brain signals, and software translates the input into preprogrammed commands to the airplane. Maybe someone got more than that bit of a gist than I did.

Right now in the digital piano forum there is a discussion on samplings, where the sound is produced through samplings of recorded actual pianos, versus a system that translates the actions of a pianist into responses like those of an acoustic piano to the playing of a pianist. There are some limitations due to the mechanical construction of digital pianos. But the remaining limitations are due to the programming, which is limited, while the combination of touches by an accomplished pianist is limitless. So if we are talking about playing and interpreting a piece of work, such a system which depends on the programmed responses to brain signals, where programming is by its nature limited to the writings of the programmer, then such a system would limit the musician rather than expanding what he can do.

Of course it depends on what you're talking about. If you just mean playing as many notes as possible as well as as fast as possible, then some limits of the human body may be removed. However, this particular system depends on the mind firing off signals of physical body movement so it is still dependent on 10 fingers, since he cannot fire signals to non-existent fingers number 14, 15, and 16. Nor could a non-pianist be able to send signals, since he has no muscle memory to fire off.


I'm not sure that I follow. Do you mean improvising?



I looked them up. I could find only one example, which was developed by Max Mathews starting in the 1950's
http://www.youtube.com/watch?v=3ZOzUVD4oLg
and then there was a "Tapper" which seems to be a takeoff of that, where you tap a single key or a foot pedal
http://www.youtube.com/watch?v=wKd4RyhivtI

As I understood it, there is already a score of an orchestral work that can be played back, which is readily done these days. I have Finale which gives me a primitive version, so I'm familiar with it to that extent. In the demos, I saw that they could affect an overall beat so that in fast music in 2/4 time one might conduct only the first beat of each measure - the music as a whole would be slower or faster, with all the notes proportionally correct. In slower music that would be on the beat. They could also make the music overall louder and softer in a very general way.

There didn't seem to be the possibility of making a note come in early or late for effect, rubato of any kind. Is there a way of highlighting a set of instruments or a single instrument to make it prominent in a given passage? How about the special effects of an instrument? When a conductor nods to the violinists, those violinists are musicians with their own minds and knowledge - they will choose a particular bow attack, choose to play closer to the bridge and a hundred subtle choices. These are the things that give performances their character. The conductor programs seemed to be limited at "overall louder and softer of the group as a whole", and "general tempo". This also seemed limiting, rather than expanding horizons.

Of course if you have no means (like most of us), and you're stuck with Finale or similar and watching the program run with no possibility of input while it's running, then this will expand your horizons. Is that what you're talking about? My familiarity with conductor programs is limited to those two clips.

Here is a discussion of Tapper.

http://www.musanim.com/tapper/

Look at the Venn diagrams and the colored charts.

The limitations are only what somebody hasn't thought of yet.

Being tied to an acoustic piano tends to limit our thoughts to what can be easily played. At least, for me. But if we free ourselves a bit from that box, potentially you could add functions to fit your desires to express yourself.

Tapper hasn't been developed because there's no market force, so it's fairly rudimentary and the user interface is not real intuitive. But yes, you can do unlimited rubato and whatever dynamics digital pianos allow. It plays the next note when you press a key. If you don't, it just waits for you. It has been used in performance (one of the Brahms concertos.)

You could potentially add features to anything digital. Rather than being "limited by programming," you are freed to add anything your brain can come up with, whether you can play it or not. Does your creativity demand a long high trill, while both hands are occupied down in the middle and low range? No problem. Can you reach a 15th? No problem. Etc.

I heard a band playing at an Oktoberfest dinner recently. There was a very clean and very tasteful bass line in the mix. But no bass player - no tuba, upright, or electric bass to be seen. The keyboard player had split his keyboard and was playing left hand bass, sounding like a bass, while his right hand was playing piano sounds. It was very effective.

But, nobody would want to do that? Well, maybe you or I wouldn't. Maybe somebody else would, and lots more.

I think limitations inspire creativity though. My piano tech grew up in Dominican Republic and had no piano. He had heard it on the radio and fell in love, then played one at his school. He memorized the sound of each note at school then went home and made a paper keyboard of all 88 keys and learned to play the song her heard that way. He would double-check his playing when he went back to school. He taught himself how to tune a piano once he was able to actually own one years later.

Not having a piano isn't a great way to learn to play piano, but because of this his ears are very well trained. There are many other ways limitations bring out our creativity. Playing a written composition, for example, is a limitation. You have only the notes on the page to work with. You could say improvisation would be better because then all 88 keys are possible at any given time. But within the notes on the page, one has to be creative in finding ways to play them to communicate effectively. To say that improv will be better/more creative than playing composed music because it's less limiting is not necessarily true.

Nothing against improv, either. When you have potential 88 keys to work with, an improviser sets his own limits in some way - a key signature, a scale, a chord progression, or even a musical form. The same would happen in the mind of someone playing music on a piano via electronic means. Even if it is set up so that things that are physically impossible to play with the human body (in interval of say a 15th in each hand, or 14 simultaneous notes all a 3rd apart), the brain would set its own limitations, or its own categories for things to make sense out of it. The brain does not handle chaos well. We like categories or some sort of structure within which we can work. Not only that, but I say within limitations are where true creativity can live. Set the limitations too close, however, and the result will be limited. Set the limitations too far, and things tend more toward chaos/not making sense.

I literally don't understand this. Surely what we imagine and hear in our minds precede what we can produce, and generally surpass what we can produce.

I'm familiar with your link including the Venn diagram since I did my research before writing.

In any case, we are discussing John's link. What's being developed depends on imagining familiar physical actions produced by your body, such as raising an arm, and it is intended for such things as moving a wheelchair forward and back, or turning it.

That's what is being developed currently. I think it's interesting to speculate about how much further these ideas might go, and how that interacts with our conceptions of piano (or instrument) playing. Even if it's not likely that the product can achieve such fine-grained control sometime in future decades or centuries, it's interesting to think about. As in the discussion between you and TimR, it reveals ideas about why each of us plays the piano, and ideas about using external products (like a software-programmed EEG cap) and to what extent such things might extend or limit our playing and our conceptions about music.

A piano has one key board. It sounds like a Yamaha or a Steinway or a Bechstein. You work within that tonal palette.

A harpsichord commonly has more than one manual.

An organ has several manuals and a large number of stops which can be played singly or in combination.

You can be just as creative with 8 crayons as 64 or 128. But some people desire more colors in their palette. (and I'm not limiting it to timbre)

Just on the heals of the above reference, I find this: The Borg take over Piano Playing

Most of us use a qwerty keyboard.

A few brave souls have opted for the dvorak.

Are they still typing?

Most of us use a linear piano layout.

But there are options, like the Wicki-Hayden.

Is that still piano playing? There are some things that are much easier to do on other keyboard layouts.

My digital has a pitch blend wheel. Hmmm.

I'm assuming that despite the title this will also not be about piano. The last one pointed toward advances in wheelchairs.

It's about a very preliminary step in brain-to-brain communication. For me, it invites speculation about what might be possible. As far as what's actually done, nothing particularly useful for anything yet. If you care about present actual results, don't bother reading it. If you like to speculate about how something like this might be useful (or misused) for piano playing, learning, and teaching (*) if it were a million times as developed as it currently is, then you might find it interesting.

(*) regardless of the first uses the researchers might suggest for it, and certainly regardless of the particular arena the researchers are using for proof of concept, which in this case is video games.

Can I assume you don't know who the Borg are?

I had another thought on this. Of course, what both articles are presenting is technological developments and where they will lead, we do not know. And their impact on music, performance, etc., is also unknown. There was an article in the Wall Street Journal a couple of days back on how many sales departments are having to train young hires on how to use the telephone. Now, if you're older than 45, this makes no sense whatsoever, but most young people have never heard a dial tone; most don't know that you just dial the number and don't have to push a button to connect. And most interesting, most don't know how to carry on a meaningful conversation with a customer, because they've been texting their entire lives. If you came of age BC, ie, before computers, the technological change over the past 40 years has been nothing short of amazing. We could not begin to imagine having piano lessons over little desktop computers or ipads. But today, we are. Reflect for a moment on what technology was available to the everyday family in 1953 and compare it with what's available now. Will we make similar progress in the next 60 years? Likely.

If your purpose is to engender discussion on technology and music learning, I can go with that. Just not those links, which were never intended for music, with the suggestion in the title that they were. I found that confusing. The subject you brought up now makes more sense to me.

All too often, I find myself squarely ensconced in the box. There's some fresh air outside of it, I recently discovered!

I try to teach my students to use their creative juices and imagination when interpreting music. What better way to stretch that imagination than to try some mind bending with new technology?

Just my warped thinking, of course!

Hang in there John ... It's a good idea to go outside and smell the roses ... too often we sit on our rear-ends watching "the box" ... only exercising in making visits to the kitchen.

But for those ancients who learnt to play the piano ... there is nothing quite like the actual playing of the piano ... I'm presently
recapturing the marvel of Stephen Sondheim's "Send in the Clowns".

"Isn't it rich? ... Are we a pair? Me here at last on the ground, you in mid-air ... Send in the clowns.
Isn't it bliss ... Don't you approve ... one who keeps tearing around, one who can't move.
Where are the clowns? Send in the clowns."

They took over Sweden, the invested themselves into a tennis player who was a baseliner who dominated tennis for awhile.

I am joking, John.

Warped thinking? What warp? Warp 3, 6 or 10?

Seriously John, it takes a few decades to understand the saying "It was the best of times, worst of times" and all the rest of what Dickens expressed with that idea. I am in turn horrified and excited by technology.

I'm here at 4:30 AM, writing to you, and we have never met. A couple centuries ago people mostly did nothing at this time of night - no electricity. A little box did not suggest that they had mis-typed a word. And if they were unable to make the "red line go away", they could not then paste the irritating word into Google, to have the correct spelling suggested for a word that was too confusing for the spell-check.

So everything is great. No problems. Nothing but new marvels ahead.

Meanwhile, three of my adult students have stopped taking lessons recently because they lost their jobs. People are getting fired just as it would be time to get permanent jobs because with another month of employment they would get insurance and benefits.

There are many things I love about this Brave New World, but I would not like to be 20-something right now and worry about supporting a family.

And I do not think this is a good time to teach something that takes many years to begin to master.

I don't know if this remotely reflects what was going through you mind as you started this thread, but it is my personal reaction to it.

I'll drink to all of that!
Creativity, flexible thinking, Stephen Sondheim, new technology, piano teachers, Bjorn Borg, and if not to The Borg, then to the Binars (who better to inspire duets?).

Perhaps I'm drinking cool aid here!

As with Gary, I'm terribly conflicted about technology. One side of my brain loves it, the other side abhors it along with the implications of more and more "artificialness" in our lives. When I graduated from college, Star Trek was all the rage. Their "communicator" was a fantastically wild dream. Today, it's not only a reality, many of them have on-board computers which out perform the biggest and best computers of that era. Nikolas & I have chatted half way around the world on our "communicator" and in fact, were able to meet each other in Frankfurt, Germany, via our "communicators." This is surely one of the blessing of technology.

OTOH, the dark side of technology has also hit the music world fairly hard. Organ building has suffered even more than piano building. First, electronic organs took over and now, many churches are just using CDs and thumb drives to provide "music" for their singing and service music. Recordings make great teaching tools, but often serve to stifle creativity and original thought. Being able to locate obscure scores on imslp is a great blessing, but misused, as it often is, denies an income to the various components of music publishing business, who cannot survive without these small but continuous royalty streams. Ditto with performance rights. I'm quite certain that when the first "home computer", the Altair, went on the market back in the early 70's, the designers had no inkling that their technological innovation would permit music teachers from around the world to share ideas, music, sounds, etc, in near real time. I'm not sure where these two technologies will end up in 50 years or more, but I suspect that any teacher looking back in the archives to our current discussion will be bemused either by our blindness or vision.

For as long as I've read your writings, you have pushed the idea not just of acoustic pianos, but grand pianos, because of what they will allow a musician to do physically with the instrument. So for you, of all people, to seem to be pushing the idea of playing the piano hands-free was incongruous. I actually thought you were spoofing the virtual world by pushing it to absurd extremes.

If I am to take the idea of mind-controlled playing literally, I'd say that it is ANTI-imagination. A small child begins his ideas with building blocks, with a stick, some stones, a stream or puddle. Ideas arise from playing around with things, and cohabit with this playing around. In music you play around with sound and all the textures you can create. We already lose a lot through digital pianos, though we may gain other things.

But if you intend the virtual world and the cyberworld, then yes, it has not been used to its full potential. When I joined, I presented the idea when teaching Baroque dance music, to have students actually watch Baroque dance. Or why, if you are introducing a composer or history, use technology to regurgitate facts in writing when so much more is possible? It's like teachers try to duplicate what they know, but this time print it instead of using a typewriter.

Teaching over the Internet has its good and bad sides. The bad one being the availability of hucksters with their "learn to play in 30 days" for $29.99 in 10 easy installments, and the fact that you can't be there physically and see everything from every angle. I had 6 months of lessons at the age of 16. The woman cooked while I played - the only "teacher" available. Access via the Internet to a decent teacher could have saved me from 30 years of ignorance (as in not-knowing). But oh, if they could invent teleporting, then we'd have the best of two worlds. Um - scratch that - how would you keep unwanted people out of your studio and living room?

You know what would be really cool for distance learning? A thin flexible glove both slip on, so that the student could feel what the teacher's hand is doing and how it's feeling the keys. Good or dumb idea?

Keystring, your comment about grands is all so true. Yesterday, one of my families who just purchased a grand is now all gaga over it and are wondering why it took them so long to accept the idea that a grand is so full of possibilities......

My original post was indeed meant to be a spoof, until I started reflecting on possibilities. Both positive and negative. As some wise person once said, technology is a genie which can either become our master or our servant.

I think one take that I have on technology is that if you start with a need, look for solutions, and then use what is out there for your purpose, you'll do well with it. But if you get led by the nose by salesmen who want to sell the technology to you with invented purposes of their choosing - well then you get the equivalent of the car that promises social popularity rather than transportation. So it's gotta be on our terms. I also thing - using our IMAGINATION.

Off topic - teleporting. When it occurred on Star Trek, we were still tied to telephones attached to the wall with cords. That was our reality. What we have now is people texting each other wherever and whenever - while you're at the dinner table; an e-mail can come in while you're asleep in bed. Now translate that into teleportation. Though it's a dream come through for checking if your student is really practising like he says he is.

Yes, great idea.

And it should go both ways.

Let the student feel how the teacher is doing it. And let the teacher also know how differently the student is doing.

In working virtually and even working in person with a teacher, I've found that they can tell what I am doing physically and how it "feels" since they have an experienced eye. But it doesn't work the other way around. A lot of playing is touch. And that is very much missing when you only have sight and sound through a camera and a microphone. And that camera is usually stationary. I know of distant masterclasses where the camera can be manipulated at both ends (pan, zoom), but that's expensive and rare.

This:

http://www.dailymail.co.uk/sciencet...-ones-hold-hands-internet-sale-year.html

Everybody thought it would be used as a sex toy.

What I think will be the future is something akin to what we have in teaching built into your casio today.

Except with more measurements. Maybe using the game kinxet cameras or something.

Right now my casio only measures my timing between notes and the actual note.

No hand position.

No pedal in put

No PP vs FF you just hit the right key.

If you had that and had a transferable to PC/tablet interface then, a lot of the things your teacher does could be done by a computer.

Talking of modern technology ... my computer came to a shuddering halt when my "wireless" mouse died on me ... I should have seen the warning signs when a winking light lit up atop the beastie .

But a visit to the local shop saw me buying two new Duracell batteries ... so I'm back in business (had the savvy to buy two spares).

It's a hard life!!

I don't know. Sometimes my teacher seems pretty puzzled by whatever unique and creative error I have discovered. Since most of these issues come from my brain, perhaps he could benefit from a special tin foil hat that would gain him access to the dusty corners of my mind.

That's a scary thought! It would probably send him running from the studio!

I think the first "communicator" was as bigger than some cell phones today. And all you have to do is to increase the range of communication, then bam, we will have the same thing.

I don't know if I am conflicted about technology - probably what is DONE with it, John.

Generally it gets dumbed down. Also we have to learn to differentiate between technology and the products.

I had to get tires put on our second car, and while waiting for it to be done I had to put up with Divorce Court. While at the dentist there was General Hospital. On the basis of those two shows I would judge TV as nothing but a blight on humanity.

Yet right now there is Newsroom on HBO, House of Cards on Netflix, the latter streamable. Both series are superb.

I believe Socrates was basically against the way relying on text was pushing away the ability to memorize by rote an incredible amount of language. I suppose he had a point, but I am also glad other people disagreed.

And I would not underestimate its power. We now have the ability to interact with groups of people who are rather large in number but who are only a very tiny percentage of the world population.

Also, don't forget that what people do with technology is linked to their intelligence, creativity and willingness to think out of the box. People who are comfortable in boxes will find a way to live there - with or without any available technology

Also a great deal of everything that was once hand-made is now gone, or nearly gone. That is a definite downside, a huge one.

I heard that complained about as far back as the 1050s. At that time I thought it was just a bunch of old people complaining about the world changing, but I now think it is an important point. If you listen to recordings from the early 20th century, the playing was more uneven, more willful, more eccentric, more mannered, but it was also more individualistic, often unique. There is a price to be paid for everyone being able to hear everyone else from a very young age.

Correct. The one thing we can be sure of is that of all the people who are predicting what will come next, most will be wrong, and those who are right will probably be more lucky than smart. Just as a stopped watch is right twice a day, if you read the predictions of enough people, SOMEONE is likely to be right.

It is a bit like the infinite monkeys with infinite typewriters idea.

Whoa!!

Yeah, just how old IS Gary? I mean, I watch True Blood, so I'm familiar with the concept, but.....................

Just think how well he plays after all this time! Even Bill Murray in Groundhog Day played respectably, but this would take it to a whole new level!!

This technology has been around since around 1970 and I see hardly any progress

Basically the interface requies tons of concentration and is very slow. You control usually 1 motion at a time. Thinking of other things than the one thought you need to control the device confuses the device

Compare that with playing piano where you control many, very fast motions. Like deforming your hand to the right chord, pressing the fingers one by one with a controlled force and motion, while also moving your hand itself up and down; and doing similar things with your other hand; and meanwhile also reading the music that is coming ahead.

There is a VERY long way to go to get anything useful out of this. I consider this idea practically dead except for extreme conditions like when any physical motion is out of question eg paralysed people or people in fighter jets turning at high G.

Brain to brain interface was available in the 1970s? I didn't know that. Could you point us in the direction of any research with actual devices from that time period? Thanks.

I was talking about brain-computer interfacing, like the OP was about.

Thanks for clarifying. I had posted two articles in the thread. But just as da Vinci had postulated about powered flight, it was some time span before the Wright brothers came along and gave us a preliminary demo. I am anxious to see what comes of this.

I can imagine an application of brain-to-brain-to-movement communication, as described in the second article you linked, in this way: the teacher thinks of the desired movement; this is communicated to the student's brain; the student's fingers move (and wrists and hands and arm and body and legs and feet... playing piano as a whole body process of course). This would be to be a way of communicating what the desired movement feels like. (Leaving aside what wouter79 points out, that currently it's only simple and isolated motor skills that can be communicated -- imagine that the technology has developed.)

But the drawback is this: the recipient in the article said it felt like an involuntary twitch or tic when his finger moved under the sender's control. So would that involuntary twitch feeling persist even as this develops? In that case I'm not so sure that this would be a useful teaching technique, because it's not clear to me that a student could progress from feeling an involuntary movement under the teacher's control, to reproducing that movement voluntarily under their own control.

Direct brain connection may be the holy grail, but there are lots of ways to add technological aids right now.

For example.

A digital piano normally has 3 pedals. Why? Because it's a copy of an acoustic.

There are all sorts of things you could program additional pedals to do. A trill pedal. A real time transpose pedal. An octave doubling pedal. A loop play pedal. A rhythm pedal, a pitch bend pedal - anything you can think of and program in software. A historical temperament toggle, an Asian scale toggle, on and on.

John, how might you picture brain to brain interface applied to piano playing?