Music Transcription of Wav files in Java

Question

I have project about music transcription using Java, the thing is,, I have created an applet that records sound and saves it into a WAV file, the player should only play the piece he/she would like to transcribe, after that I am stuck at the point were I have to get information from the saved WAV file and use this information to create a MIDI file,, and after creating the MIDI file I have to generate a Musical Score for it (Sheet Music) and all these must be done in Java.

I searched everywhere for a clear explanation on how these stuff can be done, but I did not find anything direct :( since I am new to Java, I would like anybody to help me with this please, the topics that I am having trouble programming are:

1. Getting information about notes being played in the WAV file.
2. Knowing the Notes.
3. Creating a MIDI file equivalent to the WAV file.
4. Generate a SCORE from the MIDI file.

I have also tried Fast Fourier Transform after performing Segmentation on the data being read, but I think that went absolutely in the wrong direction :(

Please if anybody can help me on the topics specified above, and how to program these only in Java, it would be really highly appreciated :)

By the way, the project is:
Player plays notes on Piano >> Records his playing >> Player gets the SCORE of his playing.

Answer 1

I am by no means an expert in this area, so I apologise in advance if this is all rubbish.

To get the notes from the file, I think you need to do an FFT (Fast Fourier Transform) on the WAV file, but with only like 10ms of sound at a time. Then you find the highest peak on the FFT for that time frame, and move on to the next "frame" of 10ms, or whatever. You do the FFT again, and if the highest peak is a different frequency to the previous peak, then it will say it's a new note. To see how long the notes are, count the number of peaks within a certain threshold of each other and multiply them by the time you are using for each frame (10ms, etc)

I will say again, I am not expert and there are probably other ways of doing it.

That aside, I hope this helps... even a little.

Answer 2

There's a very good open source sound editor called Audacity. Yes it's written in C++ but you may find the answers that you're looking for in its source code. Translating C++ code to Java is not that hard.

Answer 3

I am not an expert in that area but once I was playing a little bit with Xuggler (java ffmpeg wrapper). This library was able to extract lots of interesting data from the media file, allows file transcoding and many other cool features. Here is the link: http://www.xuggle.com/xuggler/

Answer 4

What you want to do is currently impossible. The scope of what you described is beyond anything current professional music programs can do, and they have literally hundreds of man-years of programmers' time invested.

You might achieve something if you radically cut your requirements.

• to find the pitch of sound, use FFT; this is the easiest part; you will have to restrict yourself to single notes here, you will not be able to cleanly read chords.

• you will not be able to discover what the time signature or speed (bpm) of the tune - your only chance is to include some kind of metronome in your application and force the player to adhere to the beat. If you want to support the swing time, it must also be configured by the player.

• When making midi, quantitize all the notes (=move their starts and endings to the nearest 1/4, 1/8 or 1/16 of measure)

• use a ready typesetting system to create the score; the musictex might work for you; generating a tex file will be much easier than doing the drawing for yourself; if I were you, I would ignore some of the rules of typesetting music (I would certainly drop the beams; there are to many rules concerning them)

If you restrict yourself to single melody, choose to ignore most of the music typesetting, make the piano player use your own metronome and limit his rhytmic choices, you might succeed.

Answer 5

I had the same idea in mind and tried out a very simple but incomplete solution. As fdreger's post pointed out, even professional software isn't actually able to perform this task correctly. (I tried some demo-versions of IntelliScore, AudioScore and some other software and none of these seemed to be particularly efficient when working with polyphonic music)

But if you want to try out on yourself, I used the source code found here : http://www.psychicorigami.com/2009/01/17/a-5k-java-guitar-tuner/ that helped me to find out how to determine the frequency. (FFT will be more accurate, but a bunch more complex).

To display the notes on screen, I used the abc4j library that can be found here : http://code.google.com/p/abc4j/

But, as pointed out above, it works only with monophonic music (one single voice).

Good Luck!