Can Hidden Markov Models Accurately Match Prerecorded and Live Guitar Data?

In summary: The strings on the guitar are plucked with a pick, which has a sharp, high-pitched end and a flat, low-pitched end. They are plucked open (or slightly open), which means that the pick is pressed down all the way to the string. When the string is plucked open, the higher-pitched end of the pick rubs against the string and makes a louder sound. When the string is plucked closed (or slightly closed), the lower-pitched end of the pick does not rub against the string and makes a softer sound. When a string is plucked at the correct harmonic (in other words, at the correct place
  • #1
ADDA
67
2
Hi, I'm attempting to match prerecorded data from a guitar, to data recorded, as a user plays, as shown in the following video:



The red lines are the prerecorded data, and the cyan lines are the live data. In the above example, each string is plucked open, so if you look to the lower left first, the low E string, the above that the A string and so forth.

From this pdf (no pun intended) http://www.indiana.edu/~iulg/moss/hmmcalculations.pdf posted on wikipedia, I can infer that the Baum Welch process gives an estimate of matching two sequences. My initial thoughts, were to scale the data since it is in the [0.0-1.0) range, round to an integer and then set that as a state, yet I think that there is another way of doing this. Any suggestions? I'm using the jahmm library for now.
 
Technology news on Phys.org
  • #2
I have no idea what the question is you're trying to solve. Reconstruct a melody from guitar souds?
I also have no idea what data you're using.
 
  • #3
willem2, I'm attempting to write down music to tabalture for musicians. Here is an example:

 
  • #4
So you are attempting to listen to a sound, and determine exactly what string was plucked at which fret? If you play the guitar, I'm sure that you know that many notes can be made in many different ways. For example: middle C is both the fifth fret on the G string and the first fret of the B string. They are the same note, but do not sound the same because they have different undertones. If you play two strings at the same time, it gets even harder. Here is how I would approach this problem:

Record myself playing each fret on each string, record the sound, then map it to the correct tab. I'd then build a neural network and train it using the data that I have produced. That should give you a decent start, and you can expand the training material to include things like multiple strings being played together.

I'm not sure what the second video is. If those squares are supposed to represent frets, they do not match the sound that I hear.
 
  • Like
Likes stoomart
  • #5
newjerseyrunner... thanks for your reply... I've been trying to train a network with the above data to no resolve, yet however, I've somewhat succeeded in find the root tone through signal processing alone. Here is my research:



What do you think? Some omissions and a few mistakes, yet I'll be ready to move forward, maybe after some more testing. The above video (in this post) does not include any AI or recording of tones. I've been thinking of training a network as a user might play yet am still unsure of proper error calculation. Anyrate, the above video (in the last post) is A Am B Bm C D Dm E/Em F G
 
  • #6
The calculations for neural networks are done through back propagation, they are not to be put in manually.

It's looking good. I'd think maybe the next step is to start adding limitations to how far apart the green boxes can be. Most of the tabs displayed that I saw aren't actually playable with one hand. You can only play 4 strings at a time (plus the open strings) and your hand is only so big, so it doesn't make sense to have one string on the 12th fret and the other at the first.
 
  • #7
I watch the following video on neural nets:



which change the way that I write them. I had thought that the activation function was the hyperbolic tangent. My efforts to find the root tone have been difficult because most chords involve an octave overtone. So, the root tone may have 1, 2, or 3 overtones. Each corresponding to 12, 19, or 24 halftones. I am unsure of how to train a network because, in my experimentation, there is no absolute way to know for certain whether a tone is plucked or an overtone, and I've just had another idea, of which I've forgotten.

Regarding decision tree like limitations, I can think of exceptions: bar chords (only 4 strings), and capos (only the open strings). I realize that you are offering help, and I am thankful.
 
  • #8
Do you know how to play the guitar?
What you are trying to do is VERY difficult if you want a general solution. This is one of those problems that even the human brain struggles with since there is always multiple play something on a guitar and the "best" way to play something will often not be determined by what sounds best but what is possible'/ easier to play (which will also depend on the tempo of the song; e.g. switching from a bar chord to an open chord takes more time then to just use another bar chord).

Hence, if you put two experienced musicians in a room and ask them to transcribe a piece of music it is very possible that they would come up with very different solutions (nowadays this is less of a problems since it is often possible to find a video and actually look at the hand of the guitar player when transcribing a song) .
Add the possibility of different turnings (common) and the use of a capo (also very common) into the mix and you have a very tricky problem.
 

Related to Can Hidden Markov Models Accurately Match Prerecorded and Live Guitar Data?

What is a Hidden Markov Model?

A Hidden Markov Model (HMM) is a statistical model that is used to represent a sequence of observations that are dependent on an underlying, unobservable process. It is a type of Markov chain with hidden states, where the states cannot be directly observed but are instead inferred from the observed data.

How is a Hidden Markov Model different from a regular Markov Model?

A Hidden Markov Model differs from a regular Markov model in that the states in a regular Markov model are directly observable, while in an HMM, the states are hidden and can only be inferred from the observed data. Additionally, the transition probabilities between states in an HMM may also be dependent on the hidden states, whereas in a regular Markov model, the transition probabilities are typically independent of the states.

What are some applications of Hidden Markov Models?

Hidden Markov Models have a wide range of applications, including speech recognition, natural language processing, bioinformatics, and financial modeling. They are also commonly used in areas such as signal processing, time series analysis, and computer vision.

What are the key components of a Hidden Markov Model?

The key components of a Hidden Markov Model are the hidden states, observed states, transition probabilities, emission probabilities, and initial probabilities. The hidden states represent the underlying, unobservable process, while the observed states represent the data that are observed. The transition probabilities determine the likelihood of moving from one hidden state to another, while the emission probabilities determine the likelihood of observing a particular data point from a given hidden state. The initial probabilities represent the probability distribution of starting in a particular hidden state.

How is a Hidden Markov Model trained and evaluated?

A Hidden Markov Model is typically trained using the Expectation-Maximization (EM) algorithm, which involves iteratively estimating the model parameters based on the observed data. The model can then be evaluated using various metrics, such as accuracy, precision, and recall, depending on the specific application. Cross-validation techniques can also be used to evaluate the performance of the model on unseen data.

Hot Threads

Recent Insights

Back
Top