Multi-Instrument Partitioning & Identification with Harmonic Amplitude

Abstract

Multi-instrument classification is important for music recommendation, remixing tracks, and several other applications in the music industry. Our goal is to develop an algorithm that takes a song with one or more instruments and output the names of the instruments in it. Previous research has succeeded in single-instrument classification, but multi-instrument classification remains unsuccessful due to lack of prediction specificity and generalizability to all instruments. Our new idea is, instead of training on multi-instrument examples and testing directly on multi-instrument samples like previous approaches have, we train on single-instrument examples and test on multi-instrument samples that have been first reduced into multiple single-instrument samples. We perform the reduction step by first decomposing the multi-instrument sample with Fourier transform, partitioning the signals by harmonics, then sending those partitions through our classifier which was previously trained on single instruments. We train on support vector machines (SVM), k-nearest neighbors (KNN), multilayer perceptron (MLP), and random forest classifiers, using the amplitudes of the harmonics of each instrument as features. We achieved the best results with SVM using the first 10 harmonics, which resulted in a 88% cross validation accuracy, 89% single-instrument test accuracy and 75% multi-instrument test accuracy.