University of Antwerp Operations Research Group
ANT/OR FuX, an Android app that generates counterpoint D. Herremans, K. S¨orensen IEEE SSCI, April 16-19th, 2013, Singapore CI for Creativity and Affective Computing
Overview Computer Aided Composing Quantifying Counterpoint Quality VNS Android Implementation Conclusion
Computer aided composing (CAC)
Composing music = combinatorial optimization problem I
Music → combination of notes
I
“Good” music → fits a style as well as possible
I
Formalized and quantified “rules” of a style → objective function
Counterpoint
I
Polyphonic baroque music
I
Inspired Bach, Haydn,. . .
I
One of the most formally defined musical styles → Rules written by Fux in 1725
5th species counterpoint I
Counterpoint & Cantus firmus 44
I
4 4
Represented as a vector of note objects, each with: I I I I I
Pitch: midi value Duration Beat number Measure number Tied?
Quantifying musical quality Examples of rules: I
Each large leap should be followed by stepwise motion in the opposite direction
I
Half notes should always be consonant on the first beat, unless they are suspended and continued stepwise and downward
I
All perfect intervals should be approached by contrary or oblique motion
→ 19 vertical and 19 horizontal subscores between 0 and 1
Quantifying musical quality I
Eight notes (8ths) must move in step. subscoreH 1 (s) =
I
#8ths not preceded by step + 8ths not left by step #8ths × 2 (1)
Whole notes should always be vertically consonant. subscoreV1 (s) =
#dissonant whole notes #whole notes
(2)
Quantifying musical quality
fcf (s) =
19 X
ai .subscore cfH i (s)
(3)
i=0
| fcp (s) =
19 X
ai .subscore
{z
horizontal aspect
cpH i (s) +
i=0
|
19 X
}
bj .subscoreVj (s)
j=0
{z
horizontal aspect
}
|
{z
vertical aspect
}
(4)
Variable Neigborhood Search
I
Local search with 3 neighborhoods
I
Selection: steepest descent Ni Nsw Nc1 Nc2
Name Swap Change1 Change2
Description Swap two notes Change one note Change two notes
Variable Neigborhood Search I
Excluded framents I I
I
Perturbation I
I
Tabu list Infeasible Change r% of the notes randomly
Adaptive weights mechanism I I
Increase weight of subscore with highest value Keeps the search in the right direction
Generate random s Change r% of notes randomly
A Update s best LS Swap NH
Update adaptive weights LS Change1 NH
Exit
Max iters reached?
LS Change2 NH
Optimum found?
yes
No
Yes
Current s < s at A?
No Iters ++
Yes Exit
Experiments & Results I
Full factorial experiment, n=2304
Parameter Nsw - Swap Nc1 - Change1 Nc2 - Change2 Random move Adaptive weights Max. iterations Length of music
Values 1 on with ttsw =0, ttsw = 16 , ttsw = 18 , off 1 on with ttc1 =0, ttc1 = 16 , ttc1 = 18 , off 1 on with ttc2 =0, ttc2 = 16 , ttc2 = 18 , off 1 1 4 changed, 8 changed, off on, off 5, 20, 50 16, 32 measures
Nr. of levels 4 4 4 3 2 3 2
Experiments & Results I I
Multi-Way ANOVA model with interaction effects, using R R2 = 0.98 Parameter Nc1 Nc2 Nsw randsize maxiters length adj. weights ttc1 ttc2 ttsw
Df 1 1 1 2 2 1 1 2 2 2
F value 9886.2323 15690.7234 3909.2959 1110.1724 322.6488 165.6053 4.0298 2.2575 8.271 3.2447
Prob (> F ) < 2.2e−16 < 2.2e−16 < 2.2e−16 < 2.2e−16 < 2.2e−16 < 2.2e−16 0.0448367 0.1048791 0.0002646 0.0391833
Experiments & Results Mean plot for the size of the random jump
200
1 0.8 0.6
100 0
10 20 Random size (in %)
Time (s)
1.2
Score
I
Optimal parameter settings Parameter Nsw Nc1 Nc2 Random move Adaptive weights Max. number of iterations
Value 1 on with ttsw = 16 1 on with ttc1 = 16 1 on with ttc2 = 16 1 8 changed on 50
C++ Implementation
Android App - FuX
I
Software toolkit for mobile devices
I
Dalvik Virtual Machine → Java
I
VNS is computationally expensive → Native C++ (jni) with java NDK
I
Midi files (Android-Midi-Lib & Mediaplayer)
⇒ Continuous stream of music (multitreading)
Android App - FuX Time -1 0s 16s 24s ...
Generate 16 measures (file 1) 8 measures (file 2) 8 measures (file 3) 8 measures (file 4) ...
Playback file 1 file 2 file 3 ...
→ Evaluation based on last 16 measures
Results over time
file 1 file 2
4
file 1 file 2
7
fcp (sbest )
fcf (sbest )
6 3 2 1 0
5 4 3 2
0
0.5 1 1.5 Running time VNS (seconds)
(a) Cantus firmus
2
4 6 8 Running time VNS (seconds)
(b) Counterpoint
Evolution of the objective function over time
10
Composer specific music - FuX 2.0
I
Analyzed existing music (Bach, Beethoven, Haydn)
I
3 composer classification models
I
Incorporated in objective function
Conclusion An efficient VNS has been developed and implemented as an Android app. The resulting app can play a continuous stream of counterpoint music on any Android phone or tablet. Future research: I More complex music: I I I
I I I
Different styles More parts Theme
Improved sound quality Bugfixes for multitude of devices Hit song prediction
University of Antwerp Operations Research Group
ANT/OR FuX, an Android app that generates counterpoint D. Herremans, K. S¨ orensen IEEE SSCI, April 16-19th, 2013, Singapore
[email protected]