Integer-Based Wavetable Synthesis for Low-Computational Embedded Systems Ben Wright and Somsak Sukittanon University of Tennessee at Martin Department of Engineering Martin, TN USA [email protected], [email protected] Abstract— The evolution of digital music synthesis spans from tried-and-true frequency modulation to string modeling with neural networks. This project begins from a wavetable basis. The audio waveform was modeled as a superposition of formant sinusoids at various frequencies relative to the fundamental frequency. Piano sounds were reverse-engineered to derive a basis for the final formant structure, and used to create a wavetable. The quality of reproduction hangs in a trade-off between rich notes and sampling frequency. For lowcomputational systems, this calls for an approach that avoids burdensome floating-point calculations. To speed up calculations while preserving resolution, floating-point math was avoided entirely--all numbers involved are integral. The method was built into the Laser Piano. Along its 12-foot length are 24 “keys,” each consisting of a laser aligned with a photoresistive sensor connected to 8-bit MCU. The synthesis is processed in the sensor array by four microcontrollers running a strictly synchronous wavetable synthesis algorithm. In the laser array is integrated a microcontroller that can toggle each laser, allowing the piano to play itself or limit playable keys. I. INTRODUCTION A. Background Review Throughout the years, music synthesis has branched forth from many different techniques. FM synthesis is classically favored for its simplicity but is limited in signal realism. The Karplus-Strong algorithm is favored for its rich harmonic content and authenticity particularly to the transient responses of plucked and struck strings. In [1], Karplus-Strong polynomials are used to define zeros to selectively cancel harmonics from a traditional Karplus-Strong transfer function. The time-domain convolution associated with this method, however, requires more RAM than may be practical for a low-computational system. In [2], the authors proposed an impressive system to synthesize music by modeling the vibration of a string based on Scattering Recurrent Networks with very accurate results. To implement this system, however, would still be far beyond the capacity of a typical small embedded system. Techniques like these have been combined in [3] by a control structure that dynamically selects and combines synthesis techniques to benefit from the advantages of each. This control structure could prove invaluable to a system synthesizing a wide range of timbres and pitches but would be unnecessary for a system with sufficiently limited scope. For embedded systems, wavetable synthesis offers an attractive combination of realism and speed. In this paper, a lightweight implementation of wavetable synthesis is discussed. Section B will discuss the mathematics explaining the components of the wavetables. Part II will examine the music theory used to derive them and the algorithm coded to wrap it all up. Part III will cover the applied results. B. Mathematics and Music Theory Many periodic signals can be represented by a superposition of sinusoids, more conventionally called Fourier series. Eigenfunction expansions, a superset of these, can represent solutions to partial differential equations (PDE) where Fourier series cannot. In these expansions, the frequency of each wave (here called a formant) in the superposition is a function of the properties of the string. This is said merely to emphasize that the response of a plucked string may need a model more complete than a Fourier series to accurately capture its timbre as shown in equation (1). ∂2 u ∂x 2 = ∂2 u ∂t 2 u(x,0) = f (x) ∂u (x,0) = g(x) ∂t (1) The function represents the displacement of a vibrating string as a function of position along the string, where , and time . and represent the initial displacement and speed, respectively. d’Alembert’s solution [4] to this PDE is given in the form u(x,t) = Φ(x − ct) + Θ(x + ct) , which describes the wavespeed (2) of travelling waves. and describe travelling waves that move in different directions at speed . A solution of this PDE is given by (3) Initial conditions and are used, by an appeal to orthogonality, to derive a pair of coefficients and for each eigenfunction. There exists one eigenfunction for each eigenvalue . These eigenvalues need not be associated nor even countable. Each eigenfunction is called a formant. These formants are not as easily determined for a generalized eigenfunction expansion as they might be for a Fourier series. These formants are assumed to relate to the fundamental frequency of each note in identical fashion. A vibrating string, for example, should produce the same timbre (i.e. quality of sound as described by its formant structure) even as it is tuned higher or lower. It was assumed that the relations of formants to the fundamental root would fit into the model of a diatonic scale as described by Western music theory. This limited the formants to a set of the most significant few that provide a skeleton into which the formants can fit. If not perfectly accurate, this assumption proved a fair approximation, and simplified the enforcement of periodicity. Adding formants at eigenvalue frequencies extends the fundamental period of the superposition. The wavetable is not easily truncated because it is important that no discontinuities exist in it. However, the wavetable must be small enough to fit in the limited RAM capacity of a small MCU. Once the pattern of formant relationships and gains was recognized, the waveform was reconstructed in MATLAB (as shown by the code in Fig. 4a, and plotted with the ADSR envelope in Fig. 1) using a superposition of waves with a similar mapping of frequencies and amplitudes. Multiple MATLAB files were written to attempt a simulation most like on-chip synthesis as possible. Every note was sampled from an array of 256 8-bit numbers and amplitude-modulated according to the ADSR (Attack, Decay, Sustain, Release) envelope as illustrated in Fig. 2. II. ALGORITHM DESIGN A. Waveform Construction How to produce a desired timbre could be a subject of considerably deep inquiry. Applying an idealized partial differential equation to such a pursuit would be difficult enough, but easier still than so modeling a string with considerations for its manifold non-uniformities. The limitations of this implementation make work this precise needless. For this design, spectral analyses of professionallyrecorded piano notes were studied as a first step toward reverse-engineering piano sound. To lessen RAM consumption on-chip, a wavetable only large enough to capture the note’s fundamental frequency was used. To maintain periodicity, formants were chosen that satisfied periodicity within this window, most notably the note’s perfect fifth and compound major third. This way there are no discontinuities in the final wave. Fig. 2. A set of notes plotted in MATLAB to illustrate the effect of the amplitude modulation on the repetitive waveshape. Each enlarged section shows the signal within 50-ms-wide cuts. The amplitude modulation proved to be more important to the final sound than was initially expected. The use of an ADSR envelope to modulate the amplitude of the waveform provided the striking attack characteristic of a piano note. Intuitively speaking, this envelope could be just as important to other instruments, particularly to drums and woodwinds. B. Firmware Implementation A program was written in C, using Codevision C compiler [5], to synthesize a set of simultaneous piano notes on an embedded system. A timer interrupt function was used to synchronize the wavetable sampling. The code controlling amplitude modulation was written inside an interruptible loop. The output, a superposition of all notes, was output via a byte register into a DAC, from which an analog signal was filtered and sent straight to speakers. The flowchart is shown in Fig. 3b. Fig. 1. A MATLAB plot of the waveform and ADSR envelopes. The waveform shapes the timbre of the notes and the ADSR further controls the amplitude of each note to realistically depict its attack and decay. A timer built in to the microcontroller increments a byte register every 32 clock cycles. Each time this register overflows, an interrupt subroutine is called and the counter is reinitialized to tune the frequency of such calls. Every timer interrupt, an 8-bit number for each note is sampled from an index in the wavetable, then amplitude-modulated according to its position in the ADSR table. The sum of these numbers is normalized and output to the DAC. (a) (b) Fig. 3. (a) A system diagram of the piano hardware, (b) Flowchart describing the piano synthesis algorithm. Two counters increment each interrupt to synchronize ADSR timings for both damped and undamped notes. These indices are incremented every so often; those indices controlling position in the wavetable are incremented according to the frequency of each note. The indices for higher notes, then, step through the wavetable faster than for lower notes. The indices controlling a note’s position in the ADSR table is incremented slowly enough to happen within the interruptible loop. Critical to this program’s success was the tuning of the notes. The amount by which to increment each index for the lower frequency notes can get muddled by 8-bit precision as these increments become small for lower notes and a higher sampling rate. Floating-point math, on the other hand, proved to process very slowly. Instead, a 16-bit unsigned integer was used to represent each index, but scaled up by 256. This yielded tuning more than precise enough for this application while demanding much less CPU time than floating-point math. The indices for the ADSR table, in contrast, were fairly simple to process. These indices, also 8-bit, were incremented by one every so often according to the length of the note, rather than by a certain amount every timer interrupt. The increment timings were scaled down from the interrupt frequency by incrementing a counter in the interrupt function that the interruptible amplitude-modulating code would check. Also within this loop, checks are made concerning user input and which notes should start and stop. To achieve the best performance is to strive for the best sampling frequency while leaving enough CPU time to handle amplitude modulation and sampling user inputs. Traversing the ADSR table is not as important to the user’s ear as synchronously traversing the wavetable. Output from the wavetable must be synchronous. This is why the amplitude modulation is interruptible but wavetable sampling is not. In addition, every interrupt must consume the exact same amount of CPU time to remain synchronous and consistent. For this reason, the use of control statements was avoided within the timer interrupt code. The use of Boolean numbers in formulae allowed the complete avoidance of if conditions. In this way, a logical 0 or 1 can be used as a coefficient just like a numeric 0 or 1. For example, a number to be incremented if a condition is true may always be incremented without an if condition, even when that means the number is incremented by zero. III. RESULTS The final product was a laser piano synthesizer. Along its 12-foot length, an array of lasers (each 5 mW 650 nm) shoots along the floor into an array of photoresistive sensors, one for each of its 24 fully independent notes, as Fig. 3a shows. Full 24-note polyphony from C4 to B5 is processed by four microprocessors (AVR ATMega644), each governing its own range of 6 notes. Overclocked with 27 MHz piezoelectric crystals, each chip does its job consistently at a sampling rate just above 10 KHz. The synthesis algorithm uses two separate ADSR timings. A note is stepped through the ADSR more slowly as long as the user input corresponding to that note stayed present. This way a laser blockage that remains causes a note to sustain longer like holding a key down on a real piano. Each sampling period, the amplitude of the superposition of synthesized notes is output to an 8-bit DAC (DAC-08CN). In the laser array is integrated a microcontroller (ATMega32) that can toggle each laser. Programmed with the Harry Potter theme, the Imperial March, and Für Elise, the piano can effectively play itself. This microcontroller is also programmed with a set of scales to make playing the piano easier. Because this microcontroller could not source enough current to all 24 lasers, the lasers are sourced by Darlington pairs the MCU controls. The sensors were constructed using photoresistors aligned with plastic tubes and wired in series with a static resistor. Each photoresistor has high resistance when darkened and lesser resistance when light is shone upon it. The sensor circuit uses voltage division to sense the difference between a laser shining and a blockage. The series resistor was selected to maximize the difference between the maximum and minimum voltages across the photoresistor. It can be shown the best series resistance is given by the geometric mean of the highest and lowest resistances across the photoresistor. The low and high voltages were reliably distinguishable as TTL logic so the sensors could be wired directly to tri-state pins on the chip. The DAC selected outputs a current signal, so all the DAC output signals are added simply by wiring them to a common node. This current signal from the DACs was filtered by a parallel RC filter with cutoff frequency 1500 Hz meant to smooth the discontinuities in the DAC output signal. Finally, a series capacitor was added to normalize the voltage output to the speaker and eliminate popping. IV. CONCLUSION Even quite complicated timbres can be reproduced with wavetable synthesis. On systems with more computational power, more complicated string modeling techniques can better be appreciated and produce higher quality sound. There is much left to do modeling responses of musical instruments for the purpose of synthesizing them. There may be formants at frequencies lower than the fundamental frequency of the note or outside diatonic scales. When (a) formants are not as limited, it can quickly become important to sample from a larger waveform envelope to ensure periodicity of the output signal. It is important to implement these with an algorithm that does not fail to keep wavetable output synchronous while keeping ADSR timings steady. Using integer calculations wherever possible may sacrifice some precision, but for considerable savings in CPU time. Fig. 4b shows the envelopes initialized as 8-bit integral types (unsigned char in C). These methods proved effective in an implementation on an 8-bit embedded platform. Similar techniques may be effective on larger systems. Another step to make such an algorithm more effective could be to add transient effects, e.g. for synthesizing the music of an acoustic guitar, the “scratch” when a string is plucked. At some extent, however, it may become more efficient to pursue a different algorithm. ACKNOWLEDGEMENT The authors would like to thank Robert Reeves for his discussion and help in this work. REFERENCES [1] I. A. Cummings, R. Venugopal, J. Ahmed, and D. S. Bernstein, “Generalizations of the Karplus-Strong Transfer Function for Digital Music Sound Synthesis,” in IEEE Proceedings of the American Control Conferences, 1999, pp. 2210-2214. [2] A. W. Y. Su and L. San-Fu, “Synthesis of Plucked-String Tones by Physical Modeling With Recurrent Neural Networks,” in IEEE Workshop on Multimedia Signal Processing, 1997, pp. 71-76. [3] S. D. Trautmann and N. M. Cheung. “Wavetable Synthesis for Multimedia and Beyond,” in IEEE Workshop on Multimedia Signal Processing, 1997, pp. 89-94. [4] D. L. Powers, “The Wave Equation,” in Boundary Value Problems and Partial Differential Equations, 6th ed., Academic Press, 2009, pp. 229-231 [5] http://hpinfotech.ro/html/cvavr.htm (b) Fig. 4. (a) Code snippet depicting waveform construction in MATLAB, (b) envelope initialization in C generated by MATLAB for the synthesizer firmware.

© Copyright 2021 Paperzz