DSPGateway Architecture in
Detail
Yoshiya Hirase
Nokia Research Center
Nokia Japan Co. Ltd.
1
© 2005 Nokia
V1-Filename.ppt / yyyy-mm-dd / Initials
Agenda
• DSP Gateway Features
• Demonstration Using Nokia 770
2
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
DSP Gateway Features
• Open Source Software
‐ Integrated in Nokia 770 Internet Tablet
• Simple interface for DSP
• Efficient inter-processor communication (ARM <-> DSP)
• Block data transfer via shared memory
• DSP Task management
‐ Dynamic Loader
• Fault-tolerant mechanism
• Power management
3
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
History
• 2003/6: Released the DSP Gateway version 1.0 for OMAP1510 at SourceForge
• 2004/8: Supported OMAP1610/5912
• 2004/9: Merged into the Linux OMAP tree
• 2005/2H: Nokia 770 released with DSP Gateway
The latest DSP Gateway ver 3.3 supports OMAP 15XX (including 5910) and 16XX
(including 1710 and 5912).
4
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
DSP Interface: Device Files
• DSP task devices
interface to DSP task
• DSP control device
miscellaneous DSP control
ARM(Linux)
Dynamic Loader Daemon
dspctl
utility
DSP
user
app.
“task1”
“task2”
“task3”
(e.g. DSP configuration)
DSP
control
device I/F
DSP
task
watch
device I/F
• DSP error detection device
error info delivery
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
task API
DSP/
BIOS
TASK
DSP task
device I/F
DSP/
BIOS
TASK
DSP/
BIOS
TASK
Mailbox
INT handler
DSP Gateway driver
5
/dev/dsptask/
task3
DSP
error
detection
device I/F
/dev/dsptask/
task2
DSP
memory
device I/F
/dev/dsptask/
task1
/dev/dspctl/
twch
for dynamic loader
/dev/dspctl/
ctl
• DSP task watch device
/dev/dspctl/
err
DSP memory access
/dev/dspctl/
mem
• DSP memory device
INT handler
DSP Gateway library
Inter-Processor Communication
• Communications via OMAP mailbox mechanism
OMAP
ARM
(16bit + 16bit)
interrupt
DSP
Mailbox registers
(16bit + 16bit)
interrupt
internal memory
IPBUF
memory
IPBUF
6
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Inter-Processor Communication: IPBUF
Global IPBUF:
‐ Used to transfer block data to/from DSP task
– Prepared by the system and managed
with alloc & free functions
ARM
DSP
1
task A
task A
(alloc)
– Fixed block size, restricted block count
1
(free)
2
3
Global IPBUF
4
Global IPBUF
Private IPBUF:
‐ Used to transfer block data to/from DSP task
– A task can define an exclusive buffer, with
arbitrary size
ARM
task B
DSP
task B
Private
IPBUF
System IPBUF:
‐ Used to transfer extra data between systems of
ARM and DSP
task B
Private IPBUF
ARM
system
DSP
System
IPBUF
system
System IPBUF
7
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Inter-Processor Communication:
Mailbox commands
8
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Memory Mapping
• DSP internal memory
‐
0000_0000
SDRAM
DARAM, SARAM
C000_0000
MPU MMU
• External memory
‐
SDRAM
Kernel memory area
Physical Memory
DSP MMU
00_0000
DARAM
SARAM
E000_0000
DARAM
SARAM
DSP space shadow area
PDROM
DSP Memory Space
FF_FFFF
E100_0000
Linux Virtual Address Space
9
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Task Management: Dynamic Loader
10
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Task Management: DSP Application Life Cycle
DSP Gateway
DSP
ARM
process 1
open()
Linux application
Device driver
DSP kernel
open()
activate DSP task
DSP application
DSP task
dynamically
created
complete
open() return
write()
write()
input data / control command
write() return
task running
output data
read()
read()
read() return
close()
close()
deactivate DSP task
complete
close() return
11
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
DSP task
dynamically
deleted
MP3 Pseudo program (ARM side)
int fd_mp3file, fd_dsp;
DATA buf[BUFSZ];
int Decode_MP3(void)
{
// Open MP3 decoder device
fd_dsp = open(“/dev/dsptask/mp3dec”, O_RDWR);
Issue TCTL
command to DSP
// Initialize MP3 decoder
ioctl(fd, MP3_DECODE_INIT);
// Decode MP3 audio file
while (ReadMp3File(fd_mp3file, buf, BUFSZ) != EOF) {
// Send encoded data to MP3 decoder in DSP
write(fd_dsp, buf, BUFSZ);
}
// Close MP3 decoder device
close(fd_dsp);
}
12
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Issue BKSND
command to DSP
MP3 Pseudo program (DSP side)
Uns rcv_tctl(struct dsptask *task, Uns ctlcmd, Uns *ret, Uns arg)
{
Handle
switch (ctlcmd) {
MP3_DECODE_I
case MP3_DECODE_INIT:
NIT of TCTL
InitIMDCT();
command
InitFilters();
break;
...
}
Uns rcv_bksnd(struct dsptask *task, Uns bid, Uns cnt)
{
/* Decode received MP3 data */
Dequantize();
ReduceAlias();
IMDCT();
...
/* Send decoded PCM data to the Audio Device Driver */
...
}
13
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Handle BKSND
command
Nokia 770 Internet Tablet Features
• Display
‐ High-resolution (800x480) touch screen with up to 65,536 colors
• Memory
‐ Engine: DDR RAM 64MB
‐ Memory: Flash 128MB (>64MB for user)
‐ Memory card: 64MB RS-MMC (Reduced Size - MultiMediaCard)
• Supported file formats
‐ Audio: MP3, Real Audio, AAC, WAV, AMR
‐ Image: JPEG, GIF, BMP, TIFF, PNG, Animated GIF format, SVG-tiny, ICO
‐ Video: MPEG1, MPEG4, Real Video, H.263, 3GP
(RED items are decoded on DSP, BLUE items are routed through DSP)
• Connectivity
‐
‐
‐
‐
‐
14
WLAN: 802.11b/g
Bluetooth specification: 1.2
USB device mode for PC connectivity
RS-MMC
3.5 mm stereo audio out
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Multimedia Software Architecture in Nokia 770
DSP/BIOS
Linux
Multimedia
Player
GStreamer
Plugin #1 Plugin #2 Plugin #3
(Codec)
Codec
#2
Codec
#3
DSP Gateway
ARM
15
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
DSP
Future Work
• OMAP2420 support (including IVA support)
• Driver Abstraction, and expansion for generic AMC (Asymmetric Multi-Core)
processors
16
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Demo: Play Media Files using DSP Gateway
• System log
message
• Device status
Play sound or movie
17
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
References:
• Project page:
‐ http://sourceforge.net/projects/dspgateway
• Articles:
‐ 森 英悟, 小林 俊裕, 高橋 清隆. “オープン・ソース・ソフトウェアDSPGatewayを用いた
Linuxで使うOMAP DSP部のソフトウェア開発環境.” Interface: CQ Publishing, May
2004, 156-164
http://www.cqpub.co.jp/interface/contents/2004/200405.htm
18
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Thank you
19
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Appendix: File manager Image
20
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Appendix: Audio Player Image
21
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Appendix: BKSND command in log
22
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Appendix: TDEL command in log
23
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Appendix: Task Status Transition
24
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Appendix: Log message while playing MP3
25
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo
Appendix: Performance Evaluation Example
• Memory mapping improves data transfer well
‐ Data send latency (to DSP) by 44%-68%
‐ Data receive latency (from DSP) by 27-55%
MPU bksnd := MPU sends a data blocks to DSP
MPUbkreq_DSPbksnd := MPU sends a block request to DSP -> DSP receives it & then sends the block back -> MPU receives the data block from DSP
Memory write/read := MPU writes/reads data on the mapped memory
26
© 2005 Nokia
CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo