Cosc 5/4730
Mobile Media API (MMAPI) v1.1 JSR
135 and Android media
Part 1: Audio media
Blackberry MMAPI
PLAYING AUDIO
Mobile Media API (MMAPI)
background
• The MMAPI is built on a high-level abstraction of
all the multimedia devices that are possible in a
resource-limited device.
– This abstraction is manifest in three classes that form
the bulk of operations that you do with this API. These
classes are the Player and Control interfaces, and the
Manager class.
– Another class, the DataSource abstract class, is used
to locate resources, but unless you define a new way
of reading data you will probably never need to use it
directly.
Background (2)
• you use the Manager class
to create Player instances
for different media by
specifying DataSource
instances.
• The Player instances thus
created are configurable by
using Control instances.
• For example, almost all
Player instances would
theoretically support a
VolumeControl to control
the volume of the Player
Manager
• This class is used for creating players and it
provides three methods to indicate the source of
media
–
–
–
–
These methods are all static
createPlayer(DataSource source)
createPlayer(InputStream stream, String type)
createPlayer(String locator)
• where locator is a URI
– As note, with Eclipse, you need to import manager
implicitly or it gets confused…
CreatePlayer
• The JavaDocs are very lengthy for createPlayer syntax, but here are some
examples:
Media Type
Example syntax
Capture audio
"capture://audio" to capture audio on the default audio capture device
"capture://devmic0?encoding=pcm" to capture audio on the devmic0 device in
the PCM encoding
Capture video
"capture://video" to capture video from the default video capture device
"capture://devcam0?encoding=rgb888&width=100&height=50" to capture from a
secondary camera, in rgb888 encoding mode and with a specified width and
height
Start listening in on the
built-in radio
"capture://radio?f=105.1&st=stereo" to tune into 105.1 FM frequency and stereo
mode
Start streaming
video/audio/text from an
external source
"rtp://host:port/type" where type is one of audio, video or text
Play tones and MIDI
"device://tone" will give you a player that you can use to play tones
"device://midi" will give you a player that you can use to play MIDI
playing audio/video from
the web
"http://www.yourwebsite.com/audio/song.wav"
"http://www.yourwebsite.com/video/video.mpg"
Player State Model
• Players are created in the
UNREALIZED state
• and are typically realized
• then prefetched, and then
started.
• When they reach the end
of their media file, or
stop() is called, they
return to the
PREFETCHED state.
Supported A/V
•
•
•
•
To get a list of supported protocols and Content you can use
String[] Manager.getSupportedProtocols( String contentType)
String[] Manager.getSupportedContentTypes( String protocol)
Example: (storm2 9550)
– String capture[] = Manager.getSupportedContentTypes("capture");
• returns video/mpeg and audio/x-wav
– String wav[] = Manager.getSupportedProtocols("audio/x-wav");
• returns file, http, https, capture
– String file[] = Manager.getSupportedContentTypes("file");
• returns audio/midi audio/mpeg audio/x-wav audio/amr audio/x-gsm
audio/mp4 audio/aac audio/x-ms-wma video/mp4 video/3gpp video/3gpp2
video/x-msvideo video/quicktime video/mpeg video/x-ms- sf video/x-ms-wm
video/x-ms-wmv
Playing audio
• Playing Tones
import javax.microedition.media.*;
import javax.microedition.media.Manager;
import javax.microedition.media.control.*;
…
Player player = Manager.createPlayer(Manager.TONE_DEVICE_LOCATOR);
player.realize();
//Must realize() before getting tonecontrol
ToneControl tc = (ToneControl) (player.getControl("ToneControl"));
tc.setSequence(new byte[]{ToneControl.VERSION, 1,
ToneControl.C4, 8,
ToneControl.C4 + 2,
8}); // D4
player.start();
//Now the two tones will play, NON-BLOCKING!
//Note no GUI interface is needed to play audio
Playing Tones
• See the java docs ToneControl (in the control
package) for more information on tones
• They can be stored in a file
– The ToneControl interface specifies a tone sequence
format to be used for programming a tone sequence.
In addition, this tone sequence format can also be
used as a file format for specifying tone sequences.
– A file with the ".jts" extension must be used to store
tone sequences in this format. "audio/x-tone-seq"
designates the MIME type for this format.
Playing Tones (2)
InputStream in =
getClass().getResourceAsStream("/file.jts");
Player player = Manager.createPlayer(in, "audio/x-toneseq");
player.realize();
player.prefetch();
player.start(); // non-blocking.
Playing streaming media
Player p = Manager.createPlayer(
"http://www.cs.uwyo.edu/~seker/courses/4730/example/ME
EPMEEP.WAV");
– NOTE, Takes the network parameters as well, so
;deviceside=true;interface=wifi if you are running via wifi, The above
requires the MDS-CS to on for the simulator.
p.prefetch(); //this calls realize(), so we don’t have too.
p.setLoopCount(2); //play it twice
p.start();
Listener
• Since playing audio (and media) is controlled
via a non-blocking method
– The listener allows you program to know when
something "has happened"
• such as the audio has finished playing.
– implement PlayerListener
• playerUpdate(Player player, String event, Object
eventData)
– event is the one that tells you want is going on.
playerUpdate Events
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
BUFFERING_STARTED: Posted when the Player enters into a buffering mode.
BUFFERING_STOPPED: Posted when the Player leaves the buffering mode.
CLOSED: Posted when a Player is closed.
DEVICE_AVAILABLE: Posted when the system or another higher priority application has
released an exclusive device which is now available to the Player.
DEVICE_UNAVAILABLE: Posted when the system or another higher priority application has
temporarily taken control of an exclusive device which was previously available to the Player.
DURATION_UPDATED: Posted when the duration of a Player is updated.
END_OF_MEDIA: Posted when a Player has reached the end of the media.
ERROR: Posted when an error had occurred.
RECORD_ERROR: Posted when an error occurs during the recording.
RECORD_STARTED: Posted when recording is started.
RECORD_STOPPED: Posted when recording is stopped.
SIZE_CHANGED: Posted when the size of the video is changed either because the source
video size or the display size is changed.
STARTED: Posted when a Player is started.
STOPPED: Posted when a Player stops in response to the stop method call.
STOPPED_AT_TIME: Posted when a Player is stopped as responding to the setStopTime call
using the StopTimeControl.
VOLUME_CHANGED: Posted when the volume of an audio device is changed.
PlayerListener Example
public void playerUpdate(Player player, String
event, Object eventData) {
if (event == PlayerListener.END_OF_MEDIA) {
player.deallocate();
player.close();
}
}
• Where player is created…
p.addPlayerListener(this);
Android android.media
PLAYING AUDIO
Android
• android.media is similar in nature to MMAPI
• Get a MediaPlayer
• use the MediaPlayer class, with the create() method to
get a "player".
– States: prepare()  start()
• You can pause(), then start() without calling prepare()
– Also seekTo() without needed prepare() again.
• with stop() and reset(), then you must prepare() again
• See next slide for states and where to go…
– supported media formats
http://developer.android.com/intl/zhCN/guide/appendix/media-formats.html
mediaplayer states
Example
• When it is stored as a Resource (res/raw directory)
MediaPlayer mp = new MediaPlayer.create(getBaseContext(),
R.raw.laser);
mp.start(); //create calls prepare
• For media store somewhere else
MediaPlayer mp = new MediaPlayer();
mp.setDataSource("http://www.cs.uwyo.edu/~seker/courses/47
30/example/MEEPMEEP.WAV");
• OR on filesystem, “/sdcard/file.mp3”
mp.prepare();
mp.start();
As a note.
• You can also play the audio from video files
with the method as well.
– There just won’t be any picture, just the audio.
Audio Capture
• This can not be tested with the android or blackberry
simulators
– Android simulators may through errors, saying device
doesn’t exist.
– Blackberry seems to just ignore the call to record.
• So at least you can test the interface and make sure everything
else is working.
• You have to test all the code on the phone.
– All code examples on the website were tested and worked
on the phones.
• From eclipse, plugin the phone with USB. Click Debug AS
– Blackberry: on device
– Android: eclipse will ask you if you want to use the phone or emulator.
Blackberry MMAPI
CAPTURING AUDIO
Recording Audio
• You can use the following supported locator
strings in the createPlayer( str)
– AMR: capture://audio OR
• capture://audio?encoding=amr
• capture://audio?encoding=audio/amr
– PCM: capture://audio?encoding=pcm OR
• capture://audio?encoding=audio/basic
– GSM: capture://audio?encoding=gsm OR
• capture://audio?encoding=audio/x-gsm
– QCP: capture://audio?encoding=audio/qcelp
Audio Capture
• To record via the mic
Player player =
Manager.createPlayer("capture://audio");
player.realize();
• Get the RecordControl and configure the record stream
RecordControl rc = (RecordControl)
player.getControl("RecordControl");
• Setup a ByteArrayOutput to record the audio stream
ByteArrayOutputStream output = new ByteArrayOutputStream();
rc.setRecordStream(output);
rc.startRecord();
• Now start it recording, and this is blocking, so do all this on a thread.
player.start();
Audio Capture (2)
• Stop recording
rc.commit();
• Get the data
byte data[] = output.toByteArray();
• Now close down everything
player.close();
output.close();
• You can now save the data in a file, play it back, etc…
• To play it back
ByteArrayInputStream recordedIS = new ByteArrayInputStream(data);
Player player2 = Manager.createPlayer(recordedIS, "audio/amr");
player2.prefetch();
player2.start();
Blackberry
• Blackberry adds
– net.rim.device.api.media
• which provides interfaces to bluetooth devices and
mediakeylistener.
– MediaKeyListener maps key events to media actions identical
to those implemented by the built-in media player
» So volume up/down, etc…
• These do NOT require a the app to be signed to use the
APIs.
Android android.media
CAPTURING AUDIO
Recording Audio
• Simpler then Blackberry, but also less options.
• Get a new instance of the MediaRecorder()
– Configure it for recording the mic, set the audio
type
– And configure the output FILE. (no outputStream)
– Like playing, you prepare() and start().
• No thread needed.
– Stop() when done recording and then release().
– The audio is now located in the file.
Recording Audio (2)
• You need to set the following permissions in
the AndroidManifest.xml file
– android.permission.WRITE_EXTERNAL_STORAGE
– android.permission.RECORD_AUDIO
Example Code
private void startRecording() {
• get MediaRecorder
mRecorder = new MediaRecorder();
• configure it
mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mRecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
mRecorder.setOutputFile(mFileName);
mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
• Prepare to record
try { mRecorder.prepare();
} catch (IOException e) {}
• start recording
mRecorder.start();
}
private void stopRecording() {
mRecorder.stop();
mRecorder.release();
mRecorder = null;
}
Audio settings
•
mRecorder.setAudioSource();
–
–
MediaRecorder.AudioSource.MIC
MediaRecorder.AudioSource.CAMCORDER
•
–
MediaRecorder.AudioSource.VOICE_CALL
•
–
•
Voice call uplink + downlink audio source
http://developer.android.com/reference/android/media/MediaRecorder.AudioSource.html
mRecorder.setOutputFormat();
–
MediaRecorder.OutputFormat.THREE_GPP
•
–
–
Recommended setting by Andriod.
Also, AMR_NB, AMR_WB, RAW_AMR
MediaRecorder.OutputFormat.MPEG_4
•
•
•
Audio source with same orientation as camera, otherwise mic.
Using an MPEG-4 container format may confuse some desktop players.
http://developer.android.com/reference/android/media/MediaRecorder.html#setOutputFormat%28int%29
mRecorder.setAudioEncoder();
–
MediaRecorder.AudioEncoder.AAC
•
–
MediaRecorder.AudioEncoder.AMR_NB
•
–
AAC audio codec
AMR (Narrowband) audio codec
MediaRecorder.AudioEncoder.AMR_WB
•
AMR (Wideband) audio codec
Recording incoming phone calls
• As of the current time (2010)
– It doesn't appear to be possible via
JavaME/Blackberry/Android to record the audio "from
the other end" of the conversation.
– Some phones allow you record the mic during a
phone call (Blackbery and Android), but not the
"headset speaker“
• But Android has a setting for recording, so it may work (as of
2011)
– Many JavaMe phones, simply don't run any other
programs during a phone call and so you can not
record any part of the phone call.
References
• JavaME/Blackberry
– http://java.sun.com/javame/reference/apis/jsr135/
– http://docs.blackberry.com/en/developers/deliverables/11942/CS_record_vid
eo_without_using_the_camera_app_734825_11.jsp
– http://www.forum.nokia.com/document/Java_Developers_Library_v2/?conte
nt=GUID-9CAAF2D4-D395-4021-BBE6-602DFD275B03.html
– http://www.forum.nokia.com/document/Java_Developers_Library_v2/GUID8F9BE454-FC5E-4B87-B805-765DEFA66C99//overview-summary.html
– http://www.forum.nokia.com/document/Java_Developers_Library_v2/GUID8F9BE454-FC5E-4B87-B805765DEFA66C99//javax/microedition/media/control/ToneControl.html#tone_s
equence_format
– http://developers.sun.com/mobility/midp/articles/mmapioverview/
– http://www.mobiquil.com/jme_mmapi.html
• Android
– http://developer.android.com/intl/zh-CN/guide/topics/media/index.html
– http://developer.android.com/guide/topics/media/index.html
Q&A