Multimedia Technology Course (MM301)

Study Notes

The course focuses on basic knowledge of multimedia and multimedia technology.
Grading policy: Midterm (15%), Project (15%-20%), Assignments & Participations (15%-20%), Final (50%).

A PC vendor viewpoint: a PC with sound capability, DVD ROM drive, and multimedia-enabled microprocessors.
A consumer entertainment vendor viewpoint: interactive cable TV with hundreds of digital channels or TV-like service over high-speed Internet.
In our context: computer technology for efficient communication using different media types.
A Computer Science student viewpoint: applications using multiple modalities, including text, images, drawings, animation, video, sound, and interactivity.

Developing a successful multimedia system is non-trivial.
Continuous media types like video require a lot of space to store and high bandwidth to transmit.
Tight timing constraints.
Analyzing, indexing, and organizing information in audio, image, and video is harder than in text.
Multimedia involves many research areas and needs complex and efficient algorithms and hardware platforms.

Examples: World Wide Web, multimedia authoring, hypermedia courseware, video-on-demand, interactive TV, computer games, virtual reality, digital video editing, and multimedia database systems.
Video-on-demand: systems allowing users to select and watch/listen to video/audio content when they choose.
Interactive TV: adding data services to traditional television technology, including on-demand delivery of content, online shopping, and banking.
Hypermedia: not constrained to be text-based, including graphics, images, and continuous media like sound and video.

Computer-simulated environment simulating physical presence in real or imagined worlds.
Recreating sensory experiences, including virtual taste, sight, smell, sound, touch, etc.

Video editing: manipulating video images, including cutting segments, re-sequencing clips, adding transitions, and special effects.
2D and 3D Animation: creating animated content using computer-generated imagery.

Universal access to web resources for everyone, everywhere.
HTML: a language for publishing hypermedia on the World Wide Web.
XML: a markup language for the WWW, allowing users to define tags.
XSL: similar to CSS for XML.
SMIL: synchronized multimedia integration language, allowing interaction among media types and user input.

Involves computer networks, operating systems, computer vision, image and speech processing, and human-computer interaction.

Integrates a wide range of symbol modes simultaneously into a coherent framework.
Characteristics: computer-controlled, integrated, digitally represented information, and interactive interface.

Involves multimedia data representation and compression, processing and analysis, transmitting data through communication networks, and multimedia database, indexing, and retrieval.

Automatic data analysis to extract semantic meanings from audios, images, and videos.
Methods: object tracking (face, eyes), object recognition, gesture recognition, etc.

David G. Lowe's paper on object recognition from local scale-invariant features.

Detecting actions in videos, including sign language gestures and baseball sequences.

Transmitting multimedia data across networks, involving quality of service, synchronization, error and congestion control, and session setup.

Dealing with large media files, requiring new data structures, indexing, and searching methods.
Content-based multimedia retrieval is an ongoing research topic.

Journals: IEEE Multimedia, IEEE Transactions on Multimedia, IEEE Transactions on Image Processing, and IEEE Transactions on Pattern Recognition and Machine Intelligence.
Conferences: ACM Multimedia, International Conference on Multimedia and Expo (ICME), IEEE Computer Vision and Pattern Recognition (CVPR), and International Conference on Computer Vision (ICCV).