Cs8092 computer graphics and multimedia unit 4

UNIT IV
MULTIMEDIA SYSTEM DESIGN
& MULTIMEDIA FILE HANDLING
Multimedia basics − Multimedia applications − Multimedia system architecture −
Evolving technologies for multimedia − Defining objects for multimedia systems
− Multimedia data interface standards − Multimedia databases. Compression
and decompression − Data and file format standards − Multimedia I/O
technologies − Digital voice and audio − Video image and animation − Full
motion video − Storage and retrieval technologies.

WHAT IS MULTIMEDIA?
• A Multimedia System is a system capable of processing multimedia data and
applications.
• A Multimedia System is characterized by the processing, storage, generation,
manipulation and rendition of Multimedia information.

Characteristics of a
Multimedia System
• A Multimedia system has four basic characteristics:
• Multimedia systems must be computer controlled.
• Multimedia systems are integrated.
• The information they handle must be represented digitally.
• The interface to the final presentation of media is usually
interactive.

Challenges for
Multimedia Systems
• Distributed Networks
• Temporal relationship between data
– Render different data at same time — continuously.
– Sequencing within the media
playing frames in correct order/time frame in video.
• Synchronization — inter-media scheduling E.g. Video and Audio conversation.

Key Issues for Multimedia
Systems
• The key issues multimedia systems need to deal with here are:
-How to represent and store temporal information.
-How to strictly maintain the temporal relationships on play

Key Issues for Multimedia
Systems
• back/retrieval
• What process are involved in the
above.
• Data has to be represented digitally.
• Conversion, Sampling etc.
• Large Data Requirements —
bandwidth, storage, Data
compression is usually mandatory.

Desirable Features for a
Multimedia System
• Given the above challenges the following feature a desirable (if not a
prerequisite) for a Multimedia System:
-Very High Processing Power — needed to deal with large data
-processing and real time delivery of media.

Multimedia System
• Special Hardware/Software needed.
• Data Representations — File Formats that support multimedia should be easy to
handle yet allow for compression/decompression in real-time.
• Efficient and High I/O —input and output to the file subsystem needs to be
efficient and fast. Needs to allow for real-time recording as well as playback of
data

Multimedia System
• Special Operating System —to allow access to file system and process
data efficiently and quickly.
• Storage and Memory — large storage units.
• Network Support — Client-server systems -Software Tools — user
friendly tools needed to handle media, design and develop
applications to deliver media.

Components of a
Multimedia System
• Now let us consider the Components (Hardware and
Software) required for a multimedia system:
• Capture devices — Video Camera, Video Recorder, Audio
Microphone, Keyboards, mice, graphics tablets.
• Storage Devices — Hard disks, CD-ROMs, DVD-ROM, etc..
• Communication Networks — Local Networks, Intranets,
Internet, Multimedia or other special high speed
networks

Components of a
Multimedia System
• Computer Systems — Multimedia
Desktop machines, Workstations.
• Display Devices, quality speakers,
HDTV. monitors, Colour printers etc

A Brief Look at
Multimedia Data:
Input and Format
Text and Static Data
• Source: keyboard, speech input,
optical character recognition,
• data stored on disk.
• Stored and input character by
character:

A Brief Look at
Multimedia Data:
Input and Format
Text and Static Data
• Storage of text is 1 byte per char / more bytes for Unicode.
– For other forms of data (e.g. Spreadsheet files). May store
format as text (with formatting) others may use binary
encoding.
• Format: Raw text or formatted text e.g HTML, Rich Text
Format (RTF), Word or a program language source (C,
Pascal, etc..

A Brief Look at
Multimedia Data:
Input and Format
Graphics
• Format: constructed by the
composition of primitive objects such
as lines, polygons, circles, curves and
arcs.
• Input: Graphics are usually generated
by a graphics editor program (e.g.
Illustrator).
Graphics are usually editable.

A Brief Look at
Multimedia Data:
Input and Format
• Graphics
• Graphics input devices: keyboard (for text and cursor
control), mouse, trackball or graphics tablet.
• graphics standards : OpenGL, PHIGS, GKS
• Graphics files usually store the primitive assembly
• Do not take up a very high storage overhead.

A Brief Look at
Multimedia Data:
Input and Format
Images
• Still pictures which (uncompressed) are represented as a
bitmap (a grid of pixels).
• Input: digitally scanned photographs/pictures or direct
from a digital camera.
• Input: May also be generated by programs “similar” to
graphics, or animation programs.

A Brief Look at
Multimedia Data:
Input and Format
• Images
• Stored at 1 bit per pixel (Black and White), 8 Bits per pixel
(Grey Scale, Colour Map) or 24 Bits per pixel (True Colour)
• Size: a 512x512 Grey scale image takes up 1/4 MB, a
512x512
• 24 bit image takes 3/4 MB with no compression.

A Brief Look at
Multimedia Data:
Input and Format
Images
• This overhead soon increases with
image size — modern
• high digital camera 10+ Megapixels
29MB uncompressed!
• Compression is commonly applied.

A Brief Look at
Multimedia Data:
Input and Format
Audio
• Audio signals are continuous analog
signals.
• Input: microphones and then
digitised and stored
• CD Quality Audio requires 16-bit
sampling at 44.1 KHz
• Even higher audiophile rates (e.g. 24-
bit, 96 KHz)

A Brief Look at
Multimedia Data:
Input and Format
Audio
• 1 Minute of Mono CD quality
(uncompressed) audio requires 5
MB.
• 1 Minute of Stereo CD quality
(uncompressed) audio requires 10
MB.
• Usually compressed (E.g. MP3, AAC,
Flac, Ogg Vorbis).

A Brief Look at
Multimedia Data:
Input and Format
Video
• Input: Analog Video is usually captured by a video camera
and then digitized.
• There are a variety of video (analog and digital) formats
• Raw video can be regarded as being a series of single
images.
• There are typically 25, 30 or 50 frames per second.

A Brief Look at
Multimedia Data:
Input and Format
Video
• E.g. A 512 512 size monochrome video images take
6.25MB for a second to store uncompressed.
• Typical PAL digital video (720 576 pixels per colour
frame).
• High Definition video on Blu-ray (up to 19201080 = 2
Megapixels per frame) Digital video clearly needs to be
compressed for most times.

Multimedia Data
Compression
• How can we compress data?
• Lossy v Lossless :
• Lossless : Ideal (e.g. zip, unix compress) not good enough
for MM data!
• Lossy :Throw away nonessential (perceptually less
relevant) parts of the data stream FILTER the data
somehow. Examples: MP3, JPEG, MPEG Video

Introduction
 Multimedia system is a computer
(machine) that can transmit and
accept information (multiple media)
from the external world.
 supports the integrated storage,
transmission and representation of
the discrete media types text,
graphics and image and the
continuous media types audio and
video on a digital computer.
Multimedia System

Media types:
Text Hypertext
Images Static & Dynamic
Audio Speech, Music
Video Movies, Documentaries
2D Graphics Vector Graphics
3D Graphics Games

Different form of media types:
01001000
01000101
01001100
01001100
01001111
10101110000101000010101000000011
10101111111110100000010101010001
10000000000011111010101010001011111111111
01111100000010101010001010010000010101010
10111111110100001000101010101010000100010
11110111000000101010111111111111011100000
11111111111111111111000000000000001010100
00000000000000000001000010000111111111000
1111111111111000000000011
1111010000001000100101000
1111111111000000000011110
1100000010000010001000000
1111111101000001111101001
1110100000000001000100000
11100 11000 1110
11101 11001 0001
11100 11000 1110
11101 11001 0001
11100 11000 1110
11101 11001 0001
Data file
Audio file
Video file
Image file
Graphics file

Characteristic
 Generation or capture of data
 Processing such as digitalize
 Storage in devices
 Manipulate using software such as
photoshop
 Distribution over LAN or WAN network
 Output using devices such as monitor
 Rendering
 representation,
 transmission
 several time-dependent and time-
independent media streams

Components
Devices Examples
Capture devices Video camera, audio
microphone, keyboard etc.
Software Processing
Elements
Content creation,
compression, encryption
etc.
Storage devices CD-ROMS, Hard disks,
Memories
Distribution network Ethernet, ATM, Fiber
Optics, Wireless
Display / Rendering devices Monitors, Speakers,
Projectors, Printers, CD-
quality speaker
(HDTV),SVGA, HiRes
monitor
Processing devices CPUs, Set Top boxes,
workstations, DSP
hardware

Features
 Very high processing power
 A file system capable of handling multimedia
information
 File formats that exploit the inherent properties of the
multimedia information
 Efficient and high I/O rate
 Multimedia operating system
 Storage and memory
 Network support
 Software tools

Features
 Very high processing power
- In the modern multimedia context, movement and
processing of large amounts of data in real-time are
emphasized.
- Include support hardware such as graphics and video
adapters, digital signal processors and etc.
 A file system capable of handling multimedia
information
- File systems should have the capability to deal with
streams of data, such as video and audio.

 File formats that exploit the inherent properties of the
multimedia information
- The file formats should be easy to handle from the file
system point of view.
 Efficient and high I/O rate
- The input and output subsystem should be able to
handle large storage.
- The controller hardware should be able to provide
recording and reading of data from these storages.

 Multimedia operating system
- An efficient file system and data structures are
required to provide support for direct transfers.
- Disk to NIU, real time scheduling of tasks, fast interrupt
processing and streams.
 Storage and memory
- Caches have to be larger and may have to be in a 2 or 3
level hierarchy for efficient management.

 Network support
- To apply client-server paradigm in building multimedia
application, high-speed support from the network
subsystem is needed.
- It should be able to stream the data out of the disk
directly, to minimize delay.
 Software tools
- Need to be user friendly, as they have to handle a
variety of media at the same time.
- To make things easier, tools should support object-
oriented software design and development
methodologies.

Types of Multimedia
Systems
Type I
Standalone
Multimedia
system
 Example: Authoring system and multimedia
presentation system
CPU
CPU I/O MEM
NIU
Display, Audio
Controller
Display
Disk

CPU I/O MEM
NIU
Display, Audio
Controller
Display
Disk
CPU I/O MEM
NIU
Display, Audio
Controller
Display
Disk
DEDICATED
LINK
(Physical or Logical)
Type II Peer-to-peer Multimedia system
 Example: multimedia workstation-based conferencing system

Type III Distributed Multimedia system
 Exist in a large interconnected networks
 Examples: Internet and corporate networks
CPU I/O MEM
NIU
Display, Audio
Controller
Display
Disk
CPU I/O MEM
NIU
Display, Audio
Controller
Disk
…
HIGH
SPEED
NETWORK

Examples
Digital satellite: TV program
Data
encoded and
encrypted
Data transmitted to
satellite via uplink in
Isleworth
Data travels a
round trip of
44,000 miles
before reaching
the user
Data
unencrypted,
decoded in the
Digibox and
viewed on
standard TV

 When broadcasting audio and video the following has to
happen; compression (to MPEG-2); encryption, this is
the encoding of television signals or data for security
purposes.
 Example: to restrict reception to authorised viewers only
or by online communication, to keep them private; and
Transmission.
 Transmission sees the data sent to the Astra 1 satellite in
space on a geostationary orbit.
 The satellite then fires the data back and 100%
availability over a large areas including those more
derelict places as long as you can point your minidisk at
the Astra 22.8 degrees east satellite, and do not have a
building or other obstruction in the direct line of sight.

Multimedia Home platform specification
(MHP)
is a generic interface between digital applications and user
machines, whether they happen to be set top boxes, digital TV
sets or Multimedia Pc’s. MHP extends the DVB open standards.
Addressed are MHP architecture, System core, and MHP
Profiles.

 The Radio Frequency (RF) Input (from satellite
dish/aerial) is demodulated into an MPEG-2 stream.
 spliced into its video, audio, subtitle, and data streams,
and passed on to the CPU and Operating system.
 The data stream (usually image files) has its own
dedicated rendering engine and is directly passed to the
on screen display.
 The data streams are examined for system information,
teletext or MHP data object, and then routed via device
drivers to the hardware/software modules for
interpretation and execution.

FILE FORMATS
A file format is a standard way that
information is encoded for storage in a
computer file.
It specifies how bits are used to encode
information in a digital storage medium.
For Example-image file format , video file
format , text file format , audio file format.

AUDIO FILE FORMAT
An audio file format is a format for storing audio
data on a computer .
Its used for a audio playback for different tools and
program such as the computer,mp3players,cdplayers,
another.

TYPES OF AUDIO
FORMATS
->Wave Format(.Wav)was the standard audio files for
Windows 95.
->Music Instrument Digital Interface(.MIDI)files
transmit digital data such as the pitch and intensity
of musical notes to play.
->Mp3(.Mp3)files are common for playing music on
digital audio player.
->Window Media Audio(.WMA)files are similar to
Mp3 files but can compress at a higher rate.

IMAGE FILE FORMAT
Image file format are standardized means of organizing and
storing digital images.
GIF-
1stands for Graphic Interchange
Format.
2 Support 8-bit colors.
3supported by all the browser.
4- Large file size.
5-Extension is .gif.

PNG- 1-Stands for Portable Network Graphics.
2-Support for 24-bit color.
3 not supported by all browser.
4Smaller File size as compared to GIF
and JPEG.
5 Extension is . png.
JPEG-
1Stands for Joint Photographic
Expert Group.
2 Support 24-bit colors.
3 Supported by all browser
4Smaller file size as compared to gif.
5-Extension is .jpg or .jpeg.

Video File Format
video file format is consist of two ways:
->Analog
->Digital
Example:
->MPEG
->AVI
->SWF
->MOV
->WMV
>Real Video

MPEG -
1-stands for Motion Picture
Expert Group. 2-MPEG
files are also a common
format for digital videos
and movies.
3 It uses the filename
extension of .mpg or
.mpeg.
4 latest MPEG version
,MPEG
,
uses the
.mp4 filename extension.
MOV- 1 stands for Quick Time Movie
2 Quick Time Movie uses the .mov
filename extension.
3-.qt filename extension used as an
alternative.
And other AVI(AUDIO/VODEO INTERLEAVE.).

Multimedia
Database(MMDB)?
• Multimedia database is a collection of related multimedia
data.
• MMDB stores data in the form of, text, images, graphic,
animation, audio and video.
• A multimedia database is a database that include one or
more primary media file types such as .txt (documents),
.jpg (images), .swf (videos), .mp3 (audio), etc.
TEXT
AUDIO
GRAPHIC
VIDEO
ANIMATION

EVOLUTION OF
MMDB
Since existing relational and OO databases comprise the
basic requirements of any database, but to store multimedia
data -MMDB’s were evolved, that includes:
• long bit and byte strings
• BLOBS
• paths or references of images where the actual image
stored elsewhere, such as on an optical storage
subsystem.
• Content retrieval capabilities.

Multimedia Database Management
System
(MMDBMS) ?
• It is a framework that manages different types of data
potentially represented in a wide diversity of format on a
wide array of media sources. It provides support for
multimedia data types
• Provide facilitate for
• Creation,
• Storage,
• Access,
• Query
• Control of a multimedia database.

Contents of
MMDBA
• Media data - actual data representing images, audio, video
that are captured, digitized, processes, compressed and
stored.
• Media format data - contains format information of
the media data after it goes through the acquisition,
processing, and encoding phases.
• For instance, this consists of information such as the
sampling rate, resolution, frame rate, encoding scheme etc.
• Media keyword data - contains the keyword
descriptions,
• For example, for a video, this might include the date, time, and
place of recording , the person who recorded, the scene that is
recorded, etc .This is also called as content descriptive data.
• Media feature data - contains the features derived from
the media data. For example, contain information about the
distribution of colors, the kinds of textures and the different
shapes present in an image. This is also referred to as content
dependent data.

Continue
…
• The last three types are called meta data as they
describe
several different aspects of the media data.
• The media keyword data and media feature data are
used as
indices for searching purpose.
• The media format data is used to present the
retrieved information.

Types of multimedia
data
• Text: using a standard language (HTML)
• Graphics: encoded in CGM, postscript
• Images: bitmap, JPEG, MPEG
• Video: sequenced image data at specified rates
• Audio: aural recordings in a string of bits in digitized
form

data types are categorized into 3
classes
• Static media
• time-independent
• image and graphic object
• Dynamic media
• time-dependent
• Audio, video and animation
Dimensional media
• 3D game and Computer Aided Drafting (CAD)
programs.

Characteristics of
MMDM
Corresponding Storage Media
Data must be stored & managed according to their
specific characteristics of the storage media
 Descriptive Search Methods
Query must be descriptive & content oriented
View Specific & Simultaneous Data Access
Same data can be accessed through different queries
by different applications
 Management of Large Amounts of data
 Real time Data Transfer
Data transfer of real time activity gets higher priority
than other database activities
Large Transactions
Large transactions must be done in a reliable fashion,
since it take long time.

Requirements of Multimedia
databases
• Integration
• Data items do not need to be duplicated for different
programs invocations
• Data independence
• Separate the database and the management from the
application
programs
• Concurrency control
• Allows concurrent transactions
• Persistence
• Data objects can be saved and re-used by different
transactions
and program invocations
• Privacy
• Access and authorization control

Continue
…
• Integrity control
• Ensures database consistency between transactions
• Recovery
• Failures of transactions should not affect the persistent
data storage
• Query support
• Allows easy querying of multimedia data

Design goal of
MMDB
• Manage different types of input, output, and storage devices
• Handle a variety of data compression and storage formats
• Support different computing platforms and operating systems
• Integrate different data model.-(R database , OO database)
• Offer a variety of user-friendly query systems suited to
different kinds of media.
• Handle different kinds of indices
• Provide transparent view of geographically distributed data
• Synchronize different media types while presenting to user

Multimedia DB Storage
Multimedia data storage
How the multimedia data structured?
How these data can be stored?

Data
Structures
1) Raw Data - Uncompressed Image
2) Registering Data - Size & coding details of raw data
3) Descriptive Data - Textual numerical annotations
Media Raw Registering Descriptive
Text Characters Coding scheme
(ASCII), length / end
symbol
Key words,
information for
structuring
Images Pixels Height/ Width of
picture, Mode of
Compression, if JPEG,
tables for quantization
purpose
Pic.Date =
21/04/07
Pic.Reason =
Birthday
Etc
Video Pixels Frames/second,
coding details,
frame types…
Scene description
Audio Sample
sequenc
Audio coding
(PCM,…) , resolution
Content of audio
passages in short

Querying MM
data
 A DBMS permits a user to search the database by
content e.g. give the name of the student with roll
number 123456 We would like to do the same with
multimedia data
 e.g. Consider a police investigation of a murder
case. give the picture of a this person or audio files
to multimedia data library( contain CCTV footage,
authorized telephone wiretaps, document data ,bank
transition.)
 With standard data this is easy – numeric and string
operators are well understood With multimedia data
this is more difficult and requires some method of
identifying contents of which there are two kinds:

Cont
… automatic identification
an algorithm takes the data and returns a
measure which can be compared – e.g. of
blackness
 manual identification
a person examines the data and catalogues it
– e.g. in a table of pictures, there is a column
for the picture and another for the painter

Housing Multimedia
Data
There are three kinds of DBMS that might
be used for housing multimedia data.
 Relational DBMS store everything as First
Normal Form tables
 Object-oriented DBMS store everything as
classes of objects
 Object-relational DBMS are fundamentally
relations but are not First Normal Form

Relational DBMS
• Oracle support three large object types:
• BLOB(Binary Large Objects – BLOB) –
• The BLOB domain type stores unstructured binary data
in the database. BLOBs can store up to four gigabytes of
binary data.
• CLOB – The CLOB domain type stores up to four
gigabytes of single-byte character set data
• NCLOB - The NCLOB domain type stores up to four
gigabytes of fixed-width and varying width multi-byte
national character set data
• relational databases efficient for numeric and
textual data store, but do not conveniently
support content- based searches for multimedia
content.

Storing in
Database
• Oracle InterMedia
• Example:
create table image_blob_table (
id number primary key,
image_blob BLOB);
insert into image_blob_table (id, image_blob)
values (1, “Path of image”) ;

Object-oriented DBMS
• Jasmine is an Object-Oriented database that
stores the data in form of classes and objects
• It comes with a number of built in classes include
four multimedia classes:
• Picture -
• Image –
• Video –
• Audio -
• These come with manipulation and compression
facilities.
• They also have been made to fit well with Java
Media Framework

Object-relational DBMS
• The BFILE data type in Oracle provides access to BLOB
files of up to 4 gigabytes that are stored in file systems
outside an Oracle database.
• The BFILE data type allows read-only support of large
binary files; you cannot modify a file through Oracle.
Oracle provides APIs to access file data.

Example
:• CREATE OR REPLACE PROCEDURE
insert_image_file (p_id
NUMBER, p_image_name IN VARCHAR2) IS
src_file
BFILE;
dst_file
BLOB;
BEGIN src_file := BFILENAME ('image_DIR',
p_image_name); INSERT INTO temp_image (ID,
image_name, image ) VALUES (p_id, p_image_name,
EMPTY_BLOB () ) RETURNING image INTO dst_file;
END insert_image_file;
• Execute:
EXECUTE insert_image_file(1,'test_image.jpg');

Performance Issues
MMDBMSs provides good performance for real-time
querying and updating. Some of the features that
influence this are:
• indexing - most DBs use single key access (B-trees)
whereas, MM have multidimensional indexes with two
dimensional objects have X and Y co-ordinates(R-trees)
• content-retrieval indexing - special indexes are required
for this. For example, the index for a video could contain
the frame number of the start of each clip or scene.
• query optimization - MMDBs are large and manage
many complex object providing query optimization

Databases requires
• well structured data organisation
• efficient storage of large amounts of data
• Querying
• Fast retrieval of request
• transactional support for concurrent users

Issues and
challenges
• Multimedia data consists of a variety of media formats
or file representations including TIFF, BMP, PPT, FPX,
MPEG, AVI, MID, WAV, DOC, GIF,PNG,TEC
• Because of restrictions on the conversion from one
format to the other, the use of the data in a specific
format has been limited as well.
• Usually, the data size of multimedia is large such as
video; therefore, multimedia data often require a
large storage.
• Multimedia database consume a lot of processing
time, as well as bandwidth.

• multimedia data structures takes much time in retrieval
than standard database structures
• multimedia data structures do not easily lend
themselves to
content-based searching

APPLICATIO
NS
• Digital Libraries
• Documentation and keeping Records
• Knowledge distribution
• Education and Training
• News-on-Demand
• Advertisement
• Video-on-Demand
• Entertainment
• Music database
• Geographic Information System
• Marketing

Data Compression
and
Decompression

Why Data Compression?
Make optimal use of limited storage
space
Save time and help to optimize resources
Ifcompression and decompression are done in I/O processor, less time is
required to move datato or from storage subsystem, freeingI/O bus for
other work
In sending dataover communication line:less time to transmit and less
storage to host

Data Compression- Entropy
Entropy is the measure of information content
in a message.
Messages with higher entropy carry more information than
messages with lower entropy.
How to determine the entropy
Find the probability p(x) of symbol x in the message
The entropy H(x) of the symbol x is:
H(x) = - p(x) • log2p(x)
The average entropy over the entire message is
the sum of the entropy of alln symbols in the
message

Data Compression Methods
Data compression is about
storing and sending asmaller
number ofbits.
There’re two major categories
for methods to compress data:
lossless and lossy methods

Lossless Compression Methods
In lossless methods, original data and the data
after compression and decompression are
exactly the same.
Redundant data is removed in compression and
added during decompression.
Lossless methods are used when we can’t
afford to lose any data:legaland medical
documents, computer programs.

Run-length encoding
Simplest method of compression.
How: replace consecutive repeating occurrences of asymbol by 1
occurrence of the symbol itself,then followed by the number of
occurrences.
The method can be more efficientifthe datauses only 2 symbols
(0s and 1s) in bit patterns and 1 symbol is more frequent than
another.

Huffman Coding
Assign fewer bits to symbols that occur more
frequently and more bits to symbols appearless often.
There’s no unique Huffman code and every Huffman
code has the same average code length.
Algorithm:
a. Make aleafnode for each codesymbol
Add the generation probability of each symbol to the leaf node
a. Take the two leafnodes with the smallest probability and connect
them into anew node
Add 1 or 0 to each of the two branches
The probability of the new node is the sum of the probabilities of
the two connecting nodes
a. If there is only one node left,the code construction is completed. If
not, go back to (2)

Huffman Coding
Encoding
Decoding

Lempel Ziv Encoding
It is dictionary-based encoding
Basic idea:
Create adictionary(a table) of strings used during
communication.
Ifboth sender and receiver have acopy of the
dictionary, then previously-encountered strings can
be substituted by their index in the dictionary.

Lempel Ziv Compression
Have 2 phases:
Building anindexed dictionary
Compressing astring ofsymbols
• Algorithm:
Extract the smallest substring that cannot be found
in the remaining uncompressed string.
Store that substring in the dictionary as anew
entry and assign it an index value
Substring is replaced with the index found in the
dictionary
Insert the index and the last character of the
substring into the compressed string

Lempel Ziv Compression
Compression
example:

Audio Encoding
Predictive encoding
Only the differences

Lempel Ziv
Decompression
It’s just the inverse
of compression process

Lossy Compression Methods
Used for compressing images
and video files (our eyes cannot
distinguish subtle changes, so
lossy data is acceptable).
These methods are cheaper,
less time and space.
Several methods:
JPEG:compress pictures and
graphics
MPEG: compress video
MP3: compress audio

JPEG Encoding
Used to compress pictures and graphics.
In JPEG,agrayscale picture is divided into 8x8
pixel blocks to decrease the number of
calculations.
Basic idea:
Change the picture into alinear (vector) sets of numbers that
reveals the redundancies.
The redundancies is then removed by one of lossless
compression methods.

JPEG Encoding- DCT
DCT: Discrete Concise Transform
DCT transforms the 64 values in 8x8 pixel block in a
way that the relative relationships between pixels are
kept but the redundancies are revealed.
Example:
A gradientgrayscale

Quantization & Compression
Quantization:
After T tableis created, the values are quantized to reduce
the number of bits needed for encoding.
Quantization divides the number of bits by aconstant, then
drops the fraction.This is done to optimize the number of
bits and the number of 0s for each particular application.
• Compression:
Quantized values are read from the tableand redundant 0s
are removed.
To cluster the 0s together, the tableis read diagonallyin an
zigzagfashion.The reason is ifthe tabledoesn’t have fine
changes, the bottom right corner of the tableis all 0s.
JPEG usually uses lossless run-length encoding atthe
compression phase.

MPEG Encoding
Used to compress video.
Basic idea:
Each video is a rapid sequence of a set of frames.
Each frame is a spatial combination of pixels, or a
picture.
Compressing video =
spatiallycompressing each frame
+
temporally compressing aset of frames.

MPEG Encoding
Spatial Compression
Each frameis spatially compressed by JPEG.
• Temporal Compression
Redundant frames are removed.
For example,in astatic scene in which someone is talking,
most frames are the same except for the segment around the
speaker’s lips,which changes from one frame to the next.

Audio Compression
Used for speech or music
Speech: compress a64 kHz digitizedsignal
Music: compress a1.411 MHz signal
• Two categories of techniques:
Predictive encoding
Perceptual encoding

Audio Encoding
Predictive Encoding
Only the differencesbetween samples are encoded,
not the whole sample values.
Several standards: GSM (13 kbps), G.729 (8 kbps),
and G.723.3 (6.4 or 5.3 kbps)
• Perceptual Encoding: MP3
CD-quality audio needs atleast 1.411 Mbps and
cannot be sent over the Internet without
compression.
MP3 (MPEG audio layer 3) uses perceptual
encoding technique to compress audio.

Introduction
Characteristics ofsound and digital audio
DigitalAudioSystem
MIDI
AudioFileFormats
UsingAudioin MultimediaApplicationsAudio
Digital Audio and
Video

Digital Audio
SOUND
◦ Sound issimplywavesof compressed
air that moveout from its source, similar to dropping a stone in
water.
◦ Yourears are sensitive to these compressed waves.Asthe sound
wavesinteract with yourear drum, the signal is sent to your
brain, converting them to the sounds that yourecognize.
Twotypes ofSounds
Analogsound(recorded &reproducethe signals)
Digitalsound(stored asnumericaldata&
easilyreproduced&manipulated)

Characteristics of Sound & Digital
Audio
Sound isbased on two things: Energy&Transport medium.
Energyfrom a bangingdrum is converted into pressure that is
transmittedbythe air.
Thereare two characteristic ofsoundwaves
◦ Amplitude
◦ Amplitude isthe power or intensity ofthe sound.
◦ Thelouder the sound,the larger the amplitude.
◦ Frequency
◦ Frequency isthe rate at which sound ismeasured.
◦ It ismeasured in cyclesper second or Hertz(Hz).

Voice and
Music
TheThree classesofsounds:
V
oices,
Music and
Sound effects.
V
oiceisdefined as talking, not singing byhuman. It isdoes
not havea pitch and tone.
Musiccan haverapid changes in tone and pitch. It is
usuallyrecorded and playedin stereo.
Soundeffects can be voiceor music.It isvaryin tone, pitch
and time.

Digitizing Sound
Digitizing Sound means capturing the sound
amplitude at a specificfrequency.
Thethree characteristics for recording sound:
Frequency rate
Amplitude measurement
Soundchannels(Monaural or stereo)
Formultimediacomputers, standard
frequency ratesincludes
11.025Kilohertz(KHz)
22.5 KHz
44.1 KHz
◦ Amplitude measurementisbasedon8 bits(1byte)for 256 levels
and16bits for65,536 levels

Calculating Audio Data Size
There are three characteristic is used to calculating audio filesizes,
amplitude, sound channels and frequency rates
c=number ofchannels(mono-1channel, stereo- 2channel)
s=samplingrate in Hertz(cycle persecond)
t=a time(second)
b=bytes(1for8 bits, 2for16bits)
File size=c*s*t*b

Digital Audio System
 Digitalaudioscreated whenthe characteristic ofsound waveisrepresent
using number – a process referred to as Digitizing.
 Wecan digitize anysound(Microphone,Synthesizer…)
 Digitized sound is sampledsound
 Everynth fraction ofasecond,asample ofa soundistaken &stored as
digital information in bits and bytes.
 The quality ofdigital recording depends on the howoften the samples
are taken
i.e. Higherthe sampling rate better is the quality of sound.

sampling rates are normally used in multimedia 44.1KHz(CD-
Quality),22.05 KHzand 11.025KHz
Largerthe sample size, more accurate data willdescribe the recorded
sound
Thevalueofeach sample is rounded offto the nearest integer called
Quantization.
DigitalAudiosoftwaresupport
Playback
Record
Stop
Fast forward
Reverse
Rewind

Editing Digital Audio
Trimming the clip&SplicingandAssembly

Volume adjustment & Format Conversion

Resampling or down sampling &
Fade-ins & fade-out

Equalization & Time-Stretching

Digital Signal Processing & Reversing sounds

Sound Controls
Multimediacomputes usually havesoftware that acts as a control for
the computer’s sound playbackand recording. Thissoftware includes:
◦ Selectionofcomponents(Microphones,Speakers, Line-in, Line-out)
◦ V
olumelevel,both monaural and stereo
◦ Recording input level
◦ Bassand treble controls
Thesecontrols are usually accessible
for a user from the operating systemof
the computers as wellas from within multimedia products
themselves

MIDI(Musical Instrument Digital
Interface)
It is a communicationsstandard developedin the 1980’s for
electronic instruments and computers
It allowsinstruments from different manufactures to communicate
ButMIDIdata is not digitized, its onlymusic data stored in
numeric form.
DigitalAudiois recorded where as MIDIis a score
Device Independent
Thisfileis a list oftime stamped commands
that are recordings ofmusical actions

UsingMIDI,wecan easilyand quicklycompose our ownoriginal
score
Theprocess ofcreating MIDIis quite different from digitizing a
recorded audio
DigitizedAudio – BitmapImage the MIDI –Vector graphics
Fordigitizedaudio wesimple need to playthe recording through a
computer devicethat can digitallyrecord the sound
◦ TomakeMIDI,weneed
Notation software
Sequencer software
Synthesizer
MIDI Keyboard

MIDI vs. Digital Audio
Advantages
MIDIfilesare morecompactthat audio files
MIDIissmall,canbeeasilyembeddedinto the webpages
MIDIsound source ishighquality,this sounds better than audio
files
MIDIdata iscompletely editable
MIDIdata canbeeasilyconvertedto musicalnotations and vise
versa.

Audio File Format
A sound files format is a recognized
methodology for compressing data bits
ofdigitized sound into a data file
Thestructure ofthe filemust be knownevenbefore the data
can be savedor later loadedinto a computer to be edited or
playedassound
Thefilename extension identifies the method ofstorage used
On the Macintosh,digitized sounds maybe stored as data
files,resources, or applications such asAIFFor AIFC
InWindows,digitized sounds are usuallystored asW
A
Vfiles
Bothcan use MIDIfiles(.mid)

Audio File Format
Themethod used for consumer grade music CD’sis
LinearPulseCodemodulation(LPCM shortened to
PCM)
CD-ROM/XA(ExtendedArchitecture) format for
reading and writing CD’swasdevelopedlater soone
can put several recording sessions ofmusic to be
placed on a singleCD-R
LPCMtracks from anAudioCDare usually
converted and stored on a computer in
uncompressedAIFFor W
A
Vformat
◦ AIFF-AudioInterchangeFileFormat
◦ W
A
V-WaveFormat
Both contain
Uncompressed data

Audio File Format
Thereare huge no offileformats &
“MultimediaContainers” that store sound data
There are converter to read &write sound
filesfrom one format to another
MP3wasdeveloped byMPEG(MovingPicture Expert Group)
Common method for storing consumer audio
Contains lossycompression algorithm to save space
Evolvedin early1990’s
WMA(Windows MediaAudio)isa proprietary format
OGGwasdeveloped as an open-source and royaltyfree
”Container” for sound compressed using the “Vorbis
algorithm”
Vorbissound data resides within an OGGcontainer – OGG
Vorbis

MP$format is based onApple’sQuickTime
Movies(.mov) “container”model
It is similar to the MOVformat that stores
different types of media
MP4extension is used when filestreams audio &videotogether
M4aextension is used onlywhen filecontains onlyaudio data
M4pextension contains onlyaudio,but encrypted for Digital
Rights Management(DRM)
M4r filesare used for ringtones onApple’s iPhone
◦ Other GSMphones use 3gp file format for ring tones

Advanced Audio Coding(AAC) format is adopted by
Apple’siTunes store
Default format for iPod, iPhone, PlayStation, Wii, Dsi,
Motorala, Nokia, Philips, Samsung, Siemens and Sony
Ericssion
Codecisa software that compresses a stream ofaudio or
videofor data storage or transmission and then
decompresses it for playback
Someare “lossy”– trade quality for reduced filesize
transmission speed
Someare “lossless”– originaldata isnot at all altered.

Using Audio In Multimedia
Application Audio
Addingsound to Multimediaproject &the FollowingStep:
Determine the fileformats that are compatible with MMauthoring
S/W
Determine the sound playback capabilities that the end user’ssystem
offers
Decidethe type ofsound needed
Decidewhereand whenyouwant to use the digital audio or MIDI
data
Acquire source material by creating it from scratch or buying it
Editthe sounds to fit your project
Test the soundsto be sure they are timed properly with the project’s
images.(This mayinvolvesteps 1-4until everything isinsync)

Space Considerations
Largeamount ofdigital information isrequired for high
quality sound, whichin turn requires large amount of
space.
Ifmonaural sound isadequate then one can cut the
storage space byhalfor get double the playingtime in
the same memory space.
With compression codec'swecan reduce the size to 1/8th
the space but there islossofquality &wecan also
downsample.
Formula
◦ Samplingrate* bits persample/8=bytes/sec
◦ Sample rate* sample size/8 *#sec*2 (if stereo)= file size inKB

Audio Recording
Ifwerequire CDquality recording, then digitize
sound at 44.1KHz and 16bit to store, at a sound
studio.
Ifwedecideto do it our selfthen weneed to invest
in:
◦ Acousticallytreated room
◦ Highend amplifiers
◦ Recording equipments
◦ Powerful microphones
DigitalAudioTape(DAT)systemsprovide a
tape based 44.1KHz and 16bit record an
record playback capability
DATrecordings are too accurate, precisely
recording glitches, backgroundnoises,
microphone pops, including coughs in the
next room.

Audio Recording
Agoodeditor can reduce the impact of
these noisesbut at the cost oftime &money
Mobilephones can record audio.USB,Flashmemory
recorders can be used to record and the recordings can be
downloaded directly as digital filesusing USBcables or Card
readers
AudioCD’s– the method for digitallyencodingthe high
quality stereo ofthe consumer CDmarket is an international
standard, ISO10149.
Alsoknownas “RedBookAudio”Standard
Converter &burning s/w’s such asToast &CDcreator from
Roxiocan translate digital filesofRBAfound on CDdirectly to
digital sound fileformats likeMP3&W
A
V
.

Sound For Mobile
Ringtonesare normallyassociated with
phones
Telephonesystem – Pulsatingat 90 volt– signalis
sent to copper wire to energize the hammer like
klangsabell
Butin the present systems, the mobile’ss/wtakes the
charge whencomeonecalls,depending on the
programmed options, playsthe users choiceof
ringtone
Ringtonesplayon verysmall speakers &often
compete in a noisy environment
Theother tones includeanswer tones, sing tones,
true tones, real tones, videotones &ringels

Multimedia Storage and
Retrieval

Storage and Retrieval
A data storage device is a device for recording (storing) information (data).
Recording can be done using virtually any form of energy, spanning from manual
muscle power in handwriting, to acoustic vibrations in phonographic recording, to
electromagnetic energy modulating magnetic tape and optical discs.
Data retrieval means obtaining data from a database management system such
as ODBMS.
In this case, it is considered that data is represented in a structured way, and there
is no ambiguity in data.
In order to retrieve the desired data the user present a set of criteria by a
query.

Multimedia Storage
Multimedia can be stored in mediums such as Optical Disks, Hard Drives,
Magnetic Storage Media and such.

Multimedia Retrieval
Multimedia retrieval depends on the type of multimedia file it is which may be
continuous or discrete.
Continuous media is data where there is a timing relationship between
source and destination. Video, animation and audio are examples of
continuous media.
Some media is time independent or static or discrete media: normal data, text,
single images, graphics are examples.

Magnetic Media
Magnetic storage or magnetic recording is the storage of data on a
magnetised medium.
Magnetic storage uses different patterns of magnetisation in a magnetisable
material to store data and is a form of non-volatile memory.

Optical Media
Optical media - such as the compact disk (CD) - are storage media that hold
content in digital form and that are written and read by a laser; these media
include all the various CD and DVD variations, as well as optical jukeboxes and
autochangers.

Disk Spanning
Disc spanning is a feature of CD and DVD burning software that
automatically spreads a large amount of data across many data discs if the
data set's size exceeds the storage capacity of an individual blank disc.

RAID Storage
RAID is a technology that is used to increase the performance and/or
reliability of data storage.
The abbreviation stands for Redundant Array of Inexpensive Disks.
A RAID system consists of two or more disks working in parallel.

RAID Levels
Raid Level 0
Raid Level 1
Raid Level 2
Raid Level 3
Raid Level 4
Raid Level 5
Raid Level 6

RAID Level 0
RAID 0 (also known as a stripe set or striped volume) splits
("stripes") data evenly across two or more disks, without parity
information, redundancy, or fault tolerance.
Since RAID 0 provides no fault tolerance or redundancy, the failure of
one drive will cause the entire array to fail; as a result of having data
striped across all disks, the failure will result in total data loss.
This configuration is typically implemented having speed as the
intended goal.
RAID 0 is normally used to increase performance, although it can also
be used as a way to create a large logical volume out of two or more
physical disks.

RAID Level 1
RAID 1 consists of an exact copy (or mirror) of a set of data on two
or more disks; a classic RAID 1 mirrored pair contains two disks.
This configuration offers no parity, striping, or spanning of disk space
across multiple disks, since the data is mirrored on all disks belonging
to the array, and the array can only be as big as the smallest member
disk.
This layout is useful when read performance or reliability is more
important than write performance or the resulting data storage
capacity.

RAID Level 2
RAID 2, which is rarely used in practice, stripes
data at the bit (rather than block) level, and uses
a Hamming code for error correction.
The disks are synchronized by the controller to spin
at the same angular orientation (they reach index at
the same, so it generally cannot service multiple
requests simultaneously.
Extremely high data transfer rates are possible.

RAID Level 3
RAID 3, which is rarely used in practice, consists
of byte-level striping with a dedicated parity disk.
One of the characteristics of RAID 3 is that it
generally cannot service multiple requests
simultaneously, which happens because any
single block of data will, by definition, be spread
across all members of the set and will reside in
the same location.
Therefore, any I/O operation requires activity on
every disk and usually requires synchronized
spindles.

RAID Level 4
RAID 4 consists of block-level striping with a
dedicated parity disk.
As a result of its layout, RAID 4 provides good
performance of random reads, while the
performance of random writes is low due to
the need to write all parity data to a single
disk.

RAID Level 5
RAID 5 consists of block-level striping with
distributed parity.
Unlike in RAID 4, parity information is
distributed among the drives.
It requires that all drives but one be present to
operate.
Upon failure of a single drive, subsequent reads
can be calculated from the distributed parity such
that no data is lost.
RAID 5 requires at least three disks.

RAID Level 6
RAID 6 extends RAID 5 by adding
another parity block; thus, it uses
block-level striping with two parity
blocks distributed across all member
disks.

Cs8092 computer graphics and multimedia unit 4

Recommended

More Related Content

What's hot (20)

Similar to Cs8092 computer graphics and multimedia unit 4 (20)

More from SIMONTHOMAS S (20)

Recently uploaded (20)

Cs8092 computer graphics and multimedia unit 4