Underlying principles of parallel and distributed computing
Download as pptx, pdf1 like8,050 views
Parallel and distributed computing allows problems to be broken into discrete parts that can be solved simultaneously. This approach utilizes multiple processors that work concurrently on different parts of the problem. There are several types of parallel architectures depending on how instructions and data are distributed across processors. Shared memory systems give all processors access to a common memory space while distributed memory assigns private memory to each processor requiring explicit data transfer. Large-scale systems may combine these approaches into hybrid designs. Distributed systems extend parallelism across a network and provide users with a single, integrated view of geographically dispersed resources and computers. Key challenges for distributed systems include transparency, scalability, fault tolerance and concurrency.
More Related Content
What's hot (20)
Similar to Underlying principles of parallel and distributed computing (20)
Recently uploaded (20)
Underlying principles of parallel and distributed computing
- 1. UNDERLYING PRINCIPLES OF PARALLELAND DISTRIBUTED COMPUTING
- 2. WhatisParallelComputing? Traditionally, software has been written for serial computation To be run on a single computer having a single Central Processing Unit A problemis broken intoa discrete series of instructions Instructions are executed one after another Only one instruction may execute at any momentin time
- 5. UsesforParallelComputing ScienceandEngineering Historically, parallel computing has been used to model difficult problems in many areas of science and engineering Atmosphere, Earth, Environment, Physics, Bioscience, Chemistry, Mechanical Engineering, Electrical Engineering, Circuit Design, Microelectronics, Defense, Weapons.
- 9. Named after the Hungarian mathematician John von Neumann who first authored the general requirements foran electroniccomputerinhis 1945papers Since then, virtuallyallcomputers have followed this basic design Comprises offourmain components: RAM is used tostore both program instructions anddata Control unitfetches instructions or data frommemory, Decodes instructions and then sequentially coordinates operations to accomplish the programmed task. Arithmetic Unitperforms basic arithmetic operations Input-Output is the interface tothe human operator vonNeumannArchitecture
- 10. Flynn'sClassicalTaxonomy Widely used classifications is called Flynn's Taxonomy Based upon the number of concurrent instruction and data streams available in thearchitecture
- 11. SingleInstruction,SingleData A sequential computer which exploits no parallelism in either the instruction or datastreams Single Instruction: Only one instruction stream is being acted on by the CPU during any one clock cycle Single Data: Only one data stream is being used as input during any one clock cycle Can have concurrent processing characteristics: Pipelined execution
- 12. SingleInstruction,MultipleData A computer which exploits multipledata streams against a single instruction stream toperform operations. Single Instruction: All processing units execute the same instruction at any given clock cycle. Multiple Data: Each processing unit can operate on a different data element.
- 13. MultipleInstruction,SingleData MultipleInstruction,SingleData Multiple Instruction: Each processing unit operates on the data independently via separate instruction streams. Single Data: A single data stream is fed into multiple processing units. Some conceivable uses might be: Multiple cryptography algorithms attempting to crack a single coded message.
- 14. MultipleInstruction,MultipleData Multiple autonomous processors simultaneously executing different instructions on differentdata. Multiple Instruction: Every processor may be executing a different instruction stream. Multiple Data: Every processor may be working with a differentdata stream.
- 15. SomeGeneralParallelTerminology Parallelcomputing Using parallel computer tosolve single problems faster Parallelcomputer Multiple-processor or multi-core system supporting parallel programming Parallelprogramming Programming in a language that supports concurrency explicitly SupercomputingorHighPerformanceComputing Using the world's fastest and largest computers to solve large problems
- 16. LimitsandCostsof Parallel Programming • Introducing the number of processors N performing the parallel fraction of work P, the relationship can be modeledby:
- 18. All processors access all memory as global address space Multiple processors can operate independently butshare the same memory resources Changes in a memory location effected by one processor are visible toall other processors SharedMemory
- 19. SharedMemory Classification UMAandNUMA,based uponmemoryaccesstimes UniformMemoryAccess(UMA) Most commonly represented today by Symmetric Multiprocessor (SMP) machines Identical processors Equal access and access times to memory Sometimes called CC-UMA - Cache Coherent UMA Cache coherent means if one processor updates a location in shared memory, all the other processors know about the update. Cache coherency is accomplished at the hardware level
- 20. SharedMemory Non-UniformMemoryAccess(NUMA) Often made by physically linking twoor more SMPs One SMP can directly access memory of another SMP Not all processors have equal access time toallmemories Memory access across link is slower If cache coherency is maintained, then may also be called CC- NUMA - Cache Coherent NUMA
- 21. DistributedMemory Processors have theirownlocal memory Changes to processor’s local memory have no effect on the memory of other processors. When a processor needs access to data in another processor, it is usually the task of the programmer to explicitly define how and when data is communicated. Synchronization between tasks is likewise the programmer's responsibility. The network "fabric" used for data transfer varieswidely, though itcan be as simple as Ethernet.
- 22. HybridDistributed-SharedMemory The largest and fastest computers in the world today employ both shared and distributedmemory architectures The shared memory component can be a shared memory machine or graphics processingunits. The distributed memory component is the networking of multipleshared memory or GPU machines.
- 24. Distributed Systems A collection of independent computers that appear to the users of the system as a single computer. A collection of autonomous computers, connected through a network and distribution middleware which enables computers to coordinate their activities and to share the resources of the system, so that users perceive the system as a single, integrated computing facility Eg:Internet
- 25. Example-ATM
- 27. Basic problems and challenges Basic problem and challenges Transparency Scalability Fault tolerance Concurrency Openness These challenges can also be seen as the goals or desired properties of a distributed system
- 28. REFERENCE • Mastering Cloud computing foundation and application programming rajkumar buyya,christian vecchiola,s.Tamarai selvi
- 29. THANK YOU