More Related Content
What's hot (19)
Viewers also liked (20)
Similar to Rep on grid computing (20)
More from shweta-sharma99 (13)
Recently uploaded (20)
Rep on grid computing
- 1. PUNJAB COLLEGE OF TECHNICAL EDUCATION REPORT ON GRID COMPUTING SUBMITTED TO : SUBMITTED BY: Mrs.punam rattan Shivani Faculty Harpreet singh(281) Pcte Shweta Astha
- 2. 2 GRID COMPUTING CONTENTS Sr.No Description Page no. 1. INTRODUCTION 3 2. GRID TERMS AND CONCEPTS 4–9 3. APPLICATIONS OF GRID COMPUTING 10 - 12 4. DESIGN 13 - 21 5. BENEFITS 22 - 27 6. REFERENCES 28
- 3. 3 GRID COMPUTING INTRODUCTION:- Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach a common goal. The Grid can be thought of as a distributed system with non-interactive workloads that involve a large number of files. Grid computing can be seen as a journey along a path of integrating various technologies and solutions that move us closer to the final goal. Its key values are in the underlying distributed computing infrastructure technologies that are evolving in support of cross-organizational application and resource sharing—in a word, virtualization—virtualization across technologies, platforms, and organizations. This kind of virtualization is only achievable through the use of An environment that provides the ability to share and transparently accessresources across a distributed and heterogeneous environment not only requires the technology to virtualize certain resources, but also technologies and standards in the areas of scheduling, security, accounting, management systems ect.
- 4. 4 GRID COMPUTING 1. Grid terms and concepts:- 1.1 Types Of Resources A grid is a collection of machines, sometimes referred to as nodes, resources, members, donors, clients, hosts, engines, and many other such terms. They all contribute any combination of resources to the grid as a whole. Some resources may be used by all users of the grid, while others may have specific restrictions. 1.1.1 Computation The most common resource is computing cycles provided by the processors of the machines on the grid. The processors can vary in speed, architecture, software platform, and other associated factors, such as memory, storage, and connectivity. There are three primary ways to exploit the computation resources of a grid. The first and simplest is to use it to run an existing application on an available machine on the grid rather than locally. The second is to use an application designed to split its work in such a way that the separate parts can execute in parallel on different processors. The third is to run an application, that needs to be executed many times, on many different machines in the grid. Scalability is a measure of how efficiently the multiple processors on a grid are used. If twice as many processors makes an application complete in one half the time, then it is said to be perfectly scalable. However, there may be limits to scalability when applications can only be split into a limited number of separately running parts or if those parts experience some other interdependencies such as contention for resources of some kind.
- 5. 5 GRID COMPUTING 1.1.2. Storage The second most common resource used in a grid is data storage. A grid providing an integrated view of data storage is sometimes called a data grid. Each machine on the grid usually provides some quantity of storage for grid use, even if temporary. Storage can be memory attached to the processor or it can be secondary storage, using hard disk drives or other permanent storage media. Memory attached to a processor usually has very fast access but is volatile. It would best be used to cache data or to serve as temporary storage for running applications. Secondary storage in a grid can be used in interesting ways to increase capacity, performance, sharing, and reliability of data. Many grid systems use mountable networked file systems, such as Andrew File System (AFS), Network File System (NFS), Distributed File System (DFS), or General Parallel File System(GPFS). These offer varying degrees of performance, security features, and reliability features. Capacity can be increased by using the storage on multiple machines with a unifying file system.
- 6. 6 GRID COMPUTING 1.1.3. Communication another important resource of a grid is data communication capacity. This includes communications within the grid and external to the grid. Communications within the grid are important for sending jobs and their required data to points within the grid. Some jobs require a large amount of data to be processed, and it may not always reside on the machine running the job. The bandwidth available for such communications can often be a critical resource that can limit utilization of the grid. External communication access to the Internet, for example, can be valuable when building search engines. Machines on the grid may have connections to the external Internet in addition to the connectivity among the grid machines. When these connections do not share the same communication path, then they add to the total available bandwidth for accessing the Internet. 1.1.4. Software and Licenses The grid may have software installed that may be too expensive to install on every grid machine. Using a grid, the jobs requiring this software are sent to the particular machines on which this software happens to be installed. When the licensing fees are significant, this approach can save significant expenses for an organization. Some software licensing arrangements permit the software to be installed on all of the machines of a grid but may limit the number of installations that can be simultaneously used at any given instant. License management software keeps track of how many concurrent copies of the software are being used and prevents more than that number from executing at any given time.
- 7. 7 GRID COMPUTING 1.2. Jobs And Applications:- Jobs are programs that are executed at an appropriate point on the grid. They may compute something, execute one or more system commands, move or collect data, or operate machinery. A grid application that is organized as a collection of jobs is usually designed to have these jobs execute in parallel on different machines in the grid. The jobs may have specific dependencies that may prevent them from executing in parallel in all cases. For example, they may require some specific input data that must be copied to the machine on which the job is to run. Some jobs may require the output produced by certain other jobs and cannot be executed until those prerequisite jobs have completed executing. Jobs may spawn additional sub-jobs, depending on the data they process. This work flow can create a hierarchy of jobs and sub-jobs. Finally, the results of all of the jobs must be collected and appropriately assembled to produce the ultimate output/result for the application.
- 8. 8 GRID COMPUTING 1.3. Grid Software Components:- There are many aspects to grid computing that typically are controlled through software. These functions can be handled across a spectrum of very manual procedures to process being handled automatically through sophisticated software. 1.3.1 Management Components Any grid system has some management components. First, there is a component that keeps track of the resources available to the grid and which users are members of the grid. This information is used primarily to decide where grid jobs should be assigned. Second, there are measurement components that determine both the capacities of the nodes on the grid and their current utilization rate at any given time. This information is used to schedule jobs in the grid. Third, advanced grid management software can automatically manage many aspects of the grid. This is known as autonomic computing, or recovery oriented computing. This software would automatically recover from various kinds of grid failures and outages, finding alternative ways to get the workload processed. 1.3.2. Distributed Grid Management Larger grids may have a hierarchical or other type of organizational topology usually matching the connectivity topology. That is, machines locally connected together with a LAN form a cluster of machines. The grid may be organized in a hierarchy consisting of clusters of clusters. The work involved in managing the grid is distributed to increase the scalability of the grid. The collection and grid operation and resource data as well as job scheduling is distributed to match the topology of the grid.
- 9. 9 GRID COMPUTING 1.4. Inter Grid and Intra Grid:- Grids can be built in all sizes, ranging from just a few machines in a department to groups of machines organized as a hierarchy spanning the world. The grid system is likely to include some scheduling components. File sharing may still be accomplished using networked file systems. Machines participating in the grid may include systems from multiple departments but within the same organization. Such a grid is also referred to as an intra grid An inter grid requires the dynamic integration of applications, resources, and services with patterns, customers, and any other authorized organizations that will obtain access to the grid via the internet/WAN.
- 10. 10 GRID COMPUTING 2. Applications Of Grid Computing:- 2.1 General Applications If we take a parochial view of the grid as an environment that provides access to vast amounts of computing power, one of the simplest concepts for grid utilization is to be able to run an application somewhere else when your own machine is too busy or otherwise does not have the required resources. Almost any kind of application can be executed in a grid environment this way. You may not see spectacular performance gains unless the machine it runs on is much faster than the machine you usually use. Applications that can be run in a batch mode are the easiest to execute on other resources within the grid. Applications that need interaction through graphical user interfaces are more difficult to run on a grid, but not impossible. For instance, they can use remote graphical terminal support, such as X Windows or other similar capabilities. 2.2 CPU Intensive Applications Probably the most important step in grid enabling an application is to determine whether the calculations can be done in parallel. While High Performance Computing (HPC) clusters are sometimes used to handle the execution of applications that can utilize parallel processing, grids provide the ability to run these applications across a heterogeneous, geographically dispersed set of clusters. Rather than run the application on a single homogenous cluster, the application can take advantage of the larger set of resources in the grid. If the algorithm is such that each computation depends on the prior calculation, then a new algorithm (if possible) may be beneficial. In a distributed application, partial results or data dependencies may be met by communicating among sub-jobs. That is, one job may compute some intermediate result and then transmit it to
- 11. 11 GRID COMPUTING another job in the grid. If possible, one should consider whether it might be more efficient to simply re-compute the intermediate result at the point where it is needed rather than waiting for it from another job. One should also consider the transfer time from another job, versus retrieving it from a database of prior computations. 2.3 Data Consideration When the grid application is split into sub-jobs, often the input data is a large fixed set of data. This offers the opportunity to share this data rather than staging the entire set with each sub-job. However, one must consider that even with a shared mountable file system, the data is being sent over the network. The goal is to locate the shared data closer to the jobs that need the data. If the data is going to be used more than once, it could be replicated to the degree that space permits.
- 12. 12 GRID COMPUTING When considering applications that may be split into multiple parts for execution on a grid, it is important to consider the amounts of data that are needed to be sent to the node performing a calculation and the time required to send it. If the application can be split into small work units requiring little input data and producing small amounts of output data, that would be most ideal. The data in this kind of case is said to be staged to the node doing the work. Sending this data along with the executable file to the grid node doing the work is part of the function of most grid systems. However, in many applications, larger amounts of input and/or output data are involved, and this can cause complications and in-efficiencies.
- 13. 13 GRID COMPUTING 3. Design:- 3.1 Building a Grid Architecture The foundation of a grid solution design is typically built upon an existing infrastructure investment. However, a grid solution does not come to fruition by simply installing software to allocate resources on demand. Given that grid solutions are adaptable to meet the needs of various business problems, differing types of grids are designed to meet specific usage requirements and constraints. Additionally, differing topologies are designed to meet varying geographical constraints and network connectivity requirements. The success of a grid solution is heavily dependent on the amount of thought the IT architect puts into the solution design. At a glance, the grid architecture design should offer the following: The “blueprint” for the detailed conceptual design The use of open standards prescribed by the grid framework A multi-dimensional tiered and layered view of the grid infrastructure, which demonstrates the ability to logically partition grid resources so that their service consumption does not impact other grid locations The middleware components and subsystems for a grid infrastructure integration 3.2 Building Architecture Models:- There are different types of grid architectures to fit different types of business problems. Some grids are designed to take advantage of extra processing resources, whereas some grid architectures are designed to support collaboration between various organizations.
- 14. 14 GRID COMPUTING The type of grid selected is based primarily on the business problem that is being solved. Taking the goals of the business into consideration will help you choose the proper type of grid framework. A business that wants to tap into unused resources for calculating risk analysis within their corporate data center will have a much different design than a company that wants to open their distributed network to create a federated database with one or two of their main suppliers. Such different types of grid applications will require different designs, based on their respective unique requirements. The selection of a specific grid type will have a direct impact on the grid solution design. Additionally, it should be mentioned that grid technologies are still evolving and tactical modifications to a grid reference architecture may be required to satisfy a particular business requirement. 3.2.1 Computational Grid A computational grid aggregates the processing power from a distributed collection of systems. A well-known example of a computational grid is the SETI@home grid. This type of grid is primarily comprised of low-powered computers with minimal application logic awareness and minimal storage capacity. Additional uses for a computational grid include mathematical equations, derivatives, pricing, portfolio valuation, and simulation (especially risk measurement). Note that not all algorithms are able to leverage parallel processing, data intensive and high throughput computing, order and transaction processing, market information dissemination, and enterprise risk management. In many cases, the grid architecture model is not (yet) suitable for real-time applications. 3.2.2 Data Grid While computational grids are more suited for aggregating resources, data grids focus on providing secure access to distributed, heterogeneous pools of data. Through collaboration, data grids can also include resources such as a federated database. Within a federated database, as
- 15. 15 GRID COMPUTING illustrated in Figure 8-1 on page 103, a data grid makes a group of databases available that function as a single virtual database. Through this single interface, the federated database provides a single query point, data modeling, and data consistency. Data grids also harness data, storage, and network resources located in distinct administrative domains, respect local and global policies governing how data can be used, schedule resources efficiently (again subject to local and global constraints), and provide high speed and reliable access to data. Businesses interested in data grids typically have IT initiatives to expand data- mining abilities while maximizing the utilization of an existing storage infrastructure investment, and to reduce the complexity of data management. 3.3 Grid Topologies :- Intra grids Single organizations
- 16. 16 GRID COMPUTING No partner integration A single cluster Extra grids Multiple organizations Partner integration Multiple clusters Inter grids Many organizations Multiple partners Many multiple clusters 3.3.1 Intra Grid:- A typical intra-grid topology, exists within a single organization, providing a basic set of Grid services. The single organization could be made up of a number of computers that share a common security domain, and share data internally on a private network. The primary
- 17. 17 GRID COMPUTING characteristics of an intra-grid are a single security provider, bandwidth on the private network is high and always available, and there is a single environment within a single network. Within an intra-grid, it is easier to design and operate computational and data grids. 3.3.2 Extra Grid Based on a single organization, the extra-grid expands on the concept by bringing together two or more intra-grids. An extra-grid, typically involves more than one security provider, and the level of management complexity increases. The primary characteristics of an extra-grid are dispersed security, multiple organizations, and remote/WAN connectivity. Within an extra-grid, the resources become more dynamic and your grid needs to be more reactive to failed resources and failed components. The design becomes more complicated and information services become relevant to ensure that grid resources have access to workload management at run time.
- 18. 18 GRID COMPUTING 3.3.3 Inter Grid An inter-grid requires the dynamic integration of applications, resources, and services with patterns, customers, and any other authorized organizations that will obtain access to the grid via the internet/WAN. An inter-grid topology, is primarily used by engineering firms, life science industries, manufacturers, and by businesses in the financial industry. The primary characteristics of an inter-grid include dispersed security, multiple organizations, and remote/WAN connectivity. The data in an inter-grid is global public data, and applications (both vertical and horizontal) must be modified for a global audience. A business may deem an inter-grid necessary
- 19. 19 GRID COMPUTING if there is a need for peer-to-peer computing, a collaborative computing community, or simplified end-to-end processes with the organizations that will use the inter-grid. 3.4 Phases and Activities Deciding which grid type and topology to chose from is just the first step in the grid architecture design. A mature end-to-end design methodology is comprised of distinct phases and activities. 3.4.1 Basic Methodology For building a grid architecture, using a basic methodology allows the design to follow a consistent path from beginning to end. A methodology is not a cookbook for building a grid architecture, but a way to trace the progress of the design from the kickoff meeting to the final end state. The methodology follows a reproducible set of guidelines that can be used over again
- 20. 20 GRID COMPUTING based on a set of successful guiding principles for architecture design. A methodology allows the architecture to follow a set of principles that can be documented from beginning to end throughout the design. We define one such basic design methodology for developing the grid conceptual architecture in the next three sections i> Understanding The Business Drivers ii> Requirement Gathering iii> Validate Requirement 3.4.2 Recommended Steps i> Grid Design Workshop The purpose of the grid design workshops is to help all of the parties involved to better understand the variables, options, and considerations that need to be taken into account when developing a grid infrastructure design. Many or most of the grid middleware, technologies, and system components are probably new to many people within the design team and it is always a good idea to hear firsthand from experienced IT professionals the means by which grid infrastructures can be implemented, as well as any pitfalls to watch out for when designing environments for grid computing. ii> Documentation An extremely critical means of communicating the design (your solution) of your grid infrastructure is via an architecture or solution document. The solution document should start with a high-level overview of the environment and subsequently should drill down into the most detailed configuration diagrams and descriptions possible. You will want to include things like
- 21. 21 GRID COMPUTING IP addresses, network routes, server names, server architectures, network hardware, and essentially everything you know about the infrastructure at the time your design is completed. iii> Prototype Building a prototype of a grid system can save significant time that would otherwise be spent debugging and re-tooling unforeseen system incompatibilities. Your goal in building a prototype should be to produce a small-scale, end-to-end backbone of what your production environment will look like. It should include all interoperating technologies and/or architectures, so that if any incompatibility exists, it will be apparent before the production system is implemented. When all of the kinks are ironed out of your prototype, you will be confident that all of your components will work together properly in your designed infrastructure, and, additionally, you will have some experience in the implementation of such a system.
- 22. 22 GRID COMPUTING 4. Benefits :- When you deploy a grid, it will be to meet a set of business requirements. To better match grid computing capabilities to those requirements, it is useful to keep in mind some common motivations for using grid computing. 4.1 Exploiting Under Utilized Resources One of the basic uses of grid computing is to run an existing application on a different machine. The machine on which the application is normally run might be unusually busy due to a peak in activity. The job in question could be run on an idle machine elsewhere on the grid. There are at least two prerequisites for this scenario. First, the application must be executable remotely and without undue overhead. Second, the remote machine must meet any special hardware, software, or resource requirements imposed by the application. For example, a batch job that spends a significant amount of time processing a set of input data to produce an output data set is perhaps the most ideal and simple use case for a grid. If the quantities of input and output are large, more thought and planning might be required to efficiently use the grid for such a job. It would usually not make sense to use a word processor remotely on a grid because there would probably be greater delays and more potential points of failure. 4.2 Parallel CPU Capacity The potential for massive parallel CPU capacity is one of the most common visions and attractive features of a grid. In addition to pure scientific needs, such computing power is driving a new evolution in industries such as the bio-medical field, financial modeling, oil exploration, motion picture animation, and many others. The common attribute among such uses is that the applications have been written to use algorithms that can be partitioned into independently running parts. A CPU-intensive grid
- 23. 23 GRID COMPUTING application can be thought of as many smaller subjobs, each executing on a different machine in the grid. To the extent that these subjobs do not need to communicate with each other, the more scalable the application becomes. A perfectly scalable application will, for example, finish in one tenth of the time if it uses ten times the number of processors. 4.3 Virtual Resources And Virtual Organization For Collaboration Another capability enabled by grid computing is to provide an environment for collaboration among a wider audience. In the past, distributed computing promised this collaboration and achieved it to some extent. Grid computing can take these capabilities to an even wider audience, while offering important standards that enable very heterogeneous systems to work together to form the image of a large virtual computing system offering a variety of resources. The users of the grid can be organized dynamically into a number of virtual organizations, each with different policy requirements. These virtual organizations can share their resources collectively as a larger grid.
- 24. 24 GRID COMPUTING Sharing starts with data in the form of files or databases. A data grid can expand data capabilities in several ways. First, files or databases can span many systems and thus have larger capacities than on any single system. Such spanning can improve data transfer rates through the use of striping techniques. Data can be duplicated throughout the grid to serve as a backup and can be hosted on or near the machines most likely to need the data, in conjunction with advanced scheduling techniques. Sharing is not limited to files, but also includes other resources, such as specialized devices, software, services, licenses, and so on. These resources are virtualized to give them a more uniform interoperability among heterogeneous grid participants. 4.4 Access To Additional Resources In addition to CPU and storage resources, a grid can provide access to other resources as well. The additional resources can be provided in additional numbers and/or capacity. For example, if a user needs to increase their total bandwidth to the Internet to implement a data mining search engine, the work can be split among grid machines that have independent connections to the Internet. In this way, total searching capability is multiplied, since each machine has a separate connection to the Internet. If the machines had shared the connection to the Internet, there would not have been an effective increase in bandwidth. Some machines may have expensive licensed software installed that users require. Users’ jobs can be sent to such machines, more fully exploiting the software licenses. Some machines on the grid may have special devices. Most of us have used remote printers, perhaps with advanced color capabilities or faster speeds. Similarly, a grid can be used to make use of other special equipment.
- 25. 25 GRID COMPUTING 4.5 Resource Balancing A grid federates a large number of resources contributed by individual machines into a large single-system image. For applications that are grid-enabled, the grid can offer a resource balancing effect by scheduling grid jobs on machines with low utilization. This feature can prove invaluable for handling occasional peak loads of activity in parts of a larger organization. This can happen in two ways: An unexpected peak can be routed to relatively idle machines in the grid. If the grid is already fully utilized, the lowest priority work being performed on the grid can be temporarily suspended or even cancelled and performed again later to make room for the higher priority work. Without a grid infrastructure, such balancing decisions are difficult to prioritize and execute. Occasionally, a project may suddenly rise in importance with a specific deadline. A grid cannot perform a miracle and achieve a deadline when it is already too close. However, if the size of the job is known, if it is a kind of job that can be sufficiently split into subjobs, and if enough resources are available after preempting lower priority work, a grid can bring a very large amount of processing. 4.6 Reliability High-end conventional computing systems use expensive hardware to increase reliability. They are built using chips with redundant circuits that vote on results, and contain logic to achieve graceful recovery from an assortment of hardware failures. The machines also use duplicate processors with hot pluggability so that when they fail, one can be replaced without turning the other off. Power supplies and cooling systems are duplicated. The systems are operated on special power sources that can start generators if utility power is interrupted. All of this builds a reliable system, but at a great cost, due to the duplication of expensive components.
- 26. 26 GRID COMPUTING In the future, we will see a complementary approach to reliability that relies on software and hardware. A grid is just the beginning of such technology. The systems in a grid can be relatively inexpensive and geographically dispersed. Thus, if there is a power or other kind of failure at one location, the other parts of the grid are not likely to be affected. Grid management software can automatically resubmit jobs to other machines on the grid when a failure is detected. In critical, real-time situations, multiple copies of important jobs can be run on different machines throughout the grid. Their results can be checked for any kind of inconsistency, such as computer failures, data corruption, or tampering. 4.7 Management The goal to virtualize the resources on the grid and more uniformly handle heterogeneous systems will create new opportunities to better manage a larger, more distributed IT infrastructure. It will be easier to visualize capacity and utilization, making it easier for IT departments to control expenditures for computing resources over a larger organization. The grid offers management of priorities among different projects. In the past, each project may have been responsible for its own IT resources and the associated expenses. Often these resources might be under utilized while another project finds itself in trouble, needing more resources due to unexpected events. With the larger view a grid can offer, it becomes easier to control and manage such situations. administrators can changeany number of policies that affect how the different organizations might share or compete for resources.
- 28. 28 GRID COMPUTING References’:- i> www.wikipedia.com ii> www.redbooks.ibm.com