Performance Metrics and Ontology for Describing Performance Data of Grid Workflows
0 likes646 views
Many Grid work ow middleware services require knowledge about the performance behavior of Grid applications/services in order to e ectively select, compose, and execute work ows in dynamic and complex Grid systems. To provide performance information for building such knowledge, Grid work ow performance tools have to select, measure, and analyze various performance metrics of work ows. However, there is a lack of a comprehensive study of performance metrics which can be used to evaluate the performance of a work ow executed in the Grid. Moreover, given the complexity of both Grid systems and work ows, semantics of essential performance-related concepts and relationships, and associated performance data in Grid work ows should be well described. In this paper, we analyze performance metrics that performance monitoring and analysis tools should provide during the evaluation of the performance of Grid work ows. Performance metrics are associated with multiple levels of abstraction. We introduce an ontology for describing performance data of Grid work ows and illustrate how the ontology can be utilized for monitoring and analyzing the performance of Grid work ows.
More Related Content
What's hot (13)
Viewers also liked (19)
Similar to Performance Metrics and Ontology for Describing Performance Data of Grid Workflows (20)
More from Hong-Linh Truong (20)
Recently uploaded (20)
Performance Metrics and Ontology for Describing Performance Data of Grid Workflows
- 1. Performance Metrics and Ontology for Describing Performance Data of Grid Workflows Hong-Linh Truong, Thomas Fahringer, Francesco Nerieri Distributed and Parallel Systems Group Institute for Computer Science, University of Innsbruck {truong,tf,nero}@dps.uibk.ac.at Schahram Dustdar Information Systems Institute, Vienna University of Technology dustdar@infosys.tuwien.ac.at http://dps.uibk.ac.at/projects/pma 1st Performability Workshop, CCGrid05, Cardiff 09 May, 2005
- 2. Outline Motivation Grid workflows and workflow execution model Performance metrics of Grid workflows WfPerfOnto: Ontology for describing performance data of Grid workflows Utilizing WfPerfOnto Conclusion and Future work Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 2
- 3. Motivation Lack of comprehensive study of useful performance metrics for Grid workflows A few metrics are studied and supported Most of metrics are being limited to the activity (task) level. study performance metrics at multiple levels of abstraction Describing and sharing performance data of Grid workflows Highly heterogeneous, inter-related and dynamic Inter-organizational Multiple types of performance and monitoring data provided by various tools an ontology for performance data • Can be used to describe concepts associated with workflow executions • Will facilitate the performance data sharing Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 3
- 4. Hierarchical Structure View of a Workflow <parallel> Workflow <activity name="mProject2"> <executable name="/home/truong/mProject2"/> </activity> Workflow Construct n <activity name="mProject1"> <executable name="/home/truong/mProject1"/> </activity> Activity m </parallel> mProject1.c Invoked Application m int main() { A(); while () { Code Code Code ... Region 1 Region … Region q } } Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 4
- 5. Workflow Execution Model (Simplified) Local scheduler Workflow execution Spanning multiple Grid sites Highly inter-organizational, inter-related and dynamic Multiple levels of job scheduling At workflow execution engine (part of WfMS) At Grid sites Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 5
- 6. Performance Metrics of Grid Workflows Interesting performance metrics associated with multiple levels of abstraction Metrics can be used in workflow composition, for comparing different invoked applications of a single activity, etc. Five levels of abstraction Code region, Invoked application Activity,Workflow construct, Workflow Performance metrics of a lower level can be used to construct similar metrics for the immediate higher-level By using aggregate operator Based on metric definition and structure of workflows Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 6
- 7. Performance Metrics at Code Region Level Category Metric Execution time ElapsedTIme, UserCPUTime, SystemCPUTime, SerialTime, EncodingTime Counter L2_TCM, L2_TCA, etc., (hardware counters) NCalls, NSubs, RecvMsgCount, SendMsgCount Synchronization CondSynTime, ExclSynTime Data Movement TotalCommTime, TotalTransSize Ratio MeanElapsedTime, CommPerComp, MeanTransRate, MeanTranSize CachMissRatio, MFLOPS, etc. Temporal overhead temporal overhead of parallel code regions Most existing conventional performance tools provide these metrics Existing workflow monitoring and analysis tools normally do not Challenging issues Integrate conventional performance monitoring tools into workflow monitoring tools Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 7
- 8. Performance Metrics at Invoked Application Level Most metrics can be constructed from metrics at code region level Category Metric Execution time ElapsedTime FailedTime Counter NCallFailed NCalls Ratio FailedFreq Performance Improvement SpeedupFactor Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 8
- 9. Performance Metrics at Activity Level Category Metric Execution time ElapsedTime, ProcessingTime, QueuingTime, SuspendingTime FailedTime, SharedResTime Counter RedandantActivity, NIteration, PathSelectionRatio, ResUtilization Ratio Throughput, MeanTimePerState, TransRate Synchronization SynDelay, ExecDelay Performance SlowdownFactor Improvement Metrics can be defined for both activity and activity instance Aggregate metrics of an activity can be defined based on its instances and the execution of instances at runtime Challenging problems How to monitor and correlate metrics when a resource is shared among applications Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 9
- 10. Performance Metrics at Workflow Construct Level Category Metric Execution time ElapsedTime, ProcessingTime Counter RedandantActivity, NIteration, PathSelectionRatio, ResUtilization Load balancing LoadIm (Load imbalance) Performance Improvement SpeedupFactor Generic and construct-specific metrics Resource RedundantProcessing Aggregate metrics of a workflow construct/workflow construct instance are defined based on the structure of the construct. E.g., LoadIm (load imbalance) is for parallel construct ElapsedTime/ProcessingTime is defined based on critical path Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 10
- 11. Performance Metrics at Workflow Level Category Metric Execution time ElapsedTime,ProcesingTime ParTime,SeqTime Ratio QueuingRatio, MeanProcessingTime, MeanQueuingTime, ResUtilization Correlation NAPerRes,ProcInRes,LoadImRes Performance Improvement Speedup Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 11
- 12. Performance Metrics Ontology WfMetricOnto OWL-based performance metrics ontology Metrics ontology Specifies which performance metrics a tool can provide Simplifies the access to performance metrics provided by various tools Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 12
- 13. Monitoring and Measuring Performance Metrics Performance monitoring and analysis tools Operate at multiple levels Correlate performance metrics from multiple levels Middleware and application instrumentation Instrument execution engine of WfMS • Execution engine can be distributed or centralized Instrument applications • Distributed, spanning multiple Grid sites Challenging problems: Performance tool and data complexity Integrate multiple performance monitoring tools executed on multiple Grid sites Integrate performance data produced by various tools Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 13
- 14. Ontology Describing Performance Data of Grid Workflows Objectives Understanding basic concepts associated with performance data of Grid workflows Performance data integration for Grid workflows Towards distributed/intelligent performance analysis WfPerfOnto (Ontology describing Performance data of Grid Workflows) Basic concepts • Concepts reflects the hierarchical view of a workflow • Static and dynamic performance and monitoring data of workflow Relationships • Static and dynamic relationships among concepts Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 14
- 15. Ontology for Describing Performance Data of Grid Workflows WfPerfOnto Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 15
- 16. Utilizing WfPerfOnto Describing Performance Data and Data Integration Different monitoring and analysis tools can store/export performance data in/to ontological representation High-level search and retrieval of performance data Knowledge base performance data of Grid workflows Utilized by high-level tools such as schedulers, workflow composition tools, etc. Used to re(discover) workflow patterns, interactions in workflows, to check correct execution, etc. Distributed Performance Analysis Performance analysis requests can be built based on WfPerfOnto Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 16
- 17. Utilizing WfPerfOnto: Describing Performance Data <rdf:Description rdf:about="http://dps.uibk.ac.at/wfperfonto#mImgtbl21"> <wfperfonto:hasPerfMetric rdf:resource="http://dps.uibk.ac.at/wfperfonto#ElapsedTime78"/> <rdf:type rdf:resource="http://dps.uibk.ac.at/wfperfonto#ActivityInstance"/> <wfperfonto:instanceName>mImgtbl21</wfperfonto:instanceName> <wfperfonto:hasPerfMetric rdf:resource="http://dps.uibk.ac.at/wfperfonto#QueuingTime80"/> <wfperfonto:ofActivity rdf:resource="http://dps.uibk.ac.at/wfperfonto#mImgtbl2"/> <wfperfonto:hasPerfMetric rdf:resource="http://dps.uibk.ac.at/wfperfonto#ProcessingTime79"/> </rdf:Description> Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 17
- 18. Utilizing WfPerfOnto: Checking Correct Execution <rdf:Description rdf:about="http://dps.uibk.ac.at/wfperfonto#Seq4ForkJoin5"> <wfperfonto:hasActivityInstance rdf:resource="http://dps.uibk.ac.at/wfperfonto#tRawImage4"/> <wfperfonto:hasActivityInstance rdf:resource="http://dps.uibk.ac.at/wfperfonto#tProjectedImage4"/> <wfperfonto:hasPerfMetric rdf:resource="http://dps.uibk.ac.at/wfperfonto#ElapsedTime57"/> <wfperfonto:hasPerfMetric rdf:resource="http://dps.uibk.ac.at/wfperfonto#QueuingTime59"/> <wfperfonto:hasActivityInstance rdf:resource="http://dps.uibk.ac.at/wfperfonto#mProject14"/> <wfperfonto:hasActivityInstance rdf:resource="http://dps.uibk.ac.at/wfperfonto#mImgtbl14"/> <rdf:type rdf:resource="http://dps.uibk.ac.at/wfperfonto#WorkflowConstructInstance"/> <wfperfonto:ofWorkflowConstruct rdf:resource="http://dps.uibk.ac.at/wfperfonto#SeqForkJoin"/> <wfperfonto:hasPerfMetric rdf:resource="http://dps.uibk.ac.at/wfperfonto#ProcessingTime58"/> <wfperfonto:instanceName>Seq4ForkJoin5</wfperfonto:instanceName> </rdf:Description> Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 18
- 19. Utilizing WfPerfOnto: Distributed Performance Analysis Clients External Knowledge Tools Builder Agent DIPAS Grid analysis Grid analysis agent agent Grid analysis agent GOM Grid analysis agent Grid analysis agent Monitoring Service Resources Applications Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 19
- 20. Utilizing WfPerfOnto: Analysis Request Requests based on Analysis WfPerfOnto agent Ontological data Monitoring agent Grid analysis agent To the Monitoring Service Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 20
- 21. Conclusion and Future Work Performance metrics of Grid workflows that characterize the performance and dependability of Grid workflows; metrics associated with multiple levels of abstraction Ontology describing performance data of Grid workflows Current implementation OWL-based ontologies, Jena toolkit for processing ontology-related task Store and export performance data in/to WfPerfOnto representation Future work Extend and revise performance metrics and WfPerfOnto Distributed performance analysis Reasoning performance data Shared conceptualization community work? Performance Metrics and Ontology for Describing Performance Data of Grid Workflows, CCGrid 05 21