Cloudera 5.3 Update
1 like1,188 views
This document provides a restricted overview of the Cloudera 5.3 release. It discusses key features around security, data governance, SQL enhancements, and performance improvements. The document is confidential and proprietary to Cloudera.
More Related Content
What's hot (16)
Viewers also liked (20)
Similar to Cloudera 5.3 Update (20)
More from Cloudera, Inc. (20)
Recently uploaded (20)
Cloudera 5.3 Update
- 1. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE RESTRICTED — DO NOT DISTRIBUTE oudera 5.3 Release Overview ember 2014
- 2. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE JficaJon informa,on in this document is proprietary to Cloudera. No part of this document may be reproduced, copied or transmi:ed in m for any purpose without the express prior wri:en permission of Cloudera. document is a preliminary version and not subject to your license agreement or any other agreement with Cloudera. This docum tains only intended strategies, developments and func,onali,es of Cloudera products and is not intended to be binding upon udera to any par,cular course of business, product strategy and/or development. Please note that this document is subject to ch may be changed by Cloudera at any ,me without no,ce. udera assumes no responsibility for errors or omissions in this document. Cloudera does not warrant the accuracy or completene informa,on, text, graphics, links or other items contained within this material. This document is provided without a warranty o , either express or implied, including but not limited to the implied warran,es of merchantability, fitness for a par,cular purpos -‐infringement. udera shall have no liability for damages of any kind including without limita,on direct, special, indirect or consequen,al damag may result from the use of these materials. The limita,on shall not apply in cases of gross negligence.
- 3. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE Batch, InteracJ and Real-‐Time Leading performan and usability in one plaQorm. • End-‐to-‐end analyJ workflows • Access more data • Work with data in new ways • Enable new users e PlaQorm, Many Workloads Security and AdministraJon rocess Ingest oop, Flume ransform Reduce, Hive, ig, Spark Discover AnalyJc Database Impala Search Solr Model Machine Learning SAS, R, Spark, Mahout Serve NoSQL Database HBase Streaming Spark Streaming Unlimited Storage HDFS, HBase YARN, Cloudera Manager, Cloudera Navigator
- 4. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE e importance of being mulJ-‐framework BATCH PROCESSING MR / PIG/ Hive / Spark / Crunch SQL IMPALA SEARCH SOLR MACHINE LEARNING SAS, R, 0xdata, Spark / MLlib STREAM PROCESSING SPARK STREAMING NOSQL HBASE Process Data Train & Test Models Respond to Events in Real Time Explore & Analyz Data ghly mature ide range of clients gnificant advances in peed & usability • Broad range of opJons • Most extensive integraJon with the SAS & RevoluJon product porQolio • Python / 0xdata / ML lib for advanced users • Very low (~10ms) latency • High volumes of single events • High speed • High concurrency • Workload management • Broad BI support • For unstructured & semi structured data • For business users • Low (1 second) latency • Windows (collecJons) of events
- 5. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE • Security focus – Richer experience, improved automaJon & performance • Data Governance – Rules-‐based engine for assigning business metadata and policy rules to datasets their lifecycle erprise governance • Insight into Job SLAs, improved concurrency • Fine grained authorizaJon for cluster management • Resource isolaJon across more resources red infrastructure • Deep integraJon with large public cloud providers • Work with private cloud environments • Improved integraJon with Intel chipsets to maximize performance PlaQorm choice • Expanded SQL capabiliJes – nested data types, compliance with 3rd party extensions • Expanded SQL workloads – disk-‐based joins, cost-‐based opJmizer refinements • Performance – AddiJonal performance enhancements SQL range • AutomaJc handling of in-‐memory datasets, HSM including flash storage • Reduce $$ / TB using erasure coding, WORM support for compliant archival Storage oduct investment themes
- 6. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE pped on December 2014 ckwards compaJble, incremental update on 5.x stack ease Themes ecurity and Data Management • HDFS encrypJon • Unified authorizaJon and RBAC with Sentry • Navigator policy engine for sekng, monitoring and enforcing stewardship an curaJon policies nhanced cloud support – Microsol Azure park 1.2 ross-‐stack performance and stability improvements oudera 5.3
- 7. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE e right SQL engine for the use case SQL BI and SQL AnalyJcs Batch Processing Spark developers OR
- 8. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE Security Data Management (Navigator H 2014 • Simplified Kerberos setup • Partner integraJon – pass-‐thru credenJals • Sentry coverage – Impala, Hive, Search, HDFS • Cloudera Manager fine-‐grained roles • HBase cell-‐level security • HDFS ACLs • NaJve HDFS encrypJon • Hue applicaJon to manage security • IntegraJon of Navigator Key Trustee and Navigator Encrypt • RBAC for Navigator (Administrator, Auditor) • Extensible audiJng support • Comprehensive Audit support (Impala Search, Hue, etc) • Consolidated audiJng and lineage • Policy engine – associate metadata w data • Column level lineage for MapReduce terprise governance
- 9. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE Impala 2H 2014 • DECIMAL support • ORDER BY without LIMIT • Correlated subqueries • AnalyJc window funcJons • Use in-‐memory datasets for improved performance • char / varchar support • Incremental stats gathering L range
- 10. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE Workload Management Cloudera Manager 2H 2014 • Dynamic resource management for Impala (YARN integraJon) • YARN work preserving restarts • Fair scheduler improvements – Dominant Resource Fairness, Pre-‐ empJon improvements • SAS integraJon for staJc parJJoning • Simplified upgrades with upgrade wizard • Impala, Oozie, Flume monitoring & advisors • Custom alerts / Configurable Thresho • Fine-‐grained permissions manageme for CM • Direct-‐to-‐AD integraJon • Support for Isilon ared infrastructure
- 11. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE HDFS HBase 2H 2014 • HDFS ACLs • Key Manager interface • Hardware accelerated encrypJon • Write pipeline improvements • HDFS Caching • Cell level ACLs • Transparent encrypJon • Improved WAL write performance • Medium Object Store • HBase QOS (user level and workload type) orage
- 12. © Cloudera, Inc. All rights reserRESTRICTED — DO NOT DISTRIBUTE Cloudera Director Intel Enhancements 2H 2014 • Automated provisioning of nodes on AWS and Azure • Grow / Shrink cluster • User Interface for cluster management • Security setup • MapReduce shuffle opJmizaJons • AES-‐NI for disk writes aQorm choice
- 13. Thank you.