HBASE, HIVE , ARCHITECTURE AND WORKING EXAMPLES

HBase: Overview
• HBase is a distributed column-oriented
data store built on top of HDFS
• HBase is an Apache open source project
whose goal is to provide storage for the
Hadoop Distributed Computing
• Data is logically organized into tables,
rows and columns
2

HBase: Part of Hadoop’s Ecosystem
3
HBase is built on top of HDFS
HBase files are
internally
stored in HDFS

HBase vs. HDFS
• Both are distributed systems that scale to
hundreds or thousands of nodes
• HDFS is good for batch processing
(scans over big files)
– Not good for record lookup
– Not good for incremental addition of small
batches
– Not good for updates 4

HBase vs. HDFS (Cont’d)
• HBase is designed to efficiently address
the above points
– Fast record lookup
– Support for record-level insertion
– Support for updates (not in place)
• HBase updates are done by creating new
versions of values
5

HBase vs. HDFS (Cont’d)
6
If application has neither random reads or writes  Stick to HDFS

HBase Data Model
• HBase is based on Google’s
Bigtable model
– Key-Value pairs
8
Row key
Column Family
value
TimeStamp

HBase: Keys and Column Families
10
Each row has a Key
Each record is divided into Column Families
Each column family consists of one or more Columns

• Key
– Byte array
– Serves as the
primary key for
the table
– Indexed far fast
lookup
• Column Family
– Has a name
(string)
– Contains one or
more related
columns
• Column
– Belongs to one
column family
– Included inside
the row
• familyName:col
umnName
11
Row key
Time
Stamp
Column
“content
s:”
Column “anchor:”
“com.apac
he.ww
w”
t12
“<html>
…”
t11
“<html>
…”
t10
“anchor:apache
.com”
“APACH
E”
“com.cnn.w
ww”
t15
“anchor:cnnsi.co
m”
“CNN”
t13
“anchor:my.look.
ca”
“CNN.co
m”
t6
“<html>
…”
t5
“<html>
…”
t3
“<html>
…”
Column family named “Contents”
Column family named “anchor”
Column named “apache.com”

• Version Number
– Unique within each
key
– By default
System’s
timestamp
– Data type is Long
• Value (Cell)
– Byte array
12
Row key
Time
Stamp
Column
“content
s:”
“com.apac
he.ww
w”
t12
“<html>
…”
t11
“<html>
…”
t10
“anchor:apache
.com”
“APACH
E”
“com.cnn.w
ww”
t15
“anchor:cnnsi.co
m”
“CNN”
t13
“anchor:my.look.
ca”
“CNN.co
m”
t6
“<html>
…”
t5
“<html>
…”
t3
“<html>
…”
Version number for each row
value

Notes on Data Model
• HBase schema consists of several Tables
• Each table consists of a set of Column
Families
– Columns are not part of the schema
• HBase has Dynamic Columns
– Because column names are encoded inside the
cells
– Different cells can have different columns
13
“Roles” column family
has different columns
in different cells

Notes on Data Model (Cont’d)
• The version number can be user-supplied
– Even does not have to be inserted in increasing order
– Version number are unique within each key
• Table can be very sparse
– Many cells are empty
• Keys are indexed as the primary key
Has two columns
[cnnsi.com & my.look.ca]

HBase Physical Model
• Each column family is stored in a separate file
(called HTables)
• Key & Version numbers are replicated with each
column family
• Empty cells are not stored
16
HBase maintains a multi-
level index on values:
<key, column family,
column name, timestamp>

HBase Regions
• Each HTable (column family) is
partitioned horizontally into regions
– Regions are counterpart to HDFS blocks
19
Each will be one region

Three Major Components
21
• The HBaseMaster
– One master
• The HRegionServer
– Many region
servers
• The HBase client

HBase Components
• Region
– A subset of a table’s rows, like horizontal
range partitioning
– Automatically done
• RegionServer (many slaves)
– Manages data regions
– Serves data for reads and writes (using a
log)
• Master
– Responsible for coordinating the slaves
– Assigns regions, detects failures
– Admin functions 22

ZooKeeper
• HBase depends on
ZooKeeper
• By default HBase manages
the ZooKeeper instance
– E.g., starts and stops
ZooKeeper
• HMaster and HRegionServers
register themselves with
ZooKeeper
24

Creating a Table
HBaseAdmin admin= new
HBaseAdmin(config);
HColumnDescriptor []column;
column= new HColumnDescriptor[2];
column[0]=new
HColumnDescriptor("columnFamily1:");
column[1]=new
HColumnDescriptor("columnFamily2:");
HTableDescriptor desc= new
HTableDescriptor(Bytes.toBytes("MyTable"));
desc.addFamily(column[0]);
desc.addFamily(column[1]);
admin.createTable(desc);
25

Operations On Regions: Get()
• Given a key  return corresponding record
• For each value return the highest version
26
• Can control the number of versions you want

Operations On Regions: Scan()
27

Get()
Row key
Time
Stamp
“com.apache.www”
t12
t11
t10 “anchor:apache.com” “APACHE”
“com.cnn.www”
t9 “anchor:cnnsi.com” “CNN”
t8 “anchor:my.look.ca” “CNN.com”
t6
t5
t3
Select value from table where
key=‘com.apache.www’
AND
label=‘anchor:apache.com’

Scan()
Select value from table
where
anchor=‘cnnsi.com’
Row key
Time
Stamp
“com.apache.www”
t12
t11
t10 “anchor:apache.com” “APACHE”
“com.cnn.www”
t9 “anchor:cnnsi.com” “CNN”
t8 “anchor:my.look.ca” “CNN.com”
t6
t5
t3

Operations On Regions: Put()
• Insert a new record (with a new key), Or
• Insert a record for an existing key
30
Implicit version number
(timestamp)
Explicit version number

Operations On Regions: Delete()
• Marking table cells as deleted
• Multiple levels
– Can mark an entire column family as deleted
– Can make all column families of a given row as deleted
31
• All operations are logged by the
RegionServers
• The log is flushed periodically

HBase: Joins
• HBase does not support joins
• Can be done in the application layer
– Using scan() and get() operations
32

Altering a Table
33
Disable the table before changing the
schema

HBase Deployment
35
Master
node
Slave
nodes

HBASE, HIVE , ARCHITECTURE AND WORKING EXAMPLES

Recommended

More Related Content

Similar to HBASE, HIVE , ARCHITECTURE AND WORKING EXAMPLES (20)

Recently uploaded (20)

HBASE, HIVE , ARCHITECTURE AND WORKING EXAMPLES