Chapter 6: Data Operations Management

Data Operations Management
Ahmed Alorage

Table of Content:
• 6. Data Operations Management
• 6.1 Introduction
• 6.2 Concepts and Activities
• 6.2.1 Database Support
• 6.2.2 Data Technology Management
• 6.3 Summary
• 6.3.1 Guiding Principles
• 6.3.2 Process Summary
• 6.3.3 Organizational and Cultural Issues

6 Data Operations Management
• Considers the fourth data management function in data management
framework “Figures 1.3 & 1.4 in Chapter 1”.
• Third Data Management function Interacts with and is influenced by Data
Governance function.
• In this Chapter,
• Defines the data operations management function
• Explains the concepts and activities involved

6.1 Introduction
• Data Operations Management defined as “the development, maintenance, and
support of structured data to maximize the value of the data resources to the
enterprise”
• Includes two sub-functions:
• Database Support
• Data Technology Management
• The Goals of data operations management include:
1. Protect and ensure the integrity of structured data assets.
2. Manage the availability of data throughout its lifecycle.
3. Optimize Performance of database transactions.

6.2 Concepts and Activities
• Data Operations Management is the function of Providing Support from
Data acquisition to data Purging.
• DBAs Play a key role in this critical function.
• Concepts and Activities of Data operations management and the roles of
DBA presented as follows:
6.2.1 Data Support 6.2.2 Data Technology Management
6.2.1.1 Implement and Control Database Environments 6.2.2.1 Understand Data Technology Requirements
6.2.1.2 Obtain Externally Sourced Data 6.2.2.2 Define the Data Technology Architecture
6.2.1.3 Plan for Data Recovery 6.2.2.3 Evaluate Data Technology
6.2.1.4 Backup and Recover Data 6.2.2.4 Install and Administer Data Technology
6.2.1.5 Set Database Performance Service Levels 6.2.2.5 Inventory and Track Data Technology Licenses
6.2.1.6 Monitor and Tune Database Performance 6.2.2.6 Support Data Technology Usage and Issues
6.2.1.7 Plan for Data Retention
6.2.1.8 Archive, Retain, and Purge
6.2.1.9 Support Specialized Databases

6.2.1 Database Support
• Considers “the heart of Data Management” and Provided by DBAs.
• DBAs play the dominant role in data operations management as others “Data
Security management and So on”.
• DBAs specialize as “Development / Production” DBAs.
• Production DBAs Primary responsibility for data operations Management:
• Ensure Performance and reliability of DB, including Performance Tuning,
Monitoring, and Error Reporting.
• Implement Backup and recovery mechanisms to guarantee the recoverability
of data in any circumstance.
• Implementing mechanisms for clustering and failover of the DB, if continual
data availability data is a requirement.
• Implementing mechanisms for archiving data operations management.

6.2.1 Database Support
• The Production DBAs responsible for The following Primary Deliverables:
1. A production Database Environment, include “Instance of DBMS and its
server”
2. Mechanisms and Processes for controlled implementation and changes to
DB into the production environment.
3. Appropriate Mechanisms for ensuring the availability, integrity, and
recoverability of the data in response to all possible circumstances that
could result in loss or corruption of data.
4. Appropriate Mechanisms for detecting and reporting any error that occurs
in the DB, DBMS or the data server.
5. Database availability, recovery, and performance in accordance with SLAs.
• DBAs do not perform all the activities of Data Management Operation
exclusively.
• Data Stewards, Architects, and analysts Participate in planning for recovery,
retention, and performance.
• Also, participate in obtaining and Processing data from external Sources.

Support Database
6.2.1.1 Implement and Control Database Environments
• Database Systems Administration includes the following tasks:
• Update DBMS Software – new versions and fixes in all environments
“Development and Production”
• Maintaining Multiple installations, including different DBMS versions
• Installing and Administering related data technology, including data integration
software and third-party data administration tools.
• Setting and tuning DBMS system parameters.
• Managing DB connectivity – access to DB required technical Guidance from
DBA
• Working with Programmers and Network Administrators to tune operating
systems, networks, and transaction processing middleware to work with the
DBMS.
• Working with Storge Administrators to set up and monitor effective storage
management Procedures “Dedicating appropriate storage for the DBMS”.
• In auditing, DBA should be audited by another DBA before go to production.
• DBA should have a back out plan to reverse changes in case of problems.

Support Database
6.2.1.2 Obtain Externally Sourced Data
• Most Organizations obtain some data from external third-party sources “e.g a list
of Potential Customers Purchased from an information broker, or product data
provided by a supplier”
• Either licensed or provided free of charge.
• Provided in number of different formats (CD, DVD, EDI, XML, RSS feeds, text files),
• one-time-only or regularly updated via a subscription service, required “Legal
agreements”
• A managed Approach to data acquisition centralizes responsibility for data
subscription services with data analysts:
• Data analyst will need to document the external data source in the logical
data model and data dictionary.
• A developer may design and create scripts and programs to read the data and
load it into a Database.
• DBA will be responsible for implementing the necessary process to load the
data into The DB and/or make it available to the application.

Support Database
6.2.1.3 Plan for Data Recovery
• Data Governance councils should SLAs with IT data management services
organizations for data availability and recovery.
• The scenarios That should DBA be considers about when making recovery plan for
DB and DB servers:
• Loss of the physical DB server.
• Loss of one or more disk storage devices.
• Loss of DB, include DBMS master DB, Temporary storage DB, Transaction log
segment, etc.
• Corruption of DB index or data pages.
• Loss of the DB or log segment file system.
• Loss of DB or transaction log backup files.
• Should be review and approved by the management and Business continuity
group. And DBA group must have easy access to it.
• Keep a copy of the plan “off-site location”, kept in secure.

Support Database
6.2.1.4 Backup and Recover Data
• In DB Backups:
• there should be frequent regular backup
• There should balance of the important of the data against the cost of
protecting it.
• for large DB, frequent backups can consume large amount of disk storage and
server resource.
• Furthermore:
• DB should reside on some sort of managed Storage Area “ideally a RAID array
on SAN, with daily backup to tape”
• The frequency of transaction log backups will depend on the frequency of
updating, and the amount of data involved.
• Backup files should be kept on a separate file system from the DB, daily,
secure, and off-site facility.

Support Database
6.2.1.4 Backup and Recover Data
• For extremely Critical Data, The DBA Implement some sort of replication scheme
in which Data moves to another DB on remote server. Schemes including:
• Mirroring and log shipping
• Two-phase commit process “The first DB are replicated immediately to the
secondary DB”
• Mirroring a more expensive option than log shipping.
• Other data protection options including Server Clustering, Server virtualization.
• “Hot backups”- backups taken while applications are running.
• “Cold backups”- backups taken when the DB is Off-line.
• When necessary, DBA recover lost or damaged DB by reloading them from the
necessary DB and transaction log backups to recover as much of the data as
possible.

Support Database
6.2.1.5 Set Database Performance Service Levels
• DB performance facets: availability and Performance “unavailable DB has
Performance measure of Zero”
• SLAs define expectations for DB performance. “identify the expected timeframe of
DB availability”
• Availability is percentage of time that a system or DB can be used for productive
work.
• For related factors affect availability:
• Manageability: The ability to create and maintain an effective environment.
• Recoverability: The ability to reestablish service after interruption, and
correct errors.
• Reliability: The ability to deliver service at specified levels for a stated period.
• Serviceability: The ability to determine the existence of problems, diagnose
their causes, and repair / solve the problems.

• Many things may cause a loss of DB availability, including:
• DBAs are responsible for doing everything possible to ensure DB stay online and
operational, including:
• Running DB backup utilities.
• Running DB Reorganization utilities.
• Running Statistics Gathering utilities.
• Running integrity checking utilities.
• Automating the execution of these utilities.
• Exploiting table space clustering and partitioning.
• Replicating data across mirror DB to ensure high availability.
Support Database
6.2.1.5 Set Database Performance Service Levels
Planned and Unplanned outages. Loss of the Server hardware. Disk hardware Failure.
Operating System Failure. DBMS software Failure. Application Problems.
Network Failure. Data Center site loss. Security and Authorization Problems.
Corruption of data (due to bugs, poor
design, or user error)
Loss of database objects Loss of data
Data replication failure. Severe Performance Problems Recovery Failures
Human error

Support Database
6.2.1.6 Monitor and Tune Database Performance
• DBAs should regularly respond and run activities of Monitoring Performance and
reporting analysis through The capability provided from DBMS and Server OS, to
optimize the DB performance.
• In ETL and batch Programs “required online transactions”, DBAs and Data
integration specialists monitor the performance, noting exceptional completion
times and errors, determining the root cause of errors, and resolving these
issues.
• When Performance Problems occur, DBA should use the monitoring and
administration tools of the DBMS to help identify the source of the problem.

Support Database
• The Most common possible reasons for Poor DB Performance are:
• Memory allocation (buffer/ cache for data)
• Locking and Blocking: (Process may lock DB resources “table or data pages”, or
two processes deadlock with each process locking resource)
• Failure to update DB statistics: “query optimizer” relies on stored statistics
about data & indexes to make decisions in RDMS, update these statistics
regularly and frequently.
• Poor SQL coding “most common cause”: Query coders need to know how the
SQL query optimizer works well.
• Insufficient indexing: Code complex queries and queries involve large tables to
use index built on the tables. Avoid creating too many indexes on heavily
updated table.
• Application Activity: executing application code on DB server can affect, should
be separated.
• Increase in the number, size, or use of DB: when more DBs has adverse effect on
the performance, “Create new DB server, relocate DB in very large, or use
archive data”
• Data Volatility: Inaccurate DB distribution statistics caused by large numbers of
table inserts and deletes over a short while. “turn off DB statistics for these
tables”

Support Database
• After the cause of the problem is identified, DBA should work integrative with
application developers to improve and optimize DB code and archiving or
deleting data that is no longer actively needed by application processes.
• In exceptional cases, DBA considers working with Data modeler to de-normalize
the affected portion of the DB, after these measures:
• The creation of views and indexes
• The rewriting of SQL code, have been tried.
• After careful consideration of the possible consequences: loss of data
integrity and increase complexity of SQ queries against de-normalized
tables.

Support Database
6.2.1.7 Plan for Data Retention
• Data Retention Plan “Balanced Operation”, Begin with discussing data owner at
design time, and reach agreement on how to treat data over its useful life.
• It is incorrect to assume all data will reside forever in “Primary storage”
• Not Active Data “e.g,. Not support application processes” Should be archived on
“Secondary Storage (Less expensive disk or tape or CD/DVD)”, or separate server.
• Purge data is obsolete and unnecessary, even for regulatory purposes.
• Some data may become a liability if kept longer than necessary.
• Remember, one of the principal goals of Data Management is that the cost of
maintaining data should not exceed its value to the organization.

Support Database
6.2.1.8 Archive, Retain, and Purge Data
• DBAs will work with application developers and other operations staff “including,
Server and storage administrators”, to implement the approved data retention
plan.
• This may require:
• Creating a secondary storage area.
• Building a secondary DB server.
• Replicating less-needed data to a separate DB.
• Partitioning Existing DB tables.
• Arranging for tape or disk backups.
• Creating DB jobs which periodically purge unneeded data.

Support Database
6.2.1.9 Support Specialized Databases
• Some specialized situations require specialized types of DB, different mange from
traditional Relational DB, for example:
• Object DB: used with Computer Assisted Design and Manufacturing
(CAD/CAM)
• Geospatial DB: used with Geospatial applications such as (MapQuest or
Google Map)
• XML DB: used with shopping-cart applications (copied into OLTP DB or DW)
• Own Proprietary DB: Many application did not divulge their own way of DB
building.
• Administration of DB used only to support a particular application should not
present any great difficulty.
• DBA will mostly be responsible for ensuring regular backups of DB and
performing recovery test.
• DBA may face a challenge of integration when data from these DBs needs to be
merged with other existing data.

6.2.2 Data Technology Management
• DBAs and other data professionals manage the technology related to their
field.
• The leading reference model for technology management is the Information
Technology Infrastructure Library (ITIL).
• ITIL Principles apply to managing data technology.
• Activities in this sections including Data Technology Requirements,
Architecture, Evaluations, installation and administration, Licenses, and
usage and issues.

Data Technology
6.2.2.1 Understand Data Technology Requirements
• the data & information needs of the business is important to suggest the best possible
applications of technology to solve business problems and take advantage of new
business opportunities.
• First Understand the requirements of data technology before determining what technical
solution to choose for a particular situation.
• Several Question to understanding suitability of data technology:
1. What problem does this data technology mean to solve?
2. What does this data technology do that is unavailable in other data technologies?
3. What does this data technology not do that is available in other data technologies?
4. Are there any specific hardware requirements for this data technology?
5. Are there any specific Operating System requirements for this data technology?
6. Are there any specific software requirements or additional applications required for
this data technology to perform as advertised?
7. Are there any specific storage requirements for this data technology?
8. Are there any specific network requirements for this data technology ?
9. Does this data technology include data security functionality? If not, what other
tools does this technology work with that provides for data security functionality?
10. Are there any specific skills required to be able support this data technology? Do
we have those skills in-house, or must we acquire them?

Data Technology
6.2.2.2 Define the Data Technology Architecture
• Is part of the enterprise’s overall technology architecture.
• Addresses three basic questions:
1. What technologies are standard (which are required, preferred, or
acceptable)?
2. Which technologies apply to which purposes and circumstances?
3. In a distributed environment, which technologies exist where, and how
does data move from one node to another?
• Data technologies to be included in the technology architecture include:
• DBMS software.
• Related DB management utilities “backup and recovery tools”.
• Data Modeling and Model management software.
• BI software for reporting and analysis.
• ETL and other data integration tools.
• Data Quality analysis and data cleansing tools.
• Meta-data management software, including meta-data repositories.

Data Technology
• Technology architecture components “bricks”. Several categories or views
representing facets of data management bricks are:
• Current: Products currently supported and used.
• Deployment Period: Products to be deployed for use in next 1-2 years.
• Strategic Period: Products expected to be available for use in next +2 years.
• Retirement: Products has retired or intends to retire this years.
• Preferred: Products preferred for use by most applications.
• Containment: Products limited to use by certain applications.
• Emerging: Products being researched and piloted for possible future
deployments
• The technology road map for the organization consists of these reviewed,
approved, and published bricks, and this helps govern future technology
decisions.

Data Technology
• Several things need to understood about technology:
• It is never free. Even open-source technology requires care and feeding.
• It should always be regarded as the means to an end, rather than the end
itself.
• Buying the same technology that everyone else is using, and using it in the
same way, does not create business value or competitive advantage for the
enterprise.
• After discussions with the business users and mangers, Data services group can
summarize the data technology objectives for the business in the form of a
strategic roadmap that can be used to inform and direct future data technology
research and project work.

Data Technology
6.2.2.3 Evaluate Data Technology
• Particularly the appropriate DB management technology, to meet business needs
including total cost, reliability, and integration.
• Data technologies to be researched and evaluated include:
• DBMS software.
• DB utilities, such as backup and recovery tools, and performance monitors.
• Data modeling and model management software.
• DB management tools, such as editors, schema generators, and DB object
generators.
• BI software for reporting and analysis.
• ETL and other data integration tools.
• Data quality analysis and data cleansing tools.
• Data Virtualization technology.
• Meta-data management software, including meta-data repositories.

Data Technology
• In addition, data professionals may have unique requirements for tools used in
other fields, including:
• Change management (source code library & configuration) tools.
• Problem and issue management tools.
• Test management tools.
• Test data generators.
• Criteria used to define and compare a weighted decision of technology:
1. Understand user needs, objectives, and related requirements.
2. Understand the technology alternatives.
3. Identify available technology alternatives.
4. Identify the feature required.
5. Weigh the important of each feature.
6. Understand each technology alternative.
7. Evaluate and score each technology alternative’s ability to meet
requirements.
8. Calculate total scores and rank technology alternatives by score.
9. Evaluate the results, including the weighted criteria.
10. Present the case for selecting the highest ranking alternative.

Data Technology
• Factors considers when selecting DBMS software include:
• Product architecture and complexity.
• Application profile, such as transaction processing, BI, and Personal
profiles.
• Organizational appetite for technical risk.
• Hardware platform and Operating system support.
• Availability of supporting software tools.
• Performance benchmarks.
• Scalability.
• Software, memory, and storage requirements.
• Available supply of trained technical professionals.
• Cost of ownership, such as licensing, maintenance, and computing
resources.
• Vendor reputation.
• Vendor support policy and release schedule.
• Customer references.

Data Technology
• Factors needed by DBA to assist in evaluating technology alternatives:
• The availability, stability, maturity, and cost of current products.
• The suitability of a given product to meet the current business need/
problem.
• The extensibility, of a given product to meet other business needs.
• The Product’s “fit” with the organization’s technology and architecture
roadmap.
• The product’s “fit” with other products and technology used by the
organization.
• The vendor’s reputation, stability, and expected longevity
• The degree of support expected from the vendor.
• The DBA will need to carefully test each candidate product to determine its
strengths and weakness, ease of implementation and use, applicability to current
and future business needs and problems, and whether it lives up to the vendor’s
hype.

Data Technology
6.2.2.4 Install and Administer Data Technology
• DBAs face the work of deploying new technology product in:
Development /test, QA / Certification, and production environments.
• Need to create and document processes and procedures for administering
the product with the least amount of effort and expense.
• Expense of product, including administration, licensing, and support must
not exceed the products’ value to business.
• The purchase of new products, and the implementation of new
technology, will need self-monitoring and self-administering.
• The cost and complexity of implementing new technology is usually
under-estimated. Therefore, starting with small pilot project and proof-of-
concept (POC) implementations.

Data Technology
6.2.2.5 Inventory and Track Data Technology Licenses
• Organizations must comply with all licensing agreements and regulatory
requirements.
• Carefully track and conduct yearly audits of software license and annual
support costs, with a server lease agreements and other fixed costs.
• Being out-of-compliance with licensing agreements cause serious financial
and legal risks for an organization.
• This data can also determine the total cost-of-ownership (TCO) for each type
of technology and technology product.
• Regularly evaluate technologies and products that are becoming obsolete,
unsupported, less useful, or too expensive.

Data Technology
6.2.2.6 Support Data Technology Usage and Issues
• When business need requires new technology, DBAs will work with business
users and application developers to ensure the most effective use of the
technology, to explore new applications of the technology, and to address any
problems or issues that surface from its use.
• DBAs and other data professionals serve as Level 2 technical support, working
with help desks and technology vendor support to understand, analyze and
resolve user problems.
• Training considers a key effective understanding and use of any technology.
• Training for everyone, involved in implementing, supporting, and using data and
DB technology.
• Training plans should include appropriate levels of cross training to better
support application development, especially Agile development.
• DBAs should have and take the opportunity to learn, application development
skills such as class modeling, use-case analysis, and application data access.
• Developers should learn some DB skills, especially SQL coding.

Summary
6.3.1 Guiding Principles
• Database Administration, craig Mullins offers DBAs the following rules of
thumb for data Operations Management:
1. Write everything down
2. Keep everything
3. Whenever possible, automate a procedure
4. Focus to understand the purpose of each task, mange scope, simplify,
do one thing at a time.
5. Measure twice, cut once.
6. Don’t panic; react calmly and rationally, because panic causes more
errors.
7. Understand the business, not just the technology.
8. Work together to collaborate, be accessible, audit each other’s work,
share your knowledge.
9. Use all of the resources at your disposal
10. Keep up to data.

Chapter 6: Data Operations Management

Recommended

More Related Content

What's hot (20)

Similar to Chapter 6: Data Operations Management (20)

Recently uploaded (20)

Chapter 6: Data Operations Management