Introduction to Data Governance with Unity Catalog in Databricks

4 Scenarios

1 Hour 20 Minutes

Intermediate

Industry

general

Skills

batch-etl

data-governance

access-control-security

approach

data-storage

data-modelling

Tools

databricks

Learning Objectives

Understand the importance of data governance and the challenges of managing access across multiple workspaces.

Explain key governance aspects such as access control, user & role management, data lineage, data audit, and data discovery.

Describe Unity Catalog and its role in centralizing governance, auditing, and metadata management in Databricks.

Create and manage catalogs in Unity Catalog using both Databricks UI and SQL commands.

Overview

This module introduces data governance in Databricks, focusing on challenges in managing data access across multiple workspaces. It covers key governance aspects such as access control, user & role management, data lineage, data audit, and data discovery to ensure data security, compliance, and efficiency.

The module then explores Unity Catalog, Databricks' built-in governance solution that centralizes access control, auditing, and metadata management across workspaces. It explains the benefits of a unified governance framework and the hierarchical structure of Unity Catalog.

Finally, you will learn how to create a catalog in Unity Catalog, which serves as the top-level container for organizing datasets. The module covers catalog creation using Databricks UI and SQL commands, catalog types, and storage configuration for efficient data management.

Prerequisites

Basic understanding of Databricks workspaces and how users access data.
Familiarity with database concepts, including schemas, tables