Data Engineering Glossary

A unit of compute capacity in AWS Glue providing 4 vCPUs and 16 GB of RAM, used to measure and bill Glue job c…

An AWS Glue-specific distributed data abstraction built on top of Apache Spark DataFrames that can handle semi…

An AWS Glue component that automatically scans data sources, infers schemas, and creates or updates table defi…

The central metadata repository in AWS Glue that stores database and table definitions, schemas, and connectio…

An AWS Glue feature that tracks which data has already been processed so subsequent job runs only process new …

An AWS Glue orchestration feature that groups multiple ETL jobs, crawlers, and triggers into a single visual p…

An optimization technique where filter conditions are sent to the data source before data is transferred, redu…