A collection of configuration templates for AWS Glue and AWS Glue DataBrew resources as well as security controls for monitoring and protecting AWS Glue configuration such as Config Rules, CloudWatch Alarms, EventBridge Rules, IAM policies, and more.

Glue
Amazon Kinesis Firehose: Convert Record Format (Glue Table to S3)

This template creates a Glue table and a Kinesis Firehose delivery stream. The Glue table is used to store data with specific columns and partition keys, while the Kinesis Firehose delivery stream is used to deliver data to an S3 bucket in a specific format.

CloudFormationTerraform
AWS Glue On-Demand Trigger

This template creates an on-demand trigger that triggers one job.

CloudFormationTerraform
AWS Glue Scheduled Trigger

This template creates a scheduled trigger that runs every two hours and triggers two jobs. It declares an argument for prod-job3.

CloudFormationTerraform
AWS Glue Conditional Trigger

This template creates a conditional trigger that starts a job based on the successful completion of the job run.

CloudFormationTerraform
AWS Glue Crawler

This CloudFormation template creates an AWS Glue Crawler that crawls an S3 bucket and updates a database with the crawled data. It also creates the necessary IAM roles and policies for the crawler to function properly.

CloudFormationTerraform
AWS Glue Job

This template creates an AWS Glue job with an associated role. The job is configured with a command to run a Glue ETL script located in an Amazon S3 bucket.

CloudFormationTerraform
AWS Glue Catalog Table for Athena with Parquet Format

Sets up an AWS Glue Catalog Table optimized for Athena, using Parquet format with SNAPPY compression, stored in S3.

Terraform
Custom Connector for AWS Glue using Snowflake

Defines and uses a custom connector for AWS Glue to connect to Snowflake using a JDBC driver stored in S3.

Terraform
Generate Python Script for AWS Glue

This template generates a Python script for AWS Glue using a Directed Acyclic Graph (DAG) to define data flow between nodes.

Terraform
Generate Scala Code for AWS Glue

This template generates Scala code for AWS Glue using a Directed Acyclic Graph (DAG) to define data flow between nodes.

Terraform
Custom Connector for Glue Connection

Defines and uses a custom connector for AWS Glue, specifically for a Snowflake JDBC connection, utilizing AWS Secrets Manager for credentials.

Terraform
AWS Glue Crawler for Catalog

Defines an AWS Glue Crawler for catalog targets with schema change policies.

Terraform
AWS Glue Data Catalog Encryption Settings

Configures encryption settings for AWS Glue Data Catalog, including connection password encryption and encryption at rest using AWS KMS.

Terraform
AWS Glue Data Quality Ruleset Targeting Specific Table

Creates an AWS Glue Data Quality Ruleset targeting a specific database and table with rules for data completeness.

Terraform
Ray Glue Job

Configures an AWS Glue job to use Ray framework with specific worker type and Python version.

Terraform
AWS Glue ML Transform with Catalog Database and Table

This template sets up an AWS Glue ML Transform for finding matches, along with a Glue Catalog Database and a Glue Catalog Table configured with various properties including storage, serialization, and partitioning.

Terraform
AWS Glue Catalog Database and Table with Partition Index

This template creates an AWS Glue catalog database and table, and defines a partition index on the table.

Terraform
AWS Glue Resource Policy Example

This template sets up a Glue resource policy allowing the creation of tables across all Glue resources in the specified AWS account and region.

Terraform
AWS Glue Schema Creation

Creates an AWS Glue Schema with a specified name, registry ARN, data format, compatibility setting, and schema definition.

Terraform
AWS Glue User Defined Function Setup

Creates an AWS Glue catalog database and a user defined function within that database.

Terraform
AWS Glue Workflow with Triggers and Jobs

Creates an AWS Glue workflow with two triggers and two jobs, forming a simple directed acyclic graph (DAG).

Terraform