How to Catalogue AWS RDS MySQL Database using AWS Glue Crawler

  • Posted in AWS

This blog gives you an overview on how to create data catalogues from RDS MySQL instance using AWS Glue crawler

It is assumed that you already have RDS MySQL DB instance running with few tables created in it.

In this blog, MySQL workbench is used to connect with instance database.

Once created the DB with required tables, navigate to AWS Glue console, to create crawler.

A screenshot of a computer

Description automatically generated

Enter the crawler name, click on next.

In Data Source Configuration, select ‘No’ in ‘Is your data already mapped to Glue tables?’ and click on ‘Add a data source’ .

Enter details as follows.

In ‘Include path’, keep ‘schema_name/%’ to crawl all the tables from database. In this example, my schema name is test_sql, click on Next and select the IAM role created from AWS Glue (create a new role, if not created)

Choose the Database created in Data Catalogue. (create a new database, if not created) 

Keep the crawler schedule as ‘On Demand’ frequency, click on Next and create a crawler.

Now run the crawler. It takes few minutes to complete its  running.
On its successful completion, you can check the tables in Data Catalogues in Database created in Data Catalogue of AWS Glue. There will be same tables as RDS instance in Data Catalogue Database

A screenshot of a computer

Description automatically generated
A screenshot of a computer

Description automatically generated

Addend Analytics is a Microsoft Gold Partner based in Mumbai, India, and a branch office in the U.S.

Addend has successfully implemented 100+ Microsoft Power BI and Business Central projects for 100+ clients across sectors like Financial Services, Banking, Insurance, Retail, Sales, Manufacturing, Real estate, Logistics, and Healthcare in countries like the US, Europe, Switzerland, and Australia.

Get a free consultation now by emailing us or contacting us.