Skip to content

Learn. Share. Repeat.

Day: May 16, 2020

Cross-account AWS Glue Data Catalog access with Glue ETL

Posted on May 16, 2020May 19, 2020 by Anand

To process data in AWS Glue ETL, DataFrame or DynamicFrame is required. A DataFrame is similar to a table and supports functional-style (map/reduce/filter/etc.) along with SQL operations. The AWS Glue DynamicFrame is similar to DataFrame, except that each record is self-describing, so no schema is required initially. It computes a schema on-the-fly when required, and … Continue reading Cross-account AWS Glue Data Catalog access with Glue ETL

Posted in AWS, Big Data, Data Engineering, Glue Catalog, iamTagged AWS, Big Data, cross-account, Data Engineering, glue, Glue Catalog, glue etl, iam7 Comments
DISCLAIMER: The views expressed are mine and may not necessarily reflect that of my employers.The views,opinions and comments expressed by visitors are theirs alone and may not reflect my opinion.
May 2020
S M T W T F S
 12
3456789
10111213141516
17181920212223
24252627282930
31  
« Apr   Jul »

Archives

My Tweets

  • @WANatlParkFund @MtRainierWatch Colleague and friend @iamJohnmgray is climbing! Wishing him and all others best of luck. 2 days ago
  • And Liverpool wins! Fun final #FACupFinal 2 days ago
  • Chelsea vs. Liverpool - start of extra time #FACupFinal 2 days ago
  • @driscollis A) ([1, 2], ['three', 'four', 'five']) 2 days ago
  • RT @DalianaLiu: How to do outlier detection in 2022: Outlier detection is a common use case in tech. Here are 3 techniques you should know… 5 days ago

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 1,037 other followers

Recent Posts

  • Use Amazon MSK Connect with Lenses plugin to sink data from Amazon MSK to Amazon S3
  • Guide on PySpark DataFrame Functionality
  • Amazon Athena – CTAS, UNLOAD, Parameterized Prepared statements, Partition Projection
  • Apache Kafka – Introduction & Basic Commands
  • How to retrieve credentials stored in AWS Secrets Manager from AWS Lambda running in VPC

Blog Stats

  • 1,872,241 hits
Website Powered by WordPress.com.
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
  • Follow Following
    • Learn. Share. Repeat.
    • Join 1,037 other followers
    • Already have a WordPress.com account? Log in now.
    • Learn. Share. Repeat.
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...