Engineering Decisions for Data Lakes on Amazon S3

a single decision to make or break your firm

Ask many technology executives on what their firm’s most valuabl

e asset is. You would get couple answers, but one that would come up often is: “data”. As new paradigms emerge around Data Science and new breakthroughs around Machine Learning, both of these areas rely on data, and lots of it. As a result, your firm’s data architecture becomes more important in order to empower business units that rely on data to create insights and ultimately improve customer experience, which is increasingly becoming the focus of “data-driven” Product Managers. If you look at the thriving firms, you will notice many of them are companies that derive insights from huge amounts of data. Dont fall behind avatar.

Data Lakes

If your firm already has a sizeable footprint on AWS, S3 is the natural candidate for creating your data lake. Unfortunately your data lake can implode if you do not architect it well and plan for future scaling. Generally, S3 just flat out works for many customers with basic workloads. However, there can be important decisions for some use cases, some of which we hope to take a look at today.