S3 service is used to store a variety of data for various use cases. Ideally, when there is a requirement to access the data stored as an object, the object will be accessed as a whole entity. But what if you want to retrieve the subset of data from the…

Blob storage is an object store to store vast amounts of unstructured data, blob storage is an option. It uses a flat namespace to store the data. The blob storage can have multiple containers and each container contains the blob objects/data. Suppose, there is a requirement to mount to the…

Containers is one of the preferred ways to deploy applications. It contains everything that the application needs to run. As we know that data does not persist when the container is killed or if it does not exist anymore. For some reason, if there is a need to persist data…

AWS Glue is a fully managed serverless ETL service. It makes it easy to discover, transform and load data that would be consumed by various processes and applications. If you want to learn more about AWS Glue then please refer to the video on AWS Glue Overview

Today, we generally need to do remote development every now and then for different reasons. Let’s try to understand the need for remote development with an example and is not limited to this. For instance, if you are working on a big data engineering project or data science project where…

Have a static web application hosted on S3 for a specific group and wanted to restrict the traffic to certain IP addresses? Let’s see how this can be achieved. To restrict access to whitelisted IP addresses we will use a bucket policy. …

Multiple resources are deployed and we want a mechanism to get notified when any unusual things start to appear in logs, which as a developer we do not want to see. To get notified, we need to parse the logs to track certain Keywords, Errors, and Critical information which we…

Lambda function URLs feature is the recent addition to the AWS Lambda service. With a lambda function URL, one can invoke the lambda function via a unique URL similar to the invocation of any API endpoint with respective methods. In this article, we will configure/add the functionality to validate the…

How to enable Amazon HTTP API to serve requests originating from specific IP addresses? In this article, I will share how to whitelist IP addresses within HTTP API to entertain requests originating from whitelisted IP addresses and reject the rest of the requests. Unlike REST API, HTTP API does not…