On a AWS EC2 Windows 2012 server, my goal was to write some data to S3 bucket. I was using a small Powershell Script to copy the file to the S3 bucket. For that I needed to Install AWS Tools for Powershell and I used following command at Powershell prompt running as administrator:
Wednesday, February 24, 2021
Sunday, February 21, 2021
I was trying to ram data into AWS dynamodb via Boto3 and the streaming failed due to following error:
File "C:\Program Files\Python37\lib\site-packages\boto3\dynamodb\types.py", line 102, in serialize
dynamodb_type = self._get_dynamodb_type(value)
File "C:\Program Files\Python37\lib\site-packages\boto3\dynamodb\types.py", line 115, in _get_dynamodb_type
File "C:\Program Files\Python37\lib\site-packages\boto3\dynamodb\types.py", line 160, in _is_number
'Float types are not supported. Use Decimal types instead.')
TypeError: Float types are not supported. Use Decimal types instead.
Friday, February 19, 2021
To become a data engineer in cloud requires to have a good grasp of SQL among various other things. SQL is the premier tool for interacting with data sets. At first it seems daunting to see all those SQL analytics functions, but if you start with a tiny dataset like in the examples below and understand how these functions work, then it all becomes very easy for large datasets of any volume.
Thursday, January 28, 2021
If you running docker behind proxy on CentOS and receiving timeout or network errors, then use below steps to configure proxy settings on your CentOS box where docker is installed and you are trying to build docker image:
Tuesday, January 5, 2021
Who wants to mention in their resume that one of their operation task is to tag the cloud resources? Well I did and mentioned that one of the tools I used for that purpose was Tag Editor. Interviewer was surprised to learn that there was such a thing in AWS which allowed tagging multiple resource at once. I got the job due to this most under-appreciated and largely unknown service.
Monday, October 19, 2020
Monday, October 5, 2020
In simple words, a general Database Engine is a big clunky piece of software with features for all the use cases, and its up to you to choose which features to use. Whereas in a purpose built database, you get a lean, specific database which is only suitable for the feature you want.
Saturday, September 26, 2020
Even though AWS RDS (relational database service) is a managed service which means that you won't have to worry about upgrades, patches and other tidbits, you still have the option of manually triggering the upgrade at time of your choice.