1.. _guide_error-handling: 2 3Error handling 4============== 5 6Overview 7-------- 8Boto3 provides many features to assist in navigating the errors and exceptions that you might encounter when interacting with AWS services. 9 10Specifically, this guide provides details on the following: 11 12* How to find what exceptions could be thrown by both Boto3 and AWS services 13* How to catch and handle exceptions thrown by both Boto3 and AWS services 14* How to parse error responses from AWS services 15 16Why catch exceptions from AWS and Boto 17~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 18* *Service limits and quotas* - Your call rate to an AWS service might be too frequent, or you might have reached a specific AWS service quota. In either case, without proper error handling you wouldn’t know or wouldn’t handle them. 19* *Parameter validation and checking* - API requirements can change, especially across API versions. Catching these errors helps to identify if there’s an issue with the parameters you provide to any given API call. 20* *Proper logging and messaging* - Catching errors and exceptions means you can log them. This can be instrumental in troubleshooting any code you write when interacting with AWS services. 21 22Determining what exceptions to catch 23------------------------------------ 24Exceptions that you might encounter when using Boto3 will come from one of two sources: botocore or the AWS services your client is interacting with. 25 26Botocore exceptions 27~~~~~~~~~~~~~~~~~~~ 28These exceptions are statically defined within the botocore package, a dependency of Boto3. The exceptions are related to issues with client-side behaviors, configurations, or validations. You can generate a list of the statically defined botocore exceptions using the following code: 29 30.. code-block:: python 31 32 import botocore.exceptions 33 34 for key, value in sorted(botocore.exceptions.__dict__.items()): 35 if isinstance(value, type): 36 print(key) 37 38.. tip:: 39 40 .. raw:: html 41 42 <details> 43 <summary>Click to see a full list of static exceptions</summary> 44 45 .. code-block:: text 46 47 AliasConflictParameterError 48 ApiVersionNotFoundError 49 BaseEndpointResolverError 50 BotoCoreError 51 ChecksumError 52 ClientError 53 ConfigNotFound 54 ConfigParseError 55 ConnectTimeoutError 56 ConnectionClosedError 57 ConnectionError 58 CredentialRetrievalError 59 DataNotFoundError 60 EndpointConnectionError 61 EventStreamError 62 HTTPClientError 63 ImminentRemovalWarning 64 IncompleteReadError 65 InfiniteLoopConfigError 66 InvalidConfigError 67 InvalidDNSNameError 68 InvalidExpressionError 69 InvalidMaxRetryAttemptsError 70 InvalidRetryConfigurationError 71 InvalidS3AddressingStyleError 72 InvalidS3UsEast1RegionalEndpointConfigError 73 InvalidSTSRegionalEndpointsConfigError 74 MD5UnavailableError 75 MetadataRetrievalError 76 MissingParametersError 77 MissingServiceIdError 78 NoCredentialsError 79 NoRegionError 80 OperationNotPageableError 81 PaginationError 82 ParamValidationError 83 PartialCredentialsError 84 ProfileNotFound 85 ProxyConnectionError 86 RangeError 87 ReadTimeoutError 88 RefreshWithMFAUnsupportedError 89 SSLError 90 ServiceNotInRegionError 91 StubAssertionError 92 StubResponseError 93 UnStubbedResponseError 94 UndefinedModelAttributeError 95 UnknownClientMethodError 96 UnknownCredentialError 97 UnknownEndpointError 98 UnknownKeyError 99 UnknownParameterError 100 UnknownServiceError 101 UnknownServiceStyle 102 UnknownSignatureVersionError 103 UnseekableStreamError 104 UnsupportedS3AccesspointConfigurationError 105 UnsupportedS3ArnError 106 UnsupportedSignatureVersionError 107 UnsupportedTLSVersionWarning 108 ValidationError 109 WaiterConfigError 110 WaiterError 111 112 .. raw:: html 113 114 </details> 115 116.. note:: 117 118 You can view available descriptions of the botocore static exceptions `here <https://github.com/boto/botocore/blob/develop/botocore/exceptions.py>`_. 119 120AWS service exceptions 121~~~~~~~~~~~~~~~~~~~~~~ 122AWS service exceptions are caught with the underlying botocore exception, ``ClientError``. After you catch this exception, you can parse through the response for specifics around that error, including the service-specific exception. Exceptions and errors from AWS services vary widely. You can quickly get a list of an AWS service’s exceptions using Boto3. 123 124For a complete list of error responses from the services you’re using, consult the individual service’s `AWS documentation <https://docs.aws.amazon.com/>`_, specifically the error response section of the AWS service’s API reference. These references also provide context around the exceptions and errors. 125 126Catching exceptions when using a low-level client 127------------------------------------------------- 128 129Catching botocore exceptions 130~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 131Botocore exceptions are statically defined in the botocore package. Any Boto3 clients you create will use these same statically defined exception classes. The most common botocore exception you’ll encounter is ``ClientError``. This is a general exception when an error response is provided by an AWS service to your Boto3 client’s request. 132 133Additional client-side issues with SSL negotiation, client misconfiguration, or AWS service validation errors will also throw botocore exceptions. Here’s a generic example of how you might catch botocore exceptions. 134 135.. code-block:: python 136 137 import botocore 138 import boto3 139 140 client = boto3.client('aws_service_name') 141 142 try: 143 client.some_api_call(SomeParam='some_param') 144 145 except botocore.exceptions.ClientError as error: 146 # Put your error handling logic here 147 raise error 148 149 except botocore.exceptions.ParamValidationError as error: 150 raise ValueError('The parameters you provided are incorrect: {}'.format(error)) 151 152Parsing error responses and catching exceptions from AWS services 153~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 154Unlike botocore exceptions, AWS service exceptions aren't statically defined in Boto3. This is due to errors and exceptions from AWS services varying widely and being subject to change. To properly catch an exception from an AWS service, you must parse the error response from the service. The error response provided to your client from the AWS service follows a common structure and is minimally processed and not obfuscated by Boto3. 155 156Using Boto3, the error response from an AWS service will look similar to a success response, except that an ``Error`` nested dictionary will appear with the ``ResponseMetadata`` nested dictionary. Here is an example of what an error response might look like:: 157 158 { 159 'Error': { 160 'Code': 'SomeServiceException', 161 'Message': 'Details/context around the exception or error' 162 }, 163 'ResponseMetadata': { 164 'RequestId': '1234567890ABCDEF', 165 'HostId': 'host ID data will appear here as a hash', 166 'HTTPStatusCode': 400, 167 'HTTPHeaders': {'header metadata key/values will appear here'}, 168 'RetryAttempts': 0 169 } 170 } 171 172Boto3 classifies all AWS service errors and exceptions as ``ClientError`` exceptions. When attempting to catch AWS service exceptions, one way is to catch ``ClientError`` and then parse the error response for the AWS service-specific exception. 173 174Using Amazon Kinesis as an example service, you can use Boto3 to catch the exception ``LimitExceededException`` and insert your own logging message when your code experiences request throttling from the AWS service. 175 176.. code-block:: python 177 178 import botocore 179 import boto3 180 import logging 181 182 # Set up our logger 183 logging.basicConfig(level=logging.INFO) 184 logger = logging.getLogger() 185 186 client = boto3.client('kinesis') 187 188 try: 189 logger.info('Calling DescribeStream API on myDataStream') 190 client.describe_stream(StreamName='myDataStream') 191 192 except botocore.exceptions.ClientError as error: 193 if error.response['Error']['Code'] == 'LimitExceededException': 194 logger.warn('API call limit exceeded; backing off and retrying...') 195 else: 196 raise error 197 198.. note:: 199 200 The Boto3 ``standard`` retry mode will catch throttling errors and exceptions, and will back off and retry them for you. 201 202Additionally, you can also access some of the dynamic service-side exceptions from the client’s exception property. Using the previous example, you would need to modify only the ``except`` clause. 203 204.. code-block:: python 205 206 except client.exceptions.LimitExceedException as error: 207 logger.warn('API call limit exceeded; backing off and retrying...') 208 209.. note:: 210 211 Catching exceptions through ``ClientError`` and parsing for error codes is still the best way to catch **all** service-side exceptions and errors. 212 213Catching exceptions when using a resource client 214------------------------------------------------ 215 216When using ``Resource`` classes to interact with certain AWS services, catching exceptions and errors is a similar experience to using a low-level client. 217 218Parsing for error responses uses the same exact methodology outlined in the low-level client section. Catching exceptions through the client’s ``exceptions`` property is slightly different, as you’ll need to access the client’s ``meta`` property to get to the exceptions. 219 220.. code-block:: python 221 222 client.meta.client.exceptions.SomeServiceException 223 224Using Amazon S3 as an example resource service, you can use the client’s exception property to catch the ``BucketAlreadyExists`` exception. And you can still parse the error response to get the bucket name that's passed in the original request. 225 226.. code-block:: python 227 228 import botocore 229 import boto3 230 231 client = boto3.resource('s3') 232 233 try: 234 client.create_bucket(BucketName='myTestBucket') 235 236 except client.meta.client.exceptions.BucketAlreadyExists as err: 237 print("Bucket {} already exists!".format(err.response['Error']['BucketName'])) 238 raise err 239 240Discerning useful information from error responses 241-------------------------------------------------- 242As stated previously in this guide, for details and context around specific AWS service exceptions, see the individual service’s `AWS documentation <https://docs.aws.amazon.com/>`_, specifically the error response section of the AWS service’s API reference. 243 244Botocore exceptions will have detailed error messaging when those exceptions are thrown. These error messages provide details and context around the specific exception thrown. Descriptions of these exceptions can be viewed `here <https://github.com/boto/botocore/blob/develop/botocore/exceptions.py>`_. 245 246Outside of specific error or exception details and messaging, you might want to extract additional metadata from error responses: 247 248* *Exception class and error message* - You can use this data to build logic around, or in response to, these errors and exceptions. 249* *Request ID and HTTP status code* - AWS service exceptions might still be vague or lacking in details. If this occurs, contacting customer support and providing the AWS service name, error, error message, and request ID could allow a support engineer to further look into your issue. 250 251Using a low-level Amazon SQS client, here’s an example of catching a generic or vague exception from the AWS service, and parsing out useful metadata from the error response. 252 253.. code-block:: python 254 255 import botocore 256 import boto3 257 258 client = boto3.client('sqs') 259 queue_url = 'SQS_QUEUE_URL' 260 261 try: 262 client.send_message(QueueUrl=queue_url, MessageBody=('some_message')) 263 264 except botocore.exceptions.ClientError as err: 265 if err.response['Error']['Code'] == 'InternalError': # Generic error 266 # We grab the message, request ID, and HTTP code to give to customer support 267 print('Error Message: {}'.format(err.response['Error']['Message'])) 268 print('Request ID: {}'.format(err.response['ResponseMetadata']['RequestId'])) 269 print('Http code: {}'.format(err.response['ResponseMetadata']['HTTPStatusCode'])) 270 else: 271 raise err 272