API Callers: Base Classes for Making API Calls

The comb_utils.lib.api_callers module provides base classes for making API calls, along with a few helper functions. These classes are designed to be extended for specific APIs, allowing for easy customization and reuse of code.

The API callers wrap the requests library, which is a popular library for making HTTP requests in Python. The API callers handle the details of making the requests and parsing the responses, whether successful or failed, allowing you to focus on the logic of your application.

Base classes

The main classes in this module are:

There are plans to add additional classes for remaining request calls (i.e., PUT, PATCH, etc.) in the future. For now, you can use BaseCaller and override the BaseCaller._request_call method to implement these calls. See https://github.com/crickets-and-comb/comb_utils/issues/40.

Example usage

To use the API callers, you need to create a subclass of one of the base classes and implement the required methods. Here is an example of how to create a custom API caller for a hypothetical API:

from comb_utils import GetCaller

class MyAPICaller(GetCaller):

    # Optionally create a custom member to store the response value.
    # The, override the `_handle_200` method to set this value.
    target_response_value

    # Optionally override `_handle_200` to process the response.
    def _handle_200(self):
        super()._handle_200()
        self.target_response_value = self.response_json["target_key"]

my_caller = MyAPICaller(url="https://api.example.com/data")
my_caller.call_api()
target_response_value = my_caller.target_response_value

Here we call the example.com API. We optionally extract a specific value from the JSON response. The call_api method is inherited from the base class and handles the details of making the request and processing the response.

See Helper functions examples below for more details on how to handle paginated responses.

Using a key for authentication

Using a key for authentication is common in APIs. You can override the _get_API_key method to return the key. The base class will then construct the authentication for you. Here is an example of how to do this:

from comb_utils import GetCaller

class MyAPICaller(GetCaller):

    def _get_API_key(self):
        return my_custom_key_retrieval_function()

my_caller = MyAPICaller(url="https://api.example.com/data")
my_caller.call_api()

Helper functions

The comb_utils.lib.api_callers module provides a few helper functions for working with the base callers. These functions are designed to make it easier to work with the API callers and provide additional functionality.

Example usage

Here are examples of how to use the helper functions.

Getting generic responses: get_response_dict

from comb_utils import get_response_dict

response = requests.get("https://api.example.com/data")
response_dict = get_response_dict(response)

print(response_dict["status_code"])  # 200
print(response_dict["headers"])  # {'Content-Type': 'application/json', ...}
print(response_dict["json"])  # {'key': 'value', ...}

Handling paginated responses: get_responses and concat_response_pages

from comb_utils import (
    PagedResponseGetter,
    concat_response_pages,
    get_responses,
)

# Create a custom API caller for the paginated endpoint
class MyBaseKeyRetriever:

    def _get_API_key(self):
        return my_custom_key_retrieval_function()

class MyAPICaller(BaseKeyRetriever, PagedResponseGetter):

# Get all the responses from a paginated API endpoint.
all_responses = get_responses(url="https://api.example.com/data", paged_response_class=MyAPICaller)

# Put the responses records from all the pages into a single document.
concatenated_data = concat_response_pages(page_list=all_responses, data_key="data")

print(concatenated_data)  # [{'key': 'value', ...}, {'key': 'value', ...}, ...]

Additional Notes

You may want to override the request wait and timeout times or other parameters. See the comb_utils.lib.api_callers.BaseCaller class for details on how to do this.