salt.modules.boto_kinesis¶

Connection module for Amazon Kinesis

configuration:

This module accepts explicit Kinesis credentials but can also utilize IAM roles assigned to the instance trough Instance Profiles. Dynamic credentials are then automatically obtained from AWS API and no further configuration is necessary. More Information available at:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html

If IAM roles are not used you need to specify them either in a pillar or in the minion's config file:

kinesis.keyid: GKTADJGHEIQSXMKKRBJ08H
kinesis.key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs

A region may also be specified in the configuration:

kinesis.region: us-east-1

If a region is not specified, the default is us-east-1.

It's also possible to specify key, keyid and region via a profile, either as a passed in dict, or as a string to pull from pillars or minion config:

myprofile:
    keyid: GKTADJGHEIQSXMKKRBJ08H
    key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs
    region: us-east-1

depends:

boto3

salt.modules.boto_kinesis.create_stream(stream_name, num_shards, region=None, key=None, keyid=None, profile=None)¶

Create a stream with name stream_name and initial number of shards num_shards.

CLI Example:

salt myminion boto_kinesis.create_stream my_stream N region=us-east-1

salt.modules.boto_kinesis.decrease_stream_retention_period(stream_name, retention_hours, region=None, key=None, keyid=None, profile=None)¶

Decrease stream retention period to retention_hours

CLI Example:

salt myminion boto_kinesis.decrease_stream_retention_period my_stream N region=us-east-1

salt.modules.boto_kinesis.delete_stream(stream_name, region=None, key=None, keyid=None, profile=None)¶

Delete the stream with name stream_name. This cannot be undone! All data will be lost!!

CLI Example:

salt myminion boto_kinesis.delete_stream my_stream region=us-east-1

salt.modules.boto_kinesis.disable_enhanced_monitoring(stream_name, metrics, region=None, key=None, keyid=None, profile=None)¶

Disable enhanced monitoring for the specified shard-level metrics on stream stream_name

CLI Example:

salt myminion boto_kinesis.disable_enhanced_monitoring my_stream ["metrics", "to", "disable"] region=us-east-1

salt.modules.boto_kinesis.enable_enhanced_monitoring(stream_name, metrics, region=None, key=None, keyid=None, profile=None)¶

Enable enhanced monitoring for the specified shard-level metrics on stream stream_name

CLI Example:

salt myminion boto_kinesis.enable_enhanced_monitoring my_stream ["metrics", "to", "enable"] region=us-east-1

salt.modules.boto_kinesis.exists(stream_name, region=None, key=None, keyid=None, profile=None)¶

Check if the stream exists. Returns False and the error if it does not.

CLI Example:

salt myminion boto_kinesis.exists my_stream region=us-east-1

salt.modules.boto_kinesis.get_info_for_reshard(stream_details)¶

Collect some data: number of open shards, key range, etc. Modifies stream_details to add a sorted list of OpenShards. Returns (min_hash_key, max_hash_key, stream_details)

CLI Example:

salt myminion boto_kinesis.get_info_for_reshard existing_stream_details

salt.modules.boto_kinesis.get_stream_when_active(stream_name, region=None, key=None, keyid=None, profile=None)¶

Get complete stream info from AWS, returning only when the stream is in the ACTIVE state. Continues to retry when stream is updating or creating. If the stream is deleted during retries, the loop will catch the error and break.

CLI Example:

salt myminion boto_kinesis.get_stream_when_active my_stream region=us-east-1

salt.modules.boto_kinesis.increase_stream_retention_period(stream_name, retention_hours, region=None, key=None, keyid=None, profile=None)¶

Increase stream retention period to retention_hours

CLI Example:

salt myminion boto_kinesis.increase_stream_retention_period my_stream N region=us-east-1

salt.modules.boto_kinesis.list_streams(region=None, key=None, keyid=None, profile=None)¶

Return a list of all streams visible to the current account

CLI Example:

salt myminion boto_kinesis.list_streams

salt.modules.boto_kinesis.long_int(hash_key)¶

The hash key is a 128-bit int, sent as a string. It's necessary to convert to int/long for comparison operations. This helper method handles python 2/3 incompatibility

CLI Example:

salt myminion boto_kinesis.long_int some_MD5_hash_as_string

Returns:

long object if python 2.X, int object if python 3.X

salt.modules.boto_kinesis.reshard(stream_name, desired_size, force=False, region=None, key=None, keyid=None, profile=None)¶

Reshard a kinesis stream. Each call to this function will wait until the stream is ACTIVE, then make a single split or merge operation. This function decides where to split or merge with the assumption that the ultimate goal is a balanced partition space.

For safety, user must past in force=True; otherwise, the function will dry run.

CLI Example:

salt myminion boto_kinesis.reshard my_stream N True region=us-east-1

Returns:

True if a split or merge was found/performed, False if nothing is needed