Cookbook

This cookbook will be a growing collection of tips and tricks for how to solve development challenges with the Predix Python SDK.

Installation Recipes

Recipes for installation issues with Predix Python SDK.

How-To Install the Python SDK

If you ran into issues installing the SDK with pip install predix there may be a few things that could have gone wrong.

  1. Python Version
  2. Python Installation
  3. Proxies

Python Version

There are a few minor issues before version 0.1.0 that prevent the SDK from working in a Python 3.x environment. You’ll either need to work with Python 2.7 or watch the repository until 0.1.0 is released.

Python Installation

Certain Python versions don’t have pip installed or you may be trying to use a version of python that is on your system by default.

You should have Python 2.7.10+ and pip 9.0.1+ and write permissions for the Python packages. It is also highly recommended to use Virtual Environments.

If you need help with these, check out the Hitchiker’s Guide to Python or Python docs for details.

Proxies

If you are in an industrial setting you may need to set proxies in your operating system environment. The Python interpreter will pull these variables in at runtime so there shouldn’t be too much you need to do beyond setting http_proxy and https_proxy in the session launching your python code.

There is a good Predix Proxies guide you may want to review for more details.

How-To Learn the SDK

The documentation on this site is here to help you, but sometimes you need to learn by doing. Run pip install ipython to use an interactive python shell where you can tab complete and use ? to lookup docstrings for methods.

In [6]: asset?
Type:        Asset
String form: <predix.data.asset.Asset object at 0x10adcdc10>
File:        ~/w/predixpy/j12y/predixpy/predix/data/asset.py
Docstring:
Client library for working with the Predix Asset Service.  For more details
on use of the service please see official docs:
https://www.predix.io/services/service.html?id=1171

How-To Run a Dockerized Dev Environment

If you already use Docker, you may find it easy to start up a container that has the cf cli pre-installed and ready to use.

See the following project for more details on this Docker-based Application Development Environment:

https://github.com/j12y/cf-ade

Cloud Foundry Recipes

Recipes for working effectively with a Cloud Foundry environment.

How-To Deploy an App

The Python SDK does not provide this feature. You should just use cf push.

How-To Authenticate to Cloud Foundry

The Python SDK does not provide this feature. You should just use cf login.

How-To Specify a Python Buildpack

You can run cf buildpacks to see the version of the buildpack installed in Predix, but sometimes you want to use a newer version of the Python buildpack for a new feature or bug fix. You can do so by specifying -b https://github.com/cloudfoundry/python-buildpack.git#v1.5.20 on the command line or put it in your manifest.yml.

See a post on the Predix Developer Forums for more details:

https://forum.predix.io/articles/24714/how-to-specify-a-python-buildpack.html

How-To Include Private Python Libraries in CF

If you have Python libraries in a private source code repository or cannot publish to public indexes like PyPI you may not know how to include them in your Cloud Foundry App.

See a how-to post on the Predix Developer Forums for more details: https://forum.predix.io/articles/24733/how-to-include-your-own-private-python-libraries-i.html

How-To Specify a Different Version of Python

If you add a runtime.txt file in the same directory as your manifest.yml you can specify a single line in the file to signify the version.

For example:

python-3.6.1

or

python-2.7.13

Only specific versions supported by each python_buildpack release so check the release notes for details.

https://github.com/cloudfoundry/python-buildpack/releases/tag/v1.5.20

See a post on the Predix Developer Forums for more details: https://forum.predix.io/articles/24718/how-to-specify-a-python-version-for-your-app.html

How-To Delete a Service

If you used PredixPy to create your service you may be surprised that running cf delete-service returns an error: Cannot delete service instance, service keys and bindings must first be deleted.

You can use PredixPy to delete services

import predix.admin.app

admin = predix.admin.app.Manifest()
service_name = 'my-service'

if service_name in admin.space.get_instances():
    admin.services.purge(service_name)

print(admin.space.get_instances())

This can be done with the cf cli as well of course. You just run cf service-keys to see the named key and then cf delete-service-key.

How-To Create and Delete Spaces

To test your Python apps, it may be convenient to be able to create and delete the spaces you are working in. You can automate this with bash scripts and the cf cli, or there are some utility methods.

import predix.admin.cf.spaces

# Create a randomly named space and delete it

space = predix.admin.cf.spaces.create_temp_space()
# Do stuff
space.delete_space()

# If you want to specify your own space name

space = predix.admin.cf.spaces.Space()
space.create_space('my-space-name')
# Do stuff
space.delete_space('my-space-name')

How-To Test Apps Without Pushing to Predix Cloud

You can test your application with Python SimpleHTTPServer which allows you to visit http://localhost:4000/ to test your app before you do a cf push.

python -m SimpleHTTPServer

You wouldn’t use this in production, but it is suitable for quick development testing. If used in combination with predix.app.Manifest() many of the service bindings will behave similar to how they will when deployed to Cloud Foundry. There are a few exceptions such as PostgreSQL and Blobstore that are only available when running in a Predix Cloud environment.

How-To Choose Org and Space

If you run cf target you can see the org and space that will be used for any SDK admin operations. You can use the cf target -s <other-space> to use a different space.

If you want to automatically target the same org and space when you are using a manifest you can bake this into the manifest.yml itself. The following demonstrates how to do this:

import predix.admin.app
app = predix.admin.app.Manifest()
app.lock_to_org_space()

From then on, any admin operations will focus on the same org and space from this manifest despite what cf target currently indicates.

How-To Do Common Cloud Foundry CLI Commands

In the following table the admin object is created with the following snippet of code:

import predix.admin.app
admin = predix.admin.app.Manifest()

You can then programmatically with Python do a few of the common things you might see others do on the command line. These are mostly for convenience as you might want to script tools that can handle data from these commands.

CF Command PredixPy
cf spaces admin.space.get_spaces()
cf services admin.space.get_instances()
cf apps admin.space.get_apps()
cf marketplace admin.space.get_services()
cf create-space admin.space.create_space()
cf org-users admin.space.org.get_users()
cf delete-service -f admin.services.purge(service_name)

How-To Setup GoTTY

GoTTY is an open-source, pre-built application you can run in the Predix Cloud that provides a bash shell accessible via a web browser. By using predix-gotty-util you can quickly push an app that has psql, redis-cli and some other utilities available.

Here’s what you need to do:

git clone https://github.com/j12y/predix-gotty-util
cd predix-gotty-util
cp manifest.template.yml manifest.yml

Then modify the username and password in the manifest.yml to be something only you know. At that point, you can simply cf push your app and then go to the resulting URL. You will need need to cf enable-diego gotty if you are still running DEA.

predix-gotty-util: https://github.com/j12y/predix-gotty-util

UAA Recipes

Recipes for working with Predix Python SDK and the UAA service.

It is expected you already used the SDK to create the service so that it is configured by Manifest.

How-To Get Scopes for a Given Client

You want to verify the scopes and authorities for a client.

import predix.app
app = predix.app.Manifest()

uaa = app.get_uaa()
uaa.authenticate(client_id, client_secret)
print(uaa.get_scopes())

How-To Get the Token for a Client/User

Typically this is unnecessary when using the SDK as the headers including the token are generated for you. This is how it works though:

import predix.app
app = predix.app.Manifest()

uaa = app.get_uaa()
uaa.authenticate(user_id, user_secret)
print(uaa.get_token())

How-To Add Scopes or Authorities for a Client/User

Typically this is unnecessary when using the SDK as scopes/authorities for each service are granted when they are created, but there may be cases you need to work around default behaviors. This is how you can do it:

import predix.admin.uaa

uaa = predix.admin.uaa.UserAccountAuthentication()
uaa.authenticate()
scopes = ['uaa.resource']
authorities = ['uaa.resource']
uaa.update_client_grants(client_id, scope=scopes, authorities=authorities)

If you want to completely replace scopes and authorities you can add an optional replace=True.

How-To Create a Superuser Client

You can typically get by with just the admin user account, but if you wanted to create a separate superuser that can manage clients.

import predix.admin.app
app = predix.admin.app.Manifest()
uaa = app.get_uaa()
uaa.authenticate('admin', 'your-admin-secret')

uaa.create_client('superuser', 'superuser-secret')
uaa.grant_client_permissions('superuser', admin=True)
print(uaa.get_clients())

How-To Create Users

In order to create users you must have a client with permission to do so.

import predix.admin.app
app = predix.admin.app.Manifest()
app.create_client('usermanager', 'userman-secret')
app.grant_scim_permissions('usermanager', read=True, write=True)

Once you have that client with scim permissions you can begin modifying users.

import predix.app
app = predix.app.Manifest()
uaa = app.get_uaa()
uaa.authenticate(app.get_client_id(), app.get_client_secret())
uaa.create_user('j12y', 'password123', 'DeLancey', 'Jayson',
    'volcano@ge.com')

How-To Debug a 403 Forbidden Client Error

If you get a Forbidden for url error with UAA that looks something like the following you will need to fix permissions.

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url:
https://****.predix-uaa.run.aws-usw02-pr.ice.predix.io/userinfo

This could happen if the client making the request does not have sufficient scopes and authorities. For example, a request for that endpoint requires the openid scope. You can do something like the following if UAA is setup with an admin and client already to extend that clients permissions:

import predix.app
app = predix.app.Manifest()
uaa = app.get_uaa()
uaa.authenticate('admin', 'your-admin-secret')
uaa.grant_scim_permissions(app.get_client(), openid=True)
uaa.authenticate(app.get_client_id(), app.get_client_secret(),
    use_cache=False)

Note that the use_cache=False is there because if you already have authenticated as a client the token remains cached until it expires. You’ll need a new token anytime you modify scopes and authorities.

ACS Recipes

Recipes for working with Predix Python SDK and the ACS service.

It is expected you used the SDK to create the service so that it is configured by Manifest.

How-To Add a Policy and Test Access

If you followed the basic example from the Access Control you may have subjects and resources stored. Now you want to create a policy to test if a subject can perform an action on a resource.

resource = {'uriTemplate': '/asset'}
subject = {'name': 'devangelist', 'attributes': [{"issuer": "default", "name": "role"}]}
condition = "match.single(subject.attributes('default', 'role'), 'devangelist')"

acs.add_policy('any_devangelist', 'GET', resource, subject, condition)

print(acs.is_allowed('/user/j12y', 'GET', '/asset/12'))
print(acs.is_allowed('/user/jflannery', 'GET', '/asset/12'))

This example compares the given subjects to see if their role allows access or not.

Asset Recipes

Recipes for working with Predix Python SDK and the Asset service.

It is expected you used the SDK to create the service so that it is configured by Manifest.

How-To Get a Unique URI

The Predix Asset Service requires a uri as part of the body whenever you post a collection. The predix.data.asset.AssetCollection class will help generate this for you but sometimes you want to do that for yourself.

import predix.app
app = predix.app.Manifest()
asset = app.get_asset()

guid = asset.get_guid() # e36af84e-95a9-485b-993d-e2004427d73e
uri = asset.get_guid('/volcano') # /volcano/44c99f3b-aa1b-486c-9a9d-14740b600313

TimeSeries Recipes

Recipes for working with Predix Python SDK and the Time Series service.

It is expected you used the SDK to create the service so that it is configured by Manifest.

How-To Get List of Available Aggregations

import predix.app
app = predix.app.Manifest()

ts = app.get_timeseries()
for aggregation in ts.get_aggregations():
    print(aggregation['name'])

How-To Simulate Time Series Data

See the following example for a demonstration of generating random numbers within a threshold for voltage readings across multiple sensors that can be ingested in the Time Series service.

https://github.com/j12y/predixpy-timeseries-simulator

BlobStore Recipes

Recipes for working with Predix Python SDK and the Blob Store service.

It is expected you used the SDK to create the service so that it is configured by Manifest.

How-To Interface with Blob Store

Blob Store can only be used from within the Cloud Foundry environment. In order to upload images from the Edge you may need a sample app like the following demonstration.

https://github.com/j12y/predix-flask-blobstore

Predix Cache Recipes

Recipes for working with Predix Python SDK and the Predix Cache service.

It is expected you used the SDK to create the service so that it is configured by Manifest.

How-To Use the Redis CLI

Predix Cache can only be used from within the Predix Cloud Foundry environment so you won’t be able to use Redis CLI running on your local machine. As an alternative, you can install an app into your space called GoTTY and then install the Redis CLI.

First, see How-To Setup GoTTY for instructions on setting up GoTTY. Once deployed to Predix Cloud, log into that environment and run the following commands.

./redis-cli

This is an alias to a script that will lookup the host, port, and password from your VCAP environment variables to launch redis-cli.

See the official redis-cli docs for more details on how to use it once you are connected.

How-To Build a Web App with Cache

See the following demo application for an example of a Flask app that uses Predix Cache with PredixPy.

https://github.com/j12y/predixpy-flask-cache

Predix DBaaS Recipes

Recipes for working with Predix Python SDK and the Predix PostgreSQL service.

It is expected you used the SDK to create the service so that it is configured by Manifest.

How-To Use the PostgreSQL CLI

PostgreSQL can only be used from within the Predix Cloud Foundry environment so you won’t be able to use pgsql running on your local machine. As an alternative, you can install an app into your space called GoTTY and then install the Redix CLI.

First, see How-To Setup GoTTY for instructions on setting up GoTTY. Once deployed to Predix Cloud, log into that environment and run the following commands.

psql

This is an alias to a script that will lookup the host, port, username, and password from your VCAP environment variables to launch psql.

See the official psql docs for more details on how to use it once you are connected.

Event Hub Recipes

Recipes for working with Predix Python SDK and the Event Hub Service.

It is expected that you first properly created and configured UAA and a client to work with event hub.

More to come.

Error Recipes

Recipes for debugging and resolving common errors and issues you may run into using the Predix Python SDK.

How-To Turn Logging On

You can directly use the Python logging library to set different logging levels such as DEBUG or INFO. This is a useful way to find details about the underlying service calls.

import logging
logging.basicConfig(level=logging.DEBUG)

Enabling DEBUG will output the details of the requests such as URI, Headers, Body, and Response.

DEBUG:root:URI=https://predix-asset.run.aws-usw02-pr.ice.predix.io/v1/volcano
DEBUG:root:HEADERS={'Predix-Zone-Id': '**************', 'Content-Type':
    'application/json', 'Authorization': u'Bearer ****', 'Accept': 'application/json'}
DEBUG:urllib3.connectionpool:https://predix-asset.run.aws-usw02-pr.ice.predix.io:443
"GET /v1/volcano HTTP/1.1" 200 170
DEBUG:root:STATUS=200

How-To Fix HTTPSConnectionPool Error

If you see a HTTPSConnectionPool exception like the following:

HTTPSConnectionPool(host='predix-asset.run.aws-usw02-pr.ice.predix.io',
port=443): Max retries exceeded with url: /v1/volcano (Caused by
ProxyError('Cannot connect to proxy.',
    NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object
        at 0x10c85c410>: Failed to establish a new connection: [Errno 8]
        nodename nor servname provided, or not known',)))

This is likely to be a proxy error. You may need to set or unset your HTTPS_PROXY variables to resolve.

Reference Recipes

A collection of resources and reference applications where you can find more about using Python on Predix or use the Predix SDK for multiple services.

How-To Install the Predix Volcano App

The Predix Volcano App demonstrates use of UAA, Predix Asset and Predix Time Series in a Python Flask application. The following repository has instructions for how to setup the services, ingest data, deploy, and monitor this sort of public application using the Predix Python SDK.

https://github.com/PredixDev/predix-volcano-app/

How-To Run a Flask App with Python 3.x

The trick is setting up a runtime.txt that the Cloud Foundry Python Buildpack will identify to specify a different Python runtime envrionment.

Complete reference project: https://github.com/j12y/predix-py3-flask

How-To Run TensorFlow in Cloud Foundry

Architecturally you may not want to do this, but you can find a demonstration of running a simple TensorFlow operation in Cloud Foundry with this repository.

Complete reference project: https://github.com/j12y/predix-tensorflow-example