Running Flask on Ubuntu VM

So you have a VM in Azure and want to put it to good use?

No.
Let’s set one up!

Yes.
Great!

Before we start make sure you can ssh into your machine and run

$sudo apt-get update

sudoaptgetupdate

It’s a four step process:
1. Open the appropriate port on Azure
2. Install pip, virtualenv, virtualenvwrapper, and flask
3. Write our code and run it
4. Keep it running with Gunicorn

1. Open the appropriate ports on Azure
Go to you virtual machines landing:
vmnumberonelandingpage

Then select the resource group in the top left corner:
resourcegroup

Resource groups are the way Azure breaks down how our VM interacts with the internet, other vms, storage, and public/private networks.

Top open the port we need to change our network security group, which is represented by the shield. (Underlined in the screenshot above)

Then select settings -> Inbound Security Rules:
networksecuritygroupsettings

This will allos us to open up our VM to the public internet so we can visit what’s presented at the port like a regular website.

You should see SSH already included, that’s the port we’re using in our ssh client/terminal.
defaultssh

We’re now going to add two new Inbound Security Rules one called FlaskPort where we’ll set the destination port range to 5000 and use for debugging. The second will be called FlaskProduction that we’ll use to deploy our complete app.
Here’s the configuration panel for FlaskPort:
FlaskPort
Press okay to accept the settings.

And the other panel for FlaskProduction:
flaskproduction
Again press okay to accept the settings.

Notice how the ‘Source Port Range’ is ‘*’ that just means that we’ll accept connections from the port of any machine. This tripped me up the first time.

In a couple seconds the port will be open we’ll be ready to visit it, but nothing will be there because we haven’t create an application server.

To do that we’ll install the basics.

2. Install pip, virtualenv, virtualenvwrapper, and flask

To use Python effectively we utilize virtual environments to help keep our various python project and required libraries in order.

If you get lost in these steps or want more context Gerhard Burger provides the same setup on a very helpful post on askubuntu.

First we install pip:
$ sudo apt-get install python-pip

Second we install virtualenv and virtualenvwrapper


$ sudo pip install virtualenv
$ sudo pip install virtualenvwrapper

Third we configure virtualenv and virtualenvwrapper

Create a WORKON_HOME string which will contain the directory for our virtual environments. We’ll name it vitualenvs

$ export $WORKON_HOME=~/.virtualenvs

Now we’ll create this directory.

$ mkdir $WORKON_HOME

And add this to our bashrc file so this variable is defined automatically every time we hit the terminal.

$ echo "export WORKON_HOME=$WORKON_HOME" >> ~/.bashrc

Then we’ll setup virtualenvwrapper by importing its functions with bashrc.

$ echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc

You can see the additions to our bashrc file by opening it with nano. Scrolling down to the bottom you should see two lines like this:
bashrcconfigured

Then implement your changes.

$ source ~/.bashrc

Here’s what all that looks like all together:
configurepipandvirtualenvwrapper

Fourth, let’s create our first virtualenvironment

$ mkvirtualenv venv

And take a look at the currently installed packages
$ pip list

Like so:
firstvenv

Now we can install all of the python packages we want without risk of needing to reinstall python!

Fifth, install flask:


$ pip install flask
$ pip list

pipinstallflask
currentpackages
3. Write our code and run it!
Our first app is a simple site that shares an image.

We’re going to create a folder called Photo-App that contains two folders and an app.py that will serve our clients.

To change to our home directory:
$ cd ~
And create our new folder:
$ mkdir Photo-App

Then create a static and templates folder inside of our app.

$ sudo mkdir templtes
$ sudo mkdir static

mkPhotoApp
Then create our app.py which will be our python flask server code.

$ sudo nano app.py

Here’s the code:

from flask import Flask, render_template
app = Flask(__name__)

@app.route("/")
def index():
        return render_template("index.html")

if __name__ == "__main__":
        app.run(host='0.0.0.0', debug=True)

And what it looks like in nano:
nanoapppy

Then we need to add our first template:
$ sudo nano templates/index.html

<h1>Wazzup Dog</h1>
<img style="max-width:100%;" src="{{ url_for('static', filename='photo.jpg') }}">

And what it looks like in nano:
indexinnano

Now for our photo we’re going to download an image into our static file using curl.

$ cd static
$ curl 'http://timmyreilly.azurewebsites.net/wp-content/uploads/2015/12/Snapchat-1802119159214415224.jpg' -o 'photo.jpg'

Cool! We have an app!

To run it simply enter:
$ python app.py

Visit from a browser!

Type in the IP address of your virtualmachine with a colon at the end followed by the port.
Mine looks like:
http://138.91.154.193:5000/

And this is what I see!
wazzupdogimdex

Neato! But when we close the terminal we lose the application… Hmmmm let’s fix that!

4. Keep it running with Gunicorn
Real Python provided the bulk of this portion so if you get lost or checkout their site, plus they have info about setting up continuous deployment!

First, we install dependencies
$ sudo apt-get install -y nginx gunicorn

Now that our app is ready, let’s configure nginx.

If you’d like to learn more about this Open Source HTTP Server Software checkout their website.

Second, we start nginx.
$ sudo /etc/init.d/nginx start

And begin configuration for our project
We’re naming our nginx configuration the same as the parent directly of our app.py file in this test case.
Here are the commands:

$ sudo rm /etc/nginx/sites-enabled/default
$ sudo touch /etc/nginx/sites-available/Photo-App
$ sudo ln -s /etc/nginx/sites-available/Photo-App /etc/nginx/sites-enabled/Photo-App

Now we add the config settings for our app.

$ sudo nano /etc/nginx/sites-enabled/Photo-App
nginxconfigphotoapp
Here’s the raw text:

server {
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /static {
alias /Photo-App/static/;
}
}

Then restart the server.

$ sudo /etc/init.d/nginx restart

And navigate to your Python PhotoApp project, and start it using Gunicorn:
$sudo gunicorn app:app -b 0.0.0.0:8000 --reload

You’re using gunicorn to start app hosted at 0.0.0.0:8000 with the reload tag configured.
The reload will look for changes in your code and reload the server everytime you change any server side stuff. It won’t auto-reload for HTML changes, but will reload them once you make a change to the python code.

Now try navigating to: http://138.91.154.193:8000/ or whatever your IP address is.

You can now close you’re ssh terminal and it will continue to run.

To stop it we have to kill the actual thread it’s running on.
$ pkill gunicorn

Like I said at the beginning of this step this was adapted from the realpython.org and they have some awesome next steps available for git deployment + automating the whole process.

The code for the app can be found on GitHub here: https://github.com/timmyreilly/Flask-For-UbuntuBlog

Happy Hacking!

This was a nice reminder in the busy airport.
This was a nice reminder in the busy airport.

Intro to Ubuntu Virtual Machines on Azure

When I search:
Node JS Server Azure, Ubuntu, JavaScript, Mongo, Postgres, Flask, VM
I turn up with all sorts of unhelpful results.
So I dedicated a couple days to creating a couple guides for common Cloud Stacks on Azure VMs to make it as simple as possible to start deploying your code to the cloud.

This is the introduction and at the bottom of this blog post you’ll see other workflows fill in.

So, Here’s a guide to deploying an Ubuntu VM on Azure:
1. Gather Materials
2. Create VM
3. Check VM using SSH

1. Gather Materials
Here’s what you’ll need:
An Azure Account
An SSH Client perhaps putty… or even Bash On Windows?

2. Create VM

Head into the Azure Portal: portal.azure.com

And Select Virtual Machines -> Then ‘Add’
selectVirtualmachines

You’ll then see a page like this:
selectubuntu

Select Ubuntu Server 14.04.

There are lots of configurable deployments available if you feel like exploring.

Then select Create, but make sure the deployment model is Resource Manager as its more future ready then the classic model:
createvm

We’ll then get to the basic configuration tab, fill out the info and pick a User name and Password that you’ll remember because you’ll need it later!

configurationbasics

If you’re not familiar with Resource Groups check out THIS ARTICLE

I’ve named my resource group: ResourceGroupOne

Hit Okay to go to the next configuration pane

Select the Size of your VM. To see all the options select ‘View All’
selectvmsize

We’re going to go with the cheapest option A1 Standard:
SelectAOne

Hit Okay to take us to our final configuration Pane, “Settings”.

settingstwo

There are a number of different settings presented here.

First up is Storage:
This will configure what we want to name the storage account for our vm. I’ve changed mine to ‘resourcegrouponestorage’, but I could have selected any of my previous storage account in the same region, in this case westus.

Second is Network:
We can configure a Virtual Network to allow our virtual machines to connect to other resource on our network by default. We can also change this later. So in this case I’m creating the default virtual network.

Again, I could have selected a previously created Virtual Network Called ‘Databases’ which is in the same region.
virtualnetworkdefault

Third is Extensions:
We won’t add any extensions

Fourth is Monitoring:
Which we’ll disable for simplicity sake, but is a very powerful tool one you start needing to make scaling decisions.

Fifth and finally is Availability:
We won’t use an availability set, until we need to scale out our app.

Here’s what the lower portion of our settings pane looks like:
settingstwoend

And we’ll select OK to finish with our settings. This will take us to the summary page so we can do a one more check on our machine, don’t get to anxious about making mistakes because we can always tear this one down and spin up another if we messed something up!
vmsummary

Hit Okay one last time!

You’ll then be taken to your dashboard where you’ll see a nice loading tile:
loadingdashboard

It’ll take ~5 minutes to spin up and then we’ll be ready to take on the world!

Once ready it’ll look like this:
clickthetile

Click the tile to hit the landing page for our VM:
vmnumberonelandingpage

See that public IP address?
We’ll use that to SSH into our machine.

In my case: 13.88.180.170 !

3. Check VM using SSH

Let’s SSH into our box.

Pull out your preferred SSH client. Here’s bash on Windows and Putty Side by Side:
sshintomachine

Notice ‘Timothy’ Triple underlined?
That’s the User Name we set during basic configuration and is paired with the password that we also set in Azure.

When you connect you might have to accept the ras2key fingerprint. It’ll look like this when using putty. Or it’ll be in the terminal using bash. Type ‘yes’ or Select Yes to continue.
sayyestowarning

Then type in your password and marvel and your creation:
typeinyourpassword

Let’s test our vm by installing updates! Yay Updates!

$ sudo apt-get update
sudoaptgetupdate

Now that you have a VM ready let’s put it to work!

Host a Node Server
Host a Python Flask Server

Pradeep Cruising on National Donut Day
Pradeep Cruising on National Donut Day

Python with Ubuntu on Windows

Now that bash is on Windows, I wanted to try and make all the other guides I’d writen for Python on Windows irrelevant.

So here’s how to setup an effective environment for Python on Ubuntu on Windows.

1. Install Bash on Windows
2. Check for updates
3. Check out the REPL
4. Install Pip
5. Install VirtualEnv
6. Install VirtualEnvWrapper
7. Create your first virtualenv
8. Configure bashrc to keep it working
9. Install some packages
10. Test Flask

1. Install Bash on Windows:
Here’s the announcement blog for context.
How to Geek has a good breakdown of making it happen.

Make sure you remember your password.

Now that its installed try opening a command prompt and typing bash.
The prompt should change like this:
bashtomnt

Notice that path?
/mnt/c/Users/TimReilly

That’s your user directory for windows where your OneDrive, Documents, Desktop, etc. exist.

You can go in there now and run python scripts that might already exist, but your probably won’t have all the necessary packages installed.

Before we move forward we want to make sure Ubuntu is up to date.

2. Check for updates
From another command prompt:
lxrun /update

And inside bash
sudo apt-get update

Thanks reddit for the tips!

3. Check out the REPL

Now run python!
$ python

Should look like this:
pythonrepl

4. Install Pip

Now we’ll install Pip:
sudo apt-get install python-pip

If you have permission issues try starting an elevated prompt:
$ sudo -i
$ apt-get install python-pip
$ exit

Use exit to return to the regular prompt.
Should look like this:
sudoi
Yay we’ve got pip!
Try pip list to see what comes standard.

5. Install VirtualEnv
Now we’re basically following along with the guide presented at hitchhikersguidetopython.com

Again, you might need to start an elevated prompt to install virtualenv.

$ sudo -i
$ pip install virtualenv
$ exit
$ cd my_project_folder
$ virtualenv venv

Then to use the VirtualEnvironment

$ source venv/bin/activate

You should now see a little (venv) before your prompt.
Like this:
virtualenvfolder

Now you’ve created a virtualenv inside of your my_project_folder directory. Which is cool, but can be confusing with git, sharing code, and testing package versions.
So we use VirtualEnvWrapper to keep our virtualenvs in the same place.

Before we move on make sure you deactivate your env
deactivate

6. Install VirtualEnvWrapper

http://virtualenvwrapper.readthedocs.io/en/latest/index.html


$ pip install virtualenvwrapper
$ export WORKON_HOME=~/Envs
$ source /usr/local/bin/virtualenvwrapper.sh

$ export WORKON_HOME=~/
Can be customized to whichever directory you’d like to place your virtualenvs

7. Create virtualenv using virtualenvwrapper

$ mkvirtualenv venv
$ workon venv
$ deactivate

Here’s an example of what it looks like to remove our venv directory and instead use venvv which will be stored in the directory underlined in red.

venvv

8. Configure bashrc to keep it working

This might not happen to you, but when I opened a new bash terminal I had to re-source my virtualenvwrapper.sh and WORKON_HOME.

So instead I added those lines to my bashrc script.

$ sudo nano ~/.bashrc
-- type in password --

This is what it looks like in nano for me.
Ctrl+X to exit and y-enter to save.
bashrc

Then either:

Source ~/.bashrc

Or start a new command prompt->bash and try “workon” or “lsvirtualenv”

See the next image for a simple workflow.

9. Install some packages

Now lets install ‘requests’ into our newly created virtualenv:
pipinstallrequests

Isn’t that nice!

10. Test Flask
Finally we’re going to test this with flask.
First we install the required files using pip into our activated ‘venv’
Then runserver -> navigate to the designated address -> and see our site.

Here’s what it looks like:
flaskrunning

Have fun building with Python, on Ubuntu, on Windows!

Bike with pizza tied to the back
Sometimes you gotta tie a pizza to your bike.

Setup a virtualenv for Python 3 on Windows

It’s essentially the same as Unix!

Especially, if you’ve followed my other guide to getting setup virtualenvs and virtualenvwrapper-win on Windows. And you’ve also installed python3!

Here’s the Script I run:

mkvirtualenv --python=C:\Python35-32\python.exe pythonthreeEnv

‘pythonthreeEnv’ is the name of my environment.

Now I can run:

workon pythonthreeEnv

Here’s a screenshot of a workflow:
pythonthreeenv

Bonus: To efficiently delete an env:

rmvirtualenv pythonthreeEnv -r

Happy Hacking!

THESE THINGS ARE AMAING. Thank you Alejandro and Argentina!
THESE THINGS ARE AMAZING. Thank you Alejandro and Argentina!

Python, Pip, virtualenv installation on Windows

No more struggles Windows Python development! I’ve found this is the best way to configure your dev environment.
This has made things much easier to get started and less of a headache overall.

We use Virtual Environment so we can test python code in encapsulated environments and to also avoid filling our base Python installation with a bunch of libraries we might use for only one project.

But Virtual Environments can be tricky if you don’t establish a good workflow. I’ll show you how to setup your python environment from Scratch and then do a very simple workflow using Flask.

SETUP
4 Steps:
Install Python
Install Pip
Install VirtualEnv
Install VirtualEnvWrapper-win

Install Python:

First Go to the Python Downloads Site.

As of March 2015 the download you want for a standard windows machine is Windows x86-64 MSI installer (The other download is for servers). Its circled here:

Download

Run the installer!
You’ll come across this page in the installer:

PythonInstaller

You’ll want to scroll down and add it to the path. If you don’t that’s okay. You can add it later.
Adding Python to the PATH will allow you to call if from the command line.

After the installation is complete double check to make sure you see python in your PATH. You can find your path by opening your control panel -> System and Security -> System -> Advanced System Settings -> Environment Variables -> Selecting Path -> Edit ->

Now you’re looking at your Path. Be Careful, if you delete or add to the path accidently you may break other programs.

You need to confirm that C:\Python27; and C:\Python27\Scripts; is part of your path.

If you do not see it in your path you can simply add it at the beginning or end of the variable value box. As you can see in the image below.

AdvancedSettings

Install Pip:

As of Python Version 2.7.9 Pip is installed automatically and will be available in your Scripts folder.

If you install a later version of Python I would recommend installing it according to this helpful stackoverflow post.

Pip is a Package manager for python which we will use to load in modules/libraries into our environments.

An example of one of these libraries is VirtualEnv which will help us keep our environments clean from other Libraries. This sounds really confusing but as you start using it you’ll begin to understand how valuable this encapsulation of modules/libraries can be.

To test that Pip is installed open a command prompt (win+r->’cmd’->Enter) and try ‘pip help’

You should see a list of available commands including install, which we’ll use for the next part:

Install virtualenv:

Now that you have pip installed and a command prompt open installing virtualenv to our root Python installation is as easy as typing ‘pip install virtualenv’
Like so:

pipinstallvirtualenv

Now we have virtualenv installed which will make it possible to create individual environments to test our code in. But managing all these environments can become cumbersome. So we’ll pip install another helpful package…

Install virtualenvwrapper-win:

This is the kit and caboodle of this guide.

Just as before we’ll use pip to install virtualenvwrapper-win. ‘pip install virtualenvwrapper-win’
Like so:

virtualenvwrapper-win

Excellent! Now we have everything we need to start building software using python! Now I’ll show you how buttery smooth it is to use these awesome tools!

USAGE
7 Steps:
Make a Virtual Environment
Connect our project with our Environment
Set Project Directory
Deactivate
Workon
Pip Install
Flask!

Make a Virtual Environemt:

Lets call it HelloWold. All we do in a command prompt is enter ‘mkvirtualenv HelloWold’
This will create a folder with python.exe, pip, and setuptools all ready to go in its own little environment. It will also activate the Virtual Environment which is indicated with the (HelloWold) on the left side of the prompt.

mkvirtualenv

Anything we install now will be specific to this project. And available to the projects we connect to this environment.

Connect our project with our Environment:

Now we want our code to use this environment to install packages and run/test code.

First lets create a directory with the same name as our virtual environment in our preferred development folder. In this case mine is ‘dev’

See here:

mkdir

HelloWold will be the root folder of our first project!

Set Project Directory:

Now to bind our virtualenv with our current working directory we simply enter ‘setprojectdir .’
Like so:

setprojectdir

Now next time we activate this environment we will automatically move into this directory!
Buttery smooth.

Deactivate:

Let say you’re content with the work you’ve contributed to this project and you want to move onto something else in the command line. Simply type ‘deactivate’ to deactivate your environment.
Like so:

deactivate

Notice how the parenthesis disappear.
You don’t have to deactivate your environment. Closing your command prompt will deactivate it for you. As long as the parenthesis are not there you will not be affecting your environment. But you will be able to impact your root python installation.

Workon:

Now you’ve got some work to do. Open up the command prompt and type ‘workon HelloWold’ to activate the environment and move into your root project folder.

Like so:

workon

Pretty sweet! Lets get working.

Pip Install:

To use flask we need to install the packages and to do that we can use pip to install it into our HelloWold virtual environment.

Make sure (HelloWold) is to the left of your prompt and enter ‘pip install flask’
Like so:

pipinstallflask

This will bring in all the tools required to write your first web server!

Flask:

Now that you have flask installed in your virtual environment you can start coding!

Open up your favorite text editor and create a new file called hello.py and save it in your HelloWold directory.

I’ve simply taken the sample code from Flask’s website to create a very basic ‘Hello World!’ server.

I’ve named the file hello.py.

Once the code is in place I can start the server using ‘python hello.py’ this will run the python instance from your virtual environment that has flask.

See here:

webserver

You can now navigate with your browser to http://127.0.0.1:5000/ and see your new site!

Sweet. You have everything you need to start working through tutorials on Flask without worrying about gunking up your Python installations.

Let me know if you have any questions! Happy Developing!

Art Deco From Afar
Art Deco From Afar

Using Bing Maps with Flask

I’m working on a project that involves mapping and line segments.

It turns out that using the Mapping libraries of Bing and Google make this really easy.
You create ‘polylines’ out of coordinates and colors.

Check out Microsoft Maps documentation for more info and the dev page to sign up for a key.

In this example I’m using AJAX requests.

And this documentation is great for using Azure Tables
And the home directory for Python Azure Table Usage

I was struggling with pulling the data from a Database using flask and serving it to the client for an Ajax request.
The end goal is a heat map of sorts with paths being mapped and colors being changed for certain conditions.

I’m still working out the best practices for this, but its working! So take it with a grain of salt.

The code for this project can be found here: http://github.com/timmyreilly/BumpyRoads

After a couple days of tinkering, the main pieces of code that lit this up are as follows.

Get the data from the Azure Table Storage and convert it to a list

# First connect to Azure Table Service 
def connect_to_service():
    table_service = TableService(account_name=STORAGE_ACCOUNT_NAME, account_key=TABLE_STORAGE_KEY)
    print TableService
    return table_service 

# Grab the data from our table
def get_table_list(table_service=connect_to_service(), max=10, table_name='test', partitionKey='default'):
    x = table_service.query_entities(table_name)
    #print(x)
    return x 

# return it as a list  * see below for bonus on selecting color codes 
def get_json_list(entity_list=get_table_list()):
    '''
    Takes azure table list and returns json list 
    '''
    i = 0 
    response = [] 
    for r in entity_list:
        c = convert_color_key_to_rgb(int(entity_list[i].colorKey))
        t = (entity_list[i].latA, entity_list[i].longA, entity_list[i].latB, entity_list[i].longB, entity_list[i].colorKey, c[0], c[1], c[2])
        response.append(t)
        i += 1 
    # print response 
    return response 

* BONUS to manage color codes nicely I found this neat way of doing switch statements using dictionaries.

def convert_color_key_to_rgb(colorKey):
    return {
        0: [100, 100, 100], 
        1: [240, 0, 255],
        2: [0, 0, 255],
        3: [0, 255, 0],
        4: [255, 255, 0],
        5: [255, 85, 0],
        6: [255, 0, 0],
    }.get(colorKey, [100, 100, 100] )

Take that list and pass it as JSON to a template.
Still not sure if this is best practice, as on the client/javascript side it picks it up as an array object.
But this is the route we use in flask to send our data.

@app.route('/d')
def binged():   
    data = get_json_list()
    jdata = json.dumps(data)
    return render_template(
        'binged.html',
        data=jdata,
        key=token 
    )

On the template side iterate through the given array object and build and push polylines.
This was the toughest part for me to

 var map = null;
         function GetMap()
         {             
            var y = JSON.parse('{{ data|safe }}');
            // Initialize the map
            
            map = new Microsoft.Maps.Map(document.getElementById("mapDiv"),{
                credentials:"{{ key }}",
                center: new Microsoft.Maps.Location(y[0][0],y[0][1]), 
                zoom: 10
                }); 
                       
           console.log(y);
           
           for( index = 0; index < y.length; index++ ){
               lOne = new Microsoft.Maps.Location(y[index][0], y[index][1]);
               lTwo = new Microsoft.Maps.Location(y[index][2], y[index][3]);
               //console.log(lOne);
               var lineV = new Array(lOne, lTwo);
               //console.log(y[index][4])
               var pLine = new Microsoft.Maps.Polyline(lineV, {
                   strokeColor: new Microsoft.Maps.Color(y[index][5], y[index][6], y[index][7], 100),
                   strokeThickness: 3
               });
               map.entities.push(pLine)
           }
 }

Now when I run this project locally I get something that looks like this:

Soon they won't be random.
Soon they won’t be random.

If you want to give it a shot, check out the instructions/source code on github

Feel free to reach out if you have any questions or would like to work together on a project like this.

Groovy
Groovy

Flask-SocketIO, Background Threads , Jquery, Python Demo

This week I’ve been making progress on the Huggable Cloud Pillow website.

In the process I’ve learned about some sweet stuff you can do with Javascript, Python, and Flask-SocketIO.

The first thing to take note of is Flask.

Flask is the tiny server that allows us to host websites using Python to deliver content to the client. While on the server side you can manage complicated back ends or other processes using Python in conjunction with Flask.

This type of development is nice, because you can start seeing results right away but can take on big projects easily.

It might not be the most robust Framework, but its great for small projects…

If you want to get into Flask Web Development checkout this extensive MVA.

Small and simple, Flask is static on its own. This allows us to present static content, like templates and images easily and deals with input from the user using RESTful calls to receive input. This is great for static things with lots of user actions, but if we want something a bit more dynamic we’re going to need another tool.

In this case I’ve found Flask-SocketIO, similar to Flask-Sockets but with a few key differences highlighted by the creator (Miguel Grinberg) here.

Sockets are great for is providing live information with low latency. Basically, you can get info on the webpage without reloading or waiting for long-polling.

There are lots of ways you can extend this functionality to sharing rooms and providing communication with users and all sorts of fun stuff that is highlighted on GitHub with a great chunk of sample code. The following demo is based off of these samples.

For my project, I need the webpage to regularly check for differences in the state of the cloud and present them to the client, while also changing the image the user sees.

At first I tried to implement it using sockets passing information back and forth, but that wasn’t very stable.

The solution I’ve found, uses a background thread that is constantly running while the Flask-SocketIO Application is running, it provides a loop that I use to constantly check state of our queue.

Let’s break it down…
a. I need my website to display the current state of the cloud.
b. The Flask application can get the state by query our azure queue.
c. Once we determine a change of state we can display that information to the webpage.
d. To display the new state to the webpage we need to use a socket.
e. And pass the msg to be displayed.

This demo intends to break down problem a, c, d, and e.

I’ve created this little guide to help another developer get going quickly, with a nice piece of code available on GitHub.

The five steps to this little demo project are as follows:
1. Install Flask-SocketIO into our Virtual Environment
2. Create our background thread
3. Have it emit the current state to our client
4. Call the background thread when our page render_template’s
5. Have the Javascript Catch the emit and format our HTML.
Celebrate! Its Friday!

1.

Flask-SocketIO is a python package that is available for download using

pip install Flask-SocketIO

Make sure you instal it into a Virtual Environment. Check out my earlier tutorial if you need help with this step.

*Edit Here’s the top part of the “main.py” for reference:

#main.py
from gevent import monkey
monkey.patch_all()

import time
from threading import Thread
from flask import Flask, render_template, session, request
from flask.ext.socketio import SocketIO, emit, join_room, disconnect

app = Flask(__name__)
app.debug = True
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
thread = None

2.

Create our background thread.
You’ll see in the sample code from Flask-SocketIO’s github a simple way to send data to the client regardless of their requests.

For this example we’ll be changing the current time every second and display that to our client.

Background Thread:

def background_stuff():
     """ python code in main.py """
     print 'In background_stuff'
     while True:
         time.sleep(1)
         t = str(time.clock())
         socketio.emit('message', {'data': 'This is data', 'time': t}, namespace='/test')

3

This is the emit statement from above, but is the meat of our interface with SocketIO. Notice how it breaks down…

socketio.emit('message', {'data': 'This is data', 'time': t}, namespace='/test')
socektio.emit('tag', 'data', namespace)

This emit will be sending to the client (Javascript) a message called ‘message’.

When the Javascript catches this message it will be able to pull from the python dicionary msg.data and msg.time to get the result of this package.

4

So how do we call background_stuff?

We can call it wherever we want, but for this simple example we’ll put it right in our ‘/’ base route. So when we navigate to 127.0.0.1:5000 (Local Host) we’ll see the result of our background thread call.

Here’s our route:

@app.route('/')
def index():
    global thread
    if thread is None:
        thread = Thread(target=background_stuff)
        thread.start()
    return render_template('index.html')

Pretty simple… Notice global thread and target=background_stuff

Creating different background threads is a good way to iterate through your changes.

5

Next step is catching this on the other side…

So for our Javascript…

we’ll be using the socket.on method.

socket.on('message', function(msg){
    $('#test').html('&lt;p&gt;' + msg.time + '&lt;/p&gt;');
});

When we receive the emit labeled ‘message’ we’ll pick up the msg from the second parameter and have it be available to our JQuery work.

Here’s the small piece of HTML that we’re selecting to edit.

<body>
    <p id='test'>Hello</p>
</body>

I’ve posted all of this code at github.

Feel free to download it and start working with dynamic sites using SocketIO. Please let me know if you have any questions!

Code for SF's Hackathon gave out awesome tattoos!
Code for SF’s Hackathon gave out awesome tattoos!

GitHub’s Gitignore and Keeping DB Keys Safe

I’m working with a raspberry pi and I’m learning how to collect and manage sensor data.

There is a lot of data and that’s awesome, but dealing with remote persistent storage has helped me understand some good ways of keeping stuff organized and safe.

For example: ‘Keys’ or ‘Tokens’ (Not to be confused with the Late English writer Tolkien)

These are the keys to the door of your data.
To access the contents behind these doors, you must establish that you are the key-holder. One of the ways to do this in software is to create a long complicated string that can passed to the database so you can input or output data.

As long as you keep this token secret nobody will be able to mess with your data, or use it without having to pay for it.

Now this sounds pretty simple, but what if you use that token in code that is stored in a public repository on GitHub? This is essentially like printing a bunch of keys to your house and leaving them all over town. Not safe or smart.

So…

1. We need to keep these keys separate from the rest of your code.
2. We need to keep the file that contains those keys away from your public repository.

To do this I created a separate file called tokens.py that contains the strings of my tokens and a function called getStorageKey() that returns those strings.
Cool, we’ve satisfied our first goal.

Now we need to keep this file away from our repository. To do this we create a .gitignore file. This is a special ‘git’ file that allows us to specify which files will not be added to the remote repository.
To create a .gitignore file simply enter:

touch .gitignore

touch is a Unix command to create a file and update the access data, but not make any edits. Its the same as opening and closing without saving any changes.

After you’ve created this file you can edit it with any editor you like. I do it with sublime or VS Code because it helps me keep track of the separation occurring between visual studio and GitHub.

All you need to do is add on a separate lines the files you don’t want GitHub or just git to include.

When I first started all my .gitignore file had was:

tokens.py

Its expanded to include…

tokens.py
*.pyc

# the asterisk* acts as a wild card and will match any files with .pyc at the end
# anything that starts with a # is a comment and will be ignored from .gitignore

I saved this file (.gitignore) then added and committed it to the repository.

Now any files I add that end in .pyc or match tokens.py will not be added to my repository. And you will not be prompted to add them when you check git status.

What about files we’ve already have added???

If we add the rule after we added the file we want to ignore… We need to remove it from tracking on GitHub using the rm –cache command.

This will not remove it from your computer, but it will stop if from being tracked by GitHub, and will essentially be treated as if it was removed from your repository.

To pull this off simply type this command

git rm --cached filename.py

In this case I’m removing a file named ‘filename.py’.
After checking our git status we’ll see that this file is going to be removed.

Commit the changes and push to your remote repo.

You’ll see the files are still in your local directory, but are no longer in the remote or local repository. Hooray for keeping things safe!

Please let me know if you have any questions or comments.
The documentation is excellent and I recommend you spend some time reading through the resources below to round out your learning:

touch (Unix)
GitHub .gitignore
If you enter: ‘git ignore –help’ in your gitshell you can find more helpful documentation about using .gitignore
Setting up Code for Powershell
Setting up sublime for Powershell

SomeOfTheNorCalTeam
Evangelists in their natural habitat. Behind a booth and with a computer.

Running Multiple Python Versions on Windows

So, you are learning to develop with Python and you keep hopping back and forth from Python 2.x and Python 3.x and possibly versions in between.

Beside running everything in a virtual environment, its sometimes nice to just get to the different REPLs to test tiny pieces of code.

This can be managed easily in powershell using powershell profile.
This link helped me learn about what it is and how to set it up.

But basically, you establish shortcuts by customizing a profile page with the things you need to access quickly.

Currently in my Profile I have a shortcut for starting Python27, Python34, and Sublime.

I’ll show you how I set it up.
First we’ll edit our Profile, then we’ll change the execution policy to allow this file to edit PowerShell.

Open powershell!

1. Type the following command and press ENTER:

Test-Path $profile

2. If False you need to create a profile, if true go to step 3:

New-item -type file -force $profile

3. Go to you PowerShell Directory. Step 2 will show the path to the location. For me its Users\tireilly\Documents\WindowsPowerShell.

4. Open your profile:

Notepad .\Microsoft.PowerShell_profile.ps1

5. Now in Notepad you can add your aliases.

This is what I have:

Set-Alias subl 'C:\Program Files (x86)\Sublime Text 2\sublime_text.exe'
Set-Alias python34 'C:\Python34\python.exe'
Set-Alias python27 'C:\Python27\python.exe'

Make sure you have the proper path to your existence of python. I have mine in my Root directory.

Now to activate this you need to set the execution policy to allow this to be activate each time you open powershell.

To do this follow these steps:

1. Determine your current execution policy.

Get-ExecutionPolicy

This is just to double check you are restricted either way set it again to be sure.

2. Set Execution policy for yourself.

Set-ExecutionPolicy -Scope CurrentUser

3. You’ll be prompted with “ExecutionPolicy: ”
Set it as unrestricted
It should look like this:
ExecutionPolicy: Unrestricted"
Press enter.

4. Comfirm by entering ‘Y’ and pressing enter again.

Now Restart Power shell and try typing in your alias!
Does Python Start?

If not, I’m happy to help if you have any questions.

This Photo is from 2010... Just saying
This Photo is from 2010…

Getting Django Models into SQLite3 DB with VS

I ran into some troubles migrating/configuring my tables for a new app in my Django project.

I’ve been following this excellent tutorial, and ran into a bump I thought needed some clarification/update. As I’m not sure if the guide is up to date with the current version of Django.

Things I searched:
no module named models
manage.py sqlall
django not creating db
models are not being created in sqlite3 Django
sqlite3 not creating table django
No migrations to apply
django sqlite3 unable to create tables
manage.py shell
sqlite3 python package

Do these correlate with what you’re having issues with?
If so this was my solution.

First Install SQLite3 and add it too your Environment Variables -> Path
Install Page — Select: Source Code -> Zip for windows machines
I extracted it to C:/.
Now I have SQLite.exe in my root directory.
This is what the end of my Path looks like:
C:\Program Files\Microsoft\Web Platform Installer\;C:\SQLITE3.EXE\;

Sweet, now we can use SQLite in Powershell.

Configuration of Visual Studio:
Create a new app by right clicking on your Project file.
Then “Add” -> Select “Django App”
In this case my app is named book.

DjangoApp Solution Explorer

Sweet, now we have another Python app.
Go into your settings file and add it to the INSTALLED_APPS tuple.
eg. ‘book’,

Okay, now we’re configured make sure you’re SQLiteDB is properly configured as well.

eg:
‘ENGINE’: ‘django.db.backends.sqlite3’,
‘NAME’: path.join(PROJECT_ROOT, ‘db.sqlite3’),
‘USER’: ”,
‘PASSWORD’: ”,
‘HOST’: ”,
‘PORT’: ”,

Sweet, db all locked and loaded.

Next we’ll create a model.
Following along with the tutorial.
We go into: book -> models.py and create our models.
Eg:
class Publisher(models.Model):
name = models.CharField(max_length=30)
address = models.CharField(max_length=50)
city = models.CharField(max_length=60)
state_province = models.CharField(max_length=30)
country = models.CharField(max_length=50)
website = models.URLField()

Sweet. Model made. Let’s get it into our SQLite DB.

Alright now in DjangoProject1Hack (Where ‘ls’ will show db.sqlite3 among others)
We’ll be doing the migration.

1. Validate:
Run- C:\Python27\python.exe manage.py validate
2. makemigrations
Run- C:\Python27\python.exe manage.py makemigrations
Output:
Migrations for ‘book’:
0001_initial.py:
– Create model Author
– Create model Book
– Create model Publisher
– Add field publisher to book
3. Sync (This just makes sure we add what’s missing)
Run- C:\Python27\python.exe manage.py syncdb
Output:
Operations to perform:
Apply all migrations: book
Running migrations:
Applying book.0001_initial… OK

Sweeeeeeet!

Okay now we manage the db:
Run- C:\Python27\python.exe manage.py shell

Sample workflow in Shell:

>>> from book.models import Publisher
>>> p1 = Publisher(name=’Apress’, address=’2855 Telegraph ave’, city=’berkely’, state_province=’CA’, country=’USA’, website=
‘http://www.apress.com/’)
>>> p1.save()
>>> p2 = Publisher(name=”o’reilly”, address=’10 Fawcett St.’, city=’Cambridge’, state_province=’MA’, country=’USA’, website=
‘http://www.oreilly.com/’)
>>> p2.save()
>>> publisher_list = Publisher.objects.all()
>>> publisher_list
Publisher: Publisher object, Publisher: Publisher object

Yeehaa, let me know if you have any other questions!

What is it?
If you can guess what pun this represents, I’ll venmo you a dollar.