Running Flask on Ubuntu VM

So you have a VM in Azure and want to put it to good use?

No.
Let’s set one up!

Yes.
Great!

Before we start make sure you can ssh into your machine and run

$sudo apt-get update

sudoaptgetupdate

It’s a four step process:
1. Open the appropriate port on Azure
2. Install pip, virtualenv, virtualenvwrapper, and flask
3. Write our code and run it
4. Keep it running with Gunicorn

1. Open the appropriate ports on Azure
Go to you virtual machines landing:
vmnumberonelandingpage

Then select the resource group in the top left corner:
resourcegroup

Resource groups are the way Azure breaks down how our VM interacts with the internet, other vms, storage, and public/private networks.

Top open the port we need to change our network security group, which is represented by the shield. (Underlined in the screenshot above)

Then select settings -> Inbound Security Rules:
networksecuritygroupsettings

This will allos us to open up our VM to the public internet so we can visit what’s presented at the port like a regular website.

You should see SSH already included, that’s the port we’re using in our ssh client/terminal.
defaultssh

We’re now going to add two new Inbound Security Rules one called FlaskPort where we’ll set the destination port range to 5000 and use for debugging. The second will be called FlaskProduction that we’ll use to deploy our complete app.
Here’s the configuration panel for FlaskPort:
FlaskPort
Press okay to accept the settings.

And the other panel for FlaskProduction:
flaskproduction
Again press okay to accept the settings.

Notice how the ‘Source Port Range’ is ‘*’ that just means that we’ll accept connections from the port of any machine. This tripped me up the first time.

In a couple seconds the port will be open we’ll be ready to visit it, but nothing will be there because we haven’t create an application server.

To do that we’ll install the basics.

2. Install pip, virtualenv, virtualenvwrapper, and flask

To use Python effectively we utilize virtual environments to help keep our various python project and required libraries in order.

If you get lost in these steps or want more context Gerhard Burger provides the same setup on a very helpful post on askubuntu.

First we install pip:
$ sudo apt-get install python-pip

Second we install virtualenv and virtualenvwrapper


$ sudo pip install virtualenv
$ sudo pip install virtualenvwrapper

Third we configure virtualenv and virtualenvwrapper

Create a WORKON_HOME string which will contain the directory for our virtual environments. We’ll name it vitualenvs

$ export $WORKON_HOME=~/.virtualenvs

Now we’ll create this directory.

$ mkdir $WORKON_HOME

And add this to our bashrc file so this variable is defined automatically every time we hit the terminal.

$ echo "export WORKON_HOME=$WORKON_HOME" >> ~/.bashrc

Then we’ll setup virtualenvwrapper by importing its functions with bashrc.

$ echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc

You can see the additions to our bashrc file by opening it with nano. Scrolling down to the bottom you should see two lines like this:
bashrcconfigured

Then implement your changes.

$ source ~/.bashrc

Here’s what all that looks like all together:
configurepipandvirtualenvwrapper

Fourth, let’s create our first virtualenvironment

$ mkvirtualenv venv

And take a look at the currently installed packages
$ pip list

Like so:
firstvenv

Now we can install all of the python packages we want without risk of needing to reinstall python!

Fifth, install flask:


$ pip install flask
$ pip list

pipinstallflask
currentpackages
3. Write our code and run it!
Our first app is a simple site that shares an image.

We’re going to create a folder called Photo-App that contains two folders and an app.py that will serve our clients.

To change to our home directory:
$ cd ~
And create our new folder:
$ mkdir Photo-App

Then create a static and templates folder inside of our app.

$ sudo mkdir templtes
$ sudo mkdir static

mkPhotoApp
Then create our app.py which will be our python flask server code.

$ sudo nano app.py

Here’s the code:

from flask import Flask, render_template
app = Flask(__name__)

@app.route("/")
def index():
        return render_template("index.html")

if __name__ == "__main__":
        app.run(host='0.0.0.0', debug=True)

And what it looks like in nano:
nanoapppy

Then we need to add our first template:
$ sudo nano templates/index.html

<h1>Wazzup Dog</h1>
<img style="max-width:100%;" src="{{ url_for('static', filename='photo.jpg') }}">

And what it looks like in nano:
indexinnano

Now for our photo we’re going to download an image into our static file using curl.

$ cd static
$ curl 'http://timmyreilly.azurewebsites.net/wp-content/uploads/2015/12/Snapchat-1802119159214415224.jpg' -o 'photo.jpg'

Cool! We have an app!

To run it simply enter:
$ python app.py

Visit from a browser!

Type in the IP address of your virtualmachine with a colon at the end followed by the port.
Mine looks like:
http://138.91.154.193:5000/

And this is what I see!
wazzupdogimdex

Neato! But when we close the terminal we lose the application… Hmmmm let’s fix that!

4. Keep it running with Gunicorn
Real Python provided the bulk of this portion so if you get lost or checkout their site, plus they have info about setting up continuous deployment!

First, we install dependencies
$ sudo apt-get install -y nginx gunicorn

Now that our app is ready, let’s configure nginx.

If you’d like to learn more about this Open Source HTTP Server Software checkout their website.

Second, we start nginx.
$ sudo /etc/init.d/nginx start

And begin configuration for our project
We’re naming our nginx configuration the same as the parent directly of our app.py file in this test case.
Here are the commands:

$ sudo rm /etc/nginx/sites-enabled/default
$ sudo touch /etc/nginx/sites-available/Photo-App
$ sudo ln -s /etc/nginx/sites-available/Photo-App /etc/nginx/sites-enabled/Photo-App

Now we add the config settings for our app.

$ sudo nano /etc/nginx/sites-enabled/Photo-App
nginxconfigphotoapp
Here’s the raw text:

server {
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /static {
alias /Photo-App/static/;
}
}

Then restart the server.

$ sudo /etc/init.d/nginx restart

And navigate to your Python PhotoApp project, and start it using Gunicorn:
$sudo gunicorn app:app -b 0.0.0.0:8000 --reload

You’re using gunicorn to start app hosted at 0.0.0.0:8000 with the reload tag configured.
The reload will look for changes in your code and reload the server everytime you change any server side stuff. It won’t auto-reload for HTML changes, but will reload them once you make a change to the python code.

Now try navigating to: http://138.91.154.193:8000/ or whatever your IP address is.

You can now close you’re ssh terminal and it will continue to run.

To stop it we have to kill the actual thread it’s running on.
$ pkill gunicorn

Like I said at the beginning of this step this was adapted from the realpython.org and they have some awesome next steps available for git deployment + automating the whole process.

The code for the app can be found on GitHub here: https://github.com/timmyreilly/Flask-For-UbuntuBlog

Happy Hacking!

This was a nice reminder in the busy airport.
This was a nice reminder in the busy airport.

Python with Ubuntu on Windows

Now that bash is on Windows, I wanted to try and make all the other guides I’d writen for Python on Windows irrelevant.

So here’s how to setup an effective environment for Python on Ubuntu on Windows.

1. Install Bash on Windows
2. Check for updates
3. Check out the REPL
4. Install Pip
5. Install VirtualEnv
6. Install VirtualEnvWrapper
7. Create your first virtualenv
8. Configure bashrc to keep it working
9. Install some packages
10. Test Flask

1. Install Bash on Windows:
Here’s the announcement blog for context.
How to Geek has a good breakdown of making it happen.

Make sure you remember your password.

Now that its installed try opening a command prompt and typing bash.
The prompt should change like this:
bashtomnt

Notice that path?
/mnt/c/Users/TimReilly

That’s your user directory for windows where your OneDrive, Documents, Desktop, etc. exist.

You can go in there now and run python scripts that might already exist, but your probably won’t have all the necessary packages installed.

Before we move forward we want to make sure Ubuntu is up to date.

2. Check for updates
From another command prompt:
lxrun /update

And inside bash
sudo apt-get update

Thanks reddit for the tips!

3. Check out the REPL

Now run python!
$ python

Should look like this:
pythonrepl

4. Install Pip

Now we’ll install Pip:
sudo apt-get install python-pip

If you have permission issues try starting an elevated prompt:
$ sudo -i
$ apt-get install python-pip
$ exit

Use exit to return to the regular prompt.
Should look like this:
sudoi
Yay we’ve got pip!
Try pip list to see what comes standard.

5. Install VirtualEnv
Now we’re basically following along with the guide presented at hitchhikersguidetopython.com

Again, you might need to start an elevated prompt to install virtualenv.

$ sudo -i
$ pip install virtualenv
$ exit
$ cd my_project_folder
$ virtualenv venv

Then to use the VirtualEnvironment

$ source venv/bin/activate

You should now see a little (venv) before your prompt.
Like this:
virtualenvfolder

Now you’ve created a virtualenv inside of your my_project_folder directory. Which is cool, but can be confusing with git, sharing code, and testing package versions.
So we use VirtualEnvWrapper to keep our virtualenvs in the same place.

Before we move on make sure you deactivate your env
deactivate

6. Install VirtualEnvWrapper

http://virtualenvwrapper.readthedocs.io/en/latest/index.html


$ pip install virtualenvwrapper
$ export WORKON_HOME=~/Envs
$ source /usr/local/bin/virtualenvwrapper.sh

$ export WORKON_HOME=~/
Can be customized to whichever directory you’d like to place your virtualenvs

7. Create virtualenv using virtualenvwrapper

$ mkvirtualenv venv
$ workon venv
$ deactivate

Here’s an example of what it looks like to remove our venv directory and instead use venvv which will be stored in the directory underlined in red.

venvv

8. Configure bashrc to keep it working

This might not happen to you, but when I opened a new bash terminal I had to re-source my virtualenvwrapper.sh and WORKON_HOME.

So instead I added those lines to my bashrc script.

$ sudo nano ~/.bashrc
-- type in password --

This is what it looks like in nano for me.
Ctrl+X to exit and y-enter to save.
bashrc

Then either:

Source ~/.bashrc

Or start a new command prompt->bash and try “workon” or “lsvirtualenv”

See the next image for a simple workflow.

9. Install some packages

Now lets install ‘requests’ into our newly created virtualenv:
pipinstallrequests

Isn’t that nice!

10. Test Flask
Finally we’re going to test this with flask.
First we install the required files using pip into our activated ‘venv’
Then runserver -> navigate to the designated address -> and see our site.

Here’s what it looks like:
flaskrunning

Have fun building with Python, on Ubuntu, on Windows!

Bike with pizza tied to the back
Sometimes you gotta tie a pizza to your bike.

Using Bing Maps with Flask

I’m working on a project that involves mapping and line segments.

It turns out that using the Mapping libraries of Bing and Google make this really easy.
You create ‘polylines’ out of coordinates and colors.

Check out Microsoft Maps documentation for more info and the dev page to sign up for a key.

In this example I’m using AJAX requests.

And this documentation is great for using Azure Tables
And the home directory for Python Azure Table Usage

I was struggling with pulling the data from a Database using flask and serving it to the client for an Ajax request.
The end goal is a heat map of sorts with paths being mapped and colors being changed for certain conditions.

I’m still working out the best practices for this, but its working! So take it with a grain of salt.

The code for this project can be found here: http://github.com/timmyreilly/BumpyRoads

After a couple days of tinkering, the main pieces of code that lit this up are as follows.

Get the data from the Azure Table Storage and convert it to a list

# First connect to Azure Table Service 
def connect_to_service():
    table_service = TableService(account_name=STORAGE_ACCOUNT_NAME, account_key=TABLE_STORAGE_KEY)
    print TableService
    return table_service 

# Grab the data from our table
def get_table_list(table_service=connect_to_service(), max=10, table_name='test', partitionKey='default'):
    x = table_service.query_entities(table_name)
    #print(x)
    return x 

# return it as a list  * see below for bonus on selecting color codes 
def get_json_list(entity_list=get_table_list()):
    '''
    Takes azure table list and returns json list 
    '''
    i = 0 
    response = [] 
    for r in entity_list:
        c = convert_color_key_to_rgb(int(entity_list[i].colorKey))
        t = (entity_list[i].latA, entity_list[i].longA, entity_list[i].latB, entity_list[i].longB, entity_list[i].colorKey, c[0], c[1], c[2])
        response.append(t)
        i += 1 
    # print response 
    return response 

* BONUS to manage color codes nicely I found this neat way of doing switch statements using dictionaries.

def convert_color_key_to_rgb(colorKey):
    return {
        0: [100, 100, 100], 
        1: [240, 0, 255],
        2: [0, 0, 255],
        3: [0, 255, 0],
        4: [255, 255, 0],
        5: [255, 85, 0],
        6: [255, 0, 0],
    }.get(colorKey, [100, 100, 100] )

Take that list and pass it as JSON to a template.
Still not sure if this is best practice, as on the client/javascript side it picks it up as an array object.
But this is the route we use in flask to send our data.

@app.route('/d')
def binged():   
    data = get_json_list()
    jdata = json.dumps(data)
    return render_template(
        'binged.html',
        data=jdata,
        key=token 
    )

On the template side iterate through the given array object and build and push polylines.
This was the toughest part for me to

 var map = null;
         function GetMap()
         {             
            var y = JSON.parse('{{ data|safe }}');
            // Initialize the map
            
            map = new Microsoft.Maps.Map(document.getElementById("mapDiv"),{
                credentials:"{{ key }}",
                center: new Microsoft.Maps.Location(y[0][0],y[0][1]), 
                zoom: 10
                }); 
                       
           console.log(y);
           
           for( index = 0; index < y.length; index++ ){
               lOne = new Microsoft.Maps.Location(y[index][0], y[index][1]);
               lTwo = new Microsoft.Maps.Location(y[index][2], y[index][3]);
               //console.log(lOne);
               var lineV = new Array(lOne, lTwo);
               //console.log(y[index][4])
               var pLine = new Microsoft.Maps.Polyline(lineV, {
                   strokeColor: new Microsoft.Maps.Color(y[index][5], y[index][6], y[index][7], 100),
                   strokeThickness: 3
               });
               map.entities.push(pLine)
           }
 }

Now when I run this project locally I get something that looks like this:

Soon they won't be random.
Soon they won’t be random.

If you want to give it a shot, check out the instructions/source code on github

Feel free to reach out if you have any questions or would like to work together on a project like this.

Groovy
Groovy

Flask-SocketIO, Background Threads , Jquery, Python Demo

This week I’ve been making progress on the Huggable Cloud Pillow website.

In the process I’ve learned about some sweet stuff you can do with Javascript, Python, and Flask-SocketIO.

The first thing to take note of is Flask.

Flask is the tiny server that allows us to host websites using Python to deliver content to the client. While on the server side you can manage complicated back ends or other processes using Python in conjunction with Flask.

This type of development is nice, because you can start seeing results right away but can take on big projects easily.

It might not be the most robust Framework, but its great for small projects…

If you want to get into Flask Web Development checkout this extensive MVA.

Small and simple, Flask is static on its own. This allows us to present static content, like templates and images easily and deals with input from the user using RESTful calls to receive input. This is great for static things with lots of user actions, but if we want something a bit more dynamic we’re going to need another tool.

In this case I’ve found Flask-SocketIO, similar to Flask-Sockets but with a few key differences highlighted by the creator (Miguel Grinberg) here.

Sockets are great for is providing live information with low latency. Basically, you can get info on the webpage without reloading or waiting for long-polling.

There are lots of ways you can extend this functionality to sharing rooms and providing communication with users and all sorts of fun stuff that is highlighted on GitHub with a great chunk of sample code. The following demo is based off of these samples.

For my project, I need the webpage to regularly check for differences in the state of the cloud and present them to the client, while also changing the image the user sees.

At first I tried to implement it using sockets passing information back and forth, but that wasn’t very stable.

The solution I’ve found, uses a background thread that is constantly running while the Flask-SocketIO Application is running, it provides a loop that I use to constantly check state of our queue.

Let’s break it down…
a. I need my website to display the current state of the cloud.
b. The Flask application can get the state by query our azure queue.
c. Once we determine a change of state we can display that information to the webpage.
d. To display the new state to the webpage we need to use a socket.
e. And pass the msg to be displayed.

This demo intends to break down problem a, c, d, and e.

I’ve created this little guide to help another developer get going quickly, with a nice piece of code available on GitHub.

The five steps to this little demo project are as follows:
1. Install Flask-SocketIO into our Virtual Environment
2. Create our background thread
3. Have it emit the current state to our client
4. Call the background thread when our page render_template’s
5. Have the Javascript Catch the emit and format our HTML.
Celebrate! Its Friday!

1.

Flask-SocketIO is a python package that is available for download using

pip install Flask-SocketIO

Make sure you instal it into a Virtual Environment. Check out my earlier tutorial if you need help with this step.

*Edit Here’s the top part of the “main.py” for reference:

#main.py
from gevent import monkey
monkey.patch_all()

import time
from threading import Thread
from flask import Flask, render_template, session, request
from flask.ext.socketio import SocketIO, emit, join_room, disconnect

app = Flask(__name__)
app.debug = True
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
thread = None

2.

Create our background thread.
You’ll see in the sample code from Flask-SocketIO’s github a simple way to send data to the client regardless of their requests.

For this example we’ll be changing the current time every second and display that to our client.

Background Thread:

def background_stuff():
     """ python code in main.py """
     print 'In background_stuff'
     while True:
         time.sleep(1)
         t = str(time.clock())
         socketio.emit('message', {'data': 'This is data', 'time': t}, namespace='/test')

3

This is the emit statement from above, but is the meat of our interface with SocketIO. Notice how it breaks down…

socketio.emit('message', {'data': 'This is data', 'time': t}, namespace='/test')
socektio.emit('tag', 'data', namespace)

This emit will be sending to the client (Javascript) a message called ‘message’.

When the Javascript catches this message it will be able to pull from the python dicionary msg.data and msg.time to get the result of this package.

4

So how do we call background_stuff?

We can call it wherever we want, but for this simple example we’ll put it right in our ‘/’ base route. So when we navigate to 127.0.0.1:5000 (Local Host) we’ll see the result of our background thread call.

Here’s our route:

@app.route('/')
def index():
    global thread
    if thread is None:
        thread = Thread(target=background_stuff)
        thread.start()
    return render_template('index.html')

Pretty simple… Notice global thread and target=background_stuff

Creating different background threads is a good way to iterate through your changes.

5

Next step is catching this on the other side…

So for our Javascript…

we’ll be using the socket.on method.

socket.on('message', function(msg){
    $('#test').html('&lt;p&gt;' + msg.time + '&lt;/p&gt;');
});

When we receive the emit labeled ‘message’ we’ll pick up the msg from the second parameter and have it be available to our JQuery work.

Here’s the small piece of HTML that we’re selecting to edit.

<body>
    <p id='test'>Hello</p>
</body>

I’ve posted all of this code at github.

Feel free to download it and start working with dynamic sites using SocketIO. Please let me know if you have any questions!

Code for SF's Hackathon gave out awesome tattoos!
Code for SF’s Hackathon gave out awesome tattoos!