Connecting your Azure Web App to a Database using Application Settings

You just built an application that connects with a Database on your local machine, but now you want to share it with the world. But you don’t want to share everything about your application with the world.

Azure web apps supports a number of different web platforms, but to effectively build while maintaining code on GitHub we need to keep configuration details of our app private including the connection string for our database.

In this case we’re going to be using a Node JS application connected to a Mongo Database hosted on a Virtual Machine.

Before we get started…
Do you have a MongoDB up and running?
Nope. – Read this post!
Yep! – Great keep going!

For reference, I’m starting with a chat application originally written by fellow evangelist Rami Sayar for an excellent Node Course he taught recently.

Prereqs:
Git
Azure Subscription
Connection string for a DB in Mongo – mine looks like this:
mongodb://40.83.182.555:27017/test

The Steps:
1. Clone or fork the repository
2. Create an App Service
3. Connect App Service to Repository
4. Add app settings in Azure Portal
5. Try it out!

1. Clone or fork the repository

The project I’m using for this demonstration is found here: Github
Now clone the project to your local machine, or fork it in GitHub.

Take a quick look at these files:
– package.json : checkout out the dependencies of the project, notice how we’re using the official mongodb driver
– mongoClientTest.js and dbTest.js : provide other examples/test files to run, just substitute your ip address of where mongo is being hosted to make sure you’re able to connect.
– app.js : this is the main file that will be run by Azure Web Apps (app.js or server.js will automatically be started by the cloud when deployed). Also, in app.js you’ll see we set ‘db’ using process.env app.set('db', process.env.DB ); – we’ll be using that in step 4.

Once comfortable with the files locally or in GitHub we’re ready to spin up our App Service in Azure.

2. Create an App Service

App Service is a PaaS service offered by Azure to make deployment of website/web services simple and easy to maintain, develop against, and scale.

To begin select the ‘Plus’ button in the upper left corner and select web+mobile, then select Web App:
selectnewwebapp

Now configure your web app:
configurewebapp

You may have to create a app service plan and resource group if you haven’t created one yet, for more info on this portion of the architecture click here and here

You’ll see this icon on your dashboard while the resource is spinning up:
deploying

Once running the portal should automatically take you to that resource, if not you’ll see it on your dashboard:
mongoappondashboard

Here’s the overview pane and the items we’ll be using in the rest of this tutorial highlighted:
webappoverview

3. Connect App Service to Repository

Now that we have an app service, let’s connect it to our repository.
There are a number of options for deployment.

In this case we’re going to deploy from GitHub, but we could have also used a local repository by adding Azure as a remote repo.

Azure will detect changes made to the repository on GitHub and redeploy the project. This makes deployments during hackathons super easy.

Select Deployment options from the overview pane -> Choose Source -> GitHub:
deployfromgithub

Note: You can also select Local Git Repository if you’ve liked to keep your code off of GitHub and add Azure as a remote repository git add remote azure "http://git.asdfgafdgasdfgasdf" – will be displayed in overview portal once initialized.

You’ll then be asked to login to GitHub so it can access the list of your repos.
Select your forked repo or the cloned repo you’ve committed to GitHub:
authrorize-andselectyourproject

Once selected you’ll see a notification in the top right corner with a loading bar for the deployment:
waitingfordeployment

Once that’s finished, you have code ready to roll in the cloud.
Azure has automatically installed the packages listed in your package.json and found the file named server.js to deploy. Next let’s configure it to talk to Mongo!

4. Add App Settings in Azure Portal

Now we have a project connected and deployed on Azure, but it won’t store the data anywhere, because a connection string hasn’t been included in the environment variables (aka application settings).

To add our monogodb to the project we just need to include the variable name and the connection string.
In our project, as noted before, we set db using process.env.DB

Select application settings and scroll down to until you see app settings then enter into the two empty fields “DB” and the connection string mentioned earlier: mongodb://40.83.182.555:27017/test
Like this:
appsettingpane

Then hit save!

Go back to your website and try entering some chatter!

Try it out!

If you visited the page it should look something like this:
finishedproduct

In the middle of the above screen shot you can see a terminal with a connection to the VM and same data in the MongoDB.

If you’re running into issues try connecting using dbTest.js or mongoClientTest.js after you add the specific IP for your VM hosting mongo.

Hope this made it super simple to connect your application to a database using Azure Web Apps.
Please let me know if you run into any issues in the comments below.

When Brett Stateham comes to town...
When Brett Stateham comes to town…

Running Flask on Ubuntu VM

So you have a VM in Azure and want to put it to good use?

No.
Let’s set one up!

Yes.
Great!

Before we start make sure you can ssh into your machine and run

$sudo apt-get update

sudoaptgetupdate

It’s a four step process:
1. Open the appropriate port on Azure
2. Install pip, virtualenv, virtualenvwrapper, and flask
3. Write our code and run it
4. Keep it running with Gunicorn

1. Open the appropriate ports on Azure
Go to you virtual machines landing:
vmnumberonelandingpage

Then select the resource group in the top left corner:
resourcegroup

Resource groups are the way Azure breaks down how our VM interacts with the internet, other vms, storage, and public/private networks.

Top open the port we need to change our network security group, which is represented by the shield. (Underlined in the screenshot above)

Then select settings -> Inbound Security Rules:
networksecuritygroupsettings

This will allos us to open up our VM to the public internet so we can visit what’s presented at the port like a regular website.

You should see SSH already included, that’s the port we’re using in our ssh client/terminal.
defaultssh

We’re now going to add two new Inbound Security Rules one called FlaskPort where we’ll set the destination port range to 5000 and use for debugging. The second will be called FlaskProduction that we’ll use to deploy our complete app.
Here’s the configuration panel for FlaskPort:
FlaskPort
Press okay to accept the settings.

And the other panel for FlaskProduction:
flaskproduction
Again press okay to accept the settings.

Notice how the ‘Source Port Range’ is ‘*’ that just means that we’ll accept connections from the port of any machine. This tripped me up the first time.

In a couple seconds the port will be open we’ll be ready to visit it, but nothing will be there because we haven’t create an application server.

To do that we’ll install the basics.

2. Install pip, virtualenv, virtualenvwrapper, and flask

To use Python effectively we utilize virtual environments to help keep our various python project and required libraries in order.

If you get lost in these steps or want more context Gerhard Burger provides the same setup on a very helpful post on askubuntu.

First we install pip:
$ sudo apt-get install python-pip

Second we install virtualenv and virtualenvwrapper


$ sudo pip install virtualenv
$ sudo pip install virtualenvwrapper

Third we configure virtualenv and virtualenvwrapper

Create a WORKON_HOME string which will contain the directory for our virtual environments. We’ll name it vitualenvs

$ export WORKON_HOME=~/.virtualenvs

Now we’ll create this directory.

$ mkdir $WORKON_HOME

And add this to our bashrc file so this variable is defined automatically every time we hit the terminal.

$ echo "export WORKON_HOME=$WORKON_HOME" >> ~/.bashrc

Then we’ll setup virtualenvwrapper by importing its functions with bashrc.

$ echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc

You can see the additions to our bashrc file by opening it with nano. Scrolling down to the bottom you should see two lines like this:
bashrcconfigured

Then implement your changes.

$ source ~/.bashrc

Here’s what all that looks like all together:
configurepipandvirtualenvwrapper

Fourth, let’s create our first virtualenvironment

$ mkvirtualenv venv

And take a look at the currently installed packages
$ pip list

Like so:
firstvenv

Now we can install all of the python packages we want without risk of needing to reinstall python!

Fifth, install flask:


$ pip install flask
$ pip list

pipinstallflask
currentpackages
3. Write our code and run it!
Our first app is a simple site that shares an image.

We’re going to create a folder called Photo-App that contains two folders and an app.py that will serve our clients.

To change to our home directory:
$ cd ~
And create our new folder:
$ mkdir Photo-App

Then create a static and templates folder inside of our app.

$ sudo mkdir templtes
$ sudo mkdir static

mkPhotoApp
Then create our app.py which will be our python flask server code.

$ sudo nano app.py

Here’s the code:

from flask import Flask, render_template
app = Flask(__name__)

@app.route("/")
def index():
        return render_template("index.html")

if __name__ == "__main__":
        app.run(host='0.0.0.0', debug=True)

And what it looks like in nano:
nanoapppy

Then we need to add our first template:
$ sudo nano templates/index.html

<h1>Wazzup Dog</h1>
<img style="max-width:100%;" src="{{ url_for('static', filename='photo.jpg') }}">

And what it looks like in nano:
indexinnano

Now for our photo we’re going to download an image into our static file using curl.

$ cd static
$ curl 'http://timmyreilly.azurewebsites.net/wp-content/uploads/2015/12/Snapchat-1802119159214415224.jpg' -o 'photo.jpg'

Cool! We have an app!

To run it simply enter:
$ python app.py

Visit from a browser!

Type in the IP address of your virtualmachine with a colon at the end followed by the port.
Mine looks like:
http://138.91.154.193:5000/

And this is what I see!
wazzupdogimdex

Neato! But when we close the terminal we lose the application… Hmmmm let’s fix that!

4. Keep it running with Gunicorn
Real Python provided the bulk of this portion so if you get lost or checkout their site, plus they have info about setting up continuous deployment!

First, we install dependencies
$ sudo apt-get install -y nginx gunicorn

Now that our app is ready, let’s configure nginx.

If you’d like to learn more about this Open Source HTTP Server Software checkout their website.

Second, we start nginx.
$ sudo /etc/init.d/nginx start

And begin configuration for our project
We’re naming our nginx configuration the same as the parent directly of our app.py file in this test case.
Here are the commands:

$ sudo rm /etc/nginx/sites-enabled/default
$ sudo touch /etc/nginx/sites-available/Photo-App
$ sudo ln -s /etc/nginx/sites-available/Photo-App /etc/nginx/sites-enabled/Photo-App

Now we add the config settings for our app.

$ sudo nano /etc/nginx/sites-enabled/Photo-App
nginxconfigphotoapp
Here’s the raw text:

server {
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /static {
alias /Photo-App/static/;
}
}

Then restart the server.

$ sudo /etc/init.d/nginx restart

And navigate to your Python PhotoApp project, and start it using Gunicorn:
$sudo gunicorn app:app -b 0.0.0.0:8000 --reload

You’re using gunicorn to start app hosted at 0.0.0.0:8000 with the reload tag configured.
The reload will look for changes in your code and reload the server everytime you change any server side stuff. It won’t auto-reload for HTML changes, but will reload them once you make a change to the python code.

Now try navigating to: http://138.91.154.193:8000/ or whatever your IP address is.

You can now close you’re ssh terminal and it will continue to run.

To stop it we have to kill the actual thread it’s running on.
$ pkill gunicorn

Like I said at the beginning of this step this was adapted from the realpython.org and they have some awesome next steps available for git deployment + automating the whole process.

The code for the app can be found on GitHub here: https://github.com/timmyreilly/Flask-For-UbuntuBlog

Happy Hacking!

This was a nice reminder in the busy airport.
This was a nice reminder in the busy airport.

Flask-SocketIO, Background Threads , Jquery, Python Demo

This week I’ve been making progress on the Huggable Cloud Pillow website.

In the process I’ve learned about some sweet stuff you can do with Javascript, Python, and Flask-SocketIO.

The first thing to take note of is Flask.

Flask is the tiny server that allows us to host websites using Python to deliver content to the client. While on the server side you can manage complicated back ends or other processes using Python in conjunction with Flask.

This type of development is nice, because you can start seeing results right away but can take on big projects easily.

It might not be the most robust Framework, but its great for small projects…

If you want to get into Flask Web Development checkout this extensive MVA.

Small and simple, Flask is static on its own. This allows us to present static content, like templates and images easily and deals with input from the user using RESTful calls to receive input. This is great for static things with lots of user actions, but if we want something a bit more dynamic we’re going to need another tool.

In this case I’ve found Flask-SocketIO, similar to Flask-Sockets but with a few key differences highlighted by the creator (Miguel Grinberg) here.

Sockets are great for is providing live information with low latency. Basically, you can get info on the webpage without reloading or waiting for long-polling.

There are lots of ways you can extend this functionality to sharing rooms and providing communication with users and all sorts of fun stuff that is highlighted on GitHub with a great chunk of sample code. The following demo is based off of these samples.

For my project, I need the webpage to regularly check for differences in the state of the cloud and present them to the client, while also changing the image the user sees.

At first I tried to implement it using sockets passing information back and forth, but that wasn’t very stable.

The solution I’ve found, uses a background thread that is constantly running while the Flask-SocketIO Application is running, it provides a loop that I use to constantly check state of our queue.

Let’s break it down…
a. I need my website to display the current state of the cloud.
b. The Flask application can get the state by query our azure queue.
c. Once we determine a change of state we can display that information to the webpage.
d. To display the new state to the webpage we need to use a socket.
e. And pass the msg to be displayed.

This demo intends to break down problem a, c, d, and e.

I’ve created this little guide to help another developer get going quickly, with a nice piece of code available on GitHub.

The five steps to this little demo project are as follows:
1. Install Flask-SocketIO into our Virtual Environment
2. Create our background thread
3. Have it emit the current state to our client
4. Call the background thread when our page render_template’s
5. Have the Javascript Catch the emit and format our HTML.
Celebrate! Its Friday!

1.

Flask-SocketIO is a python package that is available for download using

pip install Flask-SocketIO

Make sure you instal it into a Virtual Environment. Check out my earlier tutorial if you need help with this step.

*Edit Here’s the top part of the “main.py” for reference:

#main.py
from gevent import monkey
monkey.patch_all()

import time
from threading import Thread
from flask import Flask, render_template, session, request
from flask.ext.socketio import SocketIO, emit, join_room, disconnect

app = Flask(__name__)
app.debug = True
app.config['SECRET_KEY'] = 'secret!'
socketio = SocketIO(app)
thread = None

2.

Create our background thread.
You’ll see in the sample code from Flask-SocketIO’s github a simple way to send data to the client regardless of their requests.

For this example we’ll be changing the current time every second and display that to our client.

Background Thread:

def background_stuff():
     """ python code in main.py """
     print 'In background_stuff'
     while True:
         time.sleep(1)
         t = str(time.clock())
         socketio.emit('message', {'data': 'This is data', 'time': t}, namespace='/test')

3

This is the emit statement from above, but is the meat of our interface with SocketIO. Notice how it breaks down…

socketio.emit('message', {'data': 'This is data', 'time': t}, namespace='/test')
socektio.emit('tag', 'data', namespace)

This emit will be sending to the client (Javascript) a message called ‘message’.

When the Javascript catches this message it will be able to pull from the python dicionary msg.data and msg.time to get the result of this package.

4

So how do we call background_stuff?

We can call it wherever we want, but for this simple example we’ll put it right in our ‘/’ base route. So when we navigate to 127.0.0.1:5000 (Local Host) we’ll see the result of our background thread call.

Here’s our route:

@app.route('/')
def index():
    global thread
    if thread is None:
        thread = Thread(target=background_stuff)
        thread.start()
    return render_template('index.html')

Pretty simple… Notice global thread and target=background_stuff

Creating different background threads is a good way to iterate through your changes.

5

Next step is catching this on the other side…

So for our Javascript…

we’ll be using the socket.on method.

socket.on('message', function(msg){
    $('#test').html('<p>' + msg.time + '</p>');
});

When we receive the emit labeled ‘message’ we’ll pick up the msg from the second parameter and have it be available to our JQuery work.

Here’s the small piece of HTML that we’re selecting to edit.

<body>
    <p id='test'>Hello</p>
</body>

I’ve posted all of this code at github.

Feel free to download it and start working with dynamic sites using SocketIO. Please let me know if you have any questions!

Code for SF's Hackathon gave out awesome tattoos!
Code for SF’s Hackathon gave out awesome tattoos!