There are three parts to this tutorial:
A. Generate an SSH Key
B. Create a VM in Azure that uses the public key
C. Connect to VM using SSH keys
Prerequisites:
Bash
ssh-keygen ($ info ssh-keygen to learn more)
An Azure Subscription
A. Generate an SSH Key
Open bash and enter: $ ssh-keygen -t rsa -b 2048 -C "Ubuntu@azure-server"
Keyname: server-key
Passphrase: somethingMemorable
Copy the contents of server-key.pub $ cat server-key.pub
Should look like this: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMlUr7PCEdBmCVZHG5RqI8i7GgYAzd2G/FZ987XXa63vnqxZmZogVmmXrTnBHeM6oDv7v7g495CiiiINhJbGR4o7t4agiHOM43egDv7BbiViTlfVr3y5AxLUvRwHnC3egl8ABVX1anfXXR73x7IS3YRNWkh6gXtlhImw8UKG04UoZEmWB9BLt53lk/9c3Hxz22YZarzImrpQYy1XEUZ096B9mK/Fe+/McH78ZHUpXEgOZBIDP5KdqPk5XKznpwUDJ4/SPXPEWWCCjQ8gOoTFcFMaiMnXp5o5Udsi/DFO1TS/t8BeCRymkr5tdPvzexjkjkjkjkjkjkjkjkjkjkjkjt Ubuntu@azure-server
Here’s what it looks like for me:
Cool and you’ll also notice that there’s another file in that same directory – server-key $ ls | grep server
Here’s what that looks like for me:
Now that we have our public and private keys let’s get our VM setup.
B. Create a VM in Azure that uses the public key
1. Go to the Azure Portal
2. Select New -> Search: Ubuntu Server
(I’m using 14.04 this time)
3. Make sure you’ve selected Resource Manger and click Create:
4. Now configure the basics per our ssh-keygen parameters
Name: azure-server
VM Disk Type: Up To You
User name: Ubuntu
Authentication type: SSH public key
SSH public key: Paste the results of $ cat server-key.pub
Subscription: Depends how you want to pay for the server
Resource Group: Up to you – I’m going to create a new one so I can quickly delete it.
Location: Up to you
Should look like this:
Then select OK to go to the next section.
5. Choose Virtual Machine Size
I’m going with the smallest VM for testing.
You can also view all different VM sizes to find the right one for your use case.
6. Configure optional Features
Setting the Storage account name to something you’ll remember easily is good.
And if you want to configure ports now you can select Network Security group to allow ports specific traffic.
Here’s what that looks like:
Click okay to continue to the Summary of your VM.
Here’s our summary:
Select okay to start your VM.
7. Wait for it to be ready.
Dashboard will have an icon and you’ll get a notification when its ready:
8. Once ready select on it to see the overview and the IP address.
Should look like this:
Great! We have a VM and its IP address. Lets use our Private SSH key to connect.
C. Connect to VM using SSH Keys
1. Open bash to file location you created the keys in.
Make sure they’re there: $ ls | grep server
2. Enter this command to use SSH to connect: $ ssh -i server-key Ubuntu@52.183.31.11 -v
or more generally $ ssh -i keyname username@ip.address -v
Make sure you’re using server-key and not server-key.pub
Tip: -v is the verbose option. Not necessary, but it helps to see if the key is being accepted
3. Great, now accept the certificate, and enter your memeroablePassphrase
Whole thing should look like this:
There are four steps:
1. Open the appropriate port on Azure
2. Install Mongo
3. Configure Mongo to connect to all external IPs
4. Connect your application
1. Open the appropriate port on Azure
Go to your virtual machine’s landing page and select the resource group in the top left corner:
Resource groups are the way Azure breaks down how our VM interacts with the internet, other VMs, storage, and public/private networks.
To open the port we need to change our Network Security Group, which is represented by the shield and underlined in the screenshot above.
Then, once you’ve selected the Network Security Group, select Settings -> Inbound Security rules
This will allow us to open up our VM to the Public Internet via a port that we’ll connect our client side application to.
You’ll notice that SSH is already included, that’s what we’re using in our terminal. You may also have other ports opened if you’ve followed some of my other posts.
We’re going to create a new Inbound Security Rule called MongoPort where we’ll set the Destination port range to 27017 (the default port for MongoDB)
You can see the configuration pane in the screenshot above identified as item 3.
Once you hit ‘Okay’ or ‘Save’ the port will be opened in a couple seconds.
You should see a notification in the top right corner once completed. Now the port is available to the open internet, but Mongo isn’t installed or configured to be listening at that port. So let’s get to it.
2. Install Mongo
Installing Mongo on Ubuntu is easy and well documented by the MongoDB group. Just enter a few commands and you’ll be up and running in no time.
Make sure you’re following along with the directions for your specific instance of Ubuntu.
To check your version of Ubuntu enter: $ lsb_release -a
Once you’ve followed the directions provided by Mongo, here are some other helpful commands to make sure everything is configured properly.
See a log of what MongoDB is doing: $ cat /var/log/mongodb/mongod.log
See all the processes running on your machine: $ ps -aux
See all the processes with mongo in the listing: $ ps -aux | grep mongo
See the status of the ports: $ netstat -ntlp
Here’s what the bottom of my log file looks like as well as a double checking of the current status of Mongo and what port its running on.
Before we access this DB from our application we need to change one setting so Mongo accepts connections from different IP addresses besides the local IP address.
3. Configure Mongo to connect to all external IPs
Before we go on, I need to make it clear that this is not a best practice, and no user data should be stored on a VM with an open port like this. But for development and practice purposes we’re going to make it easy to connect.
If you’re going to go into production, please refer to MongoDBs security checklist HERE.
Okay, having said that.
Open up the mongod.conf file using nano:
$ sudo nano /etc/mongod.conf
And uncomment the binding IP by adding an octothorpe at the beginning of the line like so:
Save and close the file. ( Ctrl+X -> y -> enter )
Alright, now MongoDB should be ready to be connected to!
4. Try it out and Connect your application
Open the Mongo Command line by typing mongo in your ssh terminal.
And show your DBs and collections show dbs show collections
The connection string for the VM is the IP address and the DB you’d like to write too. To connect to the test database on the is vm the connecion string to use looks something like this:
mongodb://40.83.182.555:27017/test
In the next blog I’ll show you how to connect your Azure Web App to Mongo on this VM!
Before we start make sure you can ssh into your machine and run
$sudo apt-get update
It’s a four step process:
1. Open the appropriate port on Azure
2. Install pip, virtualenv, virtualenvwrapper, and flask
3. Write our code and run it
4. Keep it running with Gunicorn
1. Open the appropriate ports on Azure
Go to you virtual machines landing:
Then select the resource group in the top left corner:
Resource groups are the way Azure breaks down how our VM interacts with the internet, other vms, storage, and public/private networks.
Top open the port we need to change our network security group, which is represented by the shield. (Underlined in the screenshot above)
Then select settings -> Inbound Security Rules:
This will allos us to open up our VM to the public internet so we can visit what’s presented at the port like a regular website.
You should see SSH already included, that’s the port we’re using in our ssh client/terminal.
We’re now going to add two new Inbound Security Rules one called FlaskPort where we’ll set the destination port range to 5000 and use for debugging. The second will be called FlaskProduction that we’ll use to deploy our complete app.
Here’s the configuration panel for FlaskPort:
Press okay to accept the settings.
And the other panel for FlaskProduction:
Again press okay to accept the settings.
Notice how the ‘Source Port Range’ is ‘*’ that just means that we’ll accept connections from the port of any machine. This tripped me up the first time.
In a couple seconds the port will be open we’ll be ready to visit it, but nothing will be there because we haven’t create an application server.
To do that we’ll install the basics.
2. Install pip, virtualenv, virtualenvwrapper, and flask
To use Python effectively we utilize virtual environments to help keep our various python project and required libraries in order.
If you get lost in these steps or want more context Gerhard Burger provides the same setup on a very helpful post on askubuntu.
First we install pip: $ sudo apt-get install python-pip
Second we install virtualenv and virtualenvwrapper
Now for our photo we’re going to download an image into our static file using curl.
$ cd static
$ curl 'http://timmyreilly.azurewebsites.net/wp-content/uploads/2015/12/Snapchat-1802119159214415224.jpg' -o 'photo.jpg'
Cool! We have an app!
To run it simply enter: $ python app.py
Visit from a browser!
Type in the IP address of your virtualmachine with a colon at the end followed by the port.
Mine looks like:
http://138.91.154.193:5000/
And this is what I see!
Neato! But when we close the terminal we lose the application… Hmmmm let’s fix that!
If you’d like to learn more about this Open Source HTTP Server Software checkout their website.
Second, we start nginx. $ sudo /etc/init.d/nginx start
And begin configuration for our project
We’re naming our nginx configuration the same as the parent directly of our app.py file in this test case.
Here are the commands:
$ sudo rm /etc/nginx/sites-enabled/default
$ sudo touch /etc/nginx/sites-available/Photo-App
$ sudo ln -s /etc/nginx/sites-available/Photo-App /etc/nginx/sites-enabled/Photo-App
Now we add the config settings for our app.
$ sudo nano /etc/nginx/sites-enabled/Photo-App
Here’s the raw text:
server {
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /static {
alias /Photo-App/static/;
}
}
Then restart the server.
$ sudo /etc/init.d/nginx restart
And navigate to your Python PhotoApp project, and start it using Gunicorn: $sudo gunicorn app:app -b 0.0.0.0:8000 --reload
You’re using gunicorn to start app hosted at 0.0.0.0:8000 with the reload tag configured.
The reload will look for changes in your code and reload the server everytime you change any server side stuff. It won’t auto-reload for HTML changes, but will reload them once you make a change to the python code.
Now try navigating to: http://138.91.154.193:8000/ or whatever your IP address is.
You can now close you’re ssh terminal and it will continue to run.
To stop it we have to kill the actual thread it’s running on. $ pkill gunicorn
Like I said at the beginning of this step this was adapted from the realpython.org and they have some awesome next steps available for git deployment + automating the whole process.
When I search:
Node JS Server Azure, Ubuntu, JavaScript, Mongo, Postgres, Flask, VM
I turn up with all sorts of unhelpful results.
So I dedicated a couple days to creating a couple guides for common Cloud Stacks on Azure VMs to make it as simple as possible to start deploying your code to the cloud.
This is the introduction and at the bottom of this blog post you’ll see other workflows fill in.
So, Here’s a guide to deploying an Ubuntu VM on Azure:
1. Gather Materials
2. Create VM
3. Check VM using SSH
There are lots of configurable deployments available if you feel like exploring.
Then select Create, but make sure the deployment model is Resource Manager as its more future ready then the classic model:
We’ll then get to the basic configuration tab, fill out the info and pick a User name and Password that you’ll remember because you’ll need it later!
If you’re not familiar with Resource Groups check out THIS ARTICLE
I’ve named my resource group: ResourceGroupOne
Hit Okay to go to the next configuration pane
Select the Size of your VM. To see all the options select ‘View All’
We’re going to go with the cheapest option A1 Standard:
Hit Okay to take us to our final configuration Pane, “Settings”.
There are a number of different settings presented here.
First up is Storage:
This will configure what we want to name the storage account for our vm. I’ve changed mine to ‘resourcegrouponestorage’, but I could have selected any of my previous storage account in the same region, in this case westus.
Second is Network:
We can configure a Virtual Network to allow our virtual machines to connect to other resource on our network by default. We can also change this later. So in this case I’m creating the default virtual network.
Again, I could have selected a previously created Virtual Network Called ‘Databases’ which is in the same region.
Third is Extensions:
We won’t add any extensions
Fourth is Monitoring:
Which we’ll disable for simplicity sake, but is a very powerful tool one you start needing to make scaling decisions.
Fifth and finally is Availability:
We won’t use an availability set, until we need to scale out our app.
Here’s what the lower portion of our settings pane looks like:
And we’ll select OK to finish with our settings. This will take us to the summary page so we can do a one more check on our machine, don’t get to anxious about making mistakes because we can always tear this one down and spin up another if we messed something up!
Hit Okay one last time!
You’ll then be taken to your dashboard where you’ll see a nice loading tile:
It’ll take ~5 minutes to spin up and then we’ll be ready to take on the world!
Once ready it’ll look like this:
Click the tile to hit the landing page for our VM:
See that public IP address?
We’ll use that to SSH into our machine.
In my case: 13.88.180.170 !
3. Check VM using SSH
Let’s SSH into our box.
Pull out your preferred SSH client. Here’s bash on Windows and Putty Side by Side:
Notice ‘Timothy’ Triple underlined?
That’s the User Name we set during basic configuration and is paired with the password that we also set in Azure.
When you connect you might have to accept the ras2key fingerprint. It’ll look like this when using putty. Or it’ll be in the terminal using bash. Type ‘yes’ or Select Yes to continue.
Then type in your password and marvel and your creation:
Let’s test our vm by installing updates! Yay Updates!
Now that bash is on Windows, I wanted to try and make all the other guides I’d writen for Python on Windows irrelevant.
So here’s how to setup an effective environment for Python on Ubuntu on Windows.
1. Install Bash on Windows
2. Check for updates
3. Check out the REPL
4. Install Pip
5. Install VirtualEnv
6. Install VirtualEnvWrapper
7. Create your first virtualenv
8. Configure bashrc to keep it working
9. Install some packages
10. Test Flask
Now we’ll install Pip: sudo apt-get install python-pip
If you have permission issues try starting an elevated prompt: $ sudo -i $ apt-get install python-pip $ exit
Use exit to return to the regular prompt.
Should look like this:
Yay we’ve got pip!
Try pip list to see what comes standard.
5. Install VirtualEnv
Now we’re basically following along with the guide presented at hitchhikersguidetopython.com
Again, you might need to start an elevated prompt to install virtualenv.
$ sudo -i
$ pip install virtualenv
$ exit
$ cd my_project_folder
$ virtualenv venv
Then to use the VirtualEnvironment
$ source venv/bin/activate
You should now see a little (venv) before your prompt.
Like this:
Now you’ve created a virtualenv inside of your my_project_folder directory. Which is cool, but can be confusing with git, sharing code, and testing package versions.
So we use VirtualEnvWrapper to keep our virtualenvs in the same place.
Before we move on make sure you deactivate your env deactivate
Here’s an example of what it looks like to remove our venv directory and instead use venvv which will be stored in the directory underlined in red.
8. Configure bashrc to keep it working
This might not happen to you, but when I opened a new bash terminal I had to re-source my virtualenvwrapper.sh and WORKON_HOME.
So instead I added those lines to my bashrc script.
$ sudo nano ~/.bashrc
-- type in password --
This is what it looks like in nano for me.
Ctrl+X to exit and y-enter to save.
Then either:
Source ~/.bashrc
Or start a new command prompt->bash and try “workon” or “lsvirtualenv”
See the next image for a simple workflow.
9. Install some packages
Now lets install ‘requests’ into our newly created virtualenv:
Isn’t that nice!
10. Test Flask
Finally we’re going to test this with flask.
First we install the required files using pip into our activated ‘venv’
Then runserver -> navigate to the designated address -> and see our site.
Here’s what it looks like:
Have fun building with Python, on Ubuntu, on Windows!
Let’s say you’re at your first hackathon. You have an idea you want to develop quickly and manage and deploy remotely so you’re ready to scale, or aggregate data cross platform. Buzzwords.
I’ve seen it many times in my last 3 months of attending hackathons.
Students want to get started quickly without wasting time worrying about deployment and compatibility across devices this setup will also prepare them to work in a team because it’ll allow deployment from Git. And these principles carry down into innovation initiatives happening at companies around the world one solution is MEAN.
Let’s say you’re new to web development and you’re learning node but don’t want to gunk up you machine with installations and moving files around before you have a grasp of what really matters.
That’s where MEAN in the Cloud comes in.
So what we’re going to do is install the MEAN stack on a cloud server so we can mess with code, learn, restart, and more without worrying about ruining our machine.
1. Login to Azure (Create an account if you don’t have one already)
Using azure isn’t required any cloud hosting that provides and Ubuntu distro should work.
2. In the bottom left click “New”
3. Next to Compute Virtual Machine
4. Select Quick Create
5. Give it a name (eg. meanincloud)
6. In IMAGE dropdown select ‘Ubuntu 14.04 LTS’
7. Leave the size at the default – you can always scale later.
8. Provide a password and confirm it
9. Select the region nearest to you.
10. Hold tight, the cloud is provisioning resources and configuring your machine.
11. After status reads running double click on ‘meanincloud’
12. Then select ‘DASHBOARD’
13. Scroll down until you see SSH details on the left.
14. take note of those values it should be something like: ‘youservername.cloudapp.net: 22″
This is your server host name and the SSH port we’ll be using this in the next section to access your remote machine
We now have a Linux VM with the Ubuntu distribution. This is our base on which we’ll build the MEAN stack. Second: Get into Server
For this part we’ll need an SSH client. Putty is what I’ve always used and works well for this instance. Download page
1. in the box labeled Host Name (or IP address) paste ‘yourservername.cloudapp.net’ and leave the port at 22.
2. You’ll likely receive a PuTTY security alert. That’s cool for now, select Yes.
3. If you’re using azure you’re login will be ‘azureuser’
4. And you’re password will be that same that you provided at ‘7.’ in the previous section.
5. You’re now at the Ubuntu terminal (Like cmd). You’ll start at home, but you can get to root by entering: ‘cd /’ and back to home with ‘cd ~’
The Mongo installation instructions can be found here if you’re curious. But we’ll run through that here.
Note: All lines in bold will be run in the terminal. Copy and right click to paste into your putty shell. Then press enter to run the command.
1. Import the public key used by the package management system: ~$ sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
2. Create a list file for MongoDB ~$ echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
No result will display.
3. Reload local package database. ~$ sudo apt-get update
You’ll get about 15 seconds of output.
4. Install the MongoDB packages – This will download the latest stable version, but you can be specific if you liked an older version. Read more into it here. ~$ sudo apt-get install -y mongodb-org
You’ll get about 30 seconds of output.
5. Reload local package database ~$ sudo apt-get update
Alright! Mongo is installed, here’s a great tutorial if you want to get a better understanding of how Mongo works. Fourth: Install Node
Node is the web app framework that you’ll be building you application with. Although its not spelled MNEA or NMEA, this more accurately represents the priority of these tools.
1. Install Node: ~$ sudo apt-get install nodejs
2. Confirm install with ‘Y’
About 20 seconds of output.
3. Node is now installed you run node by entering nodejs this will take you to the Node REPL.
4. Try it out ~$ nodejs
5. In the REPL you can run javascript > console.log('hello world');
6. ctrl+c twice will exit the repl. Fifth: Create your first Node app
We now have everything we technically need to start development with Node, the first thing we’ll do is create a hello world app to begin putting the pieces together.
1. First install emacs24 an inline text editor for linux that will allow us to make changes to code inside our terminal. sudo apt-get install emacs24
a ‘Y’ will be required to finish
And we’ll get about a minute of output.
Now we can edit code right here! No need to worry about messing up your own file system.
2. Make sure we’re updated. ~$ sudo apt-get update
3. make a directory to hold our demo ~$ mkdir mean-practice
4. ‘ls’ will show us our newly created directory
5. Change into that directory ~$ cd mean-practice
6. now we’ll create our first file inside of that directory. /mean-practice$ emacs package.json
This will open a new emacs view that will allow us to make changes, save ctrl+x+s and close ctrl+x+c.
7. This is our basic manifest file. type this into it.
{
"name" : "mean-practice",
"version": "0.0.1"
}
This is the minimum. As you learn more about node you’ll see what else will go in there.
ctrl+x+s to save and ctrl+x+c to exit.
8. Let’s create our main file, basically our entry point for the application, and where the server will begin operating. /mean-practice$ emacs server.js
9. put this into that file: console.log("Hello World with NODE!");
Though possibly over-zealous this is our first node app!
ctrl+x+s to save and ctrl+x+c to close.
10. Type ‘ls’ to see what’s in our mean-practice directory.
Not a ton, but everything we need to start our application.
11. Let’s run our application: /mean-practice$ nodejs server
Sweet! All we’ll get as output is “Hello World with NODE!” but that’s better than a kick to the face.
12. To stop the server press ctrl+c Sixth: Install secret N, NPM
Node strengths and rapid growth are due in large par to the awesome npm. A package manager which will allow you to import a whole host of clever tools, apps, widgets, and formatting helpers to make your node development as easy as possible. So let’s install that now.
1. First install NPM. /mean-practice$ sudo apt-get install npm
You’ll need another ‘Y’
and expect about 45 seconds of output.
2. Update again /mean-practice$ sudo apt-get update
another 20 seconds of output Seventh: Now we’ll install Express and Angular using NPM.
Now that NPM is installed we can add our final packages!
1. Install express /mean-practice$ sudo npm install express --save
express is in the npm system and –save will edit our package.json file to include the express module. Click this link to find out more about express.
Expect about 15 seconds of output.
2. Enter ls to see our new directory ‘node_modules’ where our node modules are being stored.
3. Open package.json to see the changes in our manifest. /mean-practice$ emacs package.json
You’ll now see under dependencies “express”
Yay! Now express is installed.
ctrl+x+s to save and ctrl+x+c to exit.
4. Now lets install angular. /mean-practice$ sudo npm install angular
Now the angular module is part of our application! Click on the link above to learn more about Angular.
That’s it! Now you have a MEAN stack ready for development!
Let’s quickly build a simple server, configure an endpoint in azure and become a client with our browser. Eigth: Get a tiny server running
1. Open our server.js /mean-practice$ emacs server.js
2. Replace our hello world statement with:
var express = require('express');
var app = express();
var port = 3000;
app.get('/', function(req, res) {
res.send('hello from afar');
});
app.listen(port, function(){
console.log("Listening at port: " + port);
})
4. Now our server.js is actually a server but our virtual machine doesn’t know that. Go back to your azure portal.
5. Select Endpoints at the top between monitor and configure.
6. At the bottom select ‘ADD’
7. Select ‘ADD A STAND-ALONE ENDPOINT’
8. Press the arrow
9. Under the NAME dropdown select HTTP
10. Leave protocol at TCP
11. Leave public port at 80
12. Change the private port to ‘3000’
We’re mapping our endpoints improperly for this demo to keep things simple. As you learn more about web development you’ll want to change your endpoints to accurately reflect what’s happening in your server.
13. Hit the checkmark.
14. Back in your putty shell run the server! /mean-practice$ nodejs server.js
15. Open a new tab in your browser and navigate to: ‘yourservername.cloudapp.net’
16. CHECK IT OUT! You’ve got a website running node on a virtual machine! Throw your hands in the air! You’re ready to hack and learn.
