September 4, 2016

Deploying a RoR App to Heroku

Deploying a Ruby on Rails application to Heroku is simple and straight forward. 

But, what is Heroku?  Heroku is a cloud platform that lets companies build, deliver, monitor and scale apps.  This cloud platform offers tools to allow us to handle deployment, performance, monitoring, everything to do with infrastructure so we can focus on building our applications.

I am using Heroku as a learning laboratory to deploy to "production" my RoR progress Blog - Personal Site app.  So, here are some of the pointers that I need to keep in mind while deploying to Heroku.

Preparing My RoR for deployment

It is recommended to deploy to "production" early in the development process and as often as possible.  In this learning exercise I am using an "agile" development approach,  therefore, as soon as a sprint finishes a new feature, the production product is updated right away.

My RoR application requires a database.  In my development environment, my local Linux Ubuntu, the database is SQLite, which is very convenient, easy to use and it was deployed with Rails.  However, Heroku uses PostgreSQL database which means that if I deploy my app "as is" then Heroku issues an error on compilation.  To avoid this error I need to create a production group in the gem file and add to it a PostgreeSQL gem "pg" to allow Rails to talk to Postgre.

group : production do
   gem 'pg'
   gem 'rails_12factor'
end
The production group also needs 'rails_12factor' to allow static asset serving and loggin on Heroku.  Adding the production group to the gem file does not affect my local development using SQLite  because bundle install can be run with the flag '--without production', like this

$ bundle install --without production

Publish to Heroku

Publishing to the cloud from my local git repo I need to make sure that it is updated with all recent changes, therefore after changing the gem file I need to commit, then push from git to heroku.  Needless to say that the Heroku command line tool needs to be locally installed and that we need to login to Heroku.

To publish the master brach to Heroku run this command:

$ git push heroku master

Of course I could push any branch but at this time, the master branch is the one with all features.  If the push yields no errors then I am in luck!  In the case that there are errors, Heroku stores them in the logs file.

$ heroku logs

Fix the errors and continue.  In my case, my 'novice' error was that I neglected to create a production group with Postgre gem in it.  I still need the sqlite gem in the gem file to be able to use this SQLite during development.  Therefore, a development group was created containing the sqlite gem.  The reviewed development and production groups look like this:

group :development do
  gem 'sqlite3'
end
group :production do
  gem 'pg'
  gem 'rails_12factor'
end

At this time Heroku does not know anything about our Object Model and the database has not been created yet, therefore, if I run the app after pushing then my app does not run, but if I run it any way, Heroku issues an app error page and the reason for the failure is written to the log and I could view it with "heroku logs" .

To create my database I need to run the rake db migration command in the Heroku command line tool.  As a matter of fact, every time my Active Record model changes I need to run this command in the Heroku command line tool:

$ heroku run rake db:migrate

Now the app can be loaded in the browser using the command:

$ heroku open

There is no data however because that db has not been seeded, but this is a different story.  At this time everything works and I can move to other things...  mainly keep learning RoR!

Other Useful Heroku Commands

There are other commands that I needed to use and they are:

$ heroku login 
$ heroku create 
$ heroku restart

The above command are self explanatory.

Stopping and Starting the App Running on Heroku

$ heroku ps:scale web=0

Setting "web=0" scales down the app to 0, and the result is the stopped app.

$ heroku ps:scale web=1

To start it up again scale the app to "web=1".

June 29, 2016

Nodebot Blink Modified

Strobing Led and Piezo Song

This is one of the exercises for the Port Coquitlam Node and Nodebots Meetup.  This challenge was posted in order to do a quick piezo demo. 



'use strict';

var five = require('johnny-five'),    
    board = new five.Board();
  
board.on('ready', function(){
    console.log('Board is ready');
  
    led.strobe(10000, function(){                     

      var led = new five.Led(13);
      var piezo = new five.Piezo(3);

        piezo.play({           
            song: "C D F D A - A A A A G G G G - - C D F D G - G G G G F F F F - -",
            beats: 1 / 4,
            tempo: 100
        });       
    });   
});
console.log('Waiting for board...');

Hey, there is a bug in your code!

You got it!  There is a bug.  Can you spot it?  After loading the firmatta to the microcontroller and running the above program we saw in the console something like this:




The board is telling us that "pin 3 is already in use".  Hum, it is only instantiated once...  or is it? Strobe is an interval operation and every time the callback function is executed we try to create a new piezo in the scope of that function using pin 3.  During the first strobe interval there is no error but the second time the strobe interval passes then we get the warning because pin 3 is already taken.

To fix the bug we need to take the line where the new piezo is declared and place it before led.strobe()

Fixed!

'use strict';

var five = require('johnny-five'),   
    board = new five.Board();
    
board.on('ready', function(){
    console.log('Board is ready');
    
    var led = new five.Led(13);
    var piezo = new five.Piezo(3);        

    led.strobe(10000, function(){                        
        piezo.play({            
            song: "C D F D A - A A A A G G G G - - C D F D G - G G G G F F F F - -",
            beats: 1 / 4,
            tempo: 100
        });        
    });    
}); 
console.log('Waiting for board...');  



June 5, 2016

NodeBots Day-1: Getting the Environment Ready - Ubuntu 16.04 LTS

Getting the robotic NodeBot environment ready on Ubuntu worked without a glitch.  To get the environment ready follow the same sequence described in the previous note:

  1. Install Node and NPM
  2. Install Johnny-Five
  3. Install Arduino software IDE
  4. Optionally install Visual Studio Code as a Code Editor
    1. Install Git previous to the installation of Visual Studio Code


I have an old Fujitsu laptop kicking around (i3, 2.26 GHz, Dual Core, with 4 GB of RAM) running Ubuntu 16.04 LTS.  Follow the instructions described in the following how to guide for installing Nodejs on Ubuntu.

We already know how to install Johnny-Five using NPM:

$ npm install johnny-five

To install the Arduino software IDE (Sketch) on ubuntu , the easiest way, is by issuing a command from terminal:

$ sudo apt-get update
$ sudo apt-get install arduino

The first command updates the list of packages and dependencies and the second command installs the arduino software.  You will need to provide the password of the super user.  Accepts the defaults and required dependencies are installed.  Once the installation process ends, search for Arduino and lock it on the launcher for convenience.

Start the sketch IDE and a permission dialog is loaded:



Click Add and provide the super user password again.  Connect the Arduino board to your computer through the provided USB cable and verify that the correct port is selected by going to Tools > Serial Port...  however, initially "Serial Port" was not enabled.  This was strange but after restarting the computer the Tools > Serial Port item was enabled and I selected the only port available to me '/dev/ttyACM0'.

Verify that the board under Tools > Boards is Arduino Uno

Now you can load the firmata from File > Examples > Firmata > StandardFirmata.  In the Windows environment we selected StandardFirmataPlus, but this one is not available in the version which was just installed.  Uploading the firmata to the board works perfect and there is no need to edit SerialFirmata.h header file.

Now you can test Blink.js.  Optionally install Visual Studio Code and GIT which I highly recommend, it provides intellisense, you can debug your scripts, you'll love it and it is free.

Write Blink or copy the Windows version but omit the configuration object passed into the Board constructor function.  There is no need to specify the port number for this exercise in this environment.

$ node Blink.js

Enjoy.

In our meetings we will try to cover as much as possible using the Johnny-Five gude.

References

June 4, 2016

NodeBots Day-1: Getting the Environment Ready - Windows

For our "NodeBots Day" activity we are going to follow the "Johnny-Five" guide.  I feel that there is enough material to get the meetup started.  Johnny-Five is a JavaScript Robotics and Internet of Things (IoT) platform.

Pre Requisites

To be able to run the exercises we need:
  1. Install Node.js and NPM.
  2. Install Johnny-Five
  3. Arduino Software IDE
  4. Optionally install a Code Editor
    1. Visual Studio Code is excellent:  recommended over Notepad++, Notepad and even Sublime
      1. Visual Studio Code has a GIT dependency therefore you will need to install Git on your station.  Install Git Bash...  you'll love it!
  5. You might need to install Python and make sure it is included in the path in the case of Windows OS

Quickly

What is Node?  It is a server side platform built on Google's Chrome JavaScript engine (V8 Engine).  As we advance in our journey you will be exposed to other fantastic things that we can do with Node including:
  • Asynchronous
  • Event Driven
  • Very fast!
  • Single thread but it is very scalable
  • It is open source!
To install Nodejs, follow the link provided in this note and select the appropriate version for your system.  If you are using the Window installer make sure that you select to include NPM.  'npm' is the Nodejs package manager and we are going to install required modules using it.

What is Johnny-Five?
As previously mentioned; it is a JavaScript Robotic and IoT development platform.  Its is community based, open source.

Once you have installed Node with npm you can install Johnny-Five.  On my git bash window it looks like this:


Great!  We are almost there!  Next we need to install the Arduino IDE.

Do I really need the Arduino Software?
Well, no.  If you do not have an Arduino board you do not really need the software and instead you could run the Nodebot-Workshop, which provides a "stubbed" Arduino board for testing each workshop challenge...  but we have boards, and we are going to have these boards around so, you can do either!

Taking the path of the real thing requires a bit more work.  We need a protocol to communicate with the board.  Arduino provide this software and it is called "StandardFirmataPlus".


O load the firmata you need to go "File > Examples > Firmata > StandardFirmataPlus".  Once the firmata loads you need to upload it to the board by clicking the "upload" arrow.  This might not work.  I am not sure why this error is "out there".



To get rid of it you need to open the header file "SerialFirmata.h" and go to line 31 and comment the include "#include "



Save the file and upload again.  It should work now.  If it does not then write a comment here and we'll work it out together.

You are almost there!

Why do I need Python installed? 
We need Python because there are a few modules that need to be built for the specific platform.  We might not need it right away, but we will for sure.

Which programming language are we going to be using?  Can I use C for Arduino?
We are using Node, which is JavaScript.  Yes you are right, you could use a C like language for Arduino but in our meetup we are doing Node.

How do I know that I am ready to "go"?  How do I know that everything is ready?
That is a good question and it will depend on a variety of factors including:

  • Operating System
  • Version of Node
  • Version or Arduino and firmware
The classic "Hello World" for the board is the "Blink-Blink" program:


Testing "Blink-Blink"

Assume your board is ready to go.


and you try to run your code after loading the firmata and...  what!?


This was difficult to troubleshoot.  The solution to this "bug" is to explicitly indicate that the Arduino board, in my case, was connected to port COM4 and this was indicated passing a configuration object into the Board construction function.


Run again and smile!  You are on your way.  This is actually documented by Johnny-Five.  What johnny-Five says is that the platform will try to determine the port, however, the port could also be explicitly set, like we did above. 

We are going to make sure that every one can run this exercise on the available boards.

I will document this same setup for Ubuntu in a part 2.  I will add references to this note a little later, but I will publish it now so some of your can get going.

In our meetings we will try to cover as much as possible the examples posted by the Johnny-Five guide.

References



August 17, 2015

My TODO List with MongoDB, Express, Angular and Node

MEAN Is as MEAN Does!

The motivation for this paper is to continue with my training into the MEAN stack of technologies.  Recently I had the pleasure to meet and share some learning time at the Vancouver MongoDB User Group (MUG).  The meeting was sponsored by Light House Labs; they donated the space and the very much appreciated, classic-meet-up-cannot-be-without: pizza and "healthy drinks" (some sort of zero calories patented combination of salts and flavors).  

The meeting was in one word, inspiring:  there is so much to learn!  So much to share!  That is how we landed on the moon!  Talking, sharing...  oh, man!  From there, I said to myself: "Ok, ok, I am going to do the new "Hello World", The "One Ring to rule them all":  a TODO list using REST and MEAN.

TODO: Mongoose Schema

The Todo Schema has three properties: title as String, completed as Boolean and createdOn as Date.  Note that createdOn has a default value of Date.now.  Also note that the title is required and that it has custom validation.



The thought of using createdOn was to at some point allow only one Todo of a type per day, like I have breakfast every day only once, the second time one eats breakfast food it is not breakfast time but something else...  lunch perhaps?  I love breakfast food for dinner!!!

In the future I am going to play with OATH and an authentication middleware so different users can have their own Todos.

TODO: Node REST API

What is new here?  Nothing, but it is mine!  It is worth mentioning that this route controls all the REST calls.




TODO:  Server

Here is the server.  The server is using the router middleware.  It is important to note that the router needs to be setup before the 'catch all' route app.get("*", function(req, res){ ... });.  The idea is that all calls are handled with the index page and the REST AJAX done by Angular is handled by the route middleware.

Interesting is to note that the connection to MongoDB is not stating the port number.  I assume that Mongoose tries the default port 27017 first hand and if the service is running on that port then uses it and everything is hunky-dory.





A section of code in the server is commented out.  I was not able to successfully use "flash" middleware with the "jade" view engine to display error messages.  So for now I am sending a json message to the client in the case that there is an error and allow the Angular client to display it perhaps using a popup window using the Angular Modal window service...  it would be ok, I guess.

TODO Controller

The controller is pretty fat I must admit.  It does everything I need it to do however,   Note that the Mongoose Todo has been injected, you probably saw that in the REST router.  Things to note in the controller:

  • Post:  Uses find (query, callback).  If not found then Todo.save(callback) invoked.
  • Get: Gets all Todos using Todo.find({}, callback).  Note that the query is empty.  Using the empty query is optional.
  • Update:  This is to be used with REST PUT.  Obviously the item is updated only if found, so this function uses Todo.findById(id, callback).  The Todo is saved in the callback.  This update is done when the user toggles the "done" checkbox".  I decided to keep the done Todo until purged.
  • Purge:  Purge is a bit more interesting, so let us take it aside.


todoController.purge

The Angular client $http service invokes the API to purge then in the Node controller the Todos collection is queried for "{completed : true}".  If the query returns something then we call Todo.remove on the entire collection BUT note that this is done with only one call to MongoDB and with a query parameter using the $in operator.

The $in operator selects the documents where the value of a field equals any value in the specified array of Todos which have completed ===  true then we call remove on them.

This is very cool.

So, what else can I show you?  Hum...  ah!  Ok, the Angular app.

TODO:  Angular Factory and Angular Controller

Nothing to it, I like how trimmed it looks.  I wish I was as trimmed and strong as one of my factories!  I would look like Terminator, the good one, the one that John Connor sent to protect his mother and himself when he was a kid in T2.  Yeah!  I like that!


and the controller is this one:


The Jade View

If you want to put it all together here the jade view.  It is a little wonky, I have to make it work a little better with errors...  I do not like what I have there. 


I notice right now that I have to move the JS code reference to the end of the file....  hey, would do that for me pal? 

:)

My MEAN Dev Tool

It seems like months since I first downloaded and installed Visual Studio Code for Node.js related development but this journey started just a few days ago on July 30st!  I guess that we get used to good things very fast.  Since then I have uninstalled all code editors:  Visual Studio Code is here to stay on my computer and that is that...  Sure it does not have the super advanced features that other product bring, but heck, it is free and it works!



Fellows, I tell you; it has intellisense, you can debug your Node.js code, place brake points.  I hope that Microsoft continues enhancing this product!




What is Next?

I could make my view better, perhaps prettier.  I want to plug in my Mocha test and add Passport to other team members can have their own Todos.

Ah!  In the case that MUG lets me, I can use this to talk about things at the user group.

Get MEAN TODO From GitHub

You can always clone the project from GitHub; here:



References:

  1. First time I use Visual Studio Code.  This note contains a reference to where to get the tool
  2. Nodding since...  not that long ago:  NodeSchool gave me the bug!
  3. Using $in:  MongoDB $in operator

August 2, 2015

Mongoose Schema Validation

Bad Data Is No Better Than No Data:  Then, Validate Ebenezer!

After my previous post, Noding, MongoDBing With Mongoose: On the Account of the MVA and Visual Studio Code, the natural thing to do was to validate the data model before a save or an update.  Let us get to it then!

It is always good practice to validate application data in the logic layer but it should be a hard requirement for any solution to enforce data validation on the model layer just before the data is saved.  MongoDB does not provide data validation, therefore data validation is on us; we are responsible for it.

Using the same Mongoose Data Model defined in the previous note, let us see how many things can go wrong with our data before adding data validation.  So, we had BankData with a collection of Accounts.  Can BankData be saved, updated without a FirstName or LastName?  Can a new Account be added without a type?  Or perhaps add a negative amount to an account?  Oh, yes, it can be done!  The Mongoose Schema has no validation!  That's not good; darn bank!


The client was able to save BankData with no first_name and no last_name and later on it was updated with a new account with no type.  

I get it, I need validation, so what types of validation Mongoose offers?  First off, validation is defined in the schema and it takes place when a document attempts to be saved, after default values have been applied .  Therefore, if our Schema would have contain validation then the above saves, without changing the code at all, would have never work and the error would have been passed to the callback, which means that assert.equals(null, err) would have failed.

Built-in SchemaType Validation

Luckily for me built in validation into the SchemaField can be applied to make these fields required.



If the client tries save the bad data then Mongoose, before the save, executes the validation, detects the error and the data is not saved.  Nice!  But there is more, the String type take s list of allowed values therefore I could place here the currencies allowed by the bank, say USD and CAD only.



Custom Schema Validation

But, what about if this bank accepts all currencies of the world,except... the British Pound! then the above validation would be unreasonable because the enum array for currency would be really, really long.  Ah!  Instead we could use a custom validation and test only for the invalid ones!  How does that work?  Easy!



Line 13 above is passing a validation function and an error type to the model SchemaType.  The currency could be anything but GBP.  This is a bit of a problem because now anything goes but the little £; even made up currencies could be set which is truly wrong.

Mongoose Middleware

Using Mongoose middleware we can validate the account currency type by intercepting the save process in a pre-save event and test the currency set by the client against a collection currencies previously set in MongoDB.  A middleware is like an interceptor which inserts its execution before or after the process execution.  The pre-save would be like catching the save on the fly.  This would be useful for the execution of complex validation or for setting some properties of the object just before save.


A Pre-save middleware executes before the save operation and if everything is OK what happens next is the actual save.


The way to implement a pre-save middleware for the account is like this:





A few things to notice from the above middleware;  First, there is a new Mongoose Schema, the validCurrencyModel (line 2).  This model represents a valid currency used by the bank.  The bank collection of valid currencies might not change very often but I decided to store it in the database for better management.  I'll show you how I batch load them to the MongoDB in a little later.

Note that in line 4 the middleware is executing a query for currency of the account (this.currency) on Currency to find out if the currency set by the client is valid.  If the currency is not found then an error is issued of the type InvalidCurrency  and the next thing that happens is that the error is returned to the callback of the save (line 11).  If the currency type is valid then the middle ware calls next() which means "carry on to the next step, the save, this data is good".

Batch-Loading Currency

A Currency object was created.  Note three things in the following code section:  currency_symbol is required, has custom validation and is unique.



Note that the batch was done by creating an array of Currency and passing it to create.  Create returns to the callback the error and the entire collection of created documents.  Simple.



Conclusion

In this note I have used Mongoose validation in three different ways to validate the Data Model:

  • Validation at SchemaType level.  We can use required, allowed value enums for strings. Max and Min for Number types.  In the case of Currency the SchemaType for currency_symbol was also set to "unique: true" in order to avid duplicates.
  • Custom Validation: By passing a validation function to the SchemaType
  • Using Mongoose Middleware:  Validation of the model in a pre-save event.
This note shows only a taste on Mongoose validation.  You have probably noticed that to test my Mongoose model I have been using scripts which are not very useful.  I feel that the natural continuation of this Node + MongoDB + Mongoose would be to use the defined model in a system through a RESTful API using Express...  hum, I would not be surprised if I soon take on this.

References

July 30, 2015

Noding, MongoDBing With Mongoose: On the Account of the MVA and Visual Studio Code

Noding From 0-60 with Stacey Mulcahy, Rami Sayar and MVA

Today, July 29th, Microsoft Virtual Academy had in store a great course, delivered masterfully by Stacy and Rami.  It was a 7 hour event!  During seven hours these fellows were talking about Node.js, Express, databases with MongoDB, debugging and deploying Node.js and extending Node.js with Azure and Web Jobs; wow!.  This course has all it takes to become popular at MVA!

During the first hour Stacey and Rami went through the obligatory introduction to Node.js layering good arguments for event driven programming, asynchronous vs synchronous (blocking), the classic "Hello Node", etc.  They did all these, and much more, using Visual Studio Code, a very sweet editing tool which I think is going to give The Others Big Ones (Sublime, WebStorm, Notepad++, ect) a good run for their money!  Yeah, Visual Studio Code is as light as pancakes and ready to give you a sugar high!  Go get it boys and girls!


Visual Studio Code;  Note the GIT icon it is telling me that the status of the Git repository is 5 new changes.  Sweet!


After the first hour of the course, still an introduction to many different Node related subjects, the event picked up speed and intensity.  I do not intend to echo here anything that Stacey and Rami talked about nor I would change one word, but would like to add my two cents.


MongoDB Without Mongoose is Like Hot Chocolate without Marshmallows! 

I just think that they needed five to ten more minutes to really complete the material by adding Object Modeling with Mongoose to their MongoDB + Node example (chapter 13).  I feel that using Mongoose changes the nomenclature of the subject and completes it; although I understand that after 7 hours of talking and explaining and running demos 5 more minutes would feel like an eternity!

Mongoose provides a simple schema based solution to help us model our application data and includes built-in types, query building, validation and much more just out of the box.  Built-in queries help us filter, and find data very easily.  Let me show you what I mean; but first you will need to install Mongoose.

To install Mongoose you need to run on the command line:

npm install mongoose --save

The save flag modifies the package.json for Chapter 13 of the course and adds to it a new module; Mongoose.

Mongoose Schema:  Building the Model

With the help of Mongoose the following code defines two entities for our system:  Account and BankData.  I am using the same definitions that the course lays out and only providing object data modeling as an add-on.



Note that BankData has a collection of Accounts which is encapsulated by the "acconuts" array property on bankDataModel.  Note that the model contains an Account and BankData defined by Mongoose.  To add this Has-a collection of accounts to BankData Mongoose tells us that we need to define an Account Schema.  Take a look at how it is done from line 6 to 10; then on line 15 BankData Schema describes the collection of accounts.

Lines 19 and 20 tells us how this mapping is done by Mongoose and we build a model containing our Mongoose object models.  Finally the script exports this model so it can be required from the application.


 Mongoose it or Loose it

We are now ready to consume the model.  Line 3 requires it.  Note that now this app does not deal with MongoDB at all; Mongoose is doing all the heavy lifting for saving, finding, querying.  The only thing we need to do is provide the URL for the connection to the database.  This is assuming that MongoDB is installed and running on your system.




The above script is just demonstrates how to use the model to Create, Read (find), Update, Delete (CRUD) using Mongoose.  You can compare this code to the code provided at the course to realize how sweet Mongoose is.  Some facilities provided:


  • model.save(callback):  saves the entity model to the doc database.  save takes a callback.  After the item has been saved the _id of the item is set. 
  • model.find(callback):  Finds all items.  find also takes a query argument which is used by Mongoose to filter and a callback taking this form model.find(query, callback).
  • model.findById(id, callback).  Do I need to say more?
  • model.remove(callback).  You know what this does.

My two cents are that adding object modeling with Mongoose to MongoDB for Node facilitates our work, adds semantics and helps our code concentrate on our business cases.  And this concludes my little tribute to this MVA magnificent course.  This is not, by a long shot, all that Mongoose has to offer and I would not be surprised if I find myself writing more about this in the future. 


References