Document Stores vs Relational Databases for Domain Modeling

I am quite excited about the rising popularity of document stores, we have been using MongoDB and CouchDB for some time now, both for internal and client projects. Both databases are similar (although built on different technologies) as they are schema-free, scalable, document-oriented.

The concept of a document database isn't new. It was pioneered by MUMPS in the 1970s for health-care and financial applications and by IBM in 1989 with the Lotus Notes Database. It's interesting to see the main concepts behind these legacy databases evolve and reemerge in new open-source projects.

Benefits of document stores
The NoSQL argument is that data should not be stored only in SQL databases, but should utilize other forms of data-store when they are better suited for the domain model. Speed and scalability are just some benefits, but I believe that the ability to quickly and concisely model complex domains is main advantage.

Modeling a CRM
Let's take the example of a simple Customer Relationship Management system (CRM), to model a business card in a relational database you would have a number of tables, for example:
  • Contacts
  • Phone Numbers
  • Email Addresses
The data in each table would have to be joined together to display all the contact details. Even in this contrived example, the data model can quickly grow in complexity as you include more types of data.

This is where the power of a document store shines through, all the contact's information can be stored in a single document. Document stores are schema-free so you do not have to define all the possible data types, you can just add new data to the document if needed. See below for an example contact document which models the business card as it would appear in the real world:

Possible drawbacks
As with all things there is a trade off, these databases typically lack row-level database transaction and 'table joins' in order to achieve higher performance. The lack of joins is not a major issue as all the related information should be in one document. However, If you currently make heavy use of database transactions, migrating to a document store isn't a good idea.

With these drawacks in mind, you probably don't want to build a banking platform on-top of these databases, however for the vast majority of applications these constraints are not a problem.

Further Reading
My personal preference is currently MongoDB as it has better support for dynamic queries, but there isn't much between the two. Following on from my previous post on MongoDB with MongoMapper and Ruby on Rails I'll ether do a series of blog posts or a screen-cast on building a simple application with MongoDB and Rails 3. 

Loading mentions Retweet
Filed under  //  mongodb   nosql  
Comments (0)

How to Get the Most Value out of Web Development

Increasingly clients want applications to be built within a tight budget, but with a feature list that is outside of it.

One way of overcoming this issue is by building and releasing your application in stages. This helps your cash flow and lets you get user feedback quickly enabling you to decide on which features you need to build next.

However, sometimes a fully bespoke system just isn't needed as there is an off-the-shelf solution. But where does that leave you if an off-the-shelf system doesn't quite fit your needs and a fully bespoke one is out of budget? I see three options:

Open Source
Customizing an existing open source application (if one is available) would save on the cost of the bespoke system as you would only pay for the new features.

Application Platforms

You can use an existing paid application platform and we can extend it with plug-ins to fit your needs. For example, we are currently a development partner with Shopify to provide e-commerce sites. We will be looking to create other similar partnerships.

API Extensions / Plug-ins / Widgets
We can build a bespoke application and use third party APIs, this not only saves on development time but you also get a more feature complete solution. For example, MailChimp for email marketing, Intense Debate or Disqus for commenting, Get Satisfaction or Zendesk for support and help-desk functionality.

I will be updating our website with more application platforms, plug-ins and open source applications soon.

We don't just want to build applications, we want to deliver real value and there is no point in reinventing the wheel.


Ruby on Rails Development by Bit Zesty

Loading mentions Retweet
Filed under  //  API   open source   project management   SaaS   services  
Comments (0)

What are the Advantages of Scrum for the Client?

Scrum provides many benefits for developers, but what are the advantages of Scrum for the client and other stakeholders? This is a question I faced yesterday in a client meeting.

To answer that I need to define what is Scrum. In layman terms, it is a project management technique that is used with agile development projects.

Benefits for the client:

First of all, Scrum facilitates changing customer requirements. It provides a flexible framework where addition of new features or re-prioritization can take place without negatively affecting the project flow and team's morale.

Secondly, it encourages regular feedback by having short development cycles (sprints), where at the end of each sprint a review takes place. The review provides opportunity for the entire team to reflect on the last sprint and improve.

Thirdly, by defining roles for team members it promotes collaboration as well as clear and open lines of communication between developers, the client and other stakeholders.

Finally, Scrum provides a framework for work estimation, where features are estimated using points. Each point represents a relative amount of effort required. The technique is flexible enough to allow for changes in requirements as well as changes in development teams velocity.


However, Scrum is not always suitable and does not always work:

It does not work with traditional software development methodologies such as Waterfall.

It cannot be fully adopted if a team is too small or too big - ideal team would be 4 to 6 developers.

It cannot work effectively if it does not have full management/client support.

It requires a Scrum Master who understands the Scrum practices and is able to apply them.

Loading mentions Retweet
Filed under  //  agile   project management   Scrum   ScrumMaster  
Comments (0)

MongoDB with MongoMapper and Ruby on Rails

I'm sure we've all heard the pros & cons of the NoSQL movement so these will not be covered here. I've been experimenting with a number of alternatives to RDBMS for a while such as CouchDB, TokyoCabinet, Redis and recently MongoDB.

MongoDB (from "humongous") is a scalable, high-performance, open source, schema-free, document-oriented database. Written in C++, MongoDB features:

MongoDB bridges the gap between key-value stores (which are fast and highly scalable) and traditional RDBMS (which provide structured schemas and powerful queries).

- from the MongoDB site.

The reason I've settled on MongoDB is (for me) it's been the most useful, more so than straight KV stores such as Redis and Tokyo. Technically CouchDB is very similar to MongoDB being a document store too, however MongoDB seems to 'gel better' (that's a technical term).

To be able to use this MongoDB goodness in my Rails app, I'm making use of the MongoMapper gem as it's the most popular one (github forks and watchers metric). There are other ORMs which you can use with MongoDB, Mongoid seems like a good alternative and the rest are documented here: http://www.mongodb.org/display/DOCS/Ruby+Language+Center.

Installation

Download and run mongod http://www.mongodb.org/display/DOCS/Downloads

Install the gems:

Configure your environment, and remove AR:

Add a config file for your database:

And add an initializer to setup MongoMapper and friends:

Getting stuff done

Now I know what you might be thinking, 'oh no not another ORM all my favorite gems won't work!', that is not the case! (*in some cases)

Usage

These Models are stripped down but you get the idea. Let say you're using Devise, then your User model could look like this:

And if that user had comments the model could look like:

These are just to help kick-start your app development, for more complete examples see the open source apps and http://www.mongodb.org/display/DOCS/MongoDB+Data+Modeling+and+Rails :

Open Source Ruby Applications using MongoDB (and MongoMapper)

NewsMonger - A simple social news application demonstrating MongoDB and Rails


Oupsnow - A bugtracker in Rails/MongoMapper
http://github.com/shingara/oupsnow/

Watchtower - An example app built with Sinatra, Mustache and MongoDB
http://github.com/kneath/watchtower/

Shapado - stackoverflow like question and answer site: 
http://gitorious.org/shapado/shapado

mmmblog - a blogging engine by the same guys as Shapado (http://blog.ricodigo.com/)
http://gitorious.org/mmmblog

More at http://wiki.github.com/jnunemaker/mongomapper/projects-using-mongomapper

Pro Tips

  • MongoMapper uses a fork of the validatable gem - which has some differences with AR validations
  • Denormalization is needed to reduce round-trips to the DB
  • Change how you think of your data models - this is closer to an object store
  • The finders have nifty options checkout http://wiki.github.com/jnunemaker/mongomapper/whats-new (0.5.7)
  • Don't be afraid to drop down and use to the Ruby MongoDB driver directly http://www.mongodb.org/display/DOCS/Ruby+Tutorial
  • Keep an eye out on the github commit feed as it's a bit of a moving target still

Summary
No migrations (yes!), storing arrays and hashes as attribute keys (sweet), and inplace updates (ftw). We're not going to use MongoDB to build transactional systems any time soon, but for the majority of web applications it's a perfect fit.


Bit Zesty - Ruby on Rails Development (and MongoDB) UK

Loading mentions Retweet
Filed under  //  development   mongodb   nosql   ruby on rails  
Comments (9)

Encrypting Data in your Ruby on Rails Application

Whilst searching for a gem/plug-in to help with storing encrypted notes in a Rails app, I found strongbox. It uses public key encryption to encrypt and store data on a ActiveRecord attribute - Spike has a great writeup on his blog. One limitation I encountered was that it only uses one key to encrypt the data, which wasn't going to work when I have multiple users, all storing private information.

491113684_ee82e217f1_m.jpg
So I wrote a wrapper around strongbox that I am releasing called Safe - a Rails engine for storing encrypted data. Safe handles the generating of the public and private keys and uses strongbox to encrypt and decrypt the data. It's still in early stages, and it doesn't support all of the options that strongbox does, but it is on the TODO list.

So let me know if you find it useful and don't forget to symlink the keys directory in your deployment scripts!


Bit Zesty - Ruby on Rails developers in London.
 

Loading mentions Retweet
Filed under  //  ruby on rails   security  
Comment (1)

Better XML parsing with Rails

We recently built a Ruby on Rails application that was receiving a large amount of XML. This as you can imagine was pretty CPU intensive, especially as Rails (ActiveSupport) uses REXML under the covers to parse the incoming requests.

I started to wonder if it was possible to use a different XML parsing library, for an overview of some of the available libraries and benchmarks see John Nunemaker's post about parsing xml with ruby and the rubyinside post ruby xml performance benchmarks.

I've used most of these libraries at some point or another for parsing XML and I was really impressed with Nokogiri, and I desperately wanted to use it in production for the Rails backed parsing. So I started to look at the Rails source and I noticed this commit http://github.com/rails/rails/commit/694998ee4fb8d257ba78424cab630846327a0889.

YES! All you have to do (besides installing Nokogiri on your server) is specify:

ActiveSupport::XmlMini.backend = 'Nokogiri'


In your production.rb environment and your done. The results speak for themselves, here is a snapshot of our CPU usage before and after, with the same load.

Loading mentions Retweet
Filed under  //  ruby on rails   xml  
Comments (0)

Agile Web Development Process

Feature Planning Workshop
A feature planning workshop is essentially a brainstorming session. The participants should at least include developer's team manager and a person leading a project in your organization. Other developers, software users and other stakeholders could also add value. During the workshop the identified requirements are captured in the form of user stories and prioritized.

Estimate and Schedule
After the feature planning workshop, the developers estimate how long each user story will take to implement. Work is divided into one or two week cycles (iterations) and resources are allocated accordingly.  You should then receive an estimate and an iteration schedule.

Weekly/Fortnightly Demonstration and Planning Meetings
Once the project is given the go ahead, you may want to meet developers again to plan the first iteration cycle - define goals for the iteration, clarify, detail and re-prioritize features. Then the development team goes away to write the code and tests while constantly collaborating with you to make sure they are building what you want.

At the end of the iteration, developers should demonstrate the completed features for that iteration. You should provide feedback and proceed to plan the second iteration in more detail. The process should keep repeating itself until the application is complete.

Completion
Once all the iterations are completed and you are happy with the result, the developers should deploy the final version of the application to your server. But the relationship should not end here - make sure they are there to support you if needed.

For more information on how to make sure your project runs smoothly, read 5 Ways To Keep a Web Project on The Rails.

Bit Zesty Ltd

Loading mentions Retweet
Filed under  //  agile   project management  
Comments (0)

Web Development Terminology

While we are not big fans of the technical jargon, sometimes we just cannot help it. I will cover some of the key terms used on our site.

 

Web Application (webapp)

A web application is computer software that is accessible through the Internet. Social networks, e-commerce sites and wikis are just some examples of web applications.

  • Web applications enable you to deliver and update software without the need to distribute and install it.  This makes it easier to reach new users and keep them happy.
  • It is also not dependant on the user’s operating system, meaning you cover a wider market without having to make changes to your software.

Ruby on Rails (RoR)

Ruby on Rails is a framework for creating web applications using the Ruby programming language.

 

  • It enables applications to be built rapidly. Therefore you receive more features at lower cost.
  • It encourages good development practices, resulting in higher quality code. This ensures your application is extendable and scalable.
  • By having excellent support for automated testing, it allows for robust code, which reduces the amount of maintenance required.

Agile Development

Agile is a development methodology, which embraces collaboration with customers, face-to-face communication, self-organizing teams, frequent delivery and ability to respond to changing requirements.

  • Agile enables you to re-prioritise or even introduce new features at any point, allowing us to react to changes in your business and ensuring that the application is just as you wanted it to be.

Test Driven Development (TDD)

Test Driven Development is a technique where automated tests are created before the program code. This forces code to be written and improved upon until it passes the test.

  • Test Driven Development enables to eliminate the majority of bugs, ensuring you get software that is robust and properly functioning.

This list is by no means complete and I am intending to keep updating it especially as we add new terminology to our website.

Bit Zesty Ltd

Loading mentions Retweet
Filed under  //  agile   development   ruby on rails  
Comment (1)

5 Ways To Keep a Web Project on The Rails

It is amazing how many things can go wrong during the course of a web project. Here, we focus on five problems which any project can encounter and explore how they can be avoided or solved.

1. Poor Communication

It is vital to establish a good line of communication between yourself and the developmentteam as soon as possible.  Poor communication can lead to misunderstandings and slow the progress of your web project considerably.

SOLUTION:  Try to ensure that there is an assigned project manager within the development team you work. You should also appoint a dedicated person within your own company as a point of contact for the development team.

2. Requirements Not Met

At the end of a project, it is all too common for clients to be presented with a product which is very different from what they had envisioned at the beginning.

SOLUTION:  Have regular meetings with your development team and request demonstrations of the progress they have made as you move forward with the web project.

3. Undefined Priorities

When priorities become muddled, projects can drag on for long after the target deadline, with no end in sight. This confusion is often caused by the introduction of more and more requirements to the brief as the project progresses.

SOLUTION:  Be clear about your priorities. It is far more efficient to launch the product and then tailor further adjustments to specific user feedback than to delay the launch date indefinitely while making changes based on guesswork.

4. Inadequate Testing

Testing is usually the first corner to be cut when the budget is tight. However, it is a risky step which rarely pays off: inadequate testing can leave the web project vulnerable to many bugs, which in turn will slow down the project and result in greater costs in the long run.

SOLUTION:  Ask for automated testing as part of the development process, especially if the web project is complex. It is also good to have someone within your company testing the project throughout the duration, helping to identify bugs as early as possible.

5. Warning Signs

If you think that something might be wrong, the likelihood is that you're right. Trust your instincts!

SOLUTION:  As soon as you notice warning signs such as slow progress or poor quality work, get to work addressing the issues so that they don’t end up snowballing.

The best way to keep your project going along the right tracks is to fix problems as soon as they start to occur. Read on Agile Web Development Process.

Loading mentions Retweet
Filed under  //  agile   project management  
Comments (0)

Announcing our redesign and launching Tweetarium

To coincide with the launch of Tweetarium (our newly released twitter application) we have also given our old website an overhaul. The new site design will hopefully give a clearer picture of the types of applications we can build and the services we provide.

Loading mentions Retweet
Comments (0)