ActiveRecord::Migration Safe Table DROPs

06.16.2013 03:05 by kbeckman | Comments

I’ve had some notes laying around after I needed to drop some database tables a while back. Now that I have the need  to drop a few tables again in an upcoming sprint, I thought I’d get my thoughts down in a quick post and share.

A Few Things to Keep In Mind…

The normal course of development and system refactoring occasionally renders database tables obsolete. I don’t bother dropping tables until after a successful production deployment has rendered the target tables obsolete. For example, if Sprint N introduces code deprecating a table, I’ll wait for Sprint N+1 to introduce the ActiveRecord::Migration to drop it. There’s no harm in letting the obsolete table sit around for another sprint; and since you’re dropping it in the very next sprint, it should still be fresh enough in your memory to keep from forgetting about it. Waiting an extra sprint is just an overly cautious way of handling this type of thing and leaving the table gives me some peace of mind should the improbable event arise that I need to reintroduce the code again that uses the table.

 

When dropping a table, ALWAYS make sure it exists before doing so. This ensures no deployment errors are introduced if the table has already been dropped for some reason.

 

Expanding on my prior points… How would a table already be “dropped for some reason”? Let’s assume that a deployment failed for some reason after the deployment migrations have already executed. The deployment will rollback all migration scripts ensuring the database is in a pre-deployment state so the deployment can be attempted again after the error(s) have been fixed. Running the migration script again will attempt to drop the deprecated tables again, but only if they exist (and they shouldn’t)… It’s a good thing we’re checking for their existence first!

ActiveRecord::IrreversibleMigration

The ActiveRecord documentation states, “Migrations [that are irreversible] should raise an ActiveRecord::IrreversibleMigration exception in their down method.” Dropping tables would suggest this type of change, right? …especially since you wouldn’t want to recreate the table with all of its existing data (possibly moving it over from a backup table) just to drop it again. I highly discourage using an ActiveRecord::IrreversibleMigration exception for this purpose unless you’re comfortable killing your database rollbacks during a failed migration on a production deployment. That’s an unnecessary headache when you’ve already got problems with a deployment failure.

ActiveRecord::Migration Safe DROP Example

class SafeTableDropExample < ActiveRecord::Migration

  def up
    drop_table 'obsolete_table' if ActiveRecord::Base.connection.table_exists? 'obsolete_table'

    if ActiveRecord::Base.connection.table_exists? :another_obsolete_table
      drop_table :another_obsolete_table
    end
  end

  def down
    # We're not going to reverse this migration even if the current deployment fails...
    # Since a previous sprint deployment has made these tables obsolete, we're completely sure they're no longer used.
    # There's no need to recreate them in the event of a rollback just so they can be DROPped again when the current
    # deployment succeeds.
  end

end

Resources

My 2013 devLink Abstracts

04.02.2013 12:40 by kbeckman | Comments

After a little bit of prodding from a friend and colleague of mine, I submitted a couple of abstracts for talks at devLink 2013. I’ve been thinking about submitting some talks for about a year now. Well, now I don’t have to think about it anymore – but I might have some prep work to do!

 

Introduce Ruby to Your .NET Environment

[Intermediate to Advanced]

Interested in Ruby, but so far have only tinkered with it at home in your free time? How about at work? Nope… “We’re a .NET shop. …and IronRuby hasn’t had a release since early 2011. Isn’t that project dead?” Now there’s no need for excuses and no need for a Ruby implementation that runs on the CLR. This session provides some real-world examples proving the benefits of using Ruby alongside your .NET environment and helps make the case to include Ruby as part of your development toolkit.

This session includes demos, discussion and Github projects covering the following:

  • Use Ruby (in combination with PowerShell, Albacore, MSBuild and WebDeploy) to streamline your Continuous Integration and code deploy process. Build your code; exercise it with tests; and package/deploy it all without writing a single line of MSBuild XML.
  • Build a stronger relationship with your QA Team by introducing Ruby as the scripting language of choice for integration test automation. You’ll make everyone’s job easier and make automated integration testing a first class citizen in your development process.
  • Tired of spending an afternoon setting up a production or test server? Are your base virtual machines chronically outdated? Use Ruby code and Opscode Chef to bootstrap your servers and use code to setup your servers instead of doing everything manually. Infrastructure as code!

 

Demystifying Single Sign-On

[Intermediate]

Single Sign-On – SSO… If you’ve considered single sign-on or identity federation for your organization, those three letters are probably better rearranged – S.O.S. This session will help you sort out all the acronyms and protocols so you can start in the right direction to a successful implementation. We’ll take a look at Windows Identity Foundation and Windows Azure Active Directory (formerly Access Control Service) and how they fit in the overall solution.

This session covers:

The Basics…

  • Token Issuers and Relying Parties (with demo)
  • Comparison of WS-Federation, OAuth and SAML-P
  • SAML AuthN Token Overview
  • Dissection of WS-Federation Web Requests

Advanced Topics…

  • Identity Federation with Trusted Organizations (with demo
  • Important Architecture Considerations
  • Social vs Corporate Identity

 

What do you think? Do either of these sound interesting enough to make the cut? I ‘m hoping so… Either way, speaking or not, I’ll see you in Chattanooga in August!

omniauth-wsfed v0.2.0 Release

03.18.2013 03:39 by kbeckman | Comments

Recently, I pushed the official second release (v0.2.0) of omniauth-wsfed to RubyGems.org. omniauth-wsfed is a WS-Federation and WS-Trust strategy for OmniAuth enabling integration with Windows Azure Active Directory (formerly Access Control Service), Active Directory Federation Services (ADFS), custom Identity Providers built with Windows Identity Foundation (WIF) or any other Identity Provider supporting the WS-Federation protocol. Specifically, the 0.2.0 release contains several enhancements related to SAML token validation and WS-Trust + WS-* field validation as well as an overall cleaner and more thoroughly-tested codebase (see the release notes or Github repo for complete details).

 

Almost four weeks after the release, RubyGems.org reported just over 600 downloads (all time, for all versions) and 124 downloads for the new version. Well… Not bad I guess, but definitely not the type of numbers that get your gem recognized on Ruby Toolbox. Reading between the numbers however, there are a couple of positives – a) At the time of this post there were 124 downloads of the new version and only a couple of those were from my current project; b) there are definitely other developers using it! Only a couple of those downloads were related to my current project as I’m in the middle of a refactor effort to organize all enterprise AuthN logic into a Rails engine shared by all applications. Once the engine is finished, the download count should increase quite a bit as it gets promoted through our test environments and onto all of the developer installations. Overall, it’s not a bad start since single sign-on and identity federation for enterprise applications are problems more commonly solved by large corporate codebases written in Java or C#. Most times, Ruby projects are solving this problem with OAuth and using well-established social Identity Providers such as Google, Facebook, Twitter, etc.

Building Some Street Cred…

So why all the information for a Ruby gem targeting such a narrow problem space? Well for starters, omniauth-wsfed finally reached a state where I felt comfortable openly promoting it instead of just using it on internal client projects. I had been thinking about promoting it for a while, but felt the library needed just a little more polish. Additionally, I read a recent post on you’ve been Haacked that finally gave me the motivation to write this post. In the post, Phil Haack discussed his opinions about establishing trust for open source packages on NuGet. Haack discussed several possible options including package signing and using custom NuGet identity verification; but arrived at the conclusion that you’re going to get the most for your effort by simple, established techniques such as social identity verification, download counts and grass roots promotions like the one you’re reading now… :-)

 

The interesting part about the article was that it doesn’t just apply to NuGet open source packages targeting the .NET Framework. It applies to open source packages in general – whatever the language. Bottom Line: if you’ve written something that you think would be useful to others, promote it; talk about it; encourage others to contribute.

Trust Me… I Have Badges

Aside from the suggestions in Haack’s post, there are some great options for Ruby projects (either completely free or free for open source projects) that can help establish additional credibility for your project. Each of these services come with their own “badges” for display in your Github README calling attention to the fact that you and your contributors are committed to quality. If you haven’t noticed, these are starting to pop up on more and more popular Ruby Github projects. Now that there’s an omniauth-wsfed release I’m proud enough to promote, I got curious about these services and went investigating…

 

I’ve added these services…

  •   Code Climate – a code quality service for Ruby repositories that tracks metrics such as duplication, code churn and method complexity. Code Climate requires absolutely no effort other than pointing it to your Github repo. 
  •   Travis-CI.org – a cloud-based, continuous integration environment for open source projects supporting many languages including C, C++, Java, Perl, PHP, Python, Ruby, etc. Travis-CI is incredibly easy to setup and requires only a Github account link and a YAML configuration file. Specifically for Ruby projects, Travis-CI is capable of continuous integration against ALL popular Ruby implementations – an insanely useful feature because you can validate your code under conditions and  in platforms that you might otherwise never use.
  •   Version Badge – as long as you’ve released a gem on RubyGems.org, Version Badge will provide a badge indicating the most recent version number.

I’m considering these services…

  • Gemnasium – a service that evaluates your project in terms of its dependencies. Projects are scored on whether or not their dependencies are up-to-date and the service notifies you if any new dependencies are available to test with.
  • Coveralls – a code coverage service for Ruby and Python projects that integrates with Travis-CI.

I Want to Hear from You!

One final thing I wanted to call out is the importance of full disclosure in open source projects. It’s extremely important to disclose any known issues (whatever the severity) as well as the project roadmap to potential users of your software. Awareness for known issues before being discovered accidentaly by your consumers will reduce the overall friction of adding your OSS project as a dependency. Personally, I feel much better about acquiring a dependency on a project even if it has some issues as long as I’m aware of them (and how they might affect me) and know the author(s) are actively working on a resolution. Github Issues are a great way to track these details. You can checkout omniauth-wsfed’s issues / roadmap here.

Prepare the Kitchen–Linux Setup for Chef-Solo

01.10.2013 05:08 by kbeckman | Comments

It’s been an incredibly long time since my last C4SC post. I’ll blame most of that on my recent wedding a couple of months ago. The rest of the blame goes to the time crunch we’re working under on my current contract. There’s no sign of any slowdown on the horizon, but I’ve made a conscious decision to take back my blogging time in 2013. So hopefully I’ll be able to get back in the habit…

 

To kick off my return to blogging, I thought I’d start with another C4SC series that reflects the majority of my recent work in DevOps while setting up our Production, Staging, QA and Integration environments. I’ve been using a combination of capistrano and chef-solo to setup and bootstrap our servers preparing them to host our Rails sites or backend services. Our production environment Linux distribution is Red Hat Enterprise Linux (RHEL) which is very similar to the freely available CentOS. Personally, I’d much rather be using Ubuntu because it just seems so much easier to work with… With that said, I’m going to do my best to provide equivalent code samples for both RHEL and Ubuntu in this and upcoming posts.

 

This post will take you through the prerequisite steps for setting up an admin user to drive chef-solo. I’ll talk about chef-solo in detail later in the series, but right now the concentration is on Linux fundamentals.

Add a Deploy User (Context – Linux Installation / VM)

First things first. We need a sudo user account other than root that we can use to deploy code and cook up some chef recipes. During the installation process for RHEL or CentOS, you’ll set up the machine root account. An Ubuntu installation is a bit different whereas you’ll actually create an admin user account with root privileges (no root account password is ever specified). Log into the machine under the root or admin account and execute the following commands to create a chef_admin user account:

useradd chef_admin
passwd chef_admin

Configure root and chef_admin Users (Context – Linux Installation / VM)

Now that we’ve created the chef_admin account and set the password, we’ll need to make a few additional configurations. First, we need to give chef_admin sudo privileges on the machine. Here we add a line to the /etc/sudoers configuration file.

echo "chef_admin ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers

 

Next as a security measure, we should remove SSH access for the root user. This is an optional step, but something you’ll want to consider as it removes the ability for the root user to connect remotely (without having direct access to the machine.

# RHEL / CentOS
echo "DenyUsers root" >> /etc/ssh/sshd_config
echo "DenyGroups root" >> /etc/ssh/sshd_config
/sbin/service sshd restart

# Ubuntu
echo "DenyUsers root" >> /etc/ssh/ssh_config
echo "DenyGroups root" >> /etc/ssh/ssh_config
/etc/init.d/ssh restart

Copy SSH Key (Context – Host OS)

Assuming you already have an SSH key that you want to use for the chef_admin account, it now needs to be copied to the Linux installation. From your local system (or your Host OS) use the ssh-copy-id command to copy the necessary key.

ssh-copy-id -i ~/.ssh/chef_admin_rsa chef_admin@192.168.56.2

 

A quick note for OSX users: You’ll need to use homebrew to install the ssh-copy-id script before you execute this command.

brew install ssh-copy-id

[Optional] Remove Password Access for chef_admin (Context – Host OS)

Similarly to the step above where we removed SSH access for the root user, we should remove password access for the chef_admin user. This step will prohibit access for any users without the chef_admin_rsa SSH key. Create an SSH connection to the Linux installation and execute the following commands:

sudo su root
passwd --delete chef_admin

Update: Enabling SSH Server on System Startup

Depending on the installation options or the version of the OS that you’re using, you may need to ensure the SSH server starts when the machine boots. I recently ran into this issue testing against CentOS 6.5.

chkconfig sshd on

Resources

Fight Back Against Pseudo-Security

09.11.2012 13:27 by kbeckman | Comments

At least some of your online personal and financial information is AT RISK RIGHT NOW. Some of your information might already be compromised… Even worse, some of the companies hosting your information may not even know that a breach has occurred. And worst of all, your personal information might be IS in the hands of companies with  bullshit security practices.  Online security breaches are an increasingly common occurrence… Back in June, LinkedIn was hacked and 6 million of its users’ passwords were posted online – mine was one that was stolen… A month or so later, Yahoo! was hacked and 400,000 user accounts were compromised. Today, well-known hosting service and DNS registrar – GoDaddy – was hacked and the number of affected sites is in the millions. Although a leak of passwords or personal information may or may not have been part of the breach… Folks, those are just the major instances this summer.

 

Amidst all the web chaos today, I became aware of the half-assed, don’t-give-a-shit security considerations of my former web hosting vendor – Arvixe. Here are the details…

 

Arvixe Billing Users Beware!!!

Until a couple of weeks ago, coding4streetcred.com was hosted through web hosting vendor, Arvixe.  The hosting arrangement was extremely vanilla – just a single server configuration sharing an IIS instance with who-knows-how-many other sites. Arvixe’s pricing was reasonable and the initial setup was incredibly easy. In the two years coding4streetcred.com resided on their servers, I only had a couple of issues with the IIS integration of Microsoft’s WebDeploy utility (probably only because WebDeploy wasn’t a mature product yet). Their support team was awesome and I always got a quick and relevant response. Despite my satisfaction with Arvixe, I recently decided to move my site to Amazon’s Elastic Compute Cloud for a couple of reasons: a) Amazon’s free usage tier; and b) I wanted more experience with Amazon’s Web Services platform for my current and upcoming development contracts.

 

All in all, my experience with Arvixe was good until today when I received the following email from their support team as I attempted to close out my account…

Hello Keith,
I am sorry to hear that you have moved your hosting services to another provider. I can close your account [account name removed] for you, effective immediately. I'll just need you to confirm your identity by providing the last 4 characters of your billing password. If you need to reset your password, please go to [link removed].

ARE YOU SHITTING ME?! PROVIDE THE LAST 4 CHARACTERS OF MY PASSWORD?! For the first time in the history of my Internet usage, a company has asked me to verify my identity by providing details of my actual account password… Unbelievable! After I politely replied via email and tweeted about my disgust with their plain text password storage, I got the following response via Twitter (full conversation is included).

image  image

I finally reset my password to finish the account termination only to find that Arvixe actually sends plain text reset passwords through email! This was another first for me – plain text passwords through email… If you use Arvixe, STOP USING THEM IMMEDIATELY.

 

Why Would This Be a Problem?

Any time you find that all or part of your password information is stored in plain text, IT IS A BIG PROBLEM. The cryptographic details are outside the scope of this post, but I’ll tell you this…

  • Passwords stored in plain text are accessible by anyone (in this case Arvixe employees) who has access to the system database.
  • Passwords stored in plain text are available in plain text to anyone who compromises or hacks that system.
  • Passwords stored in partial plain text can provide enough information to hackers to break the encryption cipher used encrypt the rest of your password. 
  • Passwords sent via email in plain text are only as secure as your email provider’s security. Passwords should never be sent via email – NEVER. An appropriate approach is to send a secure link that allows you to reset your password through some application utility.

It’s no joke… Even worse, this concerned Arvixe’s billing system – the same system that is capable of storing credit card information allowing you to automatically renew your hosting account as needed. Thank goodness they didn’t have any of my credit card information stored. At least not to my knowledge anyway.

 

The Minimum You Should Be Doing to Protect Yourself

The sad truth here is that there are droves of clever hackers out there that can and will break into a system where your account information is stored. The companies hosting your information won’t be able to stop them every time… Fortunately, there are a few things you can do to limit your attack surface… I’ve listed them below in order of what I think are most important.

  • Use a different password for every online account. This sounds a lot more difficult than it actually is… There are several good password management services out there for free or a small fee that can help you with this. I use LastPass. It integrates with every browser and they have mobile apps so you can use them on your Android or iOS devices. A different password for every account minimizes the effect on other accounts if one is ever breached.
  • Don’t store credit card details online. Most sites don’t require you to store credit card details. The exceptions would be those sites that require it for recurring charges… Only do it when or if you have to.
  • Max out your password strength. Every site has different rules regarding password strength (i.e. password length, upper/lower case character requirements, special characters, etc.). Use the longest password with the most numbers, special characters and combination of upper/lower case characters that the site will allow.
  • Be wary of passwords in plain text. If you ever find that your account provider stores passwords in plain text (or even part of your password), terminate your account immediately. The same goes for account providers that send passwords in plain text via email.
  • Make half-assed security practices public. Tweet it. Blog about it. Tell your friends. Other people need to know too… With a little exposure to bad press, companies might actually make it a priority to change their ways.
  • Always be vigilant about your personal details. Your financial stability, credit score, free time and countless other details of your life depend on it…

 

In Closing…

Cyber security is nothing to ignore. If your personal or financial information is online, it is at risk somewhere. There are countless ways your information can get into malicious hands. Unfortunately, it is just an ugly detail of Internet usage. Do everything you can to protect yourself… Stay safe out there.