I saw this link today and was very interested in how they did it.

I love the simplicity that you can get when you deploy static files - but managing them has always been a bit of a hassle - and technically above what most of our clients are happy to use.

You see, nonfiction has a CMS that we’ve been using for a number of years. It’s PHP/MySQL based and has served us well - but it’s getting a little long in the tooth. It works great for static customer websites - but when you want to add any app type functionality it leaves a lot to be desired.

We normally do web app development in Ruby on Rails because of the rich Ruby ecosystem of gems and reusable code, but have never found a Rails CMS that we’ve liked, because they haven’t been a huge step up.

We could build all sorts of sites with them, but we normally hand off our sites to our clients, who then manage them all by themselves - with us helping as needed. Most of the Rails CMS platforms haven’t had a system that we felt could be handed off easily.

I’ve been writing this post in Prose.io and while I don’t think I could hand this off to my clients in this exact state, I’m going to be watching the changes they’ve talked about adding and see if it’s a possible solution going forward.

I was getting a demo of a new tool/toolset from the guys at Stackdriver yesterday and we were talking about some of the monitoring/logs/metrics tools I was using. I listed a few, but after we finished the call I realized that I had actually forgotten a whole bunch that we use.

In the spirit of Thomas Fuchs’ recent post, this post is an overview of some of the tools we use at nonfiction to monitor our servers:

1. Pingdom - monitors our public web and dns servers - the public status report is always available. Alerts via Pagerduty if it detects a problem.
2. Munin - on an ancient RHEL box - about to be retired - graphed basic server metrics for years.
3. Monit - deployed via Chef - restarts servers if they’re not responding - local to each server - notifies via email.
4. Datadog - deployed via Chef - creates all sorts of server utilization graphs (like Munin) by default, also ties in various integrations to show your whole environment and how it works together. You can also throw various metrics into Datadog, and they take care of storing and visualizing those metrics. My current favorite tool because of the ease of use and integrations.
5. Papertrail - log aggregator that pulls all of your server logs together in one place: syslog, Heroku, random log files, etc. You can also alert for specific log patterns using Pagerduty - very handy for so many reasons and worth every penny.
6. Servicenarc - a way to make sure various cronjobs are running as often as they’re supposed to. Based on Dead Man’s Snitch.
7. Boundary - pretty amazing network visualization tool to show your network flows in pretty much real time. Don’t use it as often as I should but it’s pretty incredible when I do look at it.
8. Denyhosts - watches for SSH password guessing and locks out IP addresses that are trying to break in.
9. Logcheck - mails out “suspicious” log files - was a great tool in the past, but has largely been replaced by Papertrail for us.
10. Airbrake - our Rails apps all have this integrated for error detection and logging

There are some tools I want to try out or take a closer look at:

1. Sensu - looks promising.
2. OSSEC - I had a basic installation running, but it was SOOOO chatty that I quickly ignored it - would like to see if I can get it to a reasonable balance of signal vs. noise.
3. Stackdriver - looks interesting.
4. Fail2ban - want to extend DenyHosts to FTP at least.
5. logstash - turning logfiles into actionable data seems interesting

Also some other notable tools I have looked at in the past but don’t use at the moment:

1. New Relic - using this on one project but not overall
2. Tracealtyics - too much noise for us to be useful - may work better in other environments
3. CopperEgg - liked it - worked pretty good for us
4. Server Density - liked the iPhone app
5. Splunk Storm - seemed super expensive but worked pretty well
6. Loggly - worked great - just liked Papertrail better
7. Scout - worked great
8. Mod Security - too much noise for us to be useful - may work better in other environments
9. Librato Metrics - powerful tool with a great team behind it

What do you guys use? Anything notable that I’ve missed that I should look into?

That sounds a bit harsh - but ultimately it’s true.

Late last year, Smashing Magazine wrote an article about SEO and basically said that:

1. As long as your website is programmed properly.
2. Don’t try to game the system - it’s not a long term strategy.
3. Write original, relevant, interesting website content for people.
4. Keep doing that - it takes time.
5. As people link to your site, your ranking will rise.

I 100% completely agree.

We’ve been saying this for years - most SEO seems to be a scam - build a site properly, then fill it with interesting website content: blogs, white papers, information, items that people are interested in.

If people are interested in your site, there will be links to it. If there are links to it, then your ranking will rise. If you’re utilizing social media to spread the word, if you’re active online where your customers are active - that will all help.

You have to make what you do interesting - and ultimately what you do will be interesting to somebody.

There isn’t an easy button - you have to work at it - and gaming the system (short term SEO hacks) will only help for so long - and may hurt you in the long run.

We’ve lost contracts because we didn’t offer those scammy SEO services - but we’re still not going to offer those services. Do like Google says:

Make pages primarily for users, not for search engines.

Isn’t that what should always be the point? Do things for your clients / users and their long term benefit - not short term snake-oil cures that ultimately don’t work.

There is no easy button - and you have to take the time to fix it.

I am amazed when I see somebody applying for a development job with us and they have a shaw.ca, hotmail or yahoo address. It sort of makes me sad right away - because it shows me that they don’t really take their digital identity seriously.

I think that you need to own and control your digital identity.

It costs about $10 / year to own a domain name - buy one and use it - make it your identity for as long as possible. I am lucky that I was able to snag froese.org a long time ago - as a result, my email address will be darron@froese.org for the foreseeable future.

When your email provider shuts down - like home.com did a while ago when it got bought by Excite - it’s no big deal. When you move, it doesn’t matter - a new ISP doesn’t mean a new email address. Actually if you want to baffle somebody at an ISP, refuse to sign up for their email accounts - just say that you don’t need it - that usually doesn’t compute.

When your blog provider shuts down, it’s less of a deal, because your blog is located at your own domain name - mine is at blog.froese.org. You just export, and move to the new provider - in this case it’s Octopress hosted on Heroku. It’s possible that some links will be broken, but with enough discipline that doesn’t have to happen - and if it’s just too painful, you can work around those problems.

When your website provider shuts down / gets acquired / sunsets / whatever - it doesn’t matter if it’s your domain name - you just move you files and presto - you’re up again.

Own your digital identity - it’s a little bit of pain and setup up front - but it’s worth it in the long run.

I’m trying to find a decent Canadian VPS provider that I can use for some of my customers.

Yes - I know that having your data in Canada is not really any safer than having it hosted in the USA, but I can’t fully convince some of my client’s legal departments.

I need:

1. The ability to start up additional instances without calling somebody or filling out a form so that they will “get back to me”. Ideally there would be an api, iPhone app or web interface where I can do just that - just like I already do with Amazon Web Services and Rackspace Cloud.

2. Ubuntu LTS is required - CentOS or Windows only need not apply.

3. Onsite backups at reasonable prices. I will take care of offsite snapshots - likely in the USA as I can’t find reasonable storage costs for 200+GB in Canada.

4. Reliability - I am not interested in the lowest price - I will pay more if the servers are stable.

5. No yearly contracts.

I want:

1. Ability to scale up those instances dynamically - just like I can currently do with Rackspace Cloud.

2. Multiple locations in Canada would be nice.

3. Would be awesome if they worked with Chef or had a knife plugin - but not a deal breaker - I can always just knife bootstrap them.

4. Pay by the hour resources - but will pay by the month / quarter if needed.

Is this too much to ask?

Anybody know of a company with Canadian server hardware that fits the bill?

Chef workflow question…

Looking for a little advice here on how to move forward with some Chef built infrastructure.

I started going through my chef repo this Christmas - and some ugly little platform version hacks started appearing - so I got to thinking about how better to structure things - and here I am.

We have about 20 Rackspace Cloud Ubuntu 10.0.4 boxes running various parts of nonfiction’s web infrastructure - they’re working great and are current with all patches, etc. I’m not likely to “dist-upgrade” them - but they’re going to be alive for a few years at least.

I built them with my first Chef recipes and am now wanting to:

1. Start deploying Ubuntu 12.0.4 boxes on Rackspace’s new OpenStack architecture.
2. Not break my existing infrastructure - they work great - even though the cookbooks are a little raw in places.
3. Update my chef cookbooks and use more community cookbooks with Berkshelf, Vagrant, Jenkins, etc.
4. Stop using roles - which were used extensively before.

These boxes are built and all installed packages and software are managed with Chef - but much of the data on them isn’t managed in Chef - clients upload their own files, etc. Between DNS that I don’t manage 100% and that customer data, it’s not super simple to just “redeploy” and rebuild everything.

So my plan is to:

1. Create an environment called “lucid”.
2. Freeze all the current cookbook versions and assign them to that environment.
3. Create a new environment called “precise”
4. Deploy new boxes using that environment.
5. Fork my own chef repo and rebuild / test with Berkshelf, Vagrant, Jenkins, etc.
6. If a Lucid box can be discarded and rebuilt with Precise - I’ll do it - but likely not all of them.

What do other people do when they’re updating their infrastructure like this? New chef repo? Keep it all in one? Any tips? I’ve been looking through all sorts of links online and haven’t seen much of that sort of discussion.

Reading through this article it looks like he converted his chef-repo and just kept moving - that sort of scares me a little bit - but I guess is an option.

Any tips / links would be appreciated.