Month: May 2009

Best Practices

Keeping Gentoo Fresh

I had a converstion with a friend about Linux distros earlier today, and I was asked why I choose to run Gentoo on my web server. He told me that Gentoo was too hard to maintain on a server, and that when it came time to upgrade something (like Apache or PHP) due to security patches that it took too long, and too often failed. I was confused by this so I asked for clarification. What he described was the pain of updating anything on a “stale” Gentoo machine.

Unlike so many of the other popular distros, Gentoo does not, by default, use pre-compiled packages. So unlike doing rpm -i or apt-get install doing emerge on Gentoo requires that the package you are installing, and any missing dependencies, are pulled in as source code and compiled. When you think about adding packages like, say, Lynx, the process takes only a few minutes on a moderately decent machine. (Mine is a PII 966 and Lynx took about 4 minutes start to finish). When you talk about upgrading something like Apache, however, the length of time it takes depends not only on the speed of the machine, but how many of its dependencies are out of date. In fact, if you fail to update regularly you can run into an issue where not only are most of your files out of date, but your system profile is out of date and you need to do some serious wrenching to get the whole thing working again. In the times that this has happened to me (twice) I was able to get the system up-to-date once, and just gave up and reinstalled a newer version the second time. (These were both rarely used VMs, and not production boxes.) However, updating the profile on a “fresh” Gentoo is (in my experience) a painless procedure of rm /etc/make.profile && ln -sf /usr/portage/profiles/profile_name /etc/make.profile && emerge -uND world (uND : update newuse deep: update, take into account new use settings from the profile and make.conf, and include deep dependencies).

So how do I avoid the “stale” Gentoo syndrome? I take a three-step approach.

  1. A daily cron job runs emerge -puvD world (puvD : pretend update verbose deep : just tells you what would be emerged, in an update, verbosely, and include deep dependencies) and emails me the output. This enables me to see each morning which packages have updates available.
  2. Every day that I have the time for it I log into the machine and run emerge -uD world and follow it up with etc-update (if needed) and revdep-rebuild if any libraries were included in the updates. (I save building new kernels for Sundays, and that doesn’t happen all that often, but I do like to always run the latest.)
  3. I check the messages from emerge to see if there are any special configuration changes that need to happen post-install that cannot be handled by etc-update. For instance, changing configurations in /etc/conf.d/packagename, new profiles or anything of that sort.

Ok, so I like to keep my system on the latest and keep a shiny new everything on it. How does that compare with something like, say, Debian? In Debian (and Debian-based distros) you can update packages to a certain point, after which the package for that version of Debian is no longer supported or updated. So you need to upgrade your version, and your kernel, which you do with apt-get upgrade dist. Seems easy enough. And how does Gentoo handle version upgrades? It doesn’t need to. If you keep your system up-to-date in the way I described your system will match whatever the latest Gentoo release has. In fact, I built my web server using Gentoo 2006.0 and have been keeping it up-to-date since then. (Gentoo seems to have stopped doing the biannual releases, btw – they are now releasing updated minimal install CDs nearly weekly for each architecture.)

git

My thoughts on Git

I have been working with Git for a while now, and I have a few thoughts about it. I am used to the SVN model, where multiple developers check out of a central repository, make changes locally and then commit them back to the repo (and repair failed merges). While SVN is not perfect, at least I am comfortable with it and know what to expect from it. While some of my issues with Git may relate solely to my relative unfamiliarity with it, at least it seems like I am not the only one with issues.

In reading through the man pages and all the documentation I can find on line (including the Git – SVN Crash Course) I was totally unprepared for what actually happens. According to the Crash Course git commit -a does the same thing as svn commit, but that is not quite the case. After a commit the files still need to be moved to the repo. So as I am looking for the best way to do that I find git push. Well, it would seem to me that firing off a git push would mean that I want the files I have modified and committed to merge into the repo I checked them out of. But that isn’t quite what happens if there are any working directories checked out the repo (meaning the one you are working on). It seems that the only way to properly get the files back into the repo is to issue a git pull from the repo aimed at your working branch. Hans Fugal said it best in his post from last year: git-push is less than worthless.

What this means is that I am unable to easily move changes from my laptop to the server. Instead, in order to reduce the number of steps I need to take I check out the repo I am working on (git clone) in my home directory on the server and then use sshfs to mount that directory locally. I do my work there, and then ssh in to the working directory, fire off git commit -a and then cd to the repo to pull from the working directory. Of course this seriously hampers my ability to work offline (as in I can’t). While I am sure there are plenty of good things about Git I am not yet really seeing them. With the exception of git stash I have yet to be wowed by anything Git has to offer. In the interest of fairness, however, I have not been incredibly wowed by anything SVN has to offer either, I just find it easier to use.

PHP

SPDO moving to SourceForge (I hope)

While school has kept me busy during my non-work hours I have not had much time to tend to other projects like SPDO. Seeing how my “Abandoned Project Takeover” has been approved on SourceForge, however, has encouraged me to set aside some time to do some house cleaning on the code to get the 1.0b ready to bump to 1.0 stable. According the information they have given me it takes 2 to 3 weeks to fully process an “APT” so I do have a little time (although not a lot). This is not the first time I have tried to do this, but last time it never made it to the approval stage and languished in the queue until it timed out. So, now that it was approved and has been in the queue for 4 business days I am hopeful.

In a lot of ways, moving the project to SourceForge is a lot like making a physical change of residence. It will require updating links, setting up the pages there including wiki, bug tracking, etc, and moving the actual code base over. (Good thing it’s small!)

As always, any suggestions, bugs, or feature requests are welcome, and (for now) should be posted at the page set aside for just that.

Gentoo

Gentoo: Spamassassin crashed! What?

Life has been rather hectic lately, and a few things have fallen by the wayside in the interim, including simple maintenance tasks like, oh, mowing the lawn, and cleaning out the folder of spam for bayseian analysis, and other fun items. So when my inbox started to be flooded with what seemed like ten times the normal amount of spam I get (very little of which Mail.app leaves in the Inbox) I decided to take a quick look at the server and see what was going on.

I ran a quick /etc/init.d/spamd status and got the response *status: crashed – Uh oh. A quick check of running processes showed that spamd (the SpamAssassin daemon) was indeed not running. I did a quick restart and thought nothing of it. Spam traffic slowed right back down. For about 10 hours. Suddenly I was getting hammered again. Checked the server and once again *status: crashed – “Ok”, I thought. I restarted it yet again and then immediately checked the status. Again it showed as crashed. Checking the running processes with ps -a | grep spamd showed something different, however. It showed the SpamAssassin daemon happily chugging along.

A bit of searching brought me to the relevant bug report for Gentoo. It seems that there is something in the way SpamAssassin starts the daemon that causes the pidfile to not be written, which leaves OpenRC with the notion that it has crashed. The fix is there in the bug report, though. By changing the spamd init file ( /etc/init.d/spamd ) with the addition of a forced pidfile SpamAssassin starts and OpenRC is happy as well. All by adding the one line you see below (in bold):

        start-stop-daemon --start --quiet \
                --nicelevel ${SPAMD_NICELEVEL:-0} \
                --pidfile ${PIDFILE} \
                --exec /usr/sbin/spamd -- -d -r ${PIDFILE} "" \
                        ${SPAMD_OPTS}

Now if I could just find a one-line fix for an overgrown lawn. And dandelions …

[Edit]:

The reason for the actual crashing was due to the overgrown spam folder for bayseian analysis. It seems that running the sa-learn utility against a 6 GB directory causes all sorts of Bad Things TM to hapen, as it parses every file (again) to determine whether it can “learn” from it. I cleaned out the directory and have had no problems since then.

Home

New Beginnings

Just a short post to update all my regular readers out there (yes, both of you) that I start my new job next Wednesday and I started school full-time today. Yes, today. I have started in the BS – IT Security Emphasis program at Western Governor’s University. I don’t have a projected finish date yet, but their competency-based model means that I can finish my degree much faster than with a standard program. (Word from one of the admissions counselors is that IT professionals going in to the IT program tend to finish their Bachelor in around 2 years.)

Although I shouldn’t have to remind anyone I will: everything I say (well, type) here is my opinion and does not reflect the views or opinions of my employer or the school which I attend. With that said I have only one small grievance about my first day at WGU: every link in the student portal has a target of _blank or uses a js call to open a new window. Grrrr. My opinion: bad design idea.