Archives for category: Configuration

Naming Variables Sanely

August 2, 09 //
0

I had a simple goal. Well, someone else had a simple goal, but it was my task to fulfill it: raise the maximum number of files that could be listed in a directory in an ftp client. Pure FTP by default limits this to 2000, a reasonable limit in my opinion. But no, them what pays the bills needed it higher.

It took me fifteen minutes to find the variable that needs to be set in Pure FTP’s config file (and not because of a deficiency in my google-fu):

LimitRecursion

Oh, the things that I googled! The terms that I grepped! Things like “max” and “num” and “files”. I finally stumbled upon “limit” and found the bugger. Yet fifteen minutes! Far too much time for such a simple task.

Naming that variable LimitRecursion makes sense when you’re programming. You’re writing an ftp server, and suddenly you realize “hey, this directory with a crapton of files is nomming my resources, thanks to recursion. Let’s limit the amount of recursion that can occur and keep the server happy in the process. Let’s see, I’m trying to limit recursion… hmm… what should I name my variable…”

But when you answer LimitRecursion you are wrong sir. Because this program will be used by people who don’t give a fig how you keep the server up (limiting recursion) but do give a fig about the maximum number of files they can display.

When you name a variable, or create an API, please, for the love of Pete and end users and sysadmins everywhere, NAME IT LIKE YOU’RE AN END USER. By doing so, you GREATLY lessen your chances of being injured by an end user with a voodoo doll.

Disabling Update Notifier Pop-up in Ubuntu 9.04

July 30, 09 //
1

It seems that some ad-wizard at Canonical decided that the tray icon that notifies you of available updates in Ubuntu was simply too well liked. Thus, in 9.04, they decided to make the update manager automatically pop open any time there are updates found no matter what you are doing. Luckily, for those of us that actually enjoy an unobtrusive user interface, restoring the update manager to it’s original tray icon glory is a single command away. Simply run the following and your web 1.0 auto pop-up annoyances with Update Notifier will decrease a full 100%!

gconftool -s --type bool /apps/update-notifier/auto_launch false

To make the change take effect you will need to restart the Update notifier, or log out and log back in.

Recompiling Apache to use mpm_worker on a CPanel Server

July 11, 09 //
0

Arguably, the most important service that runs on a server is its webserver, and for most of the world this webserver is Apache. One of Apache’s main selling points, its modularity, is also its biggest drawback if not configured properly. Apache 2.0 and newer can be compiled with Multi Process Modules, or MPMs. The default MPM is prefork. Prefork is based off of the aging Apache 1.3 and works fine, but does have its disadvantages: it doesn’t handle large amounts of traffic gracefully nor is it really designed to handle newer hardware.

This is where mpm_worker comes in. Worker is a threaded MPM that is designed for newer hardware and handles larger amounts of traffic better. While prefork is the Apache default MPM, Debian based distrubutions run worker by default. Information regarding worker is available at the Apache Documentation.

“But,” you’re asking yourself, “I run a CPanel dedicated server and/or virtual private server at my webhost. Can I use worker?” Absolutely. CPanel provides the aptly named EasyApache script that allows for recompiles of Apache and PHP using an easy to navigate interface.

Before I get into the nitty gritty, a few caveats:

  1. If you’re in a shared hosting or reseller environment using CPanel, you’re probably out of luck. Most hosting providers do not allow for software changes in such situations.
  2. The Webhost Manager does have a web interface for EasyApache. I wouldn’t use it, because if your browser crashes during the EA, it’ll Break Everything™.
  3. Unless you are running a module that requires non threaded Apache, upgrading to worker will not cause any problems. Upgrading to Worker will not affect your PHP code, or your databases or your Google Analytics.

First thing, You’ll want to check to see what version of Apache you’re running, because if you’re running worker already then you’re already set. You can figure out the version of Apache by running the following command:

/usr/local/apache/bin/httpd -V

Which will give output like the following:

version

The line you’re looking for is the “APACHE_MPM_DIR”. As the screen shot shows, this server has Apache running with prefork, so it needs an upgrade. You launch the EasyApache script with the following command.

/scripts/easyapache

If your server is in a virtual environment using Xen, the EasyApache script will require a flag when it is executed. This flag will be provided when the script is ran. The technical reason is because the Xen console doesn’t support curses, which the EasyApache script uses.

A warning, I highly recommend running the EasyApache in a screen. As stated earlier, stopping an EA while its doing its thing will Break Everything™.

You’ll want to Customize based off of current profile and make sure that your compile Apache 2.0 or 2.2. Unless you’re upgrading PHP also, the next couple of windows can be skipped. On the Short Options List, pick the Exhaustive Options List. Once you’re in the exhaustive list, scroll until you hit the MPMs. It’ll look like this.

ea1

Unselect any selected MPMs and select Worker. More often then not, no MPM will be selected, which is fine because Prefork will be installed if nothing will be selected. The screen shot is with Apache 2.2 select, this screen will be different if you’re compiling Apache 2.0.

Once you select Worker, the following screen will pop up:

ea2

Select OK and hit enter. The warning is there because the non prefork or worker MPMs are experimental will probably not work most of the time. Worker is completely stable, so its safe to ignore this warning.

After that warning, navigate to the end and click save and build. With that, you have to wait for EasyApache to do its thing. This normally takes about 30 minutes, but can take longer depending on server load and how many modules are selected in EasyApache.

WARNING: Do not close your shell window until the EasyApache is done, or it will break things in interesting and fun ways. You DID remember to run this in a screen, right?

Once it’s done, you’ll want to check that it finished correctly. I usually do this by restarting Apache. Generally, if the EasyApache failed, Apache will error on restart.  After Apache restarts, you’ll want to check to make sure that Worker was installed, running the version check command from earlier. If everything went according to plan, you should get something like this:

final

If you’ve got something like that, then everything worked correctly. Now we’ll want to check to make sure Apache has its optimization directives in place. Sometimes EasyApache includes these directives, other times it doesn’t and I’m not really sure why. You’ll want to search for a block of text towards the top of the config at /usr/local/apache/conf/httpd.conf like the following. If its not there, add it. This block of text will be before the vhosts.

Timeout 300
KeepAlive On
MaxKeepAliveRequests 150
KeepAliveTimeout 5
<IfModule worker.c>
ServerLimit 16
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 500
</IfModule>

Once you’ve added these directives, you’ll want to restart Apache so they can take effect. These directives will probably need to be scaled depending on your server’s traffic load and how much resources the server has, but they do work well as a starting point.

And you’re done! Apache is now running the mpm_worker module. If your server has a high traffic load, the difference should be immediate. Slower traffic servers may not notice it until later.

Distributed Python Development and Deployment with Git and Virtualenv

July 7, 09 //
5

I’m setting this up for my own python development, and so far it’s working LIKE A BOSS. Using my workstation as a development environment, and a server as the production environment, I’m synergizing the deliverables by maximizing throughput in the signal/noise ratio.

Erm.

That is, I’m using virtualenv (and a set of popular virtualenv wrapper scripts) along with git to keep development and deployment of python projects with varying dependencies in sync.

It’s disturbingly easy to set up:

  1. Install virtualenv
  2. Install virtualenvwrapper
  3. Put the virtualenvwrapper directory in version control
  4. create a virtualenv for each project

It’s that easy!!!

I’m a fan of using pip to install everything. It’s the future of python package management because it’s so flippin’ easy to use, as well as being sanely implemented.  The copypasta for the above setup runs something like this:

pip install virtualenv
pip install virtualenvwrapper
mkdir ~/.virtualenvs

Add the following to your .bashrc:

export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/virtualenvwrapper_bashrc

then:

source ~/.bashrc
mkvirtualenv projectname
cd ~/.virtualenvs
git init

Now feel free to start using pip to build up the particular python environment the project will need, commiting the ~/.virtualenv directory to git as needed. When you’ve got your python libraries how you like, install virtualenv on the production server, and add it to the repository for your project. I’m thinking I’ll set it as a git submodule for Django projects, but so far I’ve just been keeping it as a separate repository for non-public in-development non-website projects.