How should I move my code from dev to production?

Question

I have created a PHP web-application.

I have 3 environments: DEV, TEST, PROD.

What's a good tool / business practice for me to move my PHP web-application code from DEV to TEST to the PROD environment?

Realizing that my TEST environment still only connects to my TEST database; whereas, I need to PROD environment to connect to my PROD database. So the code is mostly the same, except that I need to change my TEST code once moved into PROD to connect to the PROD database and not TEST database.

I've heard of people taking down Apache in such away that it doesn't allow new connections and once all the existing connections are idle it simply brings down the web server.

Then people manually copy the code and then manually update the config files of the PHP application to also point to the PROD instance.

That seems terribly dangerous.

Does a best practice exists?

Answer 1

Actually, I don't see any reason why TEST environment should miraculously migrate to PROD without any server shutdowns. TEST production is supposed to be for testing purposes. And even if you are actually TESTING on production server, bring it down (shutdown apache), change one line in your main config file, that is determining what set of minor config files to use) and bring it up again (start apache). This will take not more than 1-3 mins to complete and since you surely not going to do that twelve times a day, you will be fine.

Answer 2

Use configuration files to determine what database you're connecting to. That is, have a DEV configuration file, a TEST configuration file, and a PROD configuration file. It's generally the best way to avoid costly and frustrating mistakes.

Answer 3

1. Have your code in a revision control system (I prefer Subversion (svn)). This makes it easy to keep your DEV, TEST and PROD environments in sync, you don't have to keep track of files you modified. Once you are happy with your modifications on DEV, you commit the changes to svn and then run "svn update" on the TEST and eventually after testing on PROD server. Most linux hosting providers have svn client installed or you can install it yourself.

2. I don't like having a different version of a config file for each site because it requires manually renaming one file and removing the other two. I prefer having DEV, TEST and PROD configurations in the same config file. In the config file I determine which server the code is running on by checking either the hostname or the request url. Then I can have "if" or "switch" statement that would load configuration settings based on which server is currently running the code.

3. You might also need to sync database structure between your servers. I use sqlyog for this purpose, it has a built-in database structure synchronization tool that compares 2 database structures and prepares SQL to synchronize them.

Answer 4

1) Separate configuration from the rest of the code. The rest of the code should then be able to run on all 3 locations without modifications. A typical config file could be:

<? 
$db = "main_db"; $db_user="web1"; $db_pass = "xyz123"; 
$site ="example.com"; 
$htroot = "/var/www/prod/htdocs" 
?>

And for the test environment:

<? 
$db = "test_db"; $db_user="web1"; $db_pass = "xyz123"; 
$site ="test.example.com"; 
$htroot = "/var/www/test/htdocs" 
?>

Do not include the configuration files with the passwords in the code base. The code base may be copied through insecure connections or be stored on third party code hosting servers later (see below). And you maybe don't want your passwords on all your backup disks of the code.

You could also create one single config file and use a switch depending on the environment the code runs:

<? 
$site = $_SERVER["HTTP_HOST"];

if ($site == "example.com"; ) {
  $db = "main_db"; $db_user="web1"; $db_pass = "xyz123"; 
  $htroot = "/var/www/prod/htdocs";
}
if ($site == "test.example.com") { 
  $db = "test_db"; $db_user="web1"; $db_pass = "xyz123"; 
  $htroot = "/var/www/test/htdocs";
}
?>

But now you could be tempted to put it back into the code base, which is less secure as explained above. And if you do not put it there you have to update 3 files, or use one fixed location per server, and you have to make sure the code finds the file on each server. I personally prefer the one-file-per-site solution from above.

2) You have already "versions". There is one version running on prod now. Give it a unique name and number, which will never change again. You can use that version name when you backup the code and when you refer to the version or when you move it somewhere you name the subdiectory that will contian it after the version.

The version that you will put on prod in the near future is a different version, and if you make changes again this is also a different version.

As a rule of thumb: increase the version number when you move or export the code, when you swap or exchange or upgrade between locations, when you make a demo, and after each feature or milestone and each time when you do a full backup.

Please note that the config files (3, one for prod, test and dev) are NOT part of the versions. So you can "move the versions around" but the not the config files. If you can, put the config files OUTSIDE the tree with the rest of the code, so you do not need to separate them and take care when you move around the versions later. You could move the config one directory "up" and access them from the files like this:

"include ../config.php";

3) If you want to use version control systems, they do a great job but it needs some time to get used to it and if you are in a hurry with your update it is probaby not the right time to start live with it now. But I would for the future recommend to use a latest generation distributed version control system. Distributed means you do not need to setup a server and many more advantages. I will name bazaar, if it is necessary to update over ftp it can do. Please note that a version control system makes exchanging a version very fast, because only the differences between the versions are written. Bazaar has a community and documentation which makes it easy to start. There is also Git, which has the most up to date commercial hosting site: http://github.com. You can view the code online and compare between the versions and there are many more helpful features, even if you are the only coder, but in a group it is even better. The choice between the systems is not easy. I can not recommend CVS, which is outdated. Also SVN is not the latest generation of distributed version control system, I would not recommend to use it if there is not a specific reason, and it will pollute all your subdirs with special subdirs, which can be annoying. For peolple who are used to it and have already code in it it is fine, but for a starter I would say don't.

There is also Mercurial and Darcs among the distributed and open source version control systems. Mercurial also has a great commercial site for collaboration and online code view (http://bitbucket.org).

4) As long as you do not use a version control system, how about using symlinks?

You could have a directory on the server src/versions/ somewhere and put the named versions in there, each one in their own subdirectory. You will only add versions (because a version that exists will not be changed, if you change it it becomes a new version)

You could have src/versions/v001/ src/versions/v002/ src/versions/v003/ or whatever naming scheme you use.

Now here comes the trick: /var/www/prod/htdocs is a symlink to src/versions/v001/

When you upgrade to v002 you just do the following:

• shutdown apache
• remove the old symlink /var/www/prod/htdocs (at this point the apache webroot is gone!)
• create the new symlink /var/www/prod/htdocs being a link to src/versions/v002
• start apache

You can also write a script for this with parameters and call it like this:

upgrade-web prod 002

This makes the gap even shorter.

Sometimes you have to do a "fallback", when you find out that the new version has errors in production and you have to go back. This would be easy (because you do not remove the old directories, you just stop apache, delete the symlink and re-create it to the former location, in this case src/versions/v001 )

And if test and dev is on the same server, you can of course also symlink the same directories, so there would be no any for move or copy.

5) If you do it manually without symlinks, why not move instead of copy?

(When the files are not yet on the same server, you can copy them somewhere near, and then start with the migration, so you do not have to stop the server for such a ling time)

If there are several directories on the root level of the project you could move them one at time. Be sure to NOT MOVE the config files. Or find some strategy to bring them bakc. Workflow would be:

• Stop apache
• Move away all current prod directories and files on the root level except config file(s)
• Move all new prod directories and files to the root level except config file(s)
• Start apache

6) try to find your perfect individual file and directory layout and perfect workflow. This takes maybe some time and some thinking, but it pays. Do it on a piece of paper until you find the best solution. This could mean that you have to refactor your code and change server config files, but for the future your life is easier when you do administration and upgrades. From my experience: do not wait so long with this step. Your layout shoudl nmake it easy and safe to upgrade. Upgrading is not somehting extraordinary, it is routine and it should be safe and simple to do.

7) If you name your server and workstation environments (operating system of server is linux I guess, but is it a hosted or a root server, do you have ftp acces or also shell (ssh) access, or sftp? where do you develop, on a windows machine, or a mac?) then people can name tools to do the copying and moving. Also interesting: Is the test and dev server the same machine, if not, how they are connected, or they aren't? If not, you would make a 3-way transfer (Copy it to your local workstation and then to the server).

8) Think of file permissions. If you move files around or copy them, maybe the file permissions change, and if the application depends on some of them there should be a way to check and maybe chang. Example: Some applications need writable directories where they put uploaded files or session files or template caching. Other applications do not allow some files for security to be writable.

Answer 5

When we push updates live, it's not often we have to reboot apache. This maybe a side effect of not having high traffic sites (< 1M pagedraws a month).

We have 3 branches, for various stages of development/QA: alpha (bleeding edge but available for viewing by non-developers and testers), beta (somewhat frozen for a particular release, final QA phase) and live (production).

To migrate from one branch to another, we perform a merge between say alpha and beta, commit that merge. Then run a deployment script which updates the branch from SVN on our development machine, then rsync's the code web servers to the beta document root.

As already mentioned by others, each branch can contain it's own config file with appropriate settings to cater for environment differences.

We are in the process of migrating to git to smooth out the branch merging process, which can be a little traumatic in SVN for large projects/releases.