Setting up Laravel Forge and DeployBot.

I’ve used Lar­avel Forge for quite a while in my work­flow. It’s a con­ve­nient way to set up servers, quick­ly add secu­ri­ty cer­tifi­cates, and deploy sites. I’ve got a num­ber of my own sites and client sites using it and I’ve been very pleased with it.

The Laravel Forge deployment process

The piece of the process I’ve been think­ing about recent­ly is the deploy­ment method I’m using in Forge. The deploy­ment method built into Forge works and even seemed mag­i­cal­ly when I first start­ed using it. Here’s what hap­pens.

The code that makes up a site is stored in a git repos­i­to­ry. Lar­avel Forge takes the files from the git repo and installs them into an ​“app” direc­to­ry (i.e. the web site) with either a man­u­al click on a ​“deploy” but­ton or auto­mat­i­cal­ly upon a change in the repo. After the files are in place on the serv­er, it can run com­mands on the serv­er for you. Those com­mands can do things like update your soft­ware via Com­pos­er (in my case, that would be to update my CMS soft­ware), clear cache files, reload PHP, etc. This process seems ​“mag­ic” com­pared to the old way I’d done things, upload­ing files via an FTP client and hop­ing for the best.

The hole in this ​“mag­i­cal” process only came into view after using it for a while. Dur­ing the deploy­ment process, there was a brief time when the site had a mixed state of files and a vis­i­tor to a cur­rent­ly-updat­ing site could be returned an error. It was typ­i­cal­ly a very short time peri­od, but it did exist.

The oth­er more seri­ous prob­lem was that an unsuc­cess­ful deploy­ment could bring the site to a halt. If files on the serv­er in the live direc­to­ry were being mod­i­fied and some­thing went wrong and the process stopped, it could leave the site in a bro­ken state. At that moment in time, there was no back­up because the only copy of the files on the serv­er were the ones being altered.

Atomic deployments

An atom­ic deploy­ment process is a poten­tial solu­tion. The basic idea is that the new code you’re going to deploy is put in a ​“work in progress” direc­to­ry and then com­mands that update your CMS and things like that are done on that same ​“work in progress” direc­to­ry. Only after all those steps are com­plet­ed suc­cess­ful­ly will that direc­to­ry be used as the live code base for your site. If some­thing fails along the way, your users are still served the ​“old” ver­sion of the site and you can take your time fix­ing what went wrong.

I am aware there are some free ​‘roll your own’ solu­tions for atom­ic deploy­ment. I also know I could grow corn and raise cat­tle to feed myself and my fam­i­ly.

I want­ed a ser­vice to do this for me. I know of a few options that I’ve heard good thing about: Bud­dy­Works, Envoy­er (h/​t Doug St. John), and Deploy­Bot. There are oth­ers too.

I had to start some­where so I decid­ed I’d try Deploy­Bot. I’ve had rec­om­men­da­tions that I trust for the ser­vice so I decid­ed to set it up.

I came to Deploy­Bot with­out any expe­ri­ence using it and I thought it might be help­ful to doc­u­ment the steps I went through in get­ting it work­ing with Lar­avel Forge.

Before we begin

If you’re inter­est­ed in Deploy­Bot, there is a good chance your code base is already man­aged in a git repo. If not, your code base will need to be in one to make this process work. I’ll assume you’ve got that in place already.

Set up your server

Deploy­Bot does not pro­vi­sion your serv­er for you. It expects the serv­er to be built already.

Pro­vi­sion­ing servers is what I’ve been using Lar­avel Forge to do and I’m famil­iar with it, so that’s the first stop on this jour­ney. In my case, I built a small Dig­i­tal Ocean serv­er for my test­ing. I then cre­at­ed my site that would be deployed by Deploy­Bot. By ​“cre­at­ing” I sim­ply mean I defined an app, in oth­er words, my web site, in the Forge inter­face. Dur­ing this test project, what I cre­at­ed is only the stag­ing ver­sion of my site. So basi­cal­ly ​‘stag​ing​.exam​ple​.com’. (If all goes well, I would repeat this process for the pro­duc­tion site too. I’m still exper­i­ment­ing though, so that’s to be decid­ed lat­er.)

Anoth­er dif­fer­ence from my typ­i­cal process is that I don’t actu­al­ly con­nect the serv­er to a git repo with­in Forge at all. It’s worth men­tion­ing since that is what I’d do at this point. That being said, if you’re retro­fitting an exist­ing serv­er, make sure ​“Quick deploy” is shut off with­in Forge.

While still with­in Lar­avel Forge, in the Site Details of the app (ie, the site), there is a Meta menu item which lets you set the web direc­to­ry.

Since I’m using Craft 3, the default name of the pub­lic fac­ing direc­to­ry is ​“web”. Before using Deploy­Bot, I just used ​“web” as the Web Direc­to­ry. Because of the way Deploy­Bot does the ver­sion releas­es of a site, you need to point it to /current/web instead. The ​“cur­rent” direc­to­ry is a sym­link to what­ev­er is the most recent ver­sion of your site. Deploy­Bot updates this sym­link after a suc­cess­ful deploy­ment of your code.

Side note: At this point, you’ve got your serv­er and you know it’s IP address. You may want to go set up your DNS now. It depends on your sit­u­a­tion though. DNS just takes time to prop­a­gate so I like to get it out of the way as ear­ly as makes sense for the sit­u­a­tion.

DeployBot configuration

The process of con­fig­ur­ing the Deploy­Bot set­tings for my serv­er involved quite a bit of tri­al and error. This is where I hope this post will be most help­ful.

Now it’s time to head over to Deploy­Bot. Once you’ve signed up, log into your con­trol pan­el.

Click the Con­nect a repos­i­to­ry but­ton. Any git host will do, but mine hap­pens to be GitHub. There are per­mis­sions to con­fig­ure which are detailed in Deploy­Bot’s site.

Once you con­nect your git repos­i­to­ry, you need to Add an envi­ron­ment. This is where you make the con­nec­tion between your git repo and your serv­er. Deploy­Bot will walk you through the per­mis­sions required to make this hap­pen.

Since this is the ​“stag­ing” ver­sion of my site, I did­n’t want to con­nect the default ​“mas­ter” branch of my repo. I keep a sep­a­rate branch that I call ​“stag­ing” where I work. I keep the mas­ter branch for final code only. I also chose to do auto­mat­ic deploy­ment for the stag­ing site. This means every lit­tle change in my stag­ing branch pushed to Github will auto­mat­i­cal­ly be deployed to my serv­er. You prob­a­bly don’t want to do this to a pro­duc­tion site.

The serv­er I cre­at­ed via Lar­avel Forge is basi­cal­ly emp­ty at this point. It will be pop­u­lat­ed with files from my git repo soon thanks to the process we’re set­ting up in Deploy­Bot. I need those files to live in the right place though. In my case, that is:

/home/forge/dev.example.com

Put that in the serv­er set up in the Deploy­Bot set­tings.

Fol­low­ing that there are a num­ber of fields that you can option­al­ly cus­tomize. This is where I spent much of my time con­fig­ur­ing.

Webhooks

The first sec­tion for web­hooks is one I did­n’t use. If I change that lat­er, I’ll update this post.

Add configuration files

I have 2 con­fig­u­ra­tion files that need to be added to my site upon deploy­ment. I don’t keep these files in my git repos­i­to­ry because they con­tain pass­words. My site needs the cre­den­tials in these files to func­tion so they ulti­mate­ly need to exist in the ​“cur­rent” direc­to­ry of the site. This step of the con­fig­u­ra­tion solves that prob­lem.

Let’s talk about the 2 files I need.

First I have a .env.php file because I use Andrew Welch’s Craft3-Mul­ti-Envi­ron­ment for Craft 3.x con­fig. You might just have a .env though. That’s more com­mon. This file has data­base cre­den­tials, path info for my CMS, etc.

Then I also have a .env.sh file because I use anoth­er project from Andrew called craft-scripts. The script files help me sync assets between live and stag­ing ver­sions of my site and oth­er impor­tant things.

In the past, these 2 cre­den­tials files lived in the place I need­ed them on my serv­er. When Lar­avel Forge deployed via its git hooks, it just updat­ed the need­ed files in the live direc­to­ry and left every­thing else in the direc­to­ry untouched. These 2 cre­den­tial files were exact­ly where I put them orig­i­nal­ly and all was good. This does­n’t work any­more when using Deploy­Bot. Since the direc­to­ry where the site is being served from changes behind the scenes upon a suc­cess­ful deploy­ment, those cre­den­tials files would exist only in a pre­vi­ous­ly released direc­to­ry after a suc­cess­ful deploy­ment.

In this ​“Add con­fig­u­ra­tion files” area of the Deploy­Bot inter­face, you will see a Cre­ate a new file link. Click that to cre­ate con­fig­u­ra­tion files. You can name them what you like, but I keep them the names I expect­ed to see already, .env.php and .env. Just paste the text con­tent of your real files into the Deploy­Bot con­trol pan­el and save. Note that you are cre­at­ing the files here. You’re not putting them on your serv­er yet. That is set up next.

Back in the serv­er con­fig­u­ra­tion, you’ll now be able to choose the file that was just cre­at­ed and have it placed inside a new­ly deployed ver­sion of your site.

Alternate options for config files

There are a cou­ple of oth­er options for con­fig files that are worth men­tion­ing. Deploy­Bot can also deploy via SFTP, FTP, and Shell. These deploy­ment meth­ods would update the files that have changed in your release direc­to­ry. So, if you were using any of those meth­ods, the con­fig­u­ra­tion files would not need to be added in the way described above. Since they would­n’t be changed, they’d stay just as you left them.

Anoth­er way to deal with con­fig­u­ra­tion files would be to put them in your shared fold­er and make a sym­link. (Thanks to Dar­d­an from Deploy­Bot for these great tips!)

Compile, compress, or minimize your code

In this sec­tion of Deploy­Bot, I’ve left it blank for the time being. Those process­es are han­dled in my local work­flow.

Run commands after new version is uploaded

After my Craft site is deployed from Github, it does­n’t con­tain any of Craft’s appli­ca­tion files. It also does­n’t con­tain any of the plu­g­ins in Craft. I don’t keep those Craft app files in Github because they’re just a bunch of big files that I don’t want to have in my repo.

Since those Craft files won’t be in my new site direc­to­ry after a suc­cess­ful code deploy­ment, I need to have Deploy­Bot run a com­mand to get them. Craft uses Com­pos­er to do that. Lar­avel Forge serv­er builds include Com­pos­er as well so I know it will be in installed on my serv­er too. The fol­low­ing com­mand will run com­pos­er which will install all my com­pos­er man­aged files, i.e. Craft and the plu­g­ins I want to use.

composer install --no-interaction --prefer-dist --optimize-autoloader

Onward.

Exclude certain paths from being uploaded

I leave the defaults in this sec­tion.

Run commands after new version becomes active

I added 3 dif­fer­ent com­mands to this sec­tion of the serv­er set­up. I’ll show you what I have in this field and then I’ll describe why I’ve done each one below.

# Reload php so the new DeployBot linked folder works as expected
echo "" | sudo -S service php7.2-fpm reload

# Make craft-script commands executable
chmod 700 $RELEASE/scripts/*.sh

# Link shared assets for Craft to the new release version just created
ln -s $SHARED/assets $RELEASE/web/assets

The first com­mand reloads PHP. In my case that is PHP ver­sion 7.2.

This one had me stumped at first. I was hav­ing a suc­cess­ful deploy­ment of my code but the ​“cur­rent” sym­linked direc­to­ry was not resolv­ing to my new code base. If I dug around in the ter­mi­nal I could see that the code was there in the ​“releas­es” fold­er. The sym­link to ​“cur­rent” sim­ply seemed to not be point­ing at the new code.

I hap­pen to restart my serv­er and I checked the sym­link again and it worked. I real­ized I could restart my serv­er and it worked. My guess was that it prob­a­bly some cache files were cleared. Maybe I could just reload my PHP and do the same thing. I reloaded the PHP and that also worked. That seemed like a quick­er process and restart­ing my serv­er seemed to be a more dras­tic step, so reload­ing the php-fpm process is what I did.

One addi­tion­al note about the reload­ing php-fpm. After read­ing Tru­ly Atom­ic Deploy­ments with Nginx and PHP FPM I think I may need to revis­it this par­tic­u­lar part again, but reload­ing of php is still an inprove­ment in my process, so I’ll stick with it for now.

The next com­mand is very spe­cif­ic to my use of the craft-script com­mands. I men­tioned these script files above when dis­cussing the con­fig­u­ra­tion files I added to the serv­er. The script files actu­al­ly live in my git repo. The one piece that does not live in the repo are the con­fig­u­ra­tion files which con­tain pass­words.

Ear­li­er in the deploy process I copied the env.sh con­fig­u­ra­tion file to the ​“scripts” direc­to­ry to sit along­side the script files. There turned out to be a prob­lem though. When I logged into my serv­er to exe­cute one of the script files, it didn’t have exe­cu­tion per­mis­sions. The chmod com­mand changes the file per­mis­sions of all the files in the direc­to­ry that have the exten­sion .sh. Now when I log into the serv­er, the script files can be exe­cut­ed.

The final line touch­es on anoth­er aspect of how I deploy my site. In the git repos­i­to­ry I do not keep any of the files that might have been uploaded into the site, like images for posts. Those live on the serv­er, not in git. Specif­i­cal­ly they live on my pro­duc­tion serv­er. I use Andrew’s craft – scripts to keep these user-cre­at­ed assets in sync between envi­ron­ments.

This is accom­plished by keep­ing those user files in a direc­to­ry called ​“shared”. This is cre­at­ed by Deploy­Bot. It will be sit­ting along side the ​“cur­rent” direc­to­ry along with a few oth­ers. Ini­tial­ly, the ​“shared” direc­to­ry is emp­ty, but this is where I want those user files to live. Basi­cal­ly they need to exist out­side the ever chang­ing ver­sions of my site in this ​“shared” direc­to­ry.

The ​“link” com­mand line uses a cou­ple shell vari­ables that were cre­at­ed up when the appli­ca­tion path was set. Expand­ing the ​“show paths” link up there will show them all.

This means the ​“assets” direc­to­ry I keep in my ​“shared” direc­to­ry will be symn­linked to the new ver­sion of my site files using the $SHARED shell vari­able. Deploy­Bot made sev­er­al shell vari­ables ear­li­er in the process. For exam­ple there is a $RELEASE shell vari­able too point­ing to the cur­rent release ver­sion of the files as you can see in the screen­shot above.

One way to test these com­mands is to SSH into your serv­er via the com­mand line and try them. It’s the exact same code. Basi­cal­ly Deploy­Bot will just enter it for you at the right point in the process.

Wrapping up

Those are the basic things I did to get my Deploy­Bot con­fig­u­ra­tion work­ing with Lar­avel Forge.

There are a few oth­er details I did­n’t go into but they’re worth men­tion­ing. I need­ed to update the paths in my ​“craft-script” con­fig­u­ra­tion files. Since I was bring them over from my non-Deploy­Bot ver­sion of my site, I had to updat­ed the assets paths there to point to the ​“shared” direc­to­ry.

I also did­n’t men­tion need­ing to make a data­base on the new Lar­avel Forge-cre­at­ed serv­er. You’ll prob­a­bly need a data­base. LF makes that sim­ple though. Cre­ate a data­base and upload your data. In my case, I import my data­base using the ​“craft-scripts” from the pro­duc­tion ver­sion of my site but you can use Sequel­Pro, Nav­Cat or just the com­mand line.

This is a first draft of how I did this. What would you do dif­fer­ent­ly? I’m often in the Craft Slack group. Get in touch with me there or on Twit­ter or send me an email. Good luck!

Updates

Based on feed­back from Benoît Rouleau, I men­tioned that no git con­nec­tion was need­ed inside Forge and that ​“Quick deploy” should be off if you hap­pen to have one con­nect­ed already.

Doug St. John sug­gest­ed I check out Envoy­er too.

Dar­d­an from Deploy­Bot offered the addi­tion­al oth­er sug­ges­tions for con­fig­u­ra­tion files.