Setting up Laravel Forge and DeployBot.
I’ve used Laravel Forge for quite a while in my workflow. It’s a convenient way to set up servers, quickly add security certificates, and deploy sites. I’ve got a number of my own sites and client sites using it and I’ve been very pleased with it.
The Laravel Forge deployment process
The piece of the process I’ve been thinking about recently is the deployment method I’m using in Forge. The deployment method built into Forge works and even seemed magically when I first started using it. Here’s what happens.
The code that makes up a site is stored in a git repository. Laravel Forge takes the files from the git repo and installs them into an “app” directory (i.e. the web site) with either a manual click on a “deploy” button or automatically upon a change in the repo. After the files are in place on the server, it can run commands on the server for you. Those commands can do things like update your software via Composer (in my case, that would be to update my CMS software), clear cache files, reload PHP, etc. This process seems “magic” compared to the old way I’d done things, uploading files via an FTP client and hoping for the best.
The hole in this “magical” process only came into view after using it for a while. During the deployment process, there was a brief time when the site had a mixed state of files and a visitor to a currently-updating site could be returned an error. It was typically a very short time period, but it did exist.
The other more serious problem was that an unsuccessful deployment could bring the site to a halt. If files on the server in the live directory were being modified and something went wrong and the process stopped, it could leave the site in a broken state. At that moment in time, there was no backup because the only copy of the files on the server were the ones being altered.
An atomic deployment process is a potential solution. The basic idea is that the new code you’re going to deploy is put in a “work in progress” directory and then commands that update your CMS and things like that are done on that same “work in progress” directory. Only after all those steps are completed successfully will that directory be used as the live code base for your site. If something fails along the way, your users are still served the “old” version of the site and you can take your time fixing what went wrong.
I am aware there are some free ‘roll your own’ solutions for atomic deployment. I also know I could grow corn and raise cattle to feed myself and my family.
I wanted a service to do this for me. I know of a few options that I’ve heard good thing about: BuddyWorks, Envoyer (h/t Doug St. John), and DeployBot. There are others too.
I had to start somewhere so I decided I’d try DeployBot. I’ve had recommendations that I trust for the service so I decided to set it up.
I came to DeployBot without any experience using it and I thought it might be helpful to document the steps I went through in getting it working with Laravel Forge.
Before we begin
If you’re interested in DeployBot, there is a good chance your code base is already managed in a git repo. If not, your code base will need to be in one to make this process work. I’ll assume you’ve got that in place already.
Set up your server
DeployBot does not provision your server for you. It expects the server to be built already.
Provisioning servers is what I’ve been using Laravel Forge to do and I’m familiar with it, so that’s the first stop on this journey. In my case, I built a small Digital Ocean server for my testing. I then created my site that would be deployed by DeployBot. By “creating” I simply mean I defined an app, in other words, my web site, in the Forge interface. During this test project, what I created is only the staging version of my site. So basically ‘staging.example.com’. (If all goes well, I would repeat this process for the production site too. I’m still experimenting though, so that’s to be decided later.)
Another difference from my typical process is that I don’t actually connect the server to a git repo within Forge at all. It’s worth mentioning since that is what I’d do at this point. That being said, if you’re retrofitting an existing server, make sure “Quick deploy” is shut off within Forge.
While still within Laravel Forge, in the Site Details of the app (ie, the site), there is a Meta menu item which lets you set the web directory.
Since I’m using Craft 3, the default name of the public facing directory is “web”. Before using DeployBot, I just used “web” as the Web Directory. Because of the way DeployBot does the version releases of a site, you need to point it to
/current/web instead. The “current” directory is a symlink to whatever is the most recent version of your site. DeployBot updates this symlink after a successful deployment of your code.
Side note: At this point, you’ve got your server and you know it’s IP address. You may want to go set up your DNS now. It depends on your situation though. DNS just takes time to propagate so I like to get it out of the way as early as makes sense for the situation.
The process of configuring the DeployBot settings for my server involved quite a bit of trial and error. This is where I hope this post will be most helpful.
Now it’s time to head over to DeployBot. Once you’ve signed up, log into your control panel.
Click the Connect a repository button. Any git host will do, but mine happens to be GitHub. There are permissions to configure which are detailed in DeployBot’s site.
Once you connect your git repository, you need to Add an environment. This is where you make the connection between your git repo and your server. DeployBot will walk you through the permissions required to make this happen.
Since this is the “staging” version of my site, I didn’t want to connect the default “master” branch of my repo. I keep a separate branch that I call “staging” where I work. I keep the master branch for final code only. I also chose to do automatic deployment for the staging site. This means every little change in my staging branch pushed to Github will automatically be deployed to my server. You probably don’t want to do this to a production site.
The server I created via Laravel Forge is basically empty at this point. It will be populated with files from my git repo soon thanks to the process we’re setting up in DeployBot. I need those files to live in the right place though. In my case, that is:
Put that in the server set up in the DeployBot settings.
Following that there are a number of fields that you can optionally customize. This is where I spent much of my time configuring.
The first section for webhooks is one I didn’t use. If I change that later, I’ll update this post.
Add configuration files
I have 2 configuration files that need to be added to my site upon deployment. I don’t keep these files in my git repository because they contain passwords. My site needs the credentials in these files to function so they ultimately need to exist in the “current” directory of the site. This step of the configuration solves that problem.
Let’s talk about the 2 files I need.
First I have a
.env.php file because I use Andrew Welch’s Craft3-Multi-Environment for Craft 3.x config. You might just have a
.env though. That’s more common. This file has database credentials, path info for my CMS, etc.
Then I also have a
.env.sh file because I use another project from Andrew called craft-scripts. The script files help me sync assets between live and staging versions of my site and other important things.
In the past, these 2 credentials files lived in the place I needed them on my server. When Laravel Forge deployed via its git hooks, it just updated the needed files in the live directory and left everything else in the directory untouched. These 2 credential files were exactly where I put them originally and all was good. This doesn’t work anymore when using DeployBot. Since the directory where the site is being served from changes behind the scenes upon a successful deployment, those credentials files would exist only in a previously released directory after a successful deployment.
In this “Add configuration files” area of the DeployBot interface, you will see a Create a new file link. Click that to create configuration files. You can name them what you like, but I keep them the names I expected to see already,
.env. Just paste the text content of your real files into the DeployBot control panel and save. Note that you are creating the files here. You’re not putting them on your server yet. That is set up next.
Back in the server configuration, you’ll now be able to choose the file that was just created and have it placed inside a newly deployed version of your site.
Alternate options for config files
There are a couple other options for config files that are worth mentioning. DeployBot can also deploy via SFTP, FTP, and Shell. These deployment methods would just update the files that have changed in your release directory. So, if you were using any of those methods, the configuration files would not need to be added in the way described above. Since they wouldn’t be changed, they’d stay just as you left them.
Another way to deal with configuration files would be to put them in your shared folder and make a symlink. (Thanks to Dardan from DeployBot for these great tips!)
Compile, compress, or minimize your code
In this section of DeployBot, I’ve left it blank for the time being. Those processes are handled in my local workflow.
Run commands after new version is uploaded
After my Craft site is deployed from Github, it doesn’t contain any of Craft’s application files. It also doesn’t contain any of the plugins in Craft. I don’t keep those Craft app files in Github because they’re just a bunch of big files that I don’t want to have in my repo.
Since those Craft files won’t be in my new site directory after a successful code deployment, I need to have DeployBot run a command to get them. Craft uses Composer to do that. Laravel Forge server builds include Composer as well so I know it will be in installed on my server too. The following command will run composer which will install all my composer managed files, i.e. Craft and the plugins I want to use.
composer install --no-interaction --prefer-dist --optimize-autoloader
Exclude certain paths from being uploaded
I leave the defaults in this section.
Run commands after new version becomes active
I added 3 different commands to this section of the server setup. I’ll show you what I have in this field and then I’ll describe why I’ve done each one below.
# Reload php so the new DeployBot linked folder works as expected echo "" | sudo -S service php7.2-fpm reload # Make craft-script commands executable chmod 700 $RELEASE/scripts/*.sh # Link shared assets for Craft to the new release version just created ln -s $SHARED/assets $RELEASE/web/assets
The first command reloads PHP. In my case that is PHP version 7.2.
This one had me stumped at first. I was having a successful deployment of my code but the “current” symlinked directory was not resolving to my new code base. If I dug around in the terminal I could see that the code was there in the “releases” folder. The symlink to “current” simply seemed to not be pointing at the new code.
I happen to restart my server and I checked the symlink again and it worked. I realized I could restart my server and it worked. My guess was that it probably some cache files were cleared. Maybe I could just reload my PHP and do the same thing. I reloaded the PHP and that also worked. That seemed like a quicker process and restarting my server seemed to be a more drastic step, so reloading the php-fpm process is what I did.
One additional note about the reloading php-fpm. After reading Truly Atomic Deployments with Nginx and PHP FPM I think I may need to revisit this particular part again, but reloading of php is still an inprovement in my process, so I’ll stick with it for now.
The next command is very specific to my use of the craft-script commands. I mentioned these script files above when discussing the configuration files I added to the server. The script files actually live in my git repo. The one piece that does not live in the repo are the configuration files which contain passwords.
Earlier in the deploy process I copied the
env.sh configuration file to the “scripts” directory to sit alongside the script files. There turned out to be a problem though. When I logged into my server to execute one of the script files, it didn’t have execution permissions. The
chmod command changes the file permissions of all the files in the directory that have the extension
.sh. Now when I log into the server, the script files can be executed.
The final line touches on another aspect of how I deploy my site. In the git repository I do not keep any of the files that might have been uploaded into the site, like images for posts. Those live on the server, not in git. Specifically they live on my production server. I use Andrew’s craft – scripts to keep these user-created assets in sync between environments.
This is accomplished by keeping those user files in a directory called “shared”. This is created by DeployBot. It will be sitting along side the “current” directory along with a few others. Initially, the “shared” directory is empty, but this is where I want those user files to live. Basically they need to exist outside the ever changing versions of my site in this “shared” directory.
The “link” command line uses a couple shell variables that were created up when the application path was set. Expanding the “show paths” link up there will show them all.
This means the “assets” directory I keep in my “shared” directory will be symnlinked to the new version of my site files using the $SHARED shell variable. DeployBot made several shell variables earlier in the process. For example there is a $RELEASE shell variable too pointing to the current release version of the files as you can see in the screenshot above.
One way to test these commands is to SSH into your server via the command line and try them. It’s the exact same code. Basically DeployBot will just enter it for you at the right point in the process.
Those are the basic things I did to get my DeployBot configuration working with Laravel Forge.
There are a few other details I didn’t go into but they’re worth mentioning. I needed to update the paths in my “craft-script” configuration files. Since I was bring them over from my non-DeployBot version of my site, I had to updated the assets paths there to point to the “shared” directory.
I also didn’t mention needing to make a database on the new Laravel Forge-created server. You’ll probably need a database. LF makes that simple though. Create a database and upload your data. In my case, I import my database using the “craft-scripts” from the production version of my site but you can use SequelPro, NavCat or just the command line.
This is a first draft of how I did this. What would you do differently? I’m often in the Craft Slack group. Get in touch with me there or on Twitter or send me an email. Good luck!
Based on feedback from Benoît Rouleau, I mentioned that no git connection was needed inside Forge and that “Quick deploy” should be off if you happen to have one connect already.
Doug St. John suggested I check out Envoyer too.
Dardan from DeployBot offered the additional other suggestions for configuration files.