Over at Flare we're dealing with a lot of traffic on just one endpoint backed by AWS Lambda. Our base approach here is "store first, process later" (later meaning just seconds later on a async queue). However, now more than ever it's important to keep the reporting endpoint performant.

There are a couple of interesting PHP profilers out there that can help us to to so. Xdebug is probably one of the most wide-spread solutions but for Flare we chose to use Blackfire as it features an easy solution to get production profiling data without being too disruptive.

In this blogpost we'll take a quick look at how Blackfire is set-up and how Lambda layers works. In the second part of this post we'll add the two together to get production profiling data from Lambda into Blackfire.

Blackfire basics

If you've ever installed Blackfire on a more traditional server (or even locally) you'll know that there are two important moving parts:

1. the Blackfire PHP extension or "PHP probe"

2. and the Blackfire agent.

I like to think of the agent as a proxy service to Blackfire's APIs. It reads profiling data from a unix socket or TCP address, does some processing and then sends the data off to our Blackfire account using the configured server ID and token.

The PHP probe is nothing more than a PHP extension. It will read profiling data from the PHP process and send it to the Blackfire agent. That's why its most important configuration option is the BLACKFIRE_AGENT_SOCKET value. This configures how the PHP probe can communicate with the Blackfire agent. This is important because the agent doesn't necessarily run on the same machine as the PHP probe. This will come in handy for getting Blackfire to work on Lambda.

Intro to Lambda layers

It's like Shrek once said: "Lambdas are like onions. Onions have layers. Lambdas have layers". Every layer adds some additional code, libraries or runtimes on top of your Lambda's source code.

Compared to a more traditional Lambda that uses the built-in NodeJS runtime, PHP isn't one of the default Lambda runtimes. That's why every Laravel Vapor Lambda deploys with a PHP layer that contains the necessary binaries and configuration options to execute your PHP code.

These layers may sound like magic binary files containing some low-level unix voodoo but they're actually just ZIP archives containing files. If you unzip the PHP layer you should see some familiar looking directories containing a PHP binary and various extensions and CLI tools.

Lambda layers are typically used to add extensions to the base runtime. The vapor.yml file can be configured to include up to 4 layers in the application's deployment. A typical use-case for adding layers to Vapor's Lambdas is to install extensions like Imagick. We'll use it to install the Blackfire probe PHP extension.

Putting one and two together

Now we've got the theory out of the way, here's a breakdown of the steps we'll take to install Blackfire on Laravel Vapor:

1. Get a Blackfire account (this one's obvious)

2. Install the Blackfire agent on a separate EC2 instance

3. Build the Blackfire probe as a Lambda layer

4. Add the Lambda layer to the app's deployment configuration

5. Configure the Blackfire probe layer

6. ...Profit (or debug)

1. Installing the Blackfire agent

If you skipped the intro to Blackfire you might not know that Blackfire needs more than just a PHP extension to run. It also needs the Blackfire agent to function. Sadly, we can't simply install the agent on a Lambda function. We'll need a separate EC2 instance in the same security group to host the agent.

Luckily, if you're using Laravel Vapor, you might already have a "jump box" set-up to use as a SSH proxy into various services. This jump box is nothing more than a pre-configured EC2 instance that's already set-up to communicate with other services your app might need, including AWS Lambda. This means it's perfect for hosting the Blackfire agent. If you don't already have a jump box, you can create one using the vapor jump command.

For installing the Blackfire agent on this EC2 instance you can follow the instructions provided by Blackfire. They go over adding a package repository, installing and configuring the agent and finally starting it. You can skip the part about "Installing the PHP probe" as we'll take a different approach for Lambda.

The next step is to configure the agent to listen to a TCP address instead of the default unix socket. This allows us to connect to the agent from outside of the local machine. You can do this by editing the /etc/default/blackfire-agent config file using nano or vim and changing the SOURCE value to "tcp://0.0.0.0:8307".

Don't forget to restart the agent afterwards using the following command:

sudo /etc/init.d/blackfire-agent restart

Finally we need to configure the EC2 instance to accept incoming connections from any IP on port 8307. This can be changed in the section on "inbound rules" for the EC2 instance's security group
(EC2 instance configuration > security tab > clicking through to the security group > edit inbound rules). The resulting rules should include something like this:

|-Type-------|-Protocol-|-Port-|-Source----|
|------------|----------|------|-----------|
| Custom TCP | TCP      | 8307 | 0.0.0.0/0 |
| Custom TCP | TCP      | 8307 | ::/0      |

2. Building the Blackfire probe Lambda layer

Neither Blackfire nor Laravel provide a pre-built Blackfire probe extension layer for Vapor. This means we'll have to build our own. Luckily the people behind Bref (another serverless PHP framework) made this rather easy for us.

Start by cloning the brefphp/extra-php-extensions repository from GitHub. Make sure you have Docker running locally and execute the following command to build the Blackfire layer for PHP 7.4:

make docker-images layer=blackfire php_version=7.4

After a couple of minutes you can thank the Bref team and take the layer-blackfire-php-74.zip file from the export directory.

3. Deploying a Vapor app with the Blackfire probe layer

To start using our freshly built layer, we'll have to upload it to AWS first. You can do this in the AWS Lambda service section under "Layers". Make sure to give it a descriptive name like blackfire-php-74 and add the layer's ZIP archive we built in the previous step. You don't need to select any runtimes, description or license info.

Once AWS finishes processing your new layer you can copy its "Version ARN". This is the identifier we'll have to use in your application's vapor.yml file. Following the Vapor docs we can add our new layer after the default PHP 7.4 layer. Your vapor.yml file should look a little something like this:

`id: 1337
name: flareapp
environments:
production:
runtime: php-7.4
layers:

• vapor:php-7.4
• arn:aws:lambda:eu-central-1:683058659124:layer:blackfire-php74:1

rest of the file...

`

As you can see, we also included Vapor's own PHP as this is still required to run your codebase.

4. Configuring the Blackfire probe extension

At this point you should have the Blackfire agent running on an EC2 instance and the Blackfire probe layer built and added to your Vapor app. The final step is to configure the Blackfire probe extension to connect to the agent on that separate EC2 instance.

Vapor's PHP layer will check the php/conf.d/ directory in your app's directory for any additional PHP config files. You can add the following Blackfire config file in php/conf.d/blackfire.ini to configure the probe extension to contact the agent on that external server:

extension=/opt/bref-extra/blackfire.so

; Replace the following value with your agent's EC2 domain address
blackfire.agent_socket=tcp://ip-10-0-0-61.eu-central-1.compute.internal:8307

blackfire.agent_timeout=0.25

Finally, that's everything. You're one deploy removed from profiling your production Vapor app! Check out the official docs on profiling your application to get started doing some actual work.

Conclusion

Laravel Vapor could probably do with an official Blackfire layer and some documentation on how to configure it. Luckily for you we've already done some of the research to make your job easier. We can also make your job easier by reporting and solving errors in your Lambda app for you: take a look at Flare's features!