Flare and Ignition now offer AI-powered solutions

Thursday 8th of June 20238th of June 2023

One of the unique features of Flare is that it can display solutions for your errors. In our code base, we try to detect specific error messages and display a solution when we recognize a specific error.

Today, we're adding AI-powered solutions to Flare. This means we can suggest a solution for almost any error: the AI will likely suggest a helpful suggestion for most errors.

Here's an example where we forgot to configure the region for an S3 disk. "Region must be a valid RFC host label." doesn't say much, but the AI clearly explains that we should pass a region like us-west-2 and even links to the list with all possible regions.

We don't see AI-powered solutions as a "revolutionary" feature: they are just icing on our cake. The best feature of Flare is our UI and hand-crafted optimizations for both Laravel and PHP. Together with AI solutions, this will help you better understand an error and fix them faster.

In this blog post, I'd like to share more about how we implemented AI solutions in both Flare and Ignition.

Introducing Flare

Before heading into AI-land, I'll explain what Flare is to get everybody on the same page. If you're familiar with Flare, skip to the next section.

If something goes wrong in your production environment, and you don't use an error tracker, it is straightforward to miss that an error happened. If you're lucky, one of your users will contact you about it (which is a bit embarrassing), or they won't contact you about it (which is even worse because you won't have a chance to fix things).

When you use an exception tracker like Flare, your app will send the error to Flare, and Flare will send you a notification via Mail, Slack, or ... The notification will contain a link to a full report on the error, giving you as many details as possible. You can use this information to fix your app, even if your user doesn't contact you.

At Spatie, we build and maintain huge apps ourselves. We noticed that existing error trackers have confusing UIs that bombard you with too much information. We built Flare with ease of use in mind. We aim to show you the most relevant information so that you can fix errors faster in all your PHP and JS applications.

If you use Laravel, you already know what Flare looks like. When you have an error happening in your local Laravel app, you'll see an error page. We made that error page - it's called Ignition.

In Flare, errors are displayed similarly as they appear on Ignition; you already know where the important bits of info are.

We've gone the extra mile to report errors from a Laravel app: we recognize Livewire errors; for an error in a queue job, we display information about that queue; view errors will contain the decompiled Blade view, ...

AI-powered solutions in Flare

Getting started with an AI-powered solution in Flare is very easy. Generating an AI solution requires sending your errors to a third party (in this case, OpenAI). We only want to do that with your explicit approval; you need to enable the feature in the AI settings of Flare manually.

When this feature is enabled, we will send the 500 unique incoming errors to OpenAI, where a solution will be generated. Notice that it'll always include a few helpful links to documentation.

Of course, we also display the solution in all notifications we send. Here's an example of a mail notification containing an AI solution.

And here's how a Slack notification looks like:

There's a cost involved in generating AI solutions, and that's why we have a daily limit of 500 errors that will send to OpenAI. Don't worry: if you cross this limit, we'll still accept and display incoming errors, but we'll no longer generate a solution via OpenAI.

If you want to generate more than 500 solutions per day via OpenAI, you can use your own OpenAI key. When using your own key, you can specify your own daily limit.

How this works behind the scenes

Interacting with (Open)AI sounds advanced, but it's pretty simple. In short, we convert the error to a textual prompt and make an API call to OpenAI. The service responds with a solution. The workflow is more or less the same as in this blog post by Marcel.

When AI solutions are enabled, we'll send these parts on an incoming error to Open AI:

the error message
the error class
the stack trace
the framework version
little bits of context to help the AI generate a proper response

We don't send anything revolving around environment variables or request payload to avoid sending passwords to the AI.

To convert an error to a question (aka the prompt) that we'll send to the AI, we use this Blade view. I've simplified it a bit to keep it concise.

You are a talented web programmer.

@if($error->language)
You are working on a {{ $error->language }} {{ $error->languageVersion }} app

	@if($error->framework)
	    that uses {{ $error->framework }} {{ $error->frameworkVersion }}
	@endif
@endif

Use the following context to find a possible fix for the exception message at the end. Limit your answer to 4 or 5 sentences, and include links to documentation that might help. Also, rate, on a scale from 1 to 100, how sure you are that this is the correct fix. Avoid repeating the exception message in your answer.

Use this format in your answer; make sure the links are JSON.

FIX
insert the possible fix here
ENDFIX
LINKS
{"title": "Title link 1", "URL": "URL link 1"}
{"title": "Title link 2", "URL": "URL link 2"}
ENDLINKS
SCORE
score from 1 to 100
ENDSCORE
---

Here comes the context and the exception message:

Line: {{ $error->lineNumber }}

File:
{{ $error->file }}

Snippet including line numbers:


Exception class:
{{ $error->exceptionClass }}

Exception message:
{{ $error->exceptionMessage }}

To send this prompt to Open AI, we use this excellent client.

You can see that in the prompt above, we ask the AI nicely to use the template to format the message, format the links as JSON, and score its own response. Believe it or not, the AI follows these instructions perfectly.

This allows us to parse the response and make decisions based on the score. If the AI score its own answer below a certain threshold, we'll not use the solution. Sometimes the AI responds with links to documentation that result in a 404. That's why we, behind the scenes, have some code to verify that all the links the AI returns are valid.

Because Flare already featured displaying solutions long before the dawn of AI, we already had the necessary DB tables and UI in place. The only thing we added was a flag to remember that a solution was generated by UI. Using this flag, we can visually indicate that the solution was AI-generated, and we also add a little disclaimer message.

Using AI locally in Ignition

Using Flare is probably the easiest way to work with AI solutions, as you only have to enable the feature, and you're done.

But we didn't stop with just adding AI solutions to Flare; we added them to Ignition (both the Laravel and the framework agnostic variant) as well.

In Laravel

Support for AI solutions was added in Laravel-ignition v2.1. Run composer update to pull that in.

Next, you must first install this optional dependency. We need this package to communicate with open AI.

composer require openai-php/client

Finally, you must specify an OpenAI API in the IGNITION_OPEN_AI_KEY key in your .env file. You can generate this key at OpenAI.

We use cache to minimize calls made to Open AI when your application generates similar errors.

With this in place, each error your local app makes will be transmitted to OpenAI, and an AI-generated solution will be displayed next to the error message.

In any PHP app

Laravel comes with the Laravel-specific variant of Ignition installed. That version is based on a framework-agnostic version of Ignition that can be used in any PHP application.

After installing spatie/ignition in your PHP app, you must instantiate the OpenAiSolutionProvider. The constructor expects an OpenAI API key to be passed; you should generate this key at OpenAI.

use \Spatie\Ignition\Solutions\OpenAi\OpenAiSolutionProvider;

$aiSolutionProvider = new OpenAiSolutionProvider($openAiKey);

To use the solution provider, you should pass it to addSolutionProviders when registering Ignition.

\Spatie\Ignition\Ignition::make()
    ->addSolutionProviders([
    $aiSolutionProvider,
    // other solution providers...
    ])
->register();

Let's try it out; here's a simple PHP script with Ignition installed.

use Illuminate\Cache\ArrayStore;
use Illuminate\Cache\Repository;
use Spatie\Ignition\Ignition;
use Spatie\Ignition\Solutions\OpenAi\OpenAiSolutionProvider;

include 'vendor/autoload.php';

$openAiKey = 'sk-your-open-ai-key-here';

Ignition::make()
    ->addSolutionProviders([
        new OpenAiSolutionProvider(
            $openAiKey,
            new Repository(new ArrayStore())
        ),
    ])
    ->register();

throw new Exception('Stop here');

This is what we see in the browser when we try to render a webpage using that script.

Optionally, you can add caching so similar errors will only get sent to OpenAI once. To cache errors, you can call use cache on $aiSolutionProvider. You should pass a simple-cache-implementation. Here's the signature of the use cache method.

public function useCache(CacheInterface $cache, int $cacheTtlInSeconds = 60 * 60)

To increase the quality of the suggested solutions, you can send along the application type (Symfony, Drupal, WordPress, ...) to the AI.

To send the application type, call application type on the solution provider.

$aiSolutionProvider->applicationType('Symfony 6');

In closing

We hope that you like the addition of AI solutions to Flare and Ignition. As said in the intro, we think that AI solutions will not revolutionize our service (it's already great without AI solutions), but that AI can help you better in many situations.

For now, we've only enabled AI solutions for PHP and Laravel applications. In the next few weeks, we'll polish our prompt based on our users' feedback. After that, we'll experiment if we can give good solutions for JavaScript errors.

Flare is the best error tracker for Laravel and PHP projects. We display the most important aspects of issues in your production environment in our beautiful UI.

Now is the perfect time to try out Flare. You can register here for a free ten-day trial; no credit card is required.