Custom Algolia Engine for Laravel to reduce indexing operations

In my previous post, Algolia’s new pricing for 2019: Starter vs Essential plan, I switched operation & indexing -heavy projects over to Algolia’s new Starter plan. I’m excited about the new plan, as I always felt that the included 100k Operations count in the previous Essential plan was too low.

Even with the increased 250k operations included in the Starter plan, I’ll be utilizing a custom Algolia engine for Laravel which compares the updated_at timestamp using a query.

The default Algolia Engine in Laravel will perform an update on all records (even if no changes were made, see Laravel Scout Issue #226: ->searchable() update everything to Algolia, even with no changes), each counting as an operation. As object queries do not count as operations (see sources), we can determine if the model should be updated by comparing the record’s updated_at timestamp with the timestamp in the Algolia record.

The engine is based on elfeffe’s CustomAlgoliaEngine.php, which compare’s the entire record objects recursively.

Using the custom engine

Create a new AdvancedAlgoliaEngine.php file in App\Engines (Gist):

 <?php

 namespace App\Engines;

 use Illuminate\Support\Collection;
 use Laravel\Scout\Engines\AlgoliaEngine;

 class AdvancedAlgoliaEngine extends AlgoliaEngine
 {

     protected $index;

     public function update($models)
     {
         if ($models->isEmpty()) {
             return;
         }

         $index = $this->algolia->initIndex($models->first()->searchableAs());

         if ($this->usesSoftDelete($models->first()) && config('scout.soft_delete', false)) {
             $models->each->pushSoftDeleteMetadata();
         }

         $index->addObjects($models->map(function ($model) {
             $array = array_merge(
                 $model->toSearchableArray(), $model->scoutMetadata()
             );

             if (empty($array)) {
                 return;
             }

             try {
                 $index = $this->algolia->initIndex($model->searchableAs());
                 $algoliaObject = $index->getObject($model->getKey());
                 if ($this->needsUpdate($algoliaObject, $array)) {
                     return array_merge(['objectID' => $model->getScoutKey()], $array);
                 }
             } catch (\Exception $e) {
                 if ((int)$e->getCode() == 404) {
                     return array_merge(['objectID' => $model->getScoutKey()], $array);
                 }

                 return;
             }
         })->filter()->values()->all());
     }

     public function needsUpdate($algoliaObject, $localObject)
     {
         if(!isset($algoliaObject['updated_at'])) {
             return true;
         }

         return $algoliaObject['updated_at'] != $localObject['updated_at'] ? true : false;
     }

 }

 public function boot()
 {
     resolve(EngineManager::class)->extend('advancedalgolia', function () {
         AlgoliaUserAgent::$custom_value = '; Laravel Scout (custom driver)';

         return new AdvancedAlgoliaEngine(new Algolia(
             config('scout.algolia.id'), config('scout.algolia.secret')
         ));
     });
 }

Don’t forget to include the use App\Engines\AdvancedAlgoliaEngine; statement.

Ensure each Searchable Model includes updated_at as a timestamp in the toSearchableArray():

 public function toSearchableArray()
 {
     $array = $this->toArray();
     $array['updated_at'] = $this->updated_at->timestamp;

     return $array;
 }

Sources:

How does Algolia count records and operations?

More Tips & Tricks

For more helpful Laravel tips and tricks, be sure to check out the new Coder’s Tape from Victor Gonzalez