Blog Detail


Laravel Site Search - Crawl & Index Your Website for Seo cover image

arrow_back Laravel Site Search - Crawl & Index Your Website for Seo

Laravel-site-search is a package introduced by Spatie. This package can crawl and index one or more sites. You can think of it as a private Google search. What gets crawled and indexed can be highly customized. Under the hood, Meilisearch is used to provide blazing fast search speeds. You can add any custom property that you want, and even add synonyms. When crawling your site, multiple concurrent connections are used to speed up the crawling process.

The laravel-site-search package is not tied to Eloquent models. Like Google, it will crawl your entire site and index all content that is there. You can utilize this package to index non-open-source stuff as well.

Installation & Setup

You can install the laravel-site-search via Composer:

composer require "spatie/laravel-site-search:^1.0.0"

Publish migrations

Next, you should publish the migrations and run them:

php artisan vendor:publish --tag="site-search-migrations"
php artisan migrate

Schedule the crawl command

This package contains a command that will crawl your site(s), and update the indexes. In most cases, it’s best to schedule that command, so you don’t need to run it manually.

In the example below, we schedule to run the command every three hours, but you can decide which frequency is best for you.

// in app/Console/Kernel.php
use Spatie\SiteSearch\Commands\CrawlCommand;

protected function schedule(Schedule $schedule)
    // other commands
    // ...

Publish the config file

Optionally, you can publish the config file with this command.

php artisan vendor:publish --tag="site-search-config"

Install the Meilisearch client

Next, you have to require the Meilisearch PHP client:

composer require meilisearch/meilisearch-php

Install Meilisearch

This package uses Meilisearch under the hood to provide blazing fast search results. You can head over to the Meilisearch docs to learn how to install it on your system.

Here are the steps for installing it on a Forge provisioned server. You must first download the stable release:

curl -L | sh

Next, you must change the ownership and modify permission:

chmod 755 meilisearch 
chown root:root meilisearch

After that, move the binary to a system-wide available path:

sudo mv meilisearch /usr/bin/

Finally, you can run the binary and make sure it keeps running. In the Forge Dashboard, click on “Daemons” under “Server Details”. Fill out the following for a new daemon:

  • Command: meilisearch --master-key=SOME_MASTER_KEY --env=production --http-addr --db-path ./home/forge/meilifiles
  • User: forge
  • Directory: leave blank
  • Processes: 1

Authenticating requests to Meilisearch

To avoid unauthorized persons making requests to Meilisearch, either block Meilisearch’s default port (7700) in your firewall or make sure all requests use authentication.

For more details, You can head over to its complete documentation.

Published at : 27-10-2021

Author : Rizwan Aslam
Rizwan Aslam

I am a highly results-driven professional with 12+ years of collective experience in the grounds of web application development especially in laravel, native android application development in java, and desktop application development in the dot net framework. Now managing a team of expert developers at Codebrisk.

Launch your project

Launch project