Unlocking Real-Time Reactive Search in Craft CMS with Datastar

The narration of this post was created with Bespoken plugin for Craft CMS.

This post will show you how the search page on this site works and how you can build the same thing yourself in Craft CMS.

Please note that the narrated version of this post does not contain a spoken version of the code samples, but you can view those on the original blog post.

What makes the search page interesting enough for a long blog post? The short answer is that it functions like the search you'd expect to be built with a big frontend framework like React, Vue, or Svelte. Instead, it's built with a tiny 14kb library called Datastar. The Datastar plugin for Craft CMS makes using it even easier. If you're a Craft developer, it's a no-brainer addition to your toolkit.

The search page is a "reactive search," meaning that as you type, the search results are updated on the page without a page load.

Also, each search is bookmark-able. For example, this link will show you entries on this site for "craft cms." The page also fully supports using your browser's back and forward buttons. See this for yourself by clicking this link to see blog posts mentioning "font sans". Then use your delete key to remove the word "sans" from the search field. You should see many more results show up. Then click the back button in your browser, and you'll go back through your browser history to see "font sans" results again. It works the way you'd expect it to work.

To do that takes some work from you as the developer. I'll show you how Datastar makes this easy.

Datastar?

So, I hear you asking, "What is Datastar?" Let's look at the Datastar getting started page for some help:

Datastar helps you build reactive web applications with the simplicity of server-side rendering and the power of a full-stack SPA framework.

In other words, you don't need an oversized framework to build a reactive app. (At the end of this post, I'll touch on how Alpine, a small frontend framework, differs from Datastar.)

Why is it called "Datastar?" We will use data attributes on HTML elements throughout the code below. Data attributes in HTML are custom attributes you attach to HTML elements using the data- prefix. These attributes store additional information directly in the markup without cluttering classes or IDs. In other words, we’re not introducing non-standard HTML to our code when using Datastar.

So, the word “data” makes sense. What about the “star” part of the name? The asterisk is a wildcard character. If you type ls *.txt on your command line, you’ll list all files ending with .txt. An asterisk character is sometimes called a “star”. In other words, data-* defines the scope of the Datastar language, hence, the name Datastar.

Using Datastar with Craft CMS

We will use the free Datastar plugin in the Craft CMS plugin store. If you're not a Craft developer, you will likely get a feel of how Datastar, the JavaScript framework itself, works in this post. It might make you interested in picking up Craft CMS as well. (Come on in. The water's fine.)

Thank you

Before we go further, I want to thank Ben Croker for reviewing and improving the code we'll be going over. Ben is the creator of the Craft plugin for Datastar and one of the maintainers of the Datastar project itself. His input has been invaluable. Thanks, Ben! ⭐

What we're going to build

I’ll walk you through five iterations that progressively enhance the user experience of the search. By the end, you’ll better understand Datastar’s capabilities and how it can elevate your Craft CMS projects.

About the source code

You can download all of the templates in this post from my Github repo: https://github.com/johnfmorton/craft-cms-datastar-examples.

For each step we go through, you'll be referencing two files. I've tried to organize it logically, but let's spell it out.

For step 1, the first file you'll look at is called datastar-search-v1.twig. This is the file that contains the search input field. It has a companion, _datastar/search-results-v1.twig , which we'll load through an AJAX request.

I've stripped out nearly all styling to make the source code easier to read. You'll also be able to interact with versions of these files I have hosted on my blog.

Step 1: Basic real-time search

The first iteration sets up a simple search input that dynamically filters results as the user types. This implementation will:

Use Datastar to fetch filtered data without a full-page reload.
Provide immediate feedback to users as they type.
Use Datastar's built-in debounce functionality to reduce the load on your server.
Eliminate the need for complex front-end logic to handle search requests.

This approach significantly improves the user experience compared to a traditional search that requires form submission, where the page reloads with search results. Note that we're not implementing the database search until we reach step 4, so we can focus on the search functionality's UX.

Reference files from the repo:

Open https://supergeekery.com/datastar-search-v1 in a new window and try the following:

Select the input field with your cursor or use the tab key to focus it.
Type in a word, like "cat".
Take note of the content in the ctx.signals area. This is a debugging display of the Datastar data on the search page itself.
Also note the debugger area and the search results area. This content is generated in the search results page.

What’s a signal?

Signals are the data we’re working with on the page. I initially thought of signals as variables, which is somewhat true. However, since we’re diving into Datastar, which uses signals, let’s explore the concept of a signal a little more.

A signal is an object or function that holds a value. That sounds sort of like a variable, right? The difference is that a signal has some additional behavior. When the signal’s value changes, any functions or components that depend on it are notified so they can update themselves automatically. So a signal is a variable with extra reactive abilities. With that out of the way, let’s look at the code.

The code

First look at the datastar-search-v1.twig file.

You’ll see a div element with an id of “container”. This element is the parent of all the reactive elements we will manage with Datastar. Look at the data-signals attribute on this container element. This is where we’re defining the data for our reactive search. You’ll see this attribute evaluates to a JSON-like string. In that string, we’re defining two signals (aka, “variables”) and giving them some default values. The signal “phrase” is an empty string and the signal “counter” is 42.

Next, look at the “input” field. There are two data-* attributes.

data-bind-phrase : The data-bind-* attribute binds the value of this input field to the signal we defined in the parent container. If this was data-bind-counter, this input field would have displayed “42” instead of an empty string.

data-on-input: This attribute is adding an event listener to this input field. We’re listening for the input event. This event is automatically fired when a user types a character into the input field. Although it wouldn’t make a lot of sense with our UX, we could have added a click event listener with data-on-click here.

Many data attributes may contain data when using Datastar. In our input event listener, we'll use a Twig function called the Datastar Craft plugin.

data-on-input="{{ datastar.get('_datastar/search-results-v1.twig') }}"

Our input event will trigger a GET AJAX request on your server and load the search results in the Twig file. Before we leave this Twig template, let's talk about updating the input event with debouncing. Debouncing will prevent every keystroke from issuing an AJAX request. Now you can type "cat" without searching for each individual letter if you type the word quickly enough. Datastar makes adding debouncing very easy by adding the word "debounce" to our code. We'll use a debounced input event listener from now on.

data-on-input__debounce.200ms="{{ datastar.get('_datastar/search-results-v1.twig') }}"

Let's open the second file. Look in the _datastar directory for search-results-v1.twig. This is the file that generates the content of our AJAX request.

This seems pretty straightforward if you don't think too hard about it. But, the more you think about it, the more impressive it becomes. The search-results-v1.twig receives the signals, both the unchanged and updated ones, from your datastar-search-v1.twig file and sends back HTML fragments that will be inserted into your page in the correct spots. You don't have to mess with setting up GraphQL or an Element API. It just works.

Fragments and IDs

I've mentioned "fragments" a couple of times. A fragment is basically an element in the HTML, identified by an ID, that will get swapped out with new content when the AJAX returns its data. If you look at the two Twig files, you'll see both contain a div with the ID of "searchResults." You can have multiple fragments returned from the AJAX request. You'll see another element, a pre element, with the ID of "debugger." Both fragments will be returned and placed in separate locations in the page based on where the elements with the associated ID are located in the HTML.

Are we done?

This is going to be a long blog post. If bookmarking and browser history aren't relevant to you, skip to step 4, which discusses how to make Craft use the search phrase to generate search results.

If you want to take the long journey with me, let's keep going because there is more we can do to build a richer search page. Think of steps 2 and 3 as extra credit.

Step 2: Enhanced bookmarkable search results

In the second iteration, we refined the search. This implementation will:

Use query parameters to reflect the user’s search input, enabling users to share or bookmark search results.
Make searches persist across page reloads.
Prevent unnecessary requests when there is no phrase to search.

This iteration enhances usability and accessibility, ensuring that search results are not lost when navigating away or refreshing the page. However, it introduces some edge-case UX problems we will fix in step 3.

Reference files from the repo:

Open https://supergeekery.com/datastar-search-v2 in a new window and try the following:

Notice that the search field has focus. This is a search page, so we will assume that the user is here to search and we focus input on the input field.
Type a word in the text input. Along with the live rendered search results, notice that the page URL reflects the phrase you entered.
Notice that the page title includes the phrase we're searching for.
We also track the number of searches executed. (I did this for debugging purposes, which will make more sense in step 3.)
Refresh the page, and the same search results will be displayed.
You can add another word while keeping the existing word you typed. You'll see the URL, and the search results will update again.
Finally, use the back button in your browser and return to the previous search result.

We need to review both files, but I will start with search-results-v2.twig this time. This part of the process will do the database search using Craft's search functionality. We will also update the URL parameter and the document title inside a special section called executescript that allows you to execute JavaScript when Datastar gets an update from the server.

I relied exclusively on all JavaScript here in an earlier version of this file. Ben encouraged me to lean into Twig as much as possible, which is what you see here. It makes the page logic easier to understand. (If you're using another server-side solution other than Craft CMS, check out examples in PHP, Go, C# and others in the documentation.)

{% executescript %}
    {% set phrase = signals.phrase|trim %}
    const url = new URL(window.location.href);

    {% if phrase %}
    url.searchParams.set("phrase", '{{ phrase }}');
    {% else %}
    {# Remove the query parameter if the search phrase is empty #}
    url.searchParams.delete('phrase');
    {% endif %}

    {# Update the URL without triggering a page reload #}
    window.history.pushState({}, '', url);

    {# Update the title of the page #}
    document.title = `Search: {{ phrase }}`;
{% endexecutescript %}

The JavaScript is pretty easy to understand. We get the url and set the phrase parameter to the signal.phrase. If one is not set, we remove the URL parameter entirely. We push the new URL to the window's history and update the document's title. The JavaScript inside the executescript block is executed whenever we successfully issue an AJAX request from the datastar-search-v2.twig page. We'll now take a look at that file.

We set focus on the input element using basic JavaScript. Look at the script section.

Setting the page's initial phrase based on the query parameter is not much more complicated. If it is present in the URL, we grab the phrase query parameter using Twig and then set our variable to that phrase or fall back to an empty string.

{% set phrase = craft.app.request.getQueryParam('phrase') ?? '' %}

The container element now uses that information to set its phrase signal. The counter signal has been renamed to searchCounter to better reflect what we're keeping track of.

<div id="container" data-signals="{phrase: '{{ phrase }}', searchCounter: 0 }">

The text input's event listener has been updated slightly to increment the searchCounter signal when we issue the AJAX request. This is how we track how many searches we have made.

data-on-input__debounce.200ms="{{ datastar.get('_datastar/search-results-v2.twig') }} && $searchCounter++;"

We added a new on page load event listener to the input element. It will check if the phrase has content when that page is first loaded. If it does, we'll issue the AJAX request for search results. This makes our search results bookmarkable. To see it work, input a search term on the example page and then hit the browser reload button. The same search results will be displayed.

data-on-load="$phrase.length >= 1 && {{ datastar.get('_datastar/search-results-v2.twig') }} && $searchCounter++;"

If you click the back button in your browser, it won't work as expected. The search request for the phrase won't be executed. We need to set up an event listener on the search page to listen for the popstate event. If that event is heard, we retrieve the phrase from the URL and update the signal by changing the value in the input field and manually triggering an input event.

window.addEventListener('popstate', (event) => {
  // When the browser navigates back or forward,
  // the current URL already reflects the desired state
  // so we don't need to update the URL or the input field.
  const currentUrl = new URL(window.location.href);
  const searchPhraseValue = currentUrl.searchParams.get('phrase') || '';

  // Update the input field value, which is bound to the Datastar signal
  searchPhrase.value = searchPhraseValue;
  searchPhrase.dispatchEvent(new Event('input', {bubbles: true}));
});

The script I've shared is how I initially wrote this JavaScript. However, there is an error in this code that's not obvious. Can you spot it?

The problem is the dispatched input event and the action we take based on that event in the search-results-v2.twig page.

You may wonder why I needed to dispatch the event manually. The phrase is updated in the input field when the user clicks the back button. Wouldn't that cause the input event to fire? No, it won't. The input event documentation reveals why.

The input event fires when the value of an input, select, or textarea element has been changed as a direct result of a user action (such as typing in a textbox or checking a checkbox).

If the user does not type directly into the input field, no input event will be fired, and no search will be made. We can try to fix the problem by manually triggering the input event using the dispatchEvent function.

searchPhrase.dispatchEvent(new Event('input', {bubbles: true}));

So what's wrong with this solution? Go back to the live example page at https://supergeekery.com/datastar-search-v2 and try to do the following.

Type in "cat" and see the search result.
Add "dog" to the search field and see the result.
Add "fish" to the search field and see the result.
Now use the back button in your browser and see the result for "dog." It appeared to work.
Now use the back button again. The page will skip back and miss the "cat" search result entirely.

Since we push state to the window history every time we execute a search in the search-results-v2.twig file, we unintentionally corrupt our browser history when the browser's back and forward buttons are used. We will fix this in the next step of the process by introducing custom events.

Step 3: Fix the user experience with custom events

This section will fix our broken UX when the browser buttons are used. This implementation will:

Add a custom event to prevent the unintended rewriting of browsing history.
Allow the back and forward buttons in the browser to work as expected.

The example page is at https://supergeekery.com/datastar-search-v3.

We need a way to update the browser's history conditionally and Datastar has us covered with support for custom events.

Reference files from the repo:

Let's look at the search page to start. In the container element, we're now tracking additional signals: lastPhrase and browserHistoryCounter. We will store the last successful search term and use it in the logic in the code below. The browser history counter is helpful for debugging. It allows us to differentiate between new search queries and queries based on a popstate event.

In the code, we see the input event listener has been updated.

data-on-input__debounce.200ms="($phrase.trim() !== $lastPhrase.trim()) && {{ datastar.get('_datastar/search-results-v3.twig', { updateHistory: true }) }} && $searchCounter++, $lastPhrase = $phrase.trim();"

First, we ensure the phrase and last searched phrase are not the same. This prevents a new search from being fired if only a space character is added after a word. If our conditional returns true, we'll execute the search.

In the search result request, we are leaning on another feature of the Datastar plugin, which lets us send additional data along with our request. The updateHistory variable allows us to control if we need to add a URL to our browser history. We'll update the search results template shortly to use this data. We still have work to do here first. We must differentiate between an input event triggered on the text input field and a popstate event triggered on the window when the back button is used.

We'll do that by creating a custom event that allows us to pass around extra data. Let's go over the script section of the page.

const container = document.getElementById('container');
document.addEventListener('DOMContentLoaded', () => {
  const searchPhrase = document.querySelector('[data-bind-phrase]');

  searchPhrase.addEventListener('input', (e) => {
    console.log('searchPhrase', e.target.value);
  });
  // set focus on the search input
  searchPhrase.focus();

  window.addEventListener('popstate', (event) => {
    // When the browser navigates back or forward,
    // the current URL already reflects the desired state.
    const currentUrl = new URL(window.location.href);
    const searchPhraseValue = currentUrl.searchParams.get('phrase') || '';

    // Update the input field value, which is bound to the Datastar signal
    searchPhrase.value = searchPhraseValue;

    // a new custom event that we can listen for
    const historyChangeEvent = new CustomEvent('historychange', {
      detail: {
        phrase: searchPhrase.value,
      },
    });

    // Dispatch the custom event on the search input
    searchPhrase.dispatchEvent(historyChangeEvent);
  });
});

When there is a popstate event on the window, we dispatch a custom historychange event on the input field and then listen to it with a Datastar attribute. Here, we update the phrase and make the search request with the update history value set to false.

data-on-historychange="$phrase=evt.detail.phrase; $browserHistoryCounter++; {{ datastar.get('_datastar/search-results-v3.twig', { updateHistory: false }) }}"

Let's now look in the search-results-v3.twig template file.

We use the update history value to push a new item to our browser history only when a new search term has been entered. Now our browser buttons work as expected.

{% if updateHistory %}
    window.history.pushState({}, '', url);
    // Update the title of the page
    document.title = `Search: {{ phrase }}`;
{% endif %}

These changes fix our UX bugs. Users can use the forward and back buttons as much as they like, and the page will work as expected.

Step 4: Implementing the Craft CMS search functionality

Now we will add the actual search results to our template.

The example page is at https://supergeekery.com/datastar-search-v4.

Reference files from the repo:

We're only going to change the search results template. The code is straightforward. This simple Twig code searches your entries for the phrase. I'm searching in the blog section of my site. matchon

{# update the section from 'blog' to whatever section you want to search in your own Craft site #}
{% set results = craft.entries()
    .search(signals.phrase)
    .orderBy('score')
    .section('blog')
    .collect() %}
{% else %}
    {% set results = [] %}
{% endif %}

The search results fragment loops through the search results and displays them in a list.

{% fragment %}
    <div id="searchResults">
        {% if results | length %}
        <h2 class="text-lg font-bold">Search results ({{ results | length }})</h2>
        <ul class="list-inside list-disc mt-1">
            {% for result in results %}
                <li class="ml-4">{{ result.title }} - {{ result.url }}</li>
            {% endfor %}
        </ul>
        {% elseif signals.phrase | length %}
            <h2 class="text-lg font-bold pb-2">Search results</h2>
            <p class="italic">No results found for <span class="font-bold">"{{ signals.phrase ?? '' }}"</span></p>
        {% endif %}
    </div>
{% endfragment %}

It's hard to overstate how simple this part is to create. There is no parsing of JSON from an API. You're not doing a GraphQL query. This is basic Craft Twig code. Even if you're new to Craft, this probably makes sense. You've made it this far so let's take it one step further.

Step 5: Adding complexity with additional user-selected search options

In this step, we add a new feature that lets users refine their search by adding radio buttons to alter the search behavior. I use my site for blog posts and as a bookmarking database. The user can now select which section they search on the site.

The example page is at https://supergeekery.com/datastar-search-v5.

Reference files from the repo:

We've reviewed the templates thoroughly, so let's only go over the changes. (I won't cover it in this post, but you'll find an example of Datastar's class binding in the code, too.)

At the top of the page, we set a new variable called sections, which is retrieved from the URL, and use it on the container to add it to the Datastar signals.

{% set sections = craft.app.request.getQueryParam('sections') ?? 0 %}
 <div id="container"
    data-signals="{phrase: '{{ phrase }}', searchCounter: 0, browserHistoryCounter: 0, lastPhrase: '', sections: '{{ sections }}'}">

We have added three radio buttons. The options are to search all sections, only the blog section, or the saved links section. Each radio button has a unique value and we bind the sections signal to each one.

Each radio button also has an on-change event listener. Since the event listener is identical for each one, we use a Twig expression to keep our code DRY.

{% set expression = '$phrase.trim().length && ' ~ datastar.get('_datastar/search-results-v5.twig', { updateHistory: true }) ~ ' && $searchCounter++; $lastPhrase = $phrase.trim()' %}
<input id="globalSearch" data-on-change="{{ expression }}" type="radio" data-bind-sections value="0">

When the user modifies the searched section, this expression tells Datastar to perform a new search. We've also updated the script section of this page to incorporate the sections variable for the history change event.

const historyChangeEvent = new CustomEvent('historychange', {
  detail: {
    phrase: searchPhrase.value,
    sections: sections
  }
});

Now let's examine the search results file. We need to discuss an important security feature here.

Notice that only two values are acceptable for the sections attribute. A malicious user can't alter a search URL to query sections other than those we intend because we check for acceptable values before we process the search. For more information, read the security page on the Datastar site, which discusses protecting yourself when accepting user input.

{# Default searching both sections #}
{% set sections = ['blog','savedLinks'] %}
{# Only reset the selection if it is one of the expected values #}
{% if signals.sections == '1' %}
    {% set sections = "blog" %}
{% endif %}
{% if signals.sections == '2' %}
    {% set sections = "savedLinks" %}
{% endif %}

We now have a well-built search page.

One more thing: View transitions

If you've been using the example pages, did you notice how smoothly the new search results appeared on the page?

Datastar, the JavaScript framework, lets you use view transitions when server-side events send back new fragments. Since we've used the Datastar Craft Plugin, turning this on is as simple as updating your config file.

Check out the config file from the repo we've been referencing in this post. The useViewTransition option is set to true to add view transitions to fragments. It couldn't be simpler.

/**
 * The fragment options to override the Datastar defaults. Null values will be ignored.
 */
'defaultFragmentOptions' => [
    'settleDuration' => null,
    'useViewTransition' => true, // <-- The only option changed from the default value (null) for this project
],

Closing thoughts on Datastar

I hope you share some of my excitement for Datastar. I admire its straightforward syntax and the elegance of how it works.

When I first heard about Datastar, I thought it was another Alpine.js, which has worked well for me. However, Alpine's domain only covers the front end of a site. Datastar is a tiny framework that handles both frontend and backend needs.

This site's search page was previously Alpine-powered and used an Element API endpoint. That version of the page couldn't bookmark a search phrase, which meant the browser history problem hadn't presented itself. Rebuilding the page with Datastar impressed me so much that I wrote this post to help spread the word. From now on, I plan to include Datastar in my default starter projects.

Would you have built this search differently? Hit me up on the Craft CMS Discord or social media. Cheers.