Designing Better Markdown for LLMs

One of the design goals of LLM Ready was that it should work with­out requir­ing devel­op­ers to think about AI at all.

Install the plu­g­in and your pages imme­di­ate­ly become avail­able as clean Mark­down. No dupli­cate con­tent. No sec­ond set of tem­plates. No addi­tion­al main­te­nance.

I did include one escape hatch: a cus­tom Twig tem­plate that lets you com­plete­ly con­trol the gen­er­at­ed Mark­down.

If you’ve looked at the mark­down ver­sion of this page, you may have noticed I don’t actu­al­ly use that fea­ture myself.

I assumed I’d even­tu­al­ly dis­cov­er sit­u­a­tions where the Mark­down served to an LLM should dif­fer from the HTML served to a human. Months lat­er, I’m still not con­vinced rewrit­ing the con­tent is the right answer.

That has led me to a sim­ple ques­tion:

Should the Mark­down ver­sion actu­al­ly say some­thing dif­fer­ent than the HTML ver­sion?

My cur­rent think­ing is: most­ly no.

The Mark­down ver­sion should­n’t become an ​“AI ver­sion” of your web­site. It should com­mu­ni­cate the same infor­ma­tion, just in a for­mat that’s eas­i­er for a lan­guage mod­el to under­stand.

For exam­ple, a page designed for humans might rely on cards, icons, side­bars, or visu­al hier­ar­chy to com­mu­ni­cate rela­tion­ships. A Mark­down ver­sion can make those rela­tion­ships explic­it with head­ings, lists, and clear­er orga­ni­za­tion with­out chang­ing the under­ly­ing mean­ing.

In oth­er words, the Mark­down rep­re­sen­ta­tion shouldn’t be seman­ti­cal­ly dif­fer­ent. It should be seman­ti­cal­ly rich­er.

Abstracts

The idea I like most is a short, fac­tu­al abstract near the top of every Mark­down page.

Some­thing like:

Abstract: This page doc­u­ments the LLM Ready plu­g­in for Craft CMS. It explains how the plu­g­in expos­es Mark­down rep­re­sen­ta­tions of pages through URL suf­fix­es and con­tent nego­ti­a­tion, gen­er­ates llms.txt auto­mat­i­cal­ly, sup­ports cus­tom Mark­down tem­plates, and allows devel­op­ers to cus­tomize the con­tent served to AI sys­tems with­out chang­ing the HTML pre­sent­ed to human vis­i­tors.

This is inten­tion­al­ly dif­fer­ent from a meta descrip­tion.

A meta descrip­tion exists to earn a click. It’s often mar­ket­ing copy.

Con­vert your Craft CMS con­tent into AI-friend­ly Mark­down with auto­mat­ic llms.txt gen­er­a­tion and cus­tomiz­able tem­plates.

An abstract has a dif­fer­ent pur­pose. It isn’t try­ing to per­suade any­one — it exists to accu­rate­ly describe what the page con­tains.

Humans rarely need this because they can skim head­ings, images, and page lay­out.

An LLM, how­ev­er, often has to decide with­in a few sen­tences whether a page con­tains the infor­ma­tion it’s look­ing for. A con­cise abstract func­tions much like the abstract of an aca­d­e­m­ic paper: it pro­vides high-val­ue con­text.

LLM Ready 1.4.0 already sup­ports using the descrip­tion field from sev­er­al pop­u­lar SEO plu­g­ins as the page’s abstract.

The more I’ve thought about it, though, the more I think those two fields serve dif­fer­ent pur­pos­es. They often over­lap, but I’m no longer con­vinced they’re the same thing — or that a meta descrip­tion is the ide­al source for a page’s machine-read­able abstract.

My Current POV

I still haven’t found a com­pelling rea­son to rewrite pages for AI.

But I have start­ed to won­der whether every page should have an abstract — not for search engines or human read­ers, but as a machine-read­able descrip­tion of what the page actu­al­ly con­tains.

That feels like a much more inter­est­ing direc­tion.

If you’re gen­er­at­ing Mark­down for AI con­sump­tion, I’d love to hear how you’re approach­ing it. Are you serv­ing the same con­tent as your HTML pages, or have you found cas­es where the machine-read­able rep­re­sen­ta­tion should dif­fer?

You can reach me by vis­it­ing the con­tact page.