Tales from Kagi

Kagi Small Web

As a part of our ongoing pursuit to humanize the web, we are pleased to announce the launch of the Kagi Small Web initiative.

What is Kagi Small Web?

To begin with, while there is no single definition, “small web” typically refers to the non-commercial part of the web, crafted by individuals to express themselves or share knowledge without seeking any financial gain. This concept often evokes nostalgia for the early, less commercialized days of the web, before the ad-supported business model took over the internet (and we started fighting back!)

For a deeper understanding, Ben Hoyt’s “The Small Web is Beautiful” serves as an excellent primer. Additionally, our GitHub repository links to several more articles on this topic.

Kagi Small Web offers a fresh approach by promoting recently published content from the “small web.” We gather new content, published within the last week, from a handpicked list of blogs and surface it in multiple ways:

Initially inspired by a vibrant discussion on Hacker News, we began our experiment in late July, highlighting blog posts from HN users within our search results. The positive feedback propelled the initiative forward. Today, our evolving concept boasts a curated list of nearly 6,000 genuine websites featuring people with a wide variety of interests.

The entire Kagi Small Web initiative, including the list of curated websites, is open-source and can be found on Kagi’s GitHub.

Small Web in Kagi Search: A Unique Flavor

Since inception, we’ve been featuring content from the small web through our proprietary Teclis and TinyGem search indexes. This inclusion of high-quality, lesser-known parts of the web is part of what sets Kagi’s search results apart and gives them a unique flavor.

Today we’re taking this a step further by integrating Kagi Small Web results into the index.

For instance, a search for “swiftui” now brings up this intriguing article from the small web. It discusses a noteworthy change in iOS 17 and comes from Keith Harrison’s “Use Your Loaf” blog. Yes, you’ve just met one of your “web neighbors.”

swiftui search example

(note that the example Kagi search URL has been generated using Kagi’s “Share this Search” feature, enabling users without a Kagi account to view the results)

And to compare to the above, see Google’s search results for “swiftui”.

Another example: a search for “apple watch” yields this recent piece from Timo’s ‘Screaming at My Screen’ blog.

apple watch example

Notice how it adds a fresh perspective to the prevailing opinions on Apple Watch, including a fascinating neighboring result about Apple Watch Pixels from 2015, coming from our Teclis index.

Compare this diversity to Google’s results for the same term.

We strongly believe that including the results from the “small web” not only brings us closer to a more humane experience in the web search, but makes the search genuinely better!

Introducing the Kagi Small Web Website

We are also launching the Kagi Small Web website, a new platform through which we want to surface all the recent content from the small web, that usually zips by in legacy search engines.

Kagi Small Web website

Designed in line with the ethos of the “small web,” our website operates without JavaScript.

Here, you can “appreciate” a post or jot down a temporary public note about it. These notes will vanish in about a week as we cycle in new content - emphasizing the fleeting, imperfect nature of the small web.

When we previewed the site to a select group of beta testers on our Discord channel, one user was especially excited to find their blog among the curated feeds. Though they hadn’t posted recently and weren’t featured on the KSW website, they felt inspired to write more. This enthusiasm serves as a direct counter to the creeping “adshitification” of the web, encouraging genuine content over commercial clutter.

Our selection criteria for the curated feeds are:

For a post to appear on the KSW website, additional conditions include:

The website is also open source, and we’re open to merge requests.

Discover Small YouTube

Explore our experimental “Small YouTube” feature here.

It surfaces content from emerging YouTube creators who have fewer than 400k subscribers. Your feedback on this criterion is welcome.

Access via Feed and API

Obtain real-time updates from Kagi Small Web through our RSS feed, refreshed every 5 hours.

For API enthusiasts, Kagi Small Web is integrated into our TinyGem index, offering high-quality, non-mainstream news from around the web at $2 per 1000 queries.

How to Contribute and Provide Feedback

We’re open to refinements. If you notice omissions or inclusions that don’t fit, please contribute to our curated list following the guidelines.

For bug reports and feature suggestions, head to kagifeedback.org and use the small-web tag to categorize your post.

For general discussions, join us on Kagi Discord.

Final Words

The driving question behind this initiative was simple yet profound: the web is made of millions of humans, so where are they? Why do they get overshadowed in traditional search engines, and how can we remedy this?

This project required a certain leap of faith as the content we crawl may contain anything, and we are putting our reputation on the line vouching for it.

But we also recognize that the “small web” is the lifeblood of the internet, and the web we are fighting for. Those who contribute to it have already taken their own leaps of faith, often taking time and effort to create, without the assurance of an audience.

Our goal is to change that narrative. Together with the global community of people who envision a different web, we’re committed to revitalizing a digital space abundant in creativity, self-expression, and meaningful content - a more humane web for all.

–

Discuss on Hacker News.
#kagi