Kagi Small Web
As a part of our ongoing pursuit to humanize the web, we are pleased to announce the launch of the Kagi Small Web initiative.
What is Kagi Small Web?
To begin with, while there is no single definition, “small web” typically refers to the non-commercial part of the web, crafted by individuals to express themselves or share knowledge without seeking any financial gain. This concept often evokes nostalgia for the early, less commercialized days of the web, before the ad-supported business model took over the internet (and we started fighting back!)
For a deeper understanding, Ben Hoyt’s “The Small Web is Beautiful” serves as an excellent primer. Additionally, our GitHub repository links to several more articles on this topic.
Kagi Small Web offers a fresh approach by promoting recently published content from the “small web.” We gather new content, published within the last week, from a handpicked list of blogs and surface it in multiple ways:
- Directly within Kagi search results for applicable queries (existing Kagi members do not need to do anything, this will be automatic)
- Via the new Kagi Small Web website
- Through the Kagi Small Web RSS feed
- Via our Search API, where results are now part of the news enrichment API
Initially inspired by a vibrant discussion on Hacker News, we began our experiment in late July, highlighting blog posts from HN users within our search results. The positive feedback propelled the initiative forward. Today, our evolving concept boasts a curated list of nearly 6,000 genuine websites featuring people with a wide variety of interests.
The entire Kagi Small Web initiative, including the list of curated websites, is open-source and can be found on Kagi’s GitHub.
Small Web in Kagi Search: A Unique Flavor
Since inception, we’ve been featuring content from the small web through our proprietary Teclis and TinyGem search indexes. This inclusion of high-quality, lesser-known parts of the web is part of what sets Kagi’s search results apart and gives them a unique flavor.
Today we’re taking this a step further by integrating Kagi Small Web results into the index.
For instance, a search for “swiftui” now brings up this intriguing article from the small web. It discusses a noteworthy change in iOS 17 and comes from Keith Harrison’s “Use Your Loaf” blog. Yes, you’ve just met one of your “web neighbors.”
(note that the example Kagi search URL has been generated using Kagi’s “Share this Search” feature, enabling users without a Kagi account to view the results)
And to compare to the above, see Google’s search results for “swiftui”.
Another example: a search for “apple watch” yields this recent piece from Timo’s ‘Screaming at My Screen’ blog.
Notice how it adds a fresh perspective to the prevailing opinions on Apple Watch, including a fascinating neighboring result about Apple Watch Pixels from 2015, coming from our Teclis index.
Compare this diversity to Google’s results for the same term.
We strongly believe that including the results from the “small web” not only brings us closer to a more humane experience in the web search, but makes the search genuinely better!
Introducing the Kagi Small Web Website
We are also launching the Kagi Small Web website, a new platform through which we want to surface all the recent content from the small web, that usually zips by in legacy search engines.
Here, you can “appreciate” a post or jot down a temporary public note about it. These notes will vanish in about a week as we cycle in new content - emphasizing the fleeting, imperfect nature of the small web.
When we previewed the site to a select group of beta testers on our Discord channel, one user was especially excited to find their blog among the curated feeds. Though they hadn’t posted recently and weren’t featured on the KSW website, they felt inspired to write more. This enthusiasm serves as a direct counter to the creeping “adshitification” of the web, encouraging genuine content over commercial clutter.
Our selection criteria for the curated feeds are:
- The blog must have a post at least three years old.
- Content must be in English (hard to curate non-English).
- The site should offer authentic self-expression or high informational value.
- No NSFW
For a post to appear on the KSW website, additional conditions include:
- It needs to be no older than seven days.
- The site should be compatible with iframe embedding. If it’s not, we’ll still index it for our search results.
The website is also open source, and we’re open to merge requests.
Discover Small YouTube
Explore our experimental “Small YouTube” feature here.
It surfaces content from emerging YouTube creators who have fewer than 400k subscribers. Your feedback on this criterion is welcome.
Access via Feed and API
Obtain real-time updates from Kagi Small Web through our RSS feed, refreshed every 5 hours.
For API enthusiasts, Kagi Small Web is integrated into our TinyGem index, offering high-quality, non-mainstream news from around the web at $2 per 1000 queries.
How to Contribute and Provide Feedback
We’re open to refinements. If you notice omissions or inclusions that don’t fit, please contribute to our curated list following the guidelines.
For bug reports and feature suggestions, head to kagifeedback.org and use the small-web tag to categorize your post.
For general discussions, join us on Kagi Discord.
The driving question behind this initiative was simple yet profound: the web is made of millions of humans, so where are they? Why do they get overshadowed in traditional search engines, and how can we remedy this?
This project required a certain leap of faith as the content we crawl may contain anything, and we are putting our reputation on the line vouching for it.
But we also recognize that the “small web” is the lifeblood of the internet, and the web we are fighting for. Those who contribute to it have already taken their own leaps of faith, often taking time and effort to create, without the assurance of an audience.
Our goal is to change that narrative. Together with the global community of people who envision a different web, we’re committed to revitalizing a digital space abundant in creativity, self-expression, and meaningful content - a more humane web for all.
Discuss on Hacker News.