Building a map of the GeminiNet

Table of Contents

Introduction

The small web is nice. It’s really nice. It reminds me of that era when corporations were too focused on traditional media such as television or radio and left us alone on the internet where we roamed forums, chatted on IRC or made dirty webpages with HTML and CSS we learnt through weird txt files on the internet.

Gemini is also very nice. Webpages feel fresh and discussion forums have high conversation quality. You can discuss stuff without trolls (mostly), without well organized public opinion management groups. Without ads because corporations have little to no incentive to spend money in a protocol where you can’t put flashy videos and client-side scripts to track consumption habits (well, they’re not part of the protocol specification anyways).

However, I feel that unlike other decentralized services such as Mastodon or Lemmy which you can browse through your usual browser, Gemini has not received a lot of love. Despite a lot of effort by the community and capsules (the Gemini version of webpages) that exist to track the development, it’s not super intuitive and I wanted to see how it was shaped. That’s why I attempted to build a map of Gemini capsules and how they’re interconnected with each other.

Methodology

The crawling

First of all I want to apologize to capsule owners. If someone made a lot of requests to your address, it was possibly me. Hope I didn’t bring down your server or anything.

I think it’s important to clarify that this is a partial crawl. It’s not supposed to be a complete image. I started crawling on gemini://geminiprotocol.net and went through the known capsule lists which are hosted by communities, but I’m sure there are a lot that banned me and couldn’t continue, and others that simply are not listed there.

Something very very nice about Gemini is that the specification is really small. So small, in fact, that anyone with a bit of time can make a rudimentary client and connect to all the available capsules. This is something you absolutely cannot do with the modern net in a semipractical way since it’s packed with edge cases, security concerns, headers and modes that you have to handle gracefully.

The crawler I made to crawl (…) the Geminisphere was basically a botched down version of the one I made for myself. It’s a simple rust client, supports most of the specification (I would like to think) and so it generally works.

In fact, I made 3 versions of that botched down client. * First version was for crawling the capsules and writing the information into a SQLite database. * Second version was for normalizing the database. * Third version was for fetching the root of each hostname.

I probably didn’t need the third. Oh well.

There were some edge cases with crawling I didn’t expect. The Gemininet is estimated to have around 1.042.214 URLs among 4.723 (!) capsules according to gemini.bortzmeyer.org/software/lupa/stats.gmi. It’s really small! However, the problem is precisely that it’s small. A lot of those servers are hosted in a raspi or an old laptop and you can’t make a lot of requests to them.

Which is hard to do when a lot of websites have loops and CGI URLs. They have text-based applications and games for which I had to revise the database from time to time to prune URLs and hostnames. Another pain in the ass were git repositories since they tend to be quite big. At one point the database reached 45GB with 80% of that being trash URLs.

And trash URLs there are a lot. The Gemtext specification defines that:

All the following examples are valid link lines:

=> gemini://example.org/
=> gemini://example.org/ An example link
=> gemini://example.org/foo Another example link at the same host
=> foo/bar/baz.txt  A relative link
=> gopher://example.org:70/1 A gopher link

…which means that a lot of websites do things like => wikipedia.org expecting you to handle it gracefully. I mean, that’s okay. They’re not that much. Still annoying though.

There are also search engines and archiving websites that can really fatten up your crawling if you’re not careful.

Finally, you have the dots. Ahhh, the dots. gemini://foo.com/bar/../bar/../bar/./bar/../bar. Here’s where the normalization client comes in.

Summarizing the capsules

Once I had a neat small database of around 10GB pre-pruning some trash parts, I configured a local llamacpp instance with Qwen3.5 8B for summarizing the content of those capsules and capsule subdomains. The parts I was most interested in were the description and categories, but I also wanted to know its language and more detailed categories, so I set this format in the end:

{
    "hostname": ...,
    "language": ..., # two letter code
    "category": hosting|personal|bbs|forum|others # didn't want to complicate myself too much
    "subcategories": [ ... ],
    "description": ...
}

I think you can know a lot of useful information about a capsule from that. Then I made a database search for subdomains and expanded the database(JSON, not the SQLite one) such that:

    "hostname": ...
    "links": [
        {
            # the websites that hostname points to
        }
        ...
    ],
    "size": ... # the internal number of URLs
    "error": ... # error message or null if it was ok
    "redirect_to": ... # redirect address or null if it was ok
    "language": ...
    "category": ...
    "subcategories": ...
    "description": ...
    "subdomains": [
        # same as its parent except this one doesn't have further subdomains
        # for simplicity
    ]

For simplicity I considered subdomains to be anything that has a dot before the TLD.

Curating the information

All in all the descriptions seemed to be reasonable. Although I’m sure there is some weird thing around. A lot of the tags were very similar (say, blog/blogs, or site/personal webpage), so I spent some time curating them. Categories don’t make much sense if there’s going to be only 1 member in them after all.

If you see your capsule and want to change something about it, please make a pull request and I’ll accept it with no issues (if you want to make an issue on the repo, that’s ok too).

Presenting the information in a visual way

Which was half the point of doing all this to begin with.

I picked a force graph for the representation since it looked like a reasonable way of understanding the size and outgoing links of capsules, as well as having different levels of detail. I suck at front-end so I’m sorry if it doesn’t work on phone.

You can see the map here.

Some statistics

Basic stats

The database post-pruning ended up being 6.7GB. You can download it here. The JSON ended up being 34MB. I’m sure it can be compressed in some way. I’m sorry if you’re low on data.

Some of the websites with the most URLs

Resource types

I ignored common ones such as .gmi. It makes sense that .txt is common as well since it’s frequently used as a replacement for .gmi. Other than that, pictures and zip files seem to have the highest number of resources.

Languages

It’s clear that the Geminisphere is predominantly English. Other than that, Western languages seem to be the most common.

For comparison, these are the language statistics from gemini.bortzmeyer.org/software/lupa/stats.gmi as of April 2026:

Note that my statistics are based on the hostname root instead of URLs, while Bortzmeyer’s are based on header tags. However, they are similar.

External protocols

I omitted those under 20 since they were not very common. It seems that Gopher still is the most popular alternative to Gemini. This makes sense since one of the main requirements of Gemini was to provide a more modern alternative to Gopher. Finger and Telnet are also there.

Some people left file:// with local paths, which indicates that Gemini users are also humans.

Lots of mail addresses too. I don’t know what’s with the Mozilla and Chrome extensions.

Wrap-up

Well, that was fun.

I’m bad with wrap-ups. Hopefully this helped increase the visibility of Gemini. Having alternative networks such as it is very important in a time where countries are even planning to set barriers to even start using computers or installing your own operating system.

If you haven’t tried Gemini, I suggest having a look at the specification and planning for a weekend project. I might never build a web browser, but I would like to think this helped me understand a bit more of how all this works.

Of course, I know not everyone is able or willing. For those people, I recommend this short introduction to Gemini which explains serves as an introduction in a much better way than I could.